In 2022:

◆ Moved to Lisbon 🇵🇹
◆ Joined @worldcoin 🪩
◆ Launched 65 projects 🚀
◆ Spoke at 6 conferences 🎤
◆ Traveled to 11 countries ✈️
◆ Taught myself Solidity &amp; Rust 🦀
◆ Made frens all around the world 🌍

Insane year, next one will be even better. Let's go 2023!

✨ So you want to add AI image generation to your website?

Rarely, Stable Diffusion will generate NSFW images from regular prompts. The default filter catches it, but your users getting a black image isn't much better. 😅

How can we get around this? With a new model 👇

The default filter Stable Diffusion comes with is pretty good, and usually manages to catch these "exceptions".

(it's easy to manually bypass and only filters sexual content, but that's for another thread 👀)

When it triggers, SD will replace your image with a black square. ⬛️

This essentially fixes the problem of accidentally showing NSFW when users don't expect it, but at the cost of UX (you either show an error or return a black square, none of which are expected).

Can we do better? Yes! Let's embed the filter into the model itself 💡

Introducing Stable Latent Diffusion, does exactly that.

It operates at the denoising step of SD, and "pushes" generated images to be farther away from inappropiate content (defined as: hate, harassment, violence, self-harm, sexual, shocking, illegal).

https://arxiv.org/abs/2211.05105

And the best part: it requires no extra training/tuning! 🎉

This means it'll work with your existing models (so you don't have to redownload those enoromus ckpt files).

If you're using @huggingface's transformers lib, it's literally a one line change!

I've also deployed it on @replicatehq, which provides a super simple API to make integration even easier (and a pretty nice interface to try it online). 🚀

(cog repo lives at https://github.com/m1guelpf/cog-safe-diffusion)

https://replicate.com/m1guelpf/safe-latent-diffusion

Keep in mind, the Stable SD model above is new, and might produce worse results than the original, mess things up, or miss some stuff. 🚧

(you still have the original content filter backing you up if something goes wrong, so worst case is still black square)

If you try out the model, make sure to let me know how it went!

And if you're interested on learning more about the default SD content filter, I've got a thread coming on how it works, its flaws, and how to use it for non-AI images, so stay tuned for that soon 👀

Making Stable Diffusion Safer for Third Parties

✨ Defaults are incredibly important for image models.

Most users won't spent hours tweaking prompts, they'll type the first thing that comes to their minds and run with the result.

Fortunately, it's pretty easy to optimize for that, and generate better images by default. 🤯

Image models like Stable Diffusion or DALL-E are so amazing because they're general purpose.

They can generate drawings, photographs, icons and logos, 3D environments, and much more! 🚀

However, good results require work: paragraphs of prompts and constant iteration.

If you're building a consumer-facing app using these models, most of your users won't do that.

They'll write "a picture of a cat" expecting the amazing results they've been seeing around, and churn when the result inevitably disappoints. 🥲

How do we fix this?

One solution is to make models less general-purpose (aka fine-tuning). 🔩

Similar to how you can teach AI a new concept (like your face, to make cool avatars), you can also teach it new styles.

This allows you to select for the most aesthetic results on short prompts. 🤩

I faced this issue when I made my first Stable Diffusion Twitter bot. Most people would write short prompts, and results were unimpressive.

So I rebuilt it to use a fine-tuned model (try it out below!) as an experiment.

Judge for yourself, but I'd say results are much better 😁

https://twitter.com/m1guelpf/status/1592359396629106689?s=20&t=Vqi7_iljEsN3ppfIdu5wDQ

For the tweet above, I used a model fine-tuned on Midjourney v4 images.

@midjourney is another image model, and their newest version is great at short prompts.

However, they don't have an API or are as cheap as SD. Fine-tuning on selected images gets us the same quality! ✨

The @huggingface community is full of amazing fine-tuned models which produce way better base results! Some of my favorites:

- mdjrny-v4 style (the one I used) by @prompthero: https://huggingface.co/prompthero/midjourney-v4-diffusion

- redshift style (insane 3d results) by @Nitrosocke: https://huggingface.co/nitrosocke/redshift-diffusion

And, if you have a specific style in mind, it takes 8-30 images on that style and around an hour to train a new model! ⚡️

I'm thinking of making a guide on how to do that, so let me know if you'd be interested in reading that! (by RTing the first tweet of this thread hehe 😁).

We're living in truly magical times ✨

We can create images in seconds for hardly any money at all (my twitter bot cost $10 for +1.5k images), even do it locally if we want.

And thanks to fine-tuning styles, we can target them to exactly what we want, and amaze first-timers! 🤯

I'm very new to this whole AI thing, but definitely plan to keep learning (in public 👀) and doing little experiments.

Next up, I wanna play with the newly-released DreamArtist paper, which claims to to fine-tune on styles (and concepts) with just one image.

Thread soon? 👀

Generating Better AI Images By Default

✨ I started a new app recently, and while I've used Laravel in the past for "heavy" apps, decided to go 100% with Next.js this time, as a challenge.

As I'm building all the scaffolding from the app, I've borrowed some ideas from Laravel that JS devs might find interesting... 👇

While I mostly use @laravelphp for the Queue these days (no other solution comes close, especially coupled with Vapor), you usually don't think about the work it does for you:

📨 Crafting & sending mail
🔐 Authentication & authorization

With @nextjs, you start from scratch. 😅

For authentication, you usually want to use next-auth.js.org.

Howeverrr, their DB schema is quite complex, and I like going around the database manually, so I decided to try my hand at it.

Can I build a "magic link" email login flow, with just a User table? 🤔

This is the schema I want to use (using @prisma cuz it seems like the cleanest ORM for TypeScript, missing Eloquent tho 😅).

You'll notice it doesn't include any session data or verification tokens (which you'd need for the magic login links).

So, how did I make it work? 😁

First, let's cover the session. I'm using @vvoyer's iron-session package, which lets you store a small (encrypted) object inside a cookie. 🍪

I use this to store the ID of the user (and the current team as well in my case), and then fetch it from the database when I need them.

Finally, what about the verification token?

Here's where I'm taking a page from the Laravel playground, recreating their "signed links" feature.

I'm encrypting an object (w/ Iron) with the user's email and an expiry date and using that as the token users get on their email! 🤩

Note this has the downside that links will work for as long as they are active, so I'd recommend to set it to a short period (I'm using 1h).

The upside is a much much cleaner database, and clean code I really feel like I understand (which is huge!) 😁

As a bonus note, let's talk about how I'm crafting and sending emails! 📬

I write my emails in JSX using mjml-react (which comes with a pretty nifty VSCode extension w/ live-previews).

When it's time to send, you can render to html from node, and send it using nodemailer. 🚀

And that's it!

I'll keep tweeting lil DX hacks for making Next more manageable for big apps (at least from the perspective of someone used to MVC & lots of magic), so follow if you're into that i guess.

Also, check out the app I'm building! I think it's pretty dope 😁

✌️

https://twitter.com/m1guelpf/status/1583727802905694208?s=20&t=Hdb2P4DsHW0xexSOv0SYKw

Quickly Recreating Magic Login from Scratch

✨ Yesterday, I made a tweet that converted all the replies into images, using Stable Diffusion.

It blew up, with +2k people coming together to generate all kinds of art, memes, and play around with ML.

Here's how I built it, how much it cost me, and how you can try it out! 👇

I'm a big fan of @midjourney's (another AI image tool) approach to launching:

They made a Discord, where everyone can generate images in public. If you see someone making something cool, you can see their prompt, tweak it, and put your own spin into it.

It makes AI multiplayer!

Stable Diffusion is revolutionary, because it's fully open-source, and built to be run on people's computers.

You can generate amazing images in your pc/mac, in seconds, even without an internet connection! 🤯

But I really felt like it was missing that "multiplayer" experience.

🧑‍💻 I started by writing a script that'd poll the tweet for replies every minute.

Twitter only returns the latest 100 replies, so I decided to persist them to a db, along with the prompt.

When a new record got added to the db, it dispatched a new job to generate the image. 🖼

To generate the prompts, I used @replicatehq, a service that allows you to run open-source models on the cloud. ☁️

It allowed me to run SD with an API request, and have them hit me back once the model had finished (or failed).

It's also insanely cheap, but more on that later 👀

Once I got the response, I'd save the image URL and dispatch another job to reply.

The job was in charge of two things: uploading the image to Twitter, and creating the reply. 🚀

Making it a job made it really easy to retry it something went wrong instead of just erroring out.

Initially, everything was working smoothly, but then things started to fail.

It turns out, Twitter has pretty strict rate-limits, with the endpoint you use to upload images clocking out at 429 images per hour (this is nowhere in the docs!) 🥲

Thankfully, using a job queue for this made the fix really easy.

If any jobs hit the rate limit, I just released them back into the queue automatically.

This meant a hard limit on how many images can be generated every hour, but it'd auto-retry the rest on the next hour. 🔄

With all this in place, I wrote a short thread explaining how things worked (no NSFW, need to follow me for it to work, etc.), and people started playing around with it!

If you haven't had the chance to play with AI-generated images yet, I'd highly recommend to try it out! 👇

https://twitter.com/m1guelpf/status/1569487042345861121?s=20&t=WLnu3xNuoxYPaNfAGddFeA

📊 So, how did it go?

At the time of writing, it has generated 2120 images, including deleted replies (see queue pic below).

At ~$0.012 per image on Replicate, that's a cost of $22.53 for probably my most engaged-with tweet ever (and a really cool experiment). 🤩

I definitely want to do more of these experiments in the future (there's a SD fork fine-tuned for anime bet I really wanna play with 👀).

So that's something you'd like to see, drop a follow to make sure you don't miss it!

(> '')> @m1guelpf <(''<)

How I went viral by building an AI Twitter bot

✨ As part of my quest to make my website more personal, I've now added a map showing where I am.

The cool thing? It dynamically updates the location every day when I wake up, generates an Apple Maps screenshot, and a low-res placeholder. 😮

Here's how I built it 👇

Firstly, we need a way to get the data.

There's probably a lot of ways to do this, but I went for the dumbest (and simplest) one: an iOS shortcut that runs every day after I wake up.

It reads the phone location, and sends it to an API route which saves it to the db. 🛌 ➡️ 🚀

(note that I'm only sending the name of the city and region instead of full coordinates, no one should ever get access to the database, but you never know!)

Next up, I need some way to display the data. I played with embedding a map in the site, but it didn't look great. 🥲

Inspired by @brian_lovin's website (which shows an Apple Maps screenshot of SF), I started looking at using an image instead. 🖼

I was getting ready to write a script that opened some map website and took a screenshot, and then I realized that Apple has an API exactly for this!

It's called the Maps Web Snapshot API, and it's currently in beta but works great.

You just need to build a URL with your desired options (where to center the map around, level of zoom, dark mode, etc.), sign it with a key Apple gives you, and you're set!

Here's the code 👇

(note you need to sign up for Apple's developer portal to use this!)

Now that we have our map image, let's get it on the website! 🗺

I added the trusty blue marker we all know and love over the image for extra polish, and we're good to go!

... right?

While everything works, the image takes a while to load, making things look not-great on page load 😅

Let's fix it with SSR (server side rendering)!

Instead of waiting for our API to give us the image on every page load, let's get it on the server and include it in the page. ⚡️

That helps a lot, since we no longer need to wait for two requests, and our image starts loading immediately.

But we can still do better, by generating a low-res placeholder (also on the server) to show while the real image load.

Here's all the code you need 😁

With that, the placeholder (which looks like a very blurred version of the map) loads instantly, and makes space for the image.

If you want to see it live, check it out on my website! 👇

https://miguelpiedrafita.com

I also recently added a Spotify tile (which you can see below), showing what I'm playing in real time.

Websites are an amazing opportunity for self-expression, and I wanna keep playing with ideas on how to add "personal touches" to them. 🤩

Stay tuned to see what I add next! 😁

https://twitter.com/m1guelpf/status/1566979006452441088

Adding a live map to my website

quick alpha for good twitter takes:

talk to lots of people

identify the ideas/examples you keep bringing up in every conversation

tweet those ideas to keep track of them

over time, refine and rehash those ideas over more tweets

earliest example of this I can think of is my experience with my school trying to "update" itself.

I realised I was always bringing up the same example when talking about crypto-native apps, so I made it into a thread. https://twitter.com/m1guelpf/status/1455384103922978819

"debugging your way into learning" is another example.

I've been learning this way for almost 10 years, and every time I talk about it in some way or another, it seems to really resonate with people. https://twitter.com/m1guelpf/status/1522100034875105282

How to "Discover" Good Tweets

📝 How to learn anything

Get started with a project, with a stack you're familiar with (but a new piece you want to master).

Then, debug your way into knowledge. 

This is the technique I've been using for 10 years to learn pretty much everything I know.

I wrote an short article on it a year ago, when I had just gotten started with crypto.

https://m1guelpf.blog/post/how-to-learn-crypto

Oh, and the video above was made by the amazing folks @DoinGudHQ 😁

We chatted about how to get started with web3, what's coming next from the space, and wether DAOs should be truly decentralized from day 0

🎥👇👇

https://youtu.be/oPleW0lWSr0

How to Learn Anything

🎙The last episode of @PJVogt's new crypto podcast told the story of @ConstitutionDAO.

As an addendum, I talked about my experience there, and tried to explain why I'm so excited for the web3 space.

also, purple flowers 💐

https://pjvogt.substack.com/p/miguel

✨ Wallet history is notoriously hard to "explain" to users. At best, you get a bunch of swaps, but every other tx is a "contract interaction".

I've spent the past week working on a solution (and recreating the iOS lockscreen on web to showcase it 🤪).

https://wallet.m1guelpf.me

🔎 Today, your wallet's history is a big mess of numbers for anyone non-technical.

Even if websites build UIs for submitting transactions, all context is lost once they leave the page, and they'll have to face Etherscan if they ever want to remember what that was again. 🤷

Unfortunately, I'm pretty sure a perfect solution for this can never exist.

📚 The data available for transactions depends on what the smart contract developers decide to showcase, and even that can be difficult to retrieve in a reliable and fast way.

So, we can't magically guess what the code does (and its intention).

But we do have a bunch of standards (ERC20s, NFTs) & common operations (swaps, mints), that can allow us to make some guesses. 💫

What I've been working on is an open-source library where anyone can define "modules" that explain transactions. 📦

We start with the Contract Interaction module, and from there can build more specific modules for Uniswap, OpenSea, AAVE, ENS, etc.

I'm planning to open-source everything in a few days, but for now I mostly want to make sure it works as expected.

🧑‍💻 You can help by visiting the website, connecting your wallet, and making sure everything looks good.

If a transaction doesn't get interpreted correctly, lmk!

If you have any questions about the technical aspects, or have a great use-case for something like this, my DMs are open. 😁

Let's build a public repository of transaction knowledge, and make the Ethereum UX better! 🚀