Fine-tune FLUX.1 to create images of yourself

Fine-tune FLUX.1 to create images of yourself – Replicate blog

Replicate Blog

Fine-tune FLUX.1 to create images of yourself

Posted August 30, 2024 by zeke

Info

Update (May 2025): We’ve released a faster version of the Flux trainer — try it here .

The FLUX.1 family of image generation models was released earlier this month and took the world by storm, producing images surpassing the quality of existing open-source models. The community quickly started to build new capabilities on top of Flux, and not long after the release we announced Flux fine-tuning support on Replicate .

Fine-tuning Flux on Replicate is easy: you just need a handful of images to get started. No deep technical knowledge is required. You can even create a fine-tune entirely on the web , without writing a single line of code. The community has already published hundreds of public Flux fine-tunes on Replicate, plus thousands of private fine-tunes too.

One of the most exciting things about Flux is its capability to fine-tune on faces in a way that was not easily achievable with previous generations of open-source image generation models like Stable Diffusion or SDXL. Not since Dreambooth has it been this easy to get great results from just a handful of training images.

In this blog post I’ll walk you through the process of creating your own Flux fine-tune using images of yourself, so you can create novel and imaginative images of yourself as a superhero, a cartoon character, an adventurer, or just a regular person in a variety of interesting situations.

Variants of “ZIKI on a skateboard”, generated by the ziki-flux fine-tune. Step 0: Prerequisites

Here’s what you’ll need to get started:

A Replicate account

A handful of training images

Two to three US dollars

Step 1: Gather your training images

You’ll need a few images of yourself to get started. These should be high-quality images of your face, taken from various angles and in different lighting conditions.

You can fine-tune Flux with as few as two training images, but for best results you’ll want to use at least 10 images or more. In theory you’ll get continually better results as you include more images in the training data, but the training process can take longer the more images you add.

Consider the following when gathering your training images:

WebP, JPG, and PNG formats are all supported.

Use 1024x1024 or higher resolution if possible.

Filenames don’t matter. Name your files whatever you like.

Aspect ratio doesn’t matter. Images can be square, landscape, portrait, etc.

10 images is a good minimum.

Variety is key. For best results, choose training images with different settings, clothing, lighting, and angles. Once you’ve gathered your images, put them in a zip file. Assuming you put them all in a folder called data , run this command to generate a file called data.zip :

Copy

zip -r data.zip data

Step 2: Choose a unique trigger word

Whenever you fine-tune an image model, you also choose a unique “trigger word” that you’ll use later in your text prompts when generating images:

Copy

photo of YOUR_TRIGGER_WORD_HERE looking super-cool, riding on a segway scooter

Here are some things to consider when choosing a trigger word:

It should be something unique like MY_UNIQ_TRGGR . Think “vanity license plates”, but without any length limits.

It should not be an existing word in any language, like dog or cyberpunk .

It should not be TOK , because it will clash with other fine-tunes if you ever want to combine them .

Case doesn’t matter, but capital letters can help visually distinguish the trigger word from the rest of the text prompt.

For my zeke/ziki-flux fine-tune, I chose ZIKI as a trigger word. Short, unique, and memorable.

Got your trigger word? Hold it in your mind for a second. You’ll use it in the next step.

Step 3: Create and train a model

There are a couple ways to fine-tune Flux on Replicate. You can use the web-based training form , or the API . The API is great for creating and updating fine-tunes in an automated or programmatic way, but in this guide we’ll just use the web-based form. It’s easier.

Go to replicate.com/ostris/flux-dev-lora-trainer to start the web-based training process.

For the destination input, you’ll choose a model to publish to. This can be an existing model you’ve already created, or a new model:

Your fine-tuned Flux model can public or private. For the input_images input, drag and drop the zip file you created earlier.

For the trigger_word input, enter the string you chose earlier. Make sure it’s unique!

For steps , leave it at 1000. Any less and your training process will not properly learn the concept in your training images. Any more and you could be incurring extra time and cost without much improvement in the model performance.

You’ll be billed per second for the time the training process takes to run. Trainings for the Flux model run on Nvidia H100 GPU hardware, which costs $0.001528 per second at the time of this writing. For a 20-minute training (which is typical when using about 20 training images and 1000 steps), you can expect to pay about $1.85 USD. Once your model is trained, you can run it with an API just like any other Replicate model, and you’ll only be billed for the compute time it takes to generate an image.

Leave the rest of the inputs at their default values and click Create training .

Step 4: Stand up and stretch

The training process is pretty fast, but it still takes a few minutes. If you’re using ten images and 1000 steps, it will take approximately 20 minutes. Use this opportunity to get up from your computer, stretch your arms and legs, grab a drink of water, etc.

Then come back and your model should be ready to go.

Step 5: Generate images on the web

Once the training process is complete, your model will be ready to run. The easiest way to get started is by running it on the web.

The only input you’ll need to enter is the prompt . The rest you can leave alone to start. Flux is great at following long prompts, so the more detailed and descriptive you make the prompt the better. Be sure to include your trigger_word in the prompt to activate your newly trained concept in the resulting images.

Run your new fine-tuned model from the Replicate web playground. Step 6: Generate images using the API

The web playground is a great place to start playing with your new model, but generating images one click at a time can get old pretty fast. Luckily your model is…