First time here?  We recommend at least 1 hour to test our 10 best practices (below).

Model Training Tutorial & FAQ

You are training the AI with a pattern. The training method is easy and non-technical: You will be uploading images and describing them.  The quality of the images and the quality of the image captions plays a huge role in the quality of the LoRa. This cannot be overstated. You can import great images, write bad captions, and get a bad result. Amateur mistake.

Our mission is to not only to teach you how to use our software, but also to teach you how to become a good trainer, and how to balance and optimize your data sets, which we call a Catalog.

You also don’t have to learn this alone: Get help in the LoRaMaker Beta Tester support group 

Preparing your data set

Tip #1 – Quality over Quantity

Start with five great photos. Look at our five photos critically and make sure that they are unmistakably the same person. what You will need a minimum of five (5) images. In our testing, we have found that the golden number is (26) photos. Do not repeat photos.  Especially do not repeat photos with different captions! Bad!

More photos can produce worse results, because it introduces the risk of more inconsistencies and hurried captioning. Don’t overwork yourself or cut corners.

What makes a good data set?

Our software includes a Catalog – a place to store all of your photo ideas and select the ones that go into the final model, the Data Set.  A data set must contain at least five images.

Use photos taken at different times of the day.

Different days, different lighting work best.

To use an imperfect analogy: If you do an exercise the wrong way 195 times out of 200, you don’t get bigger muscles — you get an injury. Make each rep count.

Here are some examples of good and bad photos, but even the two “good ones” have discrepancies. Does she have a chin dimple, and is her eye color still green in #2?

FIGURE A

If you are absolutely sure you have 200 good photos and the quality is still bad, you’re either wrong or your captions are the problem. We do a deep dive into captions down below.

Tip #2 – No props, no costumes

Makeup vs. Natural – If a person’s best look is a certain kind of make-up, only upload images with that makeup. Otherwise their features will be confused and subject to the biases of the base model for what makeup is supposed to look like, which can add unwanted matte or gloss, or different eyebrows and lips.

Generally speaking, Stable Diffusion makes women’s faces rounder with fuller lips. Otherwise, the person may look different. If the accessories are very bold and may cause a training distraction, consider removing them or inpainting them out of the photo before uploading.

You learned in rule #1 that it’s a bad image when the face is covered by hands. This also applies to sunglasses, facial glitter, and other foreign objects. No costumes. These things will often cause mutations. The AI already knows what costumes and glasses are, so you can prompt for those sorts of things later.

Be picky. Because training bad data is hard to pluck out. It’s not a less is more thing, it’s a quality thing. Imagine you are the robot and you are confused if these photos are actually of the same person. Obsess over this.

Tip #3 – Check for the “fat” in lights

Shadows and Light can create the illusion of slimness of heaviness, shaper facial features just as much as makeup does. Does the person appear to be roughly the same weight on all of the photos?  A certain shadow or dress can add 10 pounds or make a face look rounder or thinner!  If you have those borderline images, toss those.

Do any of the angles make facial features look sharper, rounder, bigger, smaller than the other photos?  Toss those out.

Avoid photos that are dark, blurry, or make the person look different from the other photos. Expert model makers are taking each photo into Lightroom and color correcting them, every image!  You don’t have to do this, but the best of the best do.

Tip #4 – Crop your own images

Manually trim the fat, and set the focus: Again, not a requirement, because our software will actually crop the images for you if the face is too far away. For those highly concerned with quality, doing your own 1:1 crops are a good idea.

The images should be very clear and at least 512×512. Square, 1:1 photos work best.

Tip #5 – No special guests

One person per image, always: Don’t upload photos of multiple people in one image, even if you are describing them as two different people in the caption. This will hurt the training.  You can create multiple models, so a focused model about 1 thing is best.

To add multiple people into a photo, use Inpaint instead.

Let’s look at this example:

While helping a customer improve the likeness of their model, we noticed this caption read “Person’s Name headshot”

If we are a robot and these captions taken literally, we have just learned that every photo of Person must have a (fused?) dog next to their face. (Cursed!)

Are all of your portraits in the same room, at the same hour, with the same light?  That will make a very rigid model. We call this “overtraining” – See the note in Figure A about removing backgrounds and different lighting conditions to solve this.

Don’t take anything for granted. Be detailed and throw out photos that may confuse or trick Stable Diffusion.

Tip #6 – Make backgrounds flexible

Keep the backgrounds varied, or remove them completely.

If all of your photos have the same background, it will be very difficult to prompt a different outcome. If different photos are not possible, we recommend using the background remove command on your photos first.

Backgrounds or Not?

For example, if all of your photos are in a bedroom, it will be harder to prompt for things like “cyberpunk spaceship background” because the AI will have “trained” on the bedroom for multiple steps on every image.

Tip #7: Caption your images methodically

Captions are NOT prompts, don’t add commands inside captions

We asked the team that worked on a very popular NSFW model and various LoRAs, and to our surprise they told us that they take an average of SIX HOURS to caption and rebalance and rebuild their LoRAs constantly before they are satisfied. Wow!  You don’t have to do that, but it just to show how much work goes into some of these popular models.

Commas and periods are OK. Avoid other punctuation marks in captions.  Don’t add commands or ((weights)) or [[negatives]], use natural language

It makes sense: Your favorite Stable Diffusion models were painstakingly captioned, each image tagged with great precision. The best model makers study the render results and blame the bad photos in their data sets for the bad results, and train again. If you have this mindset, you will achieve your goals. Otherwise, every model would be great, and there is clearly a difference in quality between certain models.

Don’t use more than 77 tokens. Tokens are words or parts of words that Stable Diffusion uses to understand your prompts.  This topic itself deserves its own guide. For the purposes of this tutorial, as long as your prompt isn’t a two paragraphs, you’re probably fine.  Aim for 2-3 descriptive and clear sentences.

Make your captions map to how you will prompt in the future.  If you will be prompting for a brunette, then the word brunette must go into your caption. Otherwise, it will strongly pull biases from the base model for that token.

Each image must be cataloged one by one. You can provide a caption for the photo by adding it after /catalog as you upload, and edit it again later in the Catalog system. You will have a chance to check all captions one more time before building the model.

TAG MANAGERS – FOR THE PROFESSIONAL CAPTIONER

We recommend first writing down the description of what the person looks like on a spreadsheet. Do they have a unique nose, do they have a heart-shaped face or is it more of a triangle shape, are their features more masculine or are they soft and round?  These things are important to write down. The hair color, the eye color.  Think about the unique characteristics and also the situations the photos are in.  This free tag management software pictured below is recommended as it will keep you organized and give you a clear battle plan on what to balance out if the quality of your LoRA is not what you had hoped.

Tip #8 – Balance your Data Set

Picking good images was the easy part. Tagging them was more difficult. Now comes the hardest part: Balancing what you did.

• Choose Stable Diffusion 15 for the most flexible results, the best for stylish avatars. This is not the one you want for creating a “deepfake” real person, it will bend a lot based on the base model that you pair it with.
• You can choose a fine-tuned model Realistic Vision 51 for real people, but first run one of your captions on that model to understand it’s biases.  Does “redhead woman” look like what you expect?  That woman will look totally different in Deliberate2, so know the biases and pick the right model. The tradeoff is that it will be less flexible for other art styles.

Tip #9 – Know the base model biases

Tip #10 – Optimize the token and prompt

The final step when building the Lora, the point of no return

Once the LoRA is built, there is no way to change it. You can train a new LoRA with the same data set though. We don’t throw out your saved Catalog images.

With at least five images selected, click Train Lora in the header or Create Lora in the footer. They both do the same thing. You’ll be presented with a few menu options before the training begins.

Checklist of things to complete before clicking Create Concept:

1. A friendly name — this is just for your FYI, it won’t effect the image
2. The concept name — you can’t determine this yet.  It will be your friendly name plus a few numbers, so keep your friendly name short
3. Token — this is the unique word for your the subject, not the trigger word for your LoRa.  This should be a cryptic weird word that cannot be confused with something else that Stable Diffusion knows.  A good trigger is yxheehgge.  A bad trigger is Susan, Lola, Rita.  Those already have a meaning.
4. Prompt — This is the activation phrase to use your LoRA, the thing you must repeat and the most important thing is that it must contain the Token word you just made up.
5. Don’t forget the base model.  The base model will have the biggest impact on how flexible the model is.  Check the bias info in Tip 9 about this.You may want to create multiple versions of a model until you make one that suits your goals.  There’s no penalty for experimentation. If you’re not familiar with the names from the pulldown menu, search for them in our models page to see a preview.

Important: The prompt field is what you will be writing to activate your lora.  Don’t forget that phrase. Write it down right now, that’s the activator.

And you’re almost done!

What happens next:   The system will build the model and give you a Trigger or Concept name back. At this point, we still do not have the name for the model, so we cannot use it.  Wait for it to appear in the fine tunes menu.

This is explained in the Getting Around section above (see Fine-Tunes diagram)

How to use your Lora

• Repeat the prompt section word-for-word exactly as you trained it (see Concept Configure screen above)
• Repeat the base concept in your prompt. It is not embedded in the LoRA.  (That would take hours)
• Try adding /guidance:7 — how strictly the AI should follow your idea.  A max is 20 for very strict, and at 0 its creepy.
• Correct:   a photo of zhzhzy person <happyguy134> <realvis51>
• Wrong:  a photo of Susan, a brunette <happyguy>  (did you train the word brunette or Susan? is it in the token or config prompt? if you don’t specify the base model, it will look like an illustration

Please be kind to your humble LoRA trainer servers

At the risk of sounding like one of those hotels that puts up a Save The Earth sticker to lower their laundry bills while the whole staff drives Diesel Ford F150s, there is a compute cost every time we train a LoRA, there is a compute cost (to Graydient, not you) every time a LoRA is created, so please use this resource responsibly so we can offer this product at a low price to others. Don’t smash build unless you mean it.