No matter how prosperous the virtual world is, real things have an irresistible charm, and AI painting is no exception.
Today, let’s talk about how to use ChilloutMix and Lora to create particularly “realistic” characters, pursuing realism in the virtual world, which is actually the core goal of our series.
What is ChilloutMix?
Essentially, it is one of the many StableDiffusion models (checkpoints). Its biggest feature is that it can draw super realistic Asian characters, which is why it became an internet sensation upon release.
Currently, all AI characters that can easily be mistaken for real ones online are based on this model.
At present, it remains the most popular model on platform C!
ChilloutMix has received a 5-star rating, with downloads reaching 240,000. Including secondary distribution and sharing, it is estimated that this model has reached millions of enthusiasts. The images created with it are naturally in the thousands, and just scrolling through Xiaohongshu can show you many of them, often receiving hundreds to thousands of likes.
What is Lora?
In simple terms, it allows you to customize your characters. For example, the three most popular Loras can generate nearly perfect Japanese, Korean, and Taiwanese characters. (The model seems to have disappeared from platform C, but fortunately, I have a backup!) You can also create your own Lora by incorporating celebrities, influencers, or even yourself, and then combine them in various ways to create magic!
From a professional perspective, Lora is a method for fine-tuning SD models, which is much quicker and more convenient than DreamBooth, allowing for the simultaneous training of multiple concepts and exporting smaller models for easier sharing and storage.
That’s all for the basic concepts; now let’s get into practical operations!
Before starting, make sure you have configured Stable Diffusion WebUI on your local machine. If you haven’t, you can refer to the previous article “AI Painting Step 1: Installing SD-WebUI!”.
1. Obtain the Base Model and Lora
Open the Civitai website, and the first ranked one is ChilloutMix. Click to enter the detail page, and you can download the model by clicking “Download Latest” in the upper right corner. The method for obtaining Lora is basically similar. Search for Lora, find your favorite one, and download it.
I initially wanted to screenshot a Japanese or Korean Lora for demonstration but found that the originals are no longer available. Those excellent works have disappeared o(╥﹏╥)o …. Indeed, good things are often gone before you know it!
I was careless, having neither saved the prompts nor the images. The loss is huge, and it has caused significant obstacles for my article demonstration.
2. Import the Model and Lora
Open the Stable-diffusion-webui folder we previously configured,
Find the models folder under Stable-diffusion,
Copy the downloaded chilloutmix_NiPrunedFp32Fix.safetensors file here.
Then open the Lora folder under the models folder,
Place the downloaded Lora file inside.
Once placed, double-click webui-user.bat to start the software.
Copy the URL displayed in the command line window http://127.0.0.1:7860
and open it in your browser!
3. Switch the Model and Reference Lora
Since the model and Lora need to be used together, you must switch to the corresponding base model before using Lora.
After opening the webpage, click the dropdown menu, find ChilloutMix, and click on it.
Then you can use Lora. The specific steps are shown in the image below:
First, click the purple icon below Generate, then a model list will pop up below. Click Lora switch, then click on the specific Lora card. You can then add this Lora to the prompt and combine it with other descriptions to generate images!
The structure of the Lora prompt is as follows:
4. Start Creating Real Characters
Next, we are ready to start copying assignments:
I originally wanted to copy this image, but now it seems people have gotten bad; they post images without posting parameters…. What’s the difference between that and leaving images without sources?
Alas~ Without examples to refer to, where would I find inspiration? After scrolling through platform C for a while, my “inspiration” finally arrived. Spring is here, so let’s create based on the theme of “a beautiful girl in a t-shirt and yoga pants hiking,” aiming for maximum realism!
After repeated adjustments and trials, I finally produced a few decent images. Some can be shared, while others cannot be released.
Core prompts: “beautiful girl hiking on the mountain, yoga pants, t-shirt”. For complete parameters, follow the public account “Tony Is Not Tucker” and input “小姐姐” to obtain them. Haha, just a little trick, please bear with me!
In addition to directly using Lora, we can also train our own Lora models, training specific characters or styles, like this:
I’m face-blind, please loudly tell me who this is? In the future, if I want someone to hike, I can just let them hike! Changing clothes or poses is not a hard task. Of course, everything should be based on fairness, reasonableness, and legality!
Next, we will enter the topic of “how to successfully train a Lora.” Click on all the relevant points, see you next time!