ChatGPT has been out for a while, which raises lots of discussion and interesting on using those artificial intelligence technologies to complete some work orginal done by human. Using AI to draw a realistic photo is another interesting topic which has been deeply developped for certain purpose. In this post, I will show a Github project which you can use it to create your own text-2-image Web UI page then use free Google Colab platofrm to install. You also are able to install it into your local pc  to draw the pictures based on your inputs, either a image or some texts, but you had better to have a good performance machine, especially high end graphic card. 

Introduction

Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input.

Model Details

More details can be found : https://huggingface.co/CompVis/stable-diffusion-v-1-4-original

Github Project: stable-diffusion-webui-colab

Here are steps to follow this Github project to create your own WebUI on Colab to run: 

1 Go to Github Page : https://github.com/camenduru/stable-diffusion-webui-colab

2 Got to Colab section, select one option to run

Click one button to continue to load it in Colab website.

3 Colab site with loaded code.

Click run button to run those commands in Colab environment. 

4 Once completed the running, you can scroll down to find out those two WebUI links:

5 Stable Diffusion WebUI page will be opened:

6 You can enter keywords, sentences or other related words to describe the photo you wants, then click generate to see the output.

You can generate it multiple times to see different results.

Prompt Examples

Example 1: 

In a way you gonna have experience as an artist to know what make art look good.

I want a VICTORIAN GIRL in a style of OIL PAINTING
Eye and Face are important in art so she must have PERFECT FACESEXY FACE and her eye have DETAILED PUPILS
I want she to have LARGE BREASTTONED ABS and THICK THIGH.
She must look FEMININE doing EVOCATIVE POSESMIRK and FULL BODY wearing NIGHT GOWN
The output must be INTRICATEHIGH DETAILSHARP
And in the style of {I’m not give out the artist names to avoid trouble. Apologize.}

Prompt :
VICTORIAN GIRL,FEMININE,((PERFECT FACE)),((SEXY FACE)),((DETAILED PUPILS)).(ARTIST),ARTIST,ARTIST,(ARTIST). OIL PAINTING. (((LARGE BREAST)),((TONED ABS)),(THICK THIGH).EVOCATIVE POSE, SMIRK,LOOK AT VIEWER, ((BLOUSE)).(INTRICATE),(HIGH DETAIL),SHARP

This is where you put trait that you dont want to see in the finale output. i.e nipple
Negative Prompt :
((nipple)), ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), (((tranny))), (((trans))), (((trannsexual))), (hermaphrodite), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))). (((more than 2 nipples))). [[[adult]]], out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))

TIPS:

  • prompt in () will strengthening the prompt, adding more will strengthening it even further
  • prompt in [] will weakening the prompt , adding more will weakening it even further
  • Adding more artist will create amalgamation that end up look better or worst, experiment it yourself. I use 4 artist to create most of the output.
  • Restore Face is GOD SEND, USE IT.
  • It always depend on what model you use.

The level here is whatever, feel free to experiment. This is what I use.
Steps: 42, Sampler: DDIM, CFG scale: 10

Example 2 : Manga style

Prompt : GIRL,FEMININE,((PERFECT FACE)),((SEXY FACE)),((DETAILED PUPILS)).(ARTIST),(THICK THIGH).EVOCATIVE POSE, SMIRK,LOOK AT VIEWER, ((BLOUSE)).(INTRICATE),(HIGH DETAIL),SHARP,(((loli))) 

Negative Prompt : ((nipple)), ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), (((tranny))), (((trans))), (((trannsexual))), (hermaphrodite), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))). (((more than 2 nipples))). [[[adult]]], out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))

Local Installation of Stable Diffusion Web UI

GitHub Page: https://github.com/automatic1111/stable-diffusion-webui/

Steps:

  1. Install Python 3.10.6, checking “Add Python to PATH”.
  2. Install git.
  3. Download the stable-diffusion-webui repository, for example by running git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git.
  4. Close GUI interface. 
  5. Download models (model.ckpt) file,  and put them under models directory.  
  6. Run webui-user.bat from Windows Explorer as normal, non-administrator, user.

http://127.0.0.1:7860/

Generate a photo:

Total folder size: 9.40GB

Error message:

you might be getting some python related error. You can quickly update pip and reboot your system to try it again. 

Fix: reboot system and try it again.

Another error message might relate to your GPU. 

AssertionError: Torch is not able to use GPU; add –skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check. 

If your system does not have much RAM , you might want to add som other parameters to avoid installing issues:

 “set COMMANDLINE_ARGS = –lowvram –precision full –no-half –skip-torch-cuda-test”

Models

Model: Cocoa.ckpt , yohan-diffusion.ckpt

Open In Colab Open In Colab Open In Colab yohan_diffusion_webui_colab
andite/yohan-diffusion
!aria2c –console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/yohan-diffusion/resolve/main/yohan-diffusion.ckpt -d /content/stable-diffusion-webui/models/Stable-diffusion -o yohan-diffusion.ckpt
!aria2c –console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt -d /content/stable-diffusion-webui/models/Stable-diffusion -o yohan-diffusion.vae.pt
!aria2c –console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/yohan-diffusion/resolve/main/Cocoa.ckpt -d /content/stable-diffusion-webui/models/Stable-diffusion -o Cocoa.ckpt
Prompt: a beautiful 18 years old asian girl, smile face, sitting in front of a house, looking at you, Big eyes

Model Stable Diffusion v2.1

Open In Colab Open In Colab Open In Colab stable_diffusion_v2_1_webui_colab
stabilityai/stable-diffusion-2-1

It is using Stable Diffusion checkpoint : 768-v-ema.ckpt 

Prompt: a beautiful 18 years old asian girl, smile face,  sitting in front of a house, looking at you, Big eyes

Negative Prompt: lowres, low quality, doll, bad anatomy, poorly drawn, ugly, maiden, virgin, malformed, deformed, masculine, manly, black skin, asian, blurry, out of focus, freckles, skin defects, skin marks, long neck, anime, highlights, shadows,

Some Sites to Use AI

 

The list which collected from Internet will be kept updating from time to time:

LoRA and Checkpoint

What is LoRA?

LoRA models are small Stable Diffusion models that apply tiny changes to standard checkpoint models. They are usually 10 to 100 times smaller than checkpoint models. That makes them very attractive to people having an extensive collection of models.

LoRA models are small modifiers of checkpoint models. You can easily use them in AUTOMATIC1111 by including a phrase in the prompt.

Civitai hosts a large collection of LoRA models. Apply the LORA filter to see only LoRA models. You may find that they all tend to be similar: female portraits, anime, realistic illustration styles, etc.

Hugging Face is another source of LoRA libraries. You will find more varieties of LoRA models. But there are not as many LoRA models there. Their collection is a lot smaller.

Search LoRA models in Hugging Face

LoRA Library in Hugging Face

Usage:

As an example, I’ll use Arcane LoRA and download the .safetensors file. When the file is finished downloading, put it in stable-diffusion-webui/models/Lora.

!aria2c –console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/8339 -d /content/stable-diffusion-webui/models/Lora -o ArcaneLoRA.ckpt

For the SD checkpoint: You can use any of your existing models, but for guaranteed pretty good results download the AnyLoRA model. Like the rest of your models, this goes in the folder stable-diffusion-webui/models/Stable-diffusion.

!aria2c –console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/29792 -d /content/stable-diffusion-webui/models/Stable-diffusion -o AnyLoRA.ckpt
Lets load the model and checkpoint into Colab throught Github project’s link:
Open In Colab Open In Colab Open In Colab openjourney_v2_diffusion_webui_colab
prompthero/openjourney-v2

Videos

 

References

By netsec

Leave a Reply