mirror of
https://github.com/modelscope/DiffSynth-Studio.git
synced 2026-03-18 22:08:13 +00:00
update examples
This commit is contained in:
Artiprocher
@@ -85,19 +85,3 @@ We highly recommend you to use this model in the WebUI.
|
||||
|"black car"|"red car"|
|
||||
|-|-|
|
||||
|||
|
||||
|
||||
### Example: Prompt Processing
|
||||
|
||||
If you are not native English user, we provide translation service for you. Our prompter can translate other language to English and refine it using "BeautifulPrompt" models. Please see [`sd_prompt_refining.py`](./sd_prompt_refining.py) for more details.
|
||||
|
||||
Prompt: "一个漂亮的女孩". The [translation model](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh) will translate it to English.
|
||||
|
||||
|seed=0|seed=1|seed=2|seed=3|
|
||||
|-|-|-|-|
|
||||
|||||
|
||||
|
||||
Prompt: "一个漂亮的女孩". The [translation model](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh) will translate it to English. Then the [refining model](https://huggingface.co/alibaba-pai/pai-bloom-1b1-text2prompt-sd) will refine the translated prompt for better visual quality.
|
||||
|
||||
|seed=0|seed=1|seed=2|seed=3|
|
||||
|-|-|-|-|
|
||||
|||||
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
from diffsynth import ModelManager, SDXLImagePipeline, download_models, QwenPrompt
|
||||
import torch
|
||||
|
||||
|
||||
download_models(["StableDiffusionXL_v1", "QwenPrompt"])
|
||||
|
||||
# Load models
|
||||
model_manager = ModelManager(torch_dtype=torch.float16, device="cuda")
|
||||
model_manager.load_models([
|
||||
"models/stable_diffusion_xl/sd_xl_base_1.0.safetensors",
|
||||
"models/QwenPrompt/qwen2-1.5b-instruct",
|
||||
])
|
||||
pipe = SDXLImagePipeline.from_model_manager(model_manager, prompt_refiner_classes=[QwenPrompt])
|
||||
|
||||
prompt = "一个漂亮的女孩"
|
||||
negative_prompt = ""
|
||||
|
||||
for seed in range(4):
|
||||
torch.manual_seed(seed)
|
||||
image = pipe(
|
||||
prompt=prompt, negative_prompt=negative_prompt,
|
||||
height=1024, width=1024,
|
||||
num_inference_steps=30
|
||||
)
|
||||
image.save(f"{seed}.jpg")
|
||||
34
examples/prompt_magic/README.md
Normal file
34
examples/prompt_magic/README.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Prompt Refining
|
||||
|
||||
Prompt refining is supported in DiffSynth-Studio.
|
||||
|
||||
### Example: Qwen
|
||||
|
||||
If you are not native English user, we provide LLM-based translation for you. Qwen is a typical example. See [`qwen_prompt_refining.py`](./qwen_prompt_refining.py).
|
||||
|
||||
Prompt: "鹰". This prompt will be refined:
|
||||
|
||||
* A majestic eagle soaring high above a vast expanse of open sky, its wings spread wide and its eyes fixed on the horizon.
|
||||
* A majestic eagle soaring high above the horizon, its wingspan stretching out towards the endless sky. Its sharp beak and powerful talons stand out against the azure sky, highlighting its strength and freedom. The eagle's silhouette is silhouetted against the fading sun, casting long shadows behind it.
|
||||
* A majestic eagle soaring high above a vast, open landscape, its wings spread wide and its beak pointed towards the sky. The sun casts long shadows across the ground, adding depth and texture to the image. The bird's feathers shimmer in the light, creating a sense of movement and power.
|
||||
* A majestic eagle soaring high above a vast, open landscape, its sharp talons gripping a fish effortlessly in its beak. The sun casts a warm golden glow behind it, casting long shadows across the barren earth below. The eagle's wingspan stretches out towards infinity, its feathers glistening in the light. Its eyes fixate on the distant horizon, as if sensing something important about to unfold.
|
||||
|
||||
|seed=0|seed=1|seed=2|seed=3|
|
||||
|-|-|-|-|
|
||||
|||||
|
||||
|
||||
### Example: OPUS + BeautifulPrompt
|
||||
|
||||
Our prompter can translate other language to English and refine it using "BeautifulPrompt" models. Please see [`bf_prompt_refining.py`](./bf_prompt_refining.py) for more details.
|
||||
|
||||
Prompt: "一个漂亮的女孩". The [translation model](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh) will translate it to English.
|
||||
|
||||
|seed=0|seed=1|seed=2|seed=3|
|
||||
|-|-|-|-|
|
||||
|||||
|
||||
|
||||
Prompt: "一个漂亮的女孩". The [translation model](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh) will translate it to English. Then the [refining model](https://huggingface.co/alibaba-pai/pai-bloom-1b1-text2prompt-sd) will refine the translated prompt for better visual quality.
|
||||
|
||||
|seed=0|seed=1|seed=2|seed=3|
|
||||
|-|-|-|-|
|
||||
|||||
|
||||
27
examples/prompt_magic/qwen_prompt_refining.py
Normal file
27
examples/prompt_magic/qwen_prompt_refining.py
Normal file
@@ -0,0 +1,27 @@
|
||||
from diffsynth import ModelManager, FluxImagePipeline, download_models, QwenPrompt
|
||||
import torch
|
||||
|
||||
|
||||
download_models(["FLUX.1-dev", "QwenPrompt"])
|
||||
|
||||
model_manager = ModelManager(torch_dtype=torch.bfloat16, device="cuda")
|
||||
model_manager.load_models([
|
||||
"models/FLUX/FLUX.1-dev/text_encoder/model.safetensors",
|
||||
"models/FLUX/FLUX.1-dev/text_encoder_2",
|
||||
"models/FLUX/FLUX.1-dev/ae.safetensors",
|
||||
"models/FLUX/FLUX.1-dev/flux1-dev.safetensors",
|
||||
"models/QwenPrompt/qwen2-1.5b-instruct",
|
||||
])
|
||||
pipe = FluxImagePipeline.from_model_manager(model_manager, prompt_refiner_classes=[QwenPrompt])
|
||||
|
||||
prompt = "鹰"
|
||||
negative_prompt = ""
|
||||
|
||||
for seed in range(4):
|
||||
torch.manual_seed(seed)
|
||||
image = pipe(
|
||||
prompt=prompt, negative_prompt=negative_prompt,
|
||||
height=1024, width=1024,
|
||||
num_inference_steps=30
|
||||
)
|
||||
image.save(f"{seed}.jpg")
|
||||
Reference in New Issue
Block a user