theluyuan/DiffSynth-Studio
1
0
Fork 0
mirror of https://github.com/modelscope/DiffSynth-Studio.git synced 2026-03-19 06:32:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

support artaug

Browse Source
This commit is contained in:
Artiprocher
2024-12-03 15:30:01 +08:00
parent 7c0520d029
commit b84f906964
2 changed files with 36 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
examples/ArtAug/README.md Normal file
View File

@@ -0,0 +1,12 @@
# ArtAug
ArtAug is an approach designed to improve text-to-image synthesis models through synthesis-understanding interactions. We have trained an ArtAug enhancement module for FLUX.1-dev in the format of LoRA. See [`./artaug_flux.py`](./artaug_flux.py).
|FLUX.1-dev|FLUX.1-dev + ArtAug LoRA|
|-|-|
|![image_1_base](https://github.com/user-attachments/assets/e1d5c505-b423-45fe-be01-25c2758f5417)|![image_1_enhance](https://github.com/user-attachments/assets/335908e3-d0bd-41c2-9d99-d10528a2d719)|
|![image_2_base](https://github.com/user-attachments/assets/7f38e8d4-3c62-492e-bd96-be60f0855037)|![image_2_enhance](https://github.com/user-attachments/assets/ae3a1daf-7a7c-44fd-bdbc-1d2a83bc3de3)|
|![image_3_base](https://github.com/user-attachments/assets/e2ae4879-9202-45d6-9df7-fbcbd2093d19)|![image_3_enhance](https://github.com/user-attachments/assets/4df6e5b9-65de-408b-88c6-51db39aad801)|
|![image_4_base](https://github.com/user-attachments/assets/dbc65387-60df-4a18-b1bb-45eaa5be5c1d)|![image_4_enhance](https://github.com/user-attachments/assets/fc19860d-3e28-468b-b013-8745255ac6db)|
|![image_5_base](https://github.com/user-attachments/assets/bb65c1ba-c0c6-4d3b-b3ef-bdbbb5f03a48)|![image_5_enhance](https://github.com/user-attachments/assets/03570c62-9a0b-428f-8c86-6e01c1421202)|
|![image_6_base](https://github.com/user-attachments/assets/18e9a4e7-2afd-4ca9-bc49-7736042c25dc)|![image_6_enhance](https://github.com/user-attachments/assets/aa73571f-098a-4e65-9eda-b9729ba379cd)|

24
examples/ArtAug/artaug_flux.py Normal file
View File

@@ -0,0 +1,24 @@
import torch
from diffsynth import ModelManager, FluxImagePipeline, download_customized_models
prompt = "a beautiful Asian girl."
# Generate an image using FLUX.1-dev
model_manager = ModelManager(torch_dtype=torch.bfloat16, device="cuda", model_id_list=["FLUX.1-dev"])
pipe = FluxImagePipeline.from_model_manager(model_manager)
image = pipe(prompt=prompt, seed=0)
image.save("image.jpg")
# Download ArtAug LoRA
lora_path = download_customized_models(
model_id="DiffSynth-Studio/ArtAug-lora-FLUX.1dev-v1",
origin_file_path="merged_lora.safetensors",
local_dir="models/lora",
downloading_priority=["ModelScope", "HuggingFace"]
)[0]
model_manager.load_lora(lora_path, lora_alpha=1.0)
# Generate an image using FLUX.1-dev + ArtAug
image = pipe(prompt=prompt, seed=0)
image.save("image_artaug.jpg")