Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8d442ff1cd | |||
| 31c24c7aa1 | |||
| 780b80aba1 | |||
| 7a532dcd68 | |||
| 5ab708f166 | |||
| 6914441477 |
@ -1,236 +1,392 @@
|
||||
{
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"collapsed_sections": [],
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "python",
|
||||
"display_name": "Python (Pyodide)",
|
||||
"language": "python"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "python",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8"
|
||||
},
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
|
||||
}
|
||||
}
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "aa2c1ada",
|
||||
"metadata": {
|
||||
"id": "aa2c1ada"
|
||||
},
|
||||
"source": [
|
||||
"# Dreambooth\n",
|
||||
"### Notebook implementation by Joe Penna (@MysteryGuitarM on Twitter) - Improvements by David Bielejeski\n",
|
||||
"https://github.com/JoePenna/Dreambooth-Stable-Diffusion\n",
|
||||
"\n",
|
||||
"### If on runpod / vast.ai / etc, spin up an A6000 or A100 pod using a Stable Diffusion template with Jupyter pre-installed."
|
||||
]
|
||||
},
|
||||
"nbformat_minor": 5,
|
||||
"nbformat": 4,
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "# Dreambooth\n### Notebook implementation by Joe Penna (@MysteryGuitarM on Twitter) - Improvements by David Bielejeski\nhttps://github.com/JoePenna/Dreambooth-Stable-Diffusion\n\n### If on runpod / vast.ai / etc, spin up an A6000 or A100 pod using a Stable Diffusion template with Jupyter pre-installed.",
|
||||
"metadata": {
|
||||
"id": "aa2c1ada"
|
||||
},
|
||||
"id": "aa2c1ada"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "## Build Environment",
|
||||
"metadata": {
|
||||
"id": "7b971cc0"
|
||||
},
|
||||
"id": "7b971cc0"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "#BUILD ENV\n!pip install omegaconf\n!pip install einops\n!pip install pytorch-lightning==1.6.5\n!pip install test-tube\n!pip install transformers\n!pip install kornia\n!pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers\n!pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip\n!pip install setuptools==59.5.0\n!pip install pillow==9.0.1\n!pip install torchmetrics==0.6.0\n!pip install -e .\n!pip install protobuf==3.20.1\n!pip install gdown\n!pip install pydrive\n!pip install -qq diffusers[\"training\"]==0.3.0 transformers ftfy\n!pip install -qq \"ipywidgets>=7,<8\"\n!pip install huggingface_hub\n!pip install ipywidgets",
|
||||
"metadata": {
|
||||
"id": "9e1bc458-091b-42f4-a125-c3f0df20f29d",
|
||||
"scrolled": true
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "9e1bc458-091b-42f4-a125-c3f0df20f29d"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "## Move the sd-v1-4.ckpt to the root of this directory as \"model.ckpt\"\n#actual_locations_of_model_blob = !readlink -f {downloaded_model_path}\n#!cp {actual_locations_of_model_blob[-1]} model.ckpt\n!apt-get update ; apt-get install wget\n!wget 'https://prodesk.home.thijn.ovh/sd-v1-4.ckpt'\n!cp sd-v1-4.ckpt model.ckpt",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "ddf7a43d"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "## Download pre-generated regularization images\n\nWe've created the following image sets\n\n* man_euler - provided by Niko Pueringer (Corridor Digital) - euler @ 40 steps, CFG 7.5\n* man_unsplash - pictures from various photographers\n* person_ddim\n* woman_ddim - provided by David Bielejeski - ddim @ 50 steps, CFG 10.0\n\n`person_ddim` is recommended",
|
||||
"metadata": {
|
||||
"id": "mxPL2O0OLvBW"
|
||||
},
|
||||
"id": "mxPL2O0OLvBW"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "# Grab the existing regularization images\n# Choose the dataset that best represents what you are trying to do and matches what you used for your token\n# man_euler, man_unsplash, person_ddim, woman_ddim\ndataset=\"man_euler\"\n!rm -rf Stable-Diffusion-Regularization-Images-{dataset}\n!git clone https://github.com/djbielejeski/Stable-Diffusion-Regularization-Images-{dataset}.git\n\n!mkdir -p outputs/txt2img-samples/samples/{dataset}\n!mv -v Stable-Diffusion-Regularization-Images-{dataset}/{dataset}/*.* outputs/txt2img-samples/samples/{dataset}",
|
||||
"metadata": {
|
||||
"id": "e7EydXCjOV1v"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "e7EydXCjOV1v"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "# Upload your training images\nUpload 10-20 images of someone to\n\n```\n/workspace/Dreambooth-Stable-Diffusion/training_samples\n```\n\nWARNING: Be sure to upload an *even* amount of images, otherwise the training inexplicably stops at 1500 steps.\n\n* 2-3 full body\n* 3-5 upper body \n* 5-12 close-up on face",
|
||||
"metadata": {
|
||||
"id": "zshrC_JuMXmM"
|
||||
},
|
||||
"id": "zshrC_JuMXmM"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "#@markdown Add here the URLs to the images of the concept you are adding\nurls = [\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121625.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121630.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121632.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121636.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121637.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121640.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121642.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121644.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121647.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121649.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121653.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121656.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121659.png\",\n\"https://prodesk.home.thijn.ovh/gijs/IMG_20220926_121702.png\"\n ## You can add additional images here\n]",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "60e37ee0"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "#@title Download and check the images you have just added\nimport os\nimport requests\nfrom io import BytesIO\nfrom PIL import Image\n\n\ndef image_grid(imgs, rows, cols):\n assert len(imgs) == rows*cols\n\n w, h = imgs[0].size\n grid = Image.new('RGB', size=(cols*w, rows*h))\n grid_w, grid_h = grid.size\n\n for i, img in enumerate(imgs):\n grid.paste(img, box=(i%cols*w, i//cols*h))\n return grid\n\ndef download_image(url):\n try:\n response = requests.get(url)\n except:\n return None\n return Image.open(BytesIO(response.content)).convert(\"RGB\")\n\nimages = list(filter(None,[download_image(url) for url in urls]))\nsave_path = \"./training_samples\"\nif not os.path.exists(save_path):\n os.mkdir(save_path)\n[image.save(f\"{save_path}/{i}.png\", format=\"png\") for i, image in enumerate(images)]\nimage_grid(images, 1, len(images))",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "df314e2e"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "## Training\n\nIf training a person or subject, keep an eye on your project's `logs/{folder}/images/train/samples_scaled_gs-00xxxx` generations.\n\nIf training a style, keep an eye on your project's `logs/{folder}/images/train/samples_gs-00xxxx` generations.",
|
||||
"metadata": {
|
||||
"id": "ad4e50df"
|
||||
},
|
||||
"id": "ad4e50df"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "# START THE TRAINING\nproject_name = \"gijsbert\"\nbatch_size = 1000\nclass_word = \"man\" # << match this word to the class word from regularization images above\nreg_data_root = \"/workspace/Dreambooth-Stable-Diffusion/outputs/txt2img-samples/samples/\" + dataset\n\n!rm -rf training_samples/.ipynb_checkpoints\n!python \"main.py\" \\\n --base configs/stable-diffusion/v1-finetune_unfrozen.yaml \\\n -t \\\n --actual_resume \"model.ckpt\" \\\n --reg_data_root {reg_data_root} \\\n -n {project_name} \\\n --gpus 0, \\\n --data_root \"/workspace/Dreambooth-Stable-Diffusion/training_samples\" \\\n --batch_size {batch_size} \\\n --class_word class_word",
|
||||
"metadata": {
|
||||
"id": "6fa5dd66-2ca0-4819-907e-802e25583ae6",
|
||||
"tags": []
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "6fa5dd66-2ca0-4819-907e-802e25583ae6"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "## Pruning (12GB to 2GB)\nWe are working on having this happen automatically (TODO: PR's welcome)",
|
||||
"metadata": {},
|
||||
"id": "dc49d0bd"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "directory_paths = !ls -d logs/*",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "27cea333"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "# This version should automatically prune around 10GB from the ckpt file\nlast_checkpoint_file = directory_paths[-1] + \"/checkpoints/last.ckpt\"\n!python \"prune_ckpt.py\" --ckpt {last_checkpoint_file}",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "965b4654"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "last_checkpoint_file_pruned = directory_paths[-1] + \"/checkpoints/last-pruned.ckpt\"\ntraining_samples = !ls training_samples\ndate_string = !date +\"%Y-%m-%dT%H-%M-%S\"\nfile_name = date_string[-1] + \"_\" + project_name + \"_\" + str(len(training_samples)) + \"_training_images_\" + str(batch_size) + \"_batch_size_\" + class_word + \"_class_word.ckpt\"\n!mkdir -p trained_models\n!mv {last_checkpoint_file_pruned} trained_models/{file_name}",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "b7a8cec3"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "# Download your trained model file from `trained_models` and use in your favorite Stable Diffusion repo!",
|
||||
"metadata": {},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "ff1a46d9"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": "## Generate Images With Your Trained Model!",
|
||||
"metadata": {},
|
||||
"id": "d28d0139"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "!echo python scripts/stable_txt2img.py \\\n --ddim_eta 0.0 \\\n --n_samples 1 \\\n --n_iter 4 \\\n --scale 7.0 \\\n --ddim_steps 50 \\\n --ckpt \"/workspace/Dreambooth-Stable-Diffusion/trained_models/\" + {file_name} \\\n --prompt \"gijsbert person as a masterpiece portrait painting by John Singer Sargent in the style of Rembrandt\"",
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"execution_count": 2,
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "<class 'AttributeError'>",
|
||||
"evalue": "module 'pexpect' has no attribute 'TIMEOUT'",
|
||||
"traceback": [
|
||||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",
|
||||
"Cell \u001b[0;32mIn [2], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m get_ipython()\u001b[38;5;241m.\u001b[39msystem(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mchangeme python scripts/stable_txt2img.py --ddim_eta 0.0 --n_samples 1 --n_iter 4 --scale 7.0 --ddim_steps 50 --ckpt \u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/workspace/Dreambooth-Stable-Diffusion/trained_models/\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m + \u001b[39m\u001b[38;5;132;01m{file_name}\u001b[39;00m\u001b[38;5;124m --prompt \u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mjoepenna person as a masterpiece portrait painting by John Singer Sargent in the style of Rembrandt\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m'\u001b[39m)\n",
|
||||
"File \u001b[0;32m/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2466\u001b[0m, in \u001b[0;36mInteractiveShell.system_piped\u001b[0;34m(self, cmd)\u001b[0m\n\u001b[1;32m 2461\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mOSError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mBackground processes not supported.\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 2463\u001b[0m \u001b[38;5;66;03m# we explicitly do NOT return the subprocess status code, because\u001b[39;00m\n\u001b[1;32m 2464\u001b[0m \u001b[38;5;66;03m# a non-None value would trigger :func:`sys.displayhook` calls.\u001b[39;00m\n\u001b[1;32m 2465\u001b[0m \u001b[38;5;66;03m# Instead, we store the exit_code in user_ns.\u001b[39;00m\n\u001b[0;32m-> 2466\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39muser_ns[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m_exit_code\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[43msystem\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mvar_expand\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcmd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdepth\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m1\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m/lib/python3.10/site-packages/IPython/utils/_process_posix.py:129\u001b[0m, in \u001b[0;36mProcessHandler.system\u001b[0;34m(self, cmd)\u001b[0m\n\u001b[1;32m 125\u001b[0m enc \u001b[38;5;241m=\u001b[39m DEFAULT_ENCODING\n\u001b[1;32m 127\u001b[0m \u001b[38;5;66;03m# Patterns to match on the output, for pexpect. We read input and\u001b[39;00m\n\u001b[1;32m 128\u001b[0m \u001b[38;5;66;03m# allow either a short timeout or EOF\u001b[39;00m\n\u001b[0;32m--> 129\u001b[0m patterns \u001b[38;5;241m=\u001b[39m [\u001b[43mpexpect\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mTIMEOUT\u001b[49m, pexpect\u001b[38;5;241m.\u001b[39mEOF]\n\u001b[1;32m 130\u001b[0m \u001b[38;5;66;03m# the index of the EOF pattern in the list.\u001b[39;00m\n\u001b[1;32m 131\u001b[0m \u001b[38;5;66;03m# even though we know it's 1, this call means we don't have to worry if\u001b[39;00m\n\u001b[1;32m 132\u001b[0m \u001b[38;5;66;03m# we change the above list, and forget to change this value:\u001b[39;00m\n\u001b[1;32m 133\u001b[0m EOF_index \u001b[38;5;241m=\u001b[39m patterns\u001b[38;5;241m.\u001b[39mindex(pexpect\u001b[38;5;241m.\u001b[39mEOF)\n",
|
||||
"\u001b[0;31mAttributeError\u001b[0m: module 'pexpect' has no attribute 'TIMEOUT'"
|
||||
],
|
||||
"output_type": "error"
|
||||
}
|
||||
],
|
||||
"id": "80ddb03b"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": "",
|
||||
"metadata": {},
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "0e3c10d9-2c40-4f50-9cf4-97e88a57288c"
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7b971cc0",
|
||||
"metadata": {
|
||||
"id": "7b971cc0"
|
||||
},
|
||||
"source": [
|
||||
"## Build Environment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9e1bc458-091b-42f4-a125-c3f0df20f29d",
|
||||
"metadata": {
|
||||
"id": "9e1bc458-091b-42f4-a125-c3f0df20f29d",
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#BUILD ENV\n",
|
||||
"!pip install omegaconf\n",
|
||||
"!pip install einops\n",
|
||||
"!pip install pytorch-lightning==1.6.5\n",
|
||||
"!pip install test-tube\n",
|
||||
"!pip install transformers\n",
|
||||
"!pip install kornia\n",
|
||||
"!pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers\n",
|
||||
"!pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip\n",
|
||||
"!pip install setuptools==59.5.0\n",
|
||||
"!pip install pillow==9.0.1\n",
|
||||
"!pip install torchmetrics==0.6.0\n",
|
||||
"!pip install -e .\n",
|
||||
"!pip install protobuf==3.20.1\n",
|
||||
"!pip install gdown\n",
|
||||
"!pip install pydrive\n",
|
||||
"!pip install -qq diffusers[\"training\"]==0.3.0 transformers ftfy\n",
|
||||
"!pip install -qq \"ipywidgets>=7,<8\"\n",
|
||||
"!pip install huggingface_hub\n",
|
||||
"!pip install ipywidgets"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "ddf7a43d",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"## Move the sd-v1-4.ckpt to the root of this directory as \"model.ckpt\"\n",
|
||||
"#actual_locations_of_model_blob = !readlink -f {downloaded_model_path}\n",
|
||||
"#!cp {actual_locations_of_model_blob[-1]} model.ckpt\n",
|
||||
"!wget 'https://prodesk.home.thijn.ovh/sd-v1-4.ckpt'\n",
|
||||
"!cp sd-v1-4.ckpt model.ckpt"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "mxPL2O0OLvBW",
|
||||
"metadata": {
|
||||
"id": "mxPL2O0OLvBW"
|
||||
},
|
||||
"source": [
|
||||
"## Download pre-generated regularization images\n",
|
||||
"\n",
|
||||
"We've created the following image sets\n",
|
||||
"\n",
|
||||
"* man_euler - provided by Niko Pueringer (Corridor Digital) - euler @ 40 steps, CFG 7.5\n",
|
||||
"* man_unsplash - pictures from various photographers\n",
|
||||
"* person_ddim\n",
|
||||
"* woman_ddim - provided by David Bielejeski - ddim @ 50 steps, CFG 10.0\n",
|
||||
"\n",
|
||||
"`person_ddim` is recommended"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "e7EydXCjOV1v",
|
||||
"metadata": {
|
||||
"id": "e7EydXCjOV1v"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Grab the existing regularization images\n",
|
||||
"# Choose the dataset that best represents what you are trying to do and matches what you used for your token\n",
|
||||
"# man_euler, man_unsplash, person_ddim, woman_ddim\n",
|
||||
"dataset=\"person_ddim\"\n",
|
||||
"!rm -rf Stable-Diffusion-Regularization-Images-{dataset}\n",
|
||||
"!git clone https://github.com/djbielejeski/Stable-Diffusion-Regularization-Images-{dataset}.git\n",
|
||||
"\n",
|
||||
"!mkdir -p outputs/txt2img-samples/samples/{dataset}\n",
|
||||
"!mv -v Stable-Diffusion-Regularization-Images-{dataset}/{dataset}/*.* outputs/txt2img-samples/samples/{dataset}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "zshrC_JuMXmM",
|
||||
"metadata": {
|
||||
"id": "zshrC_JuMXmM"
|
||||
},
|
||||
"source": [
|
||||
"# Upload your training images\n",
|
||||
"Upload 10-20 images of someone to\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"/workspace/Dreambooth-Stable-Diffusion/training_samples\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"WARNING: Be sure to upload an *even* amount of images, otherwise the training inexplicably stops at 1500 steps.\n",
|
||||
"\n",
|
||||
"* 2-3 full body\n",
|
||||
"* 3-5 upper body \n",
|
||||
"* 5-12 close-up on face"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "60e37ee0",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#@markdown Add here the URLs to the images of the concept you are adding\n",
|
||||
"urls = [\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121054.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121057.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121100.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121102.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121104.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121106.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121108.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121112.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121116.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121118.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121120.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121153.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121155.png\",\n",
|
||||
"\"https://prodesk.home.thijn.ovh/benji/IMG_20221011_121157.png\"\n",
|
||||
" ## You can add additional images here\n",
|
||||
"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "df314e2e",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#@title Download and check the images you have just added\n",
|
||||
"import os\n",
|
||||
"import requests\n",
|
||||
"from io import BytesIO\n",
|
||||
"from PIL import Image\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def image_grid(imgs, rows, cols):\n",
|
||||
" assert len(imgs) == rows*cols\n",
|
||||
"\n",
|
||||
" w, h = imgs[0].size\n",
|
||||
" grid = Image.new('RGB', size=(cols*w, rows*h))\n",
|
||||
" grid_w, grid_h = grid.size\n",
|
||||
"\n",
|
||||
" for i, img in enumerate(imgs):\n",
|
||||
" grid.paste(img, box=(i%cols*w, i//cols*h))\n",
|
||||
" return grid\n",
|
||||
"\n",
|
||||
"def download_image(url):\n",
|
||||
" try:\n",
|
||||
" response = requests.get(url)\n",
|
||||
" except:\n",
|
||||
" return None\n",
|
||||
" return Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
|
||||
"\n",
|
||||
"images = list(filter(None,[download_image(url) for url in urls]))\n",
|
||||
"save_path = \"./training_samples\"\n",
|
||||
"if not os.path.exists(save_path):\n",
|
||||
" os.mkdir(save_path)\n",
|
||||
"[image.save(f\"{save_path}/{i}.png\", format=\"png\") for i, image in enumerate(images)]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ad4e50df",
|
||||
"metadata": {
|
||||
"id": "ad4e50df"
|
||||
},
|
||||
"source": [
|
||||
"## Training\n",
|
||||
"\n",
|
||||
"If training a person or subject, keep an eye on your project's `logs/{folder}/images/train/samples_scaled_gs-00xxxx` generations.\n",
|
||||
"\n",
|
||||
"If training a style, keep an eye on your project's `logs/{folder}/images/train/samples_gs-00xxxx` generations."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "6fa5dd66-2ca0-4819-907e-802e25583ae6",
|
||||
"metadata": {
|
||||
"id": "6fa5dd66-2ca0-4819-907e-802e25583ae6",
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# START THE TRAINING\n",
|
||||
"project_name = \"benjiman\"\n",
|
||||
"batch_size = 1000\n",
|
||||
"class_word = \"person\" # << match this word to the class word from regularization images above\n",
|
||||
"reg_data_root = \"/workspace/Dreambooth-Stable-Diffusion/outputs/txt2img-samples/samples/\" + dataset\n",
|
||||
"\n",
|
||||
"!rm -rf training_samples/.ipynb_checkpoints\n",
|
||||
"!python \"main.py\" \\\n",
|
||||
" --base configs/stable-diffusion/v1-finetune_unfrozen.yaml \\\n",
|
||||
" -t \\\n",
|
||||
" --actual_resume \"model.ckpt\" \\\n",
|
||||
" --reg_data_root {reg_data_root} \\\n",
|
||||
" -n {project_name} \\\n",
|
||||
" --gpus 0, \\\n",
|
||||
" --data_root \"/workspace/Dreambooth-Stable-Diffusion/training_samples\" \\\n",
|
||||
" --batch_size {batch_size} \\\n",
|
||||
" --class_word class_word"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "dc49d0bd",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Pruning (12GB to 2GB)\n",
|
||||
"We are working on having this happen automatically (TODO: PR's welcome)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "27cea333",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"directory_paths = !ls -d logs/*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "965b4654",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# This version should automatically prune around 10GB from the ckpt file\n",
|
||||
"last_checkpoint_file = directory_paths[-1] + \"/checkpoints/last.ckpt\"\n",
|
||||
"!python \"scripts/prune-ckpt.py\" --ckpt {last_checkpoint_file}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "b7a8cec3",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"last_checkpoint_file_pruned = directory_paths[-1] + \"/checkpoints/last-pruned.ckpt\"\n",
|
||||
"training_samples = !ls training_samples\n",
|
||||
"date_string = !date +\"%Y-%m-%dT%H-%M-%S\"\n",
|
||||
"file_name = date_string[-1] + \"_\" + project_name + \"_\" + str(len(training_samples)) + \"_training_images_\" + str(batch_size) + \"_batch_size_\" + class_word + \"_class_word.ckpt\"\n",
|
||||
"!mkdir -p trained_models\n",
|
||||
"!mv {last_checkpoint_file_pruned} trained_models/{file_name}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "ff1a46d9",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Download your trained model file from `trained_models` and use in your favorite Stable Diffusion repo!\n",
|
||||
"!wget -nc 'https://prodesk.home.thijn.ovh/gijs/ai'\n",
|
||||
"!chmod 600 ai\n",
|
||||
"!scp -r -i ai -P 2387 -o StrictHostKeyChecking=no ./trained_models ai@home.thijn.ovh:/mnt/hdd/ai/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d28d0139",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Generate Images With Your Trained Model!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "80ddb03b",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!python scripts/stable_txt2img.py \\\n",
|
||||
" --ddim_eta 0.0 \\\n",
|
||||
" --n_samples 1 \\\n",
|
||||
" --n_iter 10 \\\n",
|
||||
" --scale 7.0 \\\n",
|
||||
" --ddim_steps 50 \\\n",
|
||||
" --ckpt \"/workspace/Dreambooth-Stable-Diffusion/trained_models/CHANGEME.ckpt\" \\\n",
|
||||
" --prompt \"beautiful oil painting of benjiman with high detail of a gorgeous wood-elf ranger from dungeons and dragons in the desert by artgerm and greg rutkowski and thomas kinkade\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0e3c10d9-2c40-4f50-9cf4-97e88a57288c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"collapsed_sections": [],
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.13"
|
||||
},
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@ -8,7 +8,7 @@ from torchvision import transforms
|
||||
import random
|
||||
|
||||
training_templates_smallest = [
|
||||
'gijsbert {}',
|
||||
'benjiman {}',
|
||||
]
|
||||
|
||||
reg_templates_smallest = [
|
||||
|
||||
Loading…
Reference in New Issue
Block a user