Kaggle是一个机器学习和数据科学社区,能够利用它提供的环境来建立StableDiffusion(每周30个小时)
注册Kaggle与发动StableDiffusion
-
翻开Kaggle官网(www.kaggle.com/)
-
注册一个Kaggle账号
-
注册好Kaggle账号后,翻开链接 www.kaggle.com/code/xinsia…
-
进入你的代码(上一步点击按钮后会跳转入你的代码页面),进行GPU装备,StableDiffusion需求GPU的算力,Kaggle的CPU算力很弱很弱
手机号验证成功后,就能够开启GPU了
-
装备好后就能够运转代码,生成一个StableDiffusion
代码履行过程中,假如看到红色的正告不必太忧虑,只要代码没中止履行,就继续等等,直到看到输出链接
处理人机身份验证的问题
在注册和验证手机号的过程中,会需求一些办法,来显现人机交互验证(谷歌提供的),你能够挑选魔法或许用插件重定向拜访
下面以微软阅读器Edge来介绍一个插件Header Editor来处理人机身份验证问题,不一定能成功,仅供参考
插件地址(microsoftedge.microsoft.com/addons/deta…
-
翻开微软Edge阅读器的扩展管理
-
查找安装插件Header Editor
-
装备Header Edit
用到的规则URL:github.azurezeng.com/static/HE-G…
-
装备好后,改写对应网页,就能够看到人机身份验证了
替换StableDiffusion模型(Kaggle版)
默许的代码下载的模型只要两个,当你想用一下其他模型的时分,应该怎么做呢?
参考下面代码剖析
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/57618 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o Counterfeit-V3.safetensors
##########
#替换上面这句代码里的链接和输出的文件名,贴会Kaggle里或许直接在Kaggle里修正,再履行代码即可下载模型
获取模型下载链接的方式,这儿以 civitai.com为例
主张不要下载太多模型,只要20GB空间,所以我把不需求的ControlNet模型,都注释了下载代码,不让它下载,削减空间占用
如何下载 Lora 和 VEA
与替换模型的原理相似,除了修正下载链接和输出文件名,还要修正模型寄存方位
# Lora
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/32988 -d /kaggle/working/stable-diffusion-webui/models/Lora -o blindbox_v1_mix.safetensors
# VAE
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/gemasai/vae-ft-mse-840000-ema-pruned/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt -d /kaggle/working/stable-diffusion-webui/models/VAE -o vae-ft-mse-840000-ema-pruned.ckpt
代码下载太杂乱?来试试civitai 的SD插件
-
检查一下,代码里有没有安装这个civitai阅读插件,能够通过ctrl + F,翻开查找框,输入civitai,快速查找相关代码
-
在运转的StableDiffusionWebUI中,切换到civitAi选项卡
-
改写并读取模型
Kaggle与HuggingFace同步数据
-
注册HuggingFace账号
-
创立Dataset,来存储装备文件Config(我创立了两个一个公开的用来存储装备文件,一个私有的存储生成图的紧缩文件)
-
生成Access Tokens,用于Kaggle 代码上传文件到HuggingFace
-
上传token到Kaggle里
仿制token到txt文件
然后就能够看到token文件
将文件途径写到代码里,用于代码上传文件到HuggingFace运用token
#####################################################################
# 同步装备
# 运用huggingface保存和载入webui装备文件
huggingface_use = True
zip_output=True
huggingface_token_file = '/kaggle/input/hgtoken/huggingfacetoken.text'
huggiingface_repo_id = 'MSa/sd-output-images'
huggiingface_config_repo_id = 'MSa/sd-config'
# 将会同步的文件
config_files = [
'ui-config.json',
'config.json',
# 'styles.csv'
]
#功用函数
from pathlib import Path
from huggingface_hub import HfApi, login
# config 上传
def hugface_upload_config(huggingface_token_file, config_files, repo_id):
if Path(huggingface_token_file).exists():
with open(huggingface_token_file, encoding="utf-8") as nkfile:
hugToken = nkfile.readline()
if hugToken != '':
# 运用您的 Hugging Face 拜访令牌登录
login(token=hugToken)
# 实例化 HfApi 类
api = HfApi()
print("HfApi 类已实例化")
%cd /kaggle/working/stable-diffusion-webui
# 运用 upload_file() 函数上传文件
print("开端上传文件...")
for config_file in config_files:
if Path(config_file).exists():
response = api.upload_file(
path_or_fileobj=config_file,
path_in_repo=config_file,
repo_id=repo_id,
repo_type="dataset"
)
print("文件上传完成")
print(f"呼应: {response}")
else:
print(f'Error: File {config_file} does not exist')
else:
print(f'Error: File {huggingface_token_file} does not exist')
代码运用的留意事项
代码太杂乱?来试试HuggingFace 的SD插件
确保代码里有下载HuggingFace 的SD插件
## Kaggle下载HuggingFace 的SD插件的代码
!git clone https://github.com/camenduru/stable-diffusion-webui-huggingface /kaggle/working/stable-diffusion-webui/extensions/stable-diffusion-webui-huggingface
插件安装成功后,StableDiffusion WebUI 里会有HuggingFace选项卡
取得Kaggle 输出文件夹途径的办法
# 其实Kaggle的文件夹途径都是相对固定的(假如你没做修正的话),能够直接用下面这个途径
/kaggle/working/stable-diffusion-webui/outputs
在HuggingFace创立Dataset
仿制HuggingFace的Dataset途径
生成HuggingFace的写入token
仿制token,然后粘贴到插件的token输入框里
推送到HuggingFace的效果
如何导出导入代码
导出代码
假如忧虑Kaggle封禁项目代码,我们能够下载自己的代码备份
新建项目,导入代码
有了代码文件,能够新建项目,再导入代码,这样,你就取得一个新的StableDiffusion项目了(装备文件会是旧的,由于是保存在HuggingFace上,不过只要不同步,就是新的装备文件)
取得一个新的项目
完好的代码文件
「zh-stable-diffusion-webui-kaggle.ipynb」
链接:pan.quark.cn/s/314fe6ed6…
个人经验与主张
-
替换模型需求留意的事项
在Kaggle编辑器中找到下载代码的方位
下载代码应该编写的方位
关于代码运转的留意事项
留意,不要下载太多模型,个人主张,每次运转Kaggle(开机关机算一次),最多两个模型就好了,太多模型占用空间过多,简单死机(其实问题也不大,仅仅死机的话,会来不及保存生成的图片)
附录:
完好代码:
# Please follow me for new updates https://twitter.com/camenduru
# Please join our discord server https://discord.gg/k5BwmmvJJU
!apt -y update -qq
!apt -y install -qq aria2
!pip install -q torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html
!pip install -q https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.16/xformers-0.0.16+814314d.d20230118-cp38-cp38-linux_x86_64.whl
!pip install -q huggingface-hub==0.11.0 -U
#!pip install -q gradio==3.23.0 -U
!git clone -b v2.1 https://github.com/camenduru/stable-diffusion-webui
!wget https://raw.githubusercontent.com/camenduru/stable-diffusion-webui-scripts/main/run_n_times.py -O /kaggle/working/stable-diffusion-webui/scripts/run_n_times.py
!git clone https://github.com/AlUlkesh/stable-diffusion-webui-images-browser /kaggle/working/stable-diffusion-webui/extensions/stable-diffusion-webui-images-browser
# 切换图片阅读插件的版别
%cd /kaggle/working/stable-diffusion-webui/extensions/stable-diffusion-webui-images-browser
!git checkout 1d5c2e7
%cd ~
!git clone https://github.com/camenduru/stable-diffusion-webui-huggingface /kaggle/working/stable-diffusion-webui/extensions/stable-diffusion-webui-huggingface
!git clone -b v2.0 https://github.com/camenduru/sd-civitai-browser /kaggle/working/stable-diffusion-webui/extensions/sd-civitai-browser
!git clone https://github.com/Mikubill/sd-webui-controlnet /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet
!git clone https://github.com/camenduru/openpose-editor /kaggle/working/stable-diffusion-webui/extensions/openpose-editor
!git clone https://github.com/kohya-ss/sd-webui-additional-networks /kaggle/working/stable-diffusion-webui/extensions/sd-webui-additional-networks
!git clone https://github.com/DominikDoom/a1111-sd-webui-tagcomplete /kaggle/working/stable-diffusion-webui/extensions/a1111-sd-webui-tagcomplete
!git clone https://github.com/toriato/stable-diffusion-webui-wd14-tagger /kaggle/working/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger
!git clone https://jihulab.com/hunter0725/stable-diffusion-webui-localization-zh_CN.git
%cd /kaggle/working/stable-diffusion-webui
# !wget https://temp-1257364845.cos.ap-nanjing.myqcloud.com/config.json
!wget https://jihulab.com/hunter0726/stable-diffusion-webui-localization-zh_CN/-/raw/582ca24da2f5ca7a155b1950471a61d38b1c9b62/localizations/zh_CN.json
!mv zh_CN.json localizations
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_canny-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_canny-fp16.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_depth-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_depth-fp16.safetensors
#!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_hed-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_hed-fp16.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_mlsd-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_mlsd-fp16.safetensors
#!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_normal-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_normal-fp16.safetensors
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_openpose-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_openpose-fp16.safetensors
#!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_scribble-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_scribble-fp16.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_seg-fp16.safetensors -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_seg-fp16.safetensors
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/hand_pose_model.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/openpose -o hand_pose_model.pth
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/body_pose_model.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/openpose -o body_pose_model.pth
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/dpt_hybrid-midas-501f0c75.pt -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/midas -o dpt_hybrid-midas-501f0c75.pt
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/mlsd_large_512_fp32.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/mlsd -o mlsd_large_512_fp32.pth
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/mlsd_tiny_512_fp32.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/mlsd -o mlsd_tiny_512_fp32.pth
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/network-bsds500.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/hed -o network-bsds500.pth
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/ControlNet/resolve/main/upernet_global_small.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/uniformer -o upernet_global_small.pth
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11f1e_sd15_tile.pth -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models -o control_v11f1e_sd15_tile.pth
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11f1e_sd15_tile.yaml -d /kaggle/working/stable-diffusion-webui/extensions/sd-webui-controlnet/models
# CheckPoint
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/55203 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o QteaMix-通用Q版模型.safetensors
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/57618 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o Counterfeit-V3.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/46846 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o revAnimated_v122.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/13069 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o fantasyWorld_v1.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/25571 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o CheckpointYesmix_v16Original.safetensors
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/69832 -d /kaggle/working/stable-diffusion-webui/models/Stable-diffusion -o disneyPixarCartoon_v10.safetensors
# Lora
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/32988 -d /kaggle/working/stable-diffusion-webui/models/Lora -o blindbox_v1_mix.safetensors
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/63077 -d /kaggle/working/stable-diffusion-webui/models/Lora -o hanfu_tangstyle_v31.safetensors
# !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/50709 -d /kaggle/working/stable-diffusion-webui/models/Lora -o fashigirl-v5.4-lora-64dim-naivae-5ep.safetensors
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/85137 -d /kaggle/working/stable-diffusion-webui/models/Lora -o ChineseDragon_v2.safetensors
# embeddings
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/datasets/gsdf/EasyNegative/resolve/main/EasyNegative.safetensors -d /kaggle/working/stable-diffusion-webui/embeddings -o EasyNegative.safetensors
# VAE
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/gemasai/vae-ft-mse-840000-ema-pruned/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt -d /kaggle/working/stable-diffusion-webui/models/VAE -o vae-ft-mse-840000-ema-pruned.ckpt
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/14459?type=VAE -d /kaggle/working/stable-diffusion-webui/models/VAE
!sed -i -e 's/numpy==1.23.3/numpy==1.21.6/g' requirements_versions.txt
!sed -i -e 's/blendmodes==2022/blendmodes==2021/g' requirements_versions.txt
!sed -i -e 's/fastapi==0.90.1/fastapi==0.89.1/g' requirements_versions.txt
!sed -i -e '''/ prepare_environment()/a\ os.system(f"""sed -i -e ''"s/dict()))/dict())).cuda()/g"'' /kaggle/working/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/util.py""")''' /kaggle/working/stable-diffusion-webui/launch.py
!npm install -g localtunnel
import subprocess
import threading
import time
import socket
def iframe_thread(port):
while True:
time.sleep(0.5)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex(('127.0.0.1', port))
if result == 0:
break
sock.close()
p = subprocess.Popen(["lt", "--port", "{}".format(port)], stdout=subprocess.PIPE)
for line in p.stdout:
print(line.decode(), end='')
threading.Thread(target=iframe_thread, daemon=True, args=(7860,)).start()
#####################################################################
# 同步装备
# 运用huggingface保存和载入webui装备文件
huggingface_use = True
zip_output=True
huggingface_token_file = '/kaggle/input/hgtoken/huggingfacetoken.text'
huggiingface_repo_id = 'MSa/sd-output-images'
huggiingface_config_repo_id = 'MSa/sd-config'
# 将会同步的文件
config_files = [
'ui-config.json',
'config.json',
# 'styles.csv'
]
#功用函数
from pathlib import Path
from huggingface_hub import HfApi, login
import subprocess
import pandas as pd
import shutil
import os
import re
import gc
import requests
import datetime
import pytz
# 生成时刻字符串
def timeStamp():
# 获取当前本地时刻
local_time = datetime.datetime.now()
# 创立北京时区目标
beijing_tz = pytz.timezone('Asia/Shanghai')
# 将任意时区时刻转换为北京时刻
beijing_time = local_time.astimezone(beijing_tz)
timeStr = beijing_time.strftime('%Y%m%d-%H:%M:%S')
return timeStr
# config 上传
def hugface_upload_config(huggingface_token_file, config_files, repo_id):
if Path(huggingface_token_file).exists():
with open(huggingface_token_file, encoding="utf-8") as nkfile:
hugToken = nkfile.readline()
if hugToken != '':
# 运用您的 Hugging Face 拜访令牌登录
login(token=hugToken)
# 实例化 HfApi 类
api = HfApi()
print("HfApi 类已实例化")
%cd /kaggle/working/stable-diffusion-webui
# 运用 upload_file() 函数上传文件
print("开端上传文件...")
for config_file in config_files:
if Path(config_file).exists():
response = api.upload_file(
path_or_fileobj=config_file,
path_in_repo=config_file,
repo_id=repo_id,
repo_type="dataset"
)
print("文件上传完成")
print(f"呼应: {response}")
else:
print(f'Error: File {config_file} does not exist')
else:
print(f'Error: File {huggingface_token_file} does not exist')
# 上传图片紧缩文件
def hugface_upload_zipfile(huggingface_token_file, zipfile, repo_id):
if Path(huggingface_token_file).exists():
with open(huggingface_token_file, encoding="utf-8") as nkfile:
hugToken = nkfile.readline()
if hugToken != '':
# 运用您的 Hugging Face 拜访令牌登录
login(token=hugToken)
# 实例化 HfApi 类
api = HfApi()
print("HfApi 类已实例化")
%cd /kaggle/working/
# 运用 upload_file() 函数上传文件
print("开端上传文件...")
if Path(zipfile).exists():
response = api.upload_file(
path_or_fileobj=zipfile,
path_in_repo='output-images/' + zipfile,
repo_id=repo_id,
repo_type="dataset"
)
print("文件上传完成")
print(f"呼应: {response}")
else:
print(f'Error: File {zipfile} does not exist')
else:
print(f'Error: File {huggingface_token_file} does not exist')
# 将生成的图片紧缩
clear_output = True # 是否整理输出文件夹
def zip_updata():
if zip_output:
output_folder = f'/kaggle/working/stable-diffusion-webui/outputs/'
if os.path.exists(output_folder):
fileName = timeStamp()
filePath = '/kaggle/working/' + fileName
zip_file_path = shutil.make_archive(filePath, 'zip', output_folder)
print('图片已紧缩到output')
if os.path.exists(zip_file_path):
# 上传紧缩包
hugface_upload_zipfile(huggingface_token_file, fileName + ".zip", huggiingface_repo_id)
print("------hugface_upload_zipfile------")
else:
print(f'文件夹 {output_folder} 不存在,越过紧缩操作')
if clear_output:
%cd /kaggle/working/stable-diffusion-webui/outputs/
clean_folder('img2img-images')
clean_folder('txt2img-images')
clean_folder('img2img-grids')
clean_folder('txt2img-grids')
clean_folder('extras-images')
print('整理结束')
if huggingface_use == True:
hugface_upload_config(huggingface_token_file,config_files,huggiingface_config_repo_id)
# 整理文件夹
def clean_folder(folder_path):
if not os.path.exists(folder_path):
return
for filename in os.listdir(folder_path):
file_path = os.path.join(folder_path, filename)
if os.path.isfile(file_path):
os.remove(file_path)
elif os.path.isdir(file_path):
shutil.rmtree(file_path)
#####################################################################
####################################################################
# 下载装备文件
def webui_config_download(config_files, huggiingface_repo_id):
print("--------开端同步Config--------")
%cd /kaggle/working/stable-diffusion-webui/
for config_file in config_files:
url = f'https://huggingface.co/datasets/{huggiingface_repo_id}/resolve/main/{config_file}'
response = requests.head(url)
if response.status_code == 200:
result = subprocess.run(['wget', '-O', config_file, url, '-q'], capture_output=True)
if result.returncode != 0:
print(f'Error: Failed to download {config_file} from {url}')
else:
print(f'{config_file} download success !!!')
else:
print(f'Error: Invalid URL {url}')
# 履行下载
webui_config_download(config_files, huggiingface_config_repo_id)
#######################################################
#发动参数
args = [
'--share',
'--xformers',
'--lowram',
'--enable-insecure-extension-access',
'--theme dark',
'--gradio-queue',
]
##################################
!python launch.py {' '.join(args)}
# 履行config上传(需求的时分才履行)
hugface_upload_config(huggingface_token_file, config_files, huggiingface_config_repo_id)
# 履行上传成品图片紧缩包(需求的时分才履行)
zip_updata()