Design tools tests and updates to SDXL, Model Plan~

After these days' design work,
I am going to improve the modeling workflow/standards/tools,
and will create about 30 characters assets in 2 months as a test to make sure they work well,
then will test the grid world/tile world builder, weapon/prop assets, gameplay, storytelling and sound...~

I love SD1.5 checkpoints which have more resources and make it easier to get good-looking stylish designs,
but as I stepped further into customizing special projects, SD1.5 was unable to control the styles(like the 1990s-2000s anime) and elements (medieval fantasy and sci-fi mecha),
So I updated the SDXL preset to achieve them, (really slow but necessary~
and replaced all XL LoRAs, to make sure all of them were trained by myself so that I had a clearer vision of the results.

Mecha Turn Test: 

Used gameplay screencap from Warframe, and renders images as data sets,
(I collected, sorted them up from the internet, and used them for testing and learning only, please don't use them in other ways.

I will use my own 3d models (created by myself I mean~) and images to train for my projects later, which I will post here too and you can use them in whatever ways you want,
Hope you guys will train yours and share them~
The last piece of the puzzle will be the open checkpoints, not sure how to but I will keep it in my mind...

(Result Preview,
love them, they are elegant but not too complex to make me crazy when working as a modeler lol~
personally will combine and tweak different parts from different images for better and reasonable results in practice as I mentioned in the last post)

mechaturnRawResult.zip

raw pictures including generation info like prompts, LoRAs, samplers, and ControlNet, help us to rebuild the generation workflow and prompts frame,
("<lora:MechTurnXLRetag_v1:0.25>" is out of date, just delete this LoRA prompt.)

Merged Checkpoint

I tested about twenty or thirty SDXL checkpoints, then merged three of them to suit my requirements and taste.
A
Chenkin Anime Impasto -Based on Animagine XL V3.1-Checkpoint-风吟WindSing-LiblibAI
B
sdxl-动漫二次元-Checkpoint-出埃及记-LiblibAI
C=0.5A+0.5B
D
AbyssOrange XL Else - v1.0 | Stable Diffusion Checkpoint | Civitai
the final checkpoint = 0.5C+0.5D
just simply average them, not test too much as a lazy man myself~

Secondary LoRA

Here is the secondary LoRA for all kinds of character turntables,
actually an old LoRA with new tags to train,
CharTurnXLRetag_v4.safetensors

SD1.5 version
CharTurnRetag_v4.safetensors

Here is the training set (scale them to 768*512 for SD1.5 training), help us to predict the effect and write prompts
CharTurnXLTrainRetagged.zip

(render images, if you are interested in the models/scenes behind them, leave a message and then I will reply ASAP, they are so large and scattered that I haven't packed them all~)

All data sets used this time were tagged by GPT-V, which is much better than WD-14 Tagger,
catch the styles and elements from the training data and better respond to the rest prompts in the meantime,
more expensive but worth it,
Here is the GPT-V add-on for WebUI
GitHub - SleeeepyZhou/sd-webui-GPT4V-Image-Captioner: Stable Diffusion WebUI extension for GPT4V-Image-Captioner

All SD15 LoRAs are trained by this kohya_ss preset (something like my personal best practice, but I know there should be something wrong/unsuitable/can be improved~)
WitcherRenderTurnRetag_v1.json

and this base checkpoint
AnyLoRA - Checkpoint - bakedVae (blessed) fp16 NOT-PRUNED | Stable Diffusion Checkpoint | Civitai

All XL LoRAs are trained by this kohya_ss preset,
WitcherRenderTurnRetagXL_v1.json

and this base checkpoint
SD XL - v1.0 VAE fix | Stable Diffusion Checkpoint | Civitai

WF Gameplay Turn LoRA

wfRedAlphaTurnXL_v1.safetensors

SD1.5 version
wfTurnRedShadedAlphaBackgroundRetag_v1.safetensors
(I am always confused about how many steps/epoch/batch...should be the best,
a lot of magic numbers/training parameters that are hard to understand,
even hard to pick one of them by comparing their results...
if any guidance please let me know, THX!)

Training data set
wfAlphaTurnXLRetagTrain.zip

(gameplay screencap Preview, red shading on green background for easy roto on Photoshop, the original color set is a nightmare/work-hour black hole when separating)

WF Render Turn LoRA

wfRenderTurnXL_v1.safetensors
wfRenderTurn_v1.safetensors
WFRenderXLTrain.zip

I also test the Medieval Fantasy character turn on SDXL,

(Medieval Character Results Preview)
Medieval Character Results Raw.zip

Witcher Render Turn LoRA
trained by The Witcher 3 characters rendering images on Blender/Marmoset
WitcherRenderTurnRetagXL_v1.safetensors
WitcherRenderTurnRetag_v1.safetensors
WitcherRenderXLTrain.zip

(witcher render training data preview)

Witcher Screencap Turn LoRA
trained by The Witcher 3 gameplay screencap
WitcherAlphaTurnRetagXL_v1.safetensors
WitcherAlphaTurnRetag_v1.safetensors
WitcherAlphaTrainXL.zip

(witcher screencap training data preview)

Medieval Ref Turn
trained by the images of medieval characters from different anime generated by SD1.5,
MedievalRefTurnRetagXL_v1.safetensors
MedievalRefTurnRemixRetag_v1.safetensors
MedievalRefXLTrainRetag.zip

(Medieval ref turn training data preview)

I didn't make the images look totally the same as their original looks from their anime, because I just want the medieval elements from them instead of fanart designs.
Most Character LoRAs are trained by
nochekaiser881 Creator Profile | Civitai
And he left a training set here,
Yuuji Sakai (坂井 悠二) - Burning-Eyed Shana (Shakugan no Shana) (灼眼のシャナ) - COMMISSION W/ DATASET - anime s1 | Stable Diffusion LoRA | Civitai
then we can try to use a similar procedural way to train characters' LoRAs,
screencap, character detections, and tags, but I haven't tried cause his work is awesome and enough for me now~

Medieval Turn
trained by the images of random medieval characters generated by SDXL, make sure to better respond to relative prompts.
MedievalTurnRetagXL_v1.safetensors
MedievalTurnRemixRetag_v1.safetensors
MedievalTurnXLRemixRetag.zip

(Medieval turn training data preview)

Workable Prompts?

I also tested which prompts are workable, like classes(knight, witch, firefighter, cook, woodcutter...), skin tone,

iris color, and hairstyle/hair color,

character faces by LoRAs and even fabric materials...

but still not sure how they work overall, hard to build a full-features look-up sheet for this, even combining different prompts and random seeds will affect the results a lot,
I think the best way is to train checkpoints by ourselves or get the raw image and tag to take a look,
and the second way is testing based on special prompts, once we find it won't work well then we should train extra LoRA or merge CKPT for them,
that's the main reason why I train things like mecha and medieval fantasy above~

Plan to align the base topology and rig,
referring to characters from Maximo(limbs and torso ratio, bone rotation, joint position...),
for easy retargeting,
because we suffered from it in practice...~

We need the design tools to work for realistic characters too and will find the presets later~

I will update the anatomy of the base body, refer to an awesome guidebook,
Anatomy for Sculptors,  

mainly about the cross-section of the torso, limbs, and toes, which I didn't realize before.

Plan to Simplify the topology for anime characters,
I suffered from the fingers and toes tweaking in practice, too dense to tweak easily (like twisting to sort up topology),
actually, even about 3 rings are enough to draw a static knuckle, but the problem is dynamic shapes mostly ( as a modeler lol~), sounds like part of the work to improve anatomy~
I would like a little rough rather than a nightmare when editing,
yup~ for happy modeling/developing (but still keeping the good enough quality lol~)

(The rest improvements like extra output UV for faces and items of clothing, and solidifying some textures' detail as meshes are still on my TODO list but haven't been done this time~)

Also some mecha quick tests here,

https://1drv.ms/u/s!Am0lZ75OjQ1mo5VfBKoxgPEERnU7Tg?e=iRGNf3


https://1drv.ms/u/s!Am0lZ75OjQ1mo5VH6yOtVLipeSO2lg?e=FKo6Iq
the current problem is how to rebuild the structure quickly/procedurally, I am not sure...
but our designs have become more elegant now, which will cause fewer problems when rebuilding their structure I think~