How to use TemporalNet (Stable Diffusion IMG2IMG VIDEO TO VIDEO ANIMATION)
A quick guide on how to use TemporalNet without the python script. Video was made because I got many questions on how to use TemporalNet, and there aren’t really any proper guides online for it.
I’m not home for a few months, so I only had access to a laptop with low vram. Even just using the 3 ControlNet units pretty much maxed out my VRam usage, so I couldn’t get better looking results for you. Just generating 1 frame also took over 5 minutes, so... yeah. The settings were not tested, just went with the first settings I tried.
Would have made a more video like guide, but due to laptop hardware and time limitations... It’s mostly pictures.
I cropped the video beforehand using davinci resolve studio 18, and then extracted frames using ffmpeg at 15fps (original was 30fps). Then I just ran the batch with the settings.
For postprocessing of the final clip, I used Davinci’s deflickering and flowframes to up the fps back up to 30.
As you can see, this method allows for quite drastic style changes whilst being pretty consistent. I believe this is currently the best method for denoising 1 that I have seen (except maybe Tokyo_Jab’s method, but this allows for longer videos and easier frame-by-frame editing in my opinion). Another model that might help is reference_only, but I couldn’t get satisfactory results from the little testing I did in combination with the TemporalNet method.
I personally always use 2 TemporalNets: current and loopback.
General Tips for making videos:
- Background masking/removal helps a lot when trying to make things consistent. The actor/actress in the video is usually fairly simple to get consistent
- Recommended (Optional) post processing methods: Davinci’s Deflicker, Flowframes, (EBSynth)
- Also recommended to go and fix select frames afterwards. Sometimes hands become cloths etc.
- Use Multi-Controlnet. Some combos that I have found great are:
-- Normal_bae(or depth) Softedge_hed (or lineart_realistic) canny (threshholds around 35-45) openpose_face
-- openpose_full, normal_bae, canny
-- normal_bae, openpose_full
As for the weights, it varies quite a lot but I usually start at having Normal_bae around , hed around , canny around 0.4 and openpose face around . Then Temporalnet (current) at and temporalnet (loopback) at half of current.
Since someone will always ask anyways, here is the generation data with controlnets used:
Positive:
(masterpiece:1.4, best quality), (intricate details), unity 8k wallpaper, ultra detailed, beautiful and aesthetic, Korean girl dancing, half body, brunette, brown hair, white clothing, white sports bra, white top, navel, belly,
Negative:
(worst quality, low quality:1.4), (zombie, sketch, interlocked fingers, comic), (mask:1.2), blurry, high contrast
Steps: 24,
Sampler: Euler,
CFG scale: 7,
Seed: 4174458460,
Size: 576x1024,
Model hash: 77b7dc4ef0, Model: meinamix_meinaV10,
Denoising strength: 1,
Clip skip: 2,
ADetailer model: , ADetailer prompt: “korean face, perfect face, green eyes“, ADetailer confidence: 0.3, ADetailer dilate/erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.4, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use inpaint width/height: True, ADetailer inpaint width: 512, ADetailer inpaint height: 512, ADetailer version: ,
ControlNet 0: “preprocessor: openpose_full, model: control_v11p_sd15_openpose [cab727d4], weight: 0.7, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (512, -1, -1)“,
ControlNet 1: “preprocessor: none, model: diff_control_sd15_temporalnet_fp16 [adc6bd97], weight: 0.6, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)“,
ControlNet 2: “preprocessor: none, model: diff_control_sd15_temporalnet_fp16 [adc6bd97], weight: , starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)“, Version:
28 views
1034
287
3 days ago 00:03:47 1
SCOOTER - HOW MUCH IS THE FISH (Metal cover)
5 days ago 00:20:57 1
Incredible Discovery In The Grand Canyon? - YouTube
5 days ago 00:29:11 1
Interview with Valentin Kozlov, Valmix Company | Potato Tour Across Russia (Part 2) 🥔🚜
7 days ago 00:57:54 1
Lost In Time : The Isolated House Where She Lost Her Mind | How I Find Abandoned Placess
7 days ago 01:01:39 1
TIME SLIP STORIES (SN 18 EP 38) TRUE UNEXPLAINABLE EXPERIENCES WITH MISSING TIME & TIME SLIPS
1 week ago 00:00:31 2
How To Train Your Dragon | Big Game Spot
1 week ago 00:05:05 59
How to Clown - Misha Usov - Professional Clown School
1 week ago 00:38:45 2
Try Not To Laugh Challenge#7 | Instant Regret Fails Compilation 2024 | Amazing People
1 week ago 00:29:31 1
Membership Perks
1 week ago 07:10:32 1
У края бездны-(2024) все серии 1-4. Освобождение Мариуполя.Причитайте ограничения в описании ниже!!
2 weeks ago 00:08:00 16
Dream Theater - Midnight Messiah (Official Video)
2 weeks ago 00:05:14 2
Clara Fernández: The Spanish Sensation Transforming Pole Vaulting
2 weeks ago 00:27:54 5
Секретные космические лазеры устраивают пожары на Земле?
2 weeks ago 01:18:02 2
The Spider’s Web: Britain’s Second Empire | The Secret World of Finance
2 weeks ago 00:03:57 46
Britney Spears - ...Baby One More Time (Official Video)
2 weeks ago 00:17:09 1
Blouse crochet pattern - how to crochet summer top
2 weeks ago 00:15:40 1
Как вязать росток кардигана Вязание укороченными рядами How to knit a cardigan back height
2 weeks ago 00:04:50 20
In The Army Now (Status Quo) • Drum Cover
2 weeks ago 00:38:56 2
Time Team Podcast | New excavations at Pompeii
2 weeks ago 00:56:53 3
DOGMAN, MIRRORED MEN AND HIGHWAY GHOSTS: TALES FROM THE ROAD (SN 18 EP 36) TRUE SPOOKY STORIES
2 weeks ago 00:15:46 2
🐹 КРОТЫ УХОДЯТ НАВСЕГДА! Как прогнать кротов с участка дымовой самодельной шашкой
2 weeks ago 01:12:12 1
Crochet Lacey Vest Tutorial | Chenda DIY
2 weeks ago 00:09:56 1
Mother Tucker: The Donald Trump & Jared Kushner Family Business Suckled From The Tit Of Saudi Arabia
2 weeks ago 00:18:46 1
I Crocheted a Houndstooth Vest | Pattern Walk-through