Personal project ยท Local AI studio
CinematicAgent
Local AI studio: songs, stories, videos, images and posts are queued from Django and rendered by Temporal workflows on one shared GPU.
5000+songs created
1 GPUone heavy model at a time
- Type
- Personal project
- Status
- Ongoing
- Runs on
- One NVIDIA RTX 5060 Ti
- Orchestration
- Django + Temporal
- My role
- Designed and built end to end
01
Context
- Five kinds of output: songs, stories, videos, images, social posts
- Dozens of open models, each with its own dependencies
- One GPU, shared by all of them
02
How it works
1Request
- Created in the Django admin
- Song, story, video, image or post
2Orchestrate
- Temporal workflow in the background
- One heavy model on the GPU at a time
- Each large model in its own venv or server
3Generate
- Lyrics-writer agents and songs
- Narration and voice
- Keyframes and video
4Finish
- Post-production and mastering
- Scenes joined into an mp4
- Posts scheduled to social platforms
03
What we built
- Songs
- Lyrics-writer agents on local LLMs: Gemma 4 26B (A4B), gpt-oss, Qwen Coder and Nemotron
- Lyrics converted to Devanagari or Gurmukhi
- YuE2-3B with YuE2-VAE, served over gRPC from its own repo
- Or ACE-Step 1.5: 2B or XL 4B, with a 1.7B or 4B LM, or the RagaLoRA variant
- Songs come with vocals
- Post-production
- Demucs splits the stems, which are then cleaned up
- The hook is moved into the first 5 to 10 seconds
- Mastered with Matchering or pyloudnorm
- Narration and voice
- Edge neural voices by default
- Veena and Chatterbox as the local Hindi and English engines
- XTTS-v2, IndexTTS-2 and Svara as options; Piper and MMS as offline fallbacks
- Seed-VC for voice conversion, Whisper Hindi for speech-to-text
- Images and characters
- FLUX.2-klein-4B by default; FLUX.2-klein-9B, Qwen-Image-2512 or SDXL-Turbo as options
- RealVisXL V5.0 for realism
- IP-Adapter for face matching
- Trained character LoRAs keep the same character across images and videos
- Video
- Wan 2.2 TI2V-5B from keyframes by default
- HunyuanVideo 1.5 straight from text
- LTX-2.3 when a clip needs its own sound or lip-sync
- Wan 2.2 S2V-14B for talking characters, Animate-14B for character animation
- Wan 2.2 VACE turns live-action clips into anime or 3D (an AniSora slot falls back to Wan 2.2 until it is set up)
- Stories
- A story is split into scenes
- Each scene gets narration, a keyframe and a clip
- The clips are joined into one mp4
- Publishing
- An LLM writes titles and captions
- Posts are scheduled to YouTube, Instagram, Pinterest and LinkedIn through their APIs
- Agents
- Chat agents built with LangGraph on local Ollama models such as qwen3:30b-a3b
- Experiments
- Avatar models tried: EchoMimicV3, InfiniteTalk, LatentSync, LongCat, OmniAvatar and Ovi
03
Key insight
On one shared GPU, scheduling is the real work: load one heavy model, run it, free it, and give every big model its own environment.
04
My part
My own project: I designed and built the whole system, from the Django app and Temporal workflows to the GPU scheduling, model choices and post-production.
Want the detail behind the numbers - architecture, failure modes, what I’d change? Happy to walk through it.
Talk about this project →