Text to Video Ghibli style 🎞️ (Text2Video-Ghibli-style)

Transform text into captivating videos with the enchanting aesthetic of Studio Ghibli. This project leverages a fine-tuned zeroscope_v2_576w model to generate high-quality, Ghibli-inspired video content from text prompts. Built upon the foundation of MotionDirector, it includes optimized code and enhanced stability for seamless performance. 🌟

• 🎨 Generate videos in the iconic Studio Ghibli art style
• ⚡ Optimized for efficiency with fine-tuned zeroscope_v2_576w
• 🛠️ User-friendly notebooks for training and inference
• 🌐 Interactive Gradio demo for real-time video generation
• 🔧 Enhanced stability and performance over the original MotionDirector

Explore the project with our comprehensive notebook:

You can explore more in this HuggingFace Dataset available at the given link for further details: .

The model is trained on a curated dataset inspired by Studio Ghibli's visual style, ensuring authentic and high-quality video outputs. Details are available in the dataset processing script. 📊

This project builds upon the zeroscope_v2_576w model, fine-tuned with LoRA for Ghibli-style video generation, forked from MotionDirector. 🚀

Experience the magic of Ghibli-style video generation:

• HuggingFace Space:

• Demo GUI:
  

To run the Gradio app locally (localhost:7860):

python apps/gradio_app.py

Clone the project repository and navigate to the project directory:

git clone https://github.com/danhtran2mind/Text2Video-Ghibli-style.git
cd Text2Video-Ghibli-style

Install the required Python packages:

pip install -r requirements/requirements.txt

Run the following scripts to set up the project:

• Install Third-Party Dependencies

  python scripts/setup_third_party.py

• Download Model Checkpoints

  python scripts/download_ckpts.py

• Prepare Dataset (Optional, for Training)

  python scripts/process_dataset.py

• To train the model:

python src/text2video_ghibli_style/train.py

• Training Hyperparameters Refer to the Training Documents for detailed hyperparameters used in fine-tuning the model. ⚙️

To generate videos using the trained model:

python src/text2video_ghibli_style/inference.py

Prompt	Video
A dog is running with Ghibli style
A girl is walking with Ghibli style
Studio Ghibli style. Young man contemplates, walks away from ivy-covered yellow building.
Studio Ghibli style. Two women walk down coastal village path toward sea, passing colorful houses, sailboats visible.

• Python: 3.10 or higher
• Key Libraries: See requirements_compatible.txt for compatible versions

This repository is trained from , a fork of , with numerous bug fixes and rewritten code for improved performance and stability. You can download the zeroscope_v2_576w model from the . Explore more models on .