Image generator

Deep floyd

DeepFloyd IF is a modular composed of a frozen text encoder and three cascaded pixel diffusion modules: a base model that generates 64x64 px image based on text prompt and two super-resolution models, each designed to generate images of increasing resolution: 256x256 px and 1024x1024 px

Visit Site
Views 416.3M
Monthly visits 416.3M
Global rank ##76
Votes 0

About this tool

Core Features & Characteristics: DeepFloyd IF is a modular composed of a frozen text encoder and three cascaded pixel diffusion modules: a base model that generates 64x64 px images based on text prompt, and two super-resolution models designed to generate images of increasing resolution: 256x256 px and 1024x1024 px. All stages utilize a frozen text encoder based on the T5 transformer to extract text embeddings, which are then fed into a UNet architecture enhanced with cross-attention and attention pooling. The model outperforms current state-of-the-art models, achieving a zero-shot FID score of 6.66 on the COCO dataset, underscoring the potential of larger UNet architectures in the first stage of cascaded diffusion models.

Main Modes & Use Cases:

  1. Dream (Text-to-Image): Generates images based on text prompts with customizable parameters like guidance_scale and sample_timestep_respacing.
  2. Zero-shot Image-to-Image Translation (Style Transfer): The output of the prompt comes out in the style of the support_pil_img, supporting styles like professional origami, oil art, plastic building bricks, and classic anime.
  3. Super Resolution: Users can run IF-II and IF-III or 'Stable x4' on an image not necessarily generated by IF to upscale low-resolution images to high-resolution.
  4. Zero-shot Inpainting: Performs local image repainting based on the provided original image, inpainting mask, and text prompt.

Usage Instructions & Integration:

  • Integrated with the Hugging Face Diffusers library, utilizing model cpu offloading to run the whole IF pipeline with as little as 14 GB of VRAM. If using torch>=2.0.0, all enable_xformers_memory_efficient_attention() functions must be deleted.
  • Users must have a Hugging Face account, accept the license on the model card, and login locally using huggingface_hub with an access token.
  • Local installation via deepfloyd_if Python library is also available, requiring xformers and CLIP.

Hardware Requirements:

  • Minimum 16GB VRAM for IF-I-XL (4.3B) & IF-II-L (1.2B); 24GB VRAM required to also include Stable x4 upscaler (to 1024x1024). Requires xformers and setting env variable FORCE_MEM_EFFICIENT_ATTN=1.

Model Zoo & Scale: Includes models of various parameter sizes such as IF-I-M (400M), IF-I-L (900M), IF-I-XL (4.3B), IF-II-M (450M), IF-II-L (1.2B), and IF-III-L (700M).

Pricing & License: The code is released under a bespoke license (with specific restricted points). The initial release of the IF model is under a restricted research-purposes-only license temporarily, with the intention to release a fully open-source model later. Model weights are accessible for free via Hugging Face.

Tool NameDeep floyd
CategoryImage generator , Resources
Pricing ModelFree Trial
Listed Date2023-04-02
Top AlternativesAIAvatarGPT.com , NextThreeBooks , Getimg.ai

Traffic analysis

Traffic, ranking, and engagement signals for this tool.

Global rank ##76
Country rank #113
Monthly visits 416.3M

Visit trend

No trend data

Engagement

  • Bounce rate--
  • Pages per visit--
  • Average duration--

Traffic sources

No source data

Top countries

No country data

Interface preview

User reviews

Embed "Featured on ToolAI.io" Badge

Copy the HTML snippet below to display your ToolAI featured badge on your website or docs.

Featured on ToolAI.io