This is an unofficial implementation of HiMat (https://arxiv.org/abs/2508.07011) generating materials only at 1K resolution. It was implemented without the supplementary material so there can be discrepancies when compared the original work.
Note: This codebase was built on top of SANA.
First create a conda environment, activate it, install ninja, and verify the installation as below.
conda create -n <env_name> python=3.10.0 -y
conda activate <env_name>
pip install ninja
ninja --version
echo $? (should return exit code 0)
If exit code is not 0, please refer to flash-attn Github.
Once the ninja is successfully installed, run the following command to install essential dependencies including flash-attn:
bash environment_setup.sh
There can be problems with flash-attn installation, in that case please refer to the flash-attn Github. Moreover, do not forget to check the issues section especially this one.
The text prompts for materials were generated using Gemini-2.5. Both tags from MatSynth and the rendered material images were used to capture all the essential material properties.
In approximately 50-70 words, describe a [{tags}] surface. Include specific colors, texture details, arrangement patterns, and notable features. Write one detailed paragraph that captures all essential visual properties. Make sure the properties like shape, color, and texture details are consistent with the provided image.
Generate latents for SVBRDF maps using DC-AE and store them locally in a .tar along with text prompts which can be used with SANA's SanaWebDatasetMS dataloader.
To generate features:
python myscripts/create_features.py
Please change the datapaths inside the file and feel free to make other necessary changes.
The latents and text prompts are stored in .tar files but we still need to generate metadata wids-meta.json using the following command:
python tools/convert_scripts/convert_ImgDataset_to_WebDatasetMS_format.py
Once you have created the metadata, please verify it has information about all of the tar files you have created and it should look as follows:
{
"name": "sana-dev",
"__kind__": "SANA-WebDataset",
"wids_version": 1,
"shardlist": [
{
"url": "train-00000-of-00431.tar",
"nsamples": 9,
"filesize": 3563520
},
{
"url": "train-00001-of-00431.tar",
"nsamples": 3,
"filesize": 1198080
},
.....
.....
.....
]
}
To run the training, please edit train.sh as per requirement and run it as follows:
bash train_scripts/train.sh
Please remember to edit Sana_1600M_1024px_allqknorm_bf16_lr2e5.yaml to update the paths to the pretrained models such as Gemma-2, DC-AE, and SANA1.5@1024px.
Note: To train HiMat, you need at least 2 GPUs each having a VRAM of
$\ge$ 40GB.
To run inference, please first download the himat model from Hugging Face, e.g., as below or some other way:
hf download zeahmd/himat_unofficial --repo-type model --local-dir .
Please update the model path in the aforementioned Sana_1600M_1024px_allqknorm_bf16_lr2e5.yaml config file. Apart from himat, you will also have to specify the paths for Gemma-2 and DC-AE.
Once the environment is setup and the models are downloaded, you simply need to run inference as below:
python generate_maps.py
| Text Prompt | Rendering | Basecolor | Normal | Roughness | Metallic | Height |
|---|---|---|---|---|---|---|
| a surface showing white bathroom tiles with an exquisite pattern | ||||||
| closeup detail of red leather texture background | ||||||
| a sci-fi golden circuit agains the dark background | ||||||
| a stone with shiny patches | ||||||
| a brown wood texture background |