MiShape: Shape Modelling of Mitochondria in 3D for Microscopy

This repository contains the code to reproduce the results from the paper MiShape: Shape Modelling of Mitochondria in 3D for Microscopy. It borrows heavily from the repository Occupancy Network.

Authors: Abhinanda R. Punnakkal¹, Suyog S. Jadhav¹, Biswajoy Ghosh², Alexander Horsch¹, Krishna Agarwal², Dilip K. Prasad¹

¹ Department of Computer Science, UiT The Arctic University of Norway, Tromsø, Norway ² Department of Physics and Technology, UiT The Arctic University of Norway, Tromsø, Norway

Corresponding author: abhinanda.r.punnakkal@uit.no

The problem of accurate reconstruction of the shape of mitochondria is important for understanding the life-sustaining mechanisms at the cellular level that are indicative of many illnesses and poor health conditions. Current approaches are restricted to the problem of 2D segmentation of fluorescent images and do not address the inherent 3D structure. We present MiShape, a 3D shape prior for modeling mitochondrial shapes in fluorescence microscopy images. MiShape is representative of the real distribution of mitochondrial shapes as it is trained using shapes sourced from high-resolution Electron Microscopy images. MiShape uses implicit representation for modeling shapes, which offers a versatile solution to accurately model the diverse shapes of sub-cellular structures. MiShape is the first mitochondria analysis tool that can reconstruct 3D shapes given a single 2D image.

First you have to make sure that you have all dependencies in place. The simplest way to do so, is to use anaconda.

You can create an anaconda environment called mesh_funcspace using

conda env create -f environment.yaml
conda activate mesh_funcspace2
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

Next, compile the extension modules. You can do this via

python setup.py build_ext --inplace

This code has been tested on python version 3.6.7.

To use MiShape, you have to install all binary dependencies (steps in installation), fetch the pre-trained models and converted the data into the correct input format.

You can downlod the pretrained models for unconditional generation, single image and stack to 3D reconstruction from (the Zenodo repository.)[10.5281/zenodo.21159675]. Please unzip the folder to the 'out' directory. You'll find models for uniconditional generation, single image to shape reconstruction, 3 image, 5 image and 7 image stack models.

For example, to generate 3D reconstructions from single images, you need images/stacks of xy dimension 128x128 pixels as input.

To generate meshes using a trained model, use

python generate.py CONFIG.yaml

where you replace CONFIG.yaml with the correct config file. Adjust the 'data/path' variable in the config file as needed.

The easiest way is to use a pretrained model.

You can do this by using one of the config files

configs/img_onet_EMPIAR10791_real_test.yaml
configs/stack_onet_EMPIAR10791_real_test.yaml
configs/unconditional_onet_EMPIAR_10791.yaml

To evaluate a pretrained model or train a new model from scratch, you have to obtain the dataset. To this end, there are two options:

1. you can download our preprocessed data
2. you can run the preprocessing pipeline yourself for new data

Unless your data differes significantly from usual epi-fluoresence or confocal mitochondria images we recommmend to use the first option.

For evaluation of the models, we provide the script eval_meshes.py. You can run it using

python eval_meshes.py CONFIG.yaml

The script takes the meshes generated in the previous step and evaluates them using a standardized protocol. The output will be written to .pkl/.csv files in the corresponding generation folder which can be processed using pandas.

Finally, to train a new network from scratch, run

python train.py CONFIG.yaml

where you replace CONFIG.yaml with the name of the configuration file you want to use.

You can monitor on http://localhost:6006 the training process using tensorboard:

cd OUTPUT_DIR
tensorboard --logdir ./logs --port 6006

where you replace OUTPUT_DIR with the respective output directory.

For available training options, please take a look at configs/default.yaml.

If you use this code, please cite:

MiShape: Shape Modelling of Mitochondria in 3D for Microscopy

MIT License. See LICENSE file.