NeuralNJ

Code implementation for "Accurate and efficient phylogenetic inference through end-to-end deep learning".

To set up the environment, run the following commands:

conda env create -f environment.yaml

Note: raxmlpy needs to be installed separately. Please refer to the instructions in the RAxMLpy folder.

Activate the environment with:

conda activate NeuralNJ

• IQTree: A tool for simulating evolutionary processes and generating MSA.
• RAxML: A solver for phylogenetic inference.

To generate synthetic data with MSA lengths from 128 to 1024 and 50 species, navigate to the data_gen directory and run:

cd data_gen/
# Make sure to set the `excuting_dir` variable in `generate.py` to the path of the IQTree executable
python generate.py

By default, generate.py utilizes ALISM with its default parameter settings for tree simulation. For empirical parameter-based simulations, refer to generate_empirical.py, which employs parameters derived from Naser-Khdour, S., Minh, B. Q., & Lanfear, R. (2021). The influence of model violation on phylogenetic inference: a simulation study. bioRxiv, 2021.09.22.461455. https://doi.org/10.1101/2021.09.22.461455. These parameters are used to configure ALISM for data generation.

Note: NeuralNJ employs datasets generated via generate_empirical.py for both training and validation.

Note: Pre-generated datasets under the GTR+I+G model used in the paper are available on Zenodo: https://doi.org/10.5281/zenodo.16912077. Please download these datasets and place them in the ./data_gen/data directory for training, validation, and testing.

For real data, refer to the work described in the article Evaluating Fast Maximum Likelihood-Based Phylogenetic Programs Using Empirical Phylogenomic Data Sets.

To use the datasets provided in ./data_gen/data for training or testing, please modify the data directory paths in the corresponding YAML configuration files.

To train the model using synthetic data under the evolution model GTR+I+G, use the following command:

python train.py --config_path config/pretrain_mix.yaml

You can monitor training and validation curves using TensorBoard with the following command:

tensorboard --logdir tb_logs/

To perform inference for NeuralNJ, run:

python finetune_rl_search.py --config ./config/finetune_reinforce_search_example.yaml --infer_opt Argmax

python finetune_rl_search.py --config ./config/finetune_reinforce_search_example.yaml --infer_opt Search

python finetune_rl_search.py --config ./config/finetune_reinforce_search_example.yaml --infer_opt Finetune

The example configuration file ./config/finetune_reinforce_search_example.yaml uses test cases located in the examples folder. Specifically, two cases in examples/len1024taxa50:

• G_l_1024_n_50_0_0.03_73.phy: Used in the phylogenetic analysis case study in the paper
• G_l_1024_n_50_0_0.02_71.phy: Used in the topology construction process analysis

Additionally, examples/cal_rf_distance.py provides a utility to calculate Robinson-Foulds distance between two phylogenetic trees. Use it with python cal_rf_distance.py --reftree <reference_tree_file> --inftree <inferred_tree_file> to evaluate topological differences between trees.