An RF-based pipeline for binary classification of molecular profiles and clinical outcomes was developed using the Scikit-learn Python library with balanced class weight to tackle dataset imbalance.
The pipeline based on RF is structured into two main steps: data selection and pre-processing, followed by model creation for binary classification.
It allows to choose relevant patient characteristics and cell types with their corresponding pre-processed markers, morphological, and neighbourhood features. Next, a user defines the pair of target classes, either two molecular profiles or short-term and long-term survival. The pipeline employs nested cross-validation, where the outer loop splits patients and their corresponding cells into training (80% of samples) and test sets. The training set is utilized then by an inner loop of nested cross-validation for Bayesian hyperparameter tuning, facilitating the creation of the final RF model. Next, this model’s predictive features are extracted. The test set is used to evaluate the performance of the created model on unseen data and extract single-cell level predictions of target classes.
Files:
- Code is available in mol_profiles_bin_class_pipeline.py for molecular profiles' prediction and in 'clinical_outcome_bin_class_pipeline.py' for clinical outcomes' prediction
- Conda environment is in env.yml
- 'Cancer_BRCAloss_vs_HRP_PDS.csv' is example data for molecular profiles' prediction and 'Generate_sample_df.ipynb' - code to generate it
To run the code: Set up a conda environment with the help of env.yml file
conda env create -f env.yml
conda activate cefiira_env
- Prepare a CSV file with desired experiments
| experiment | channels | channels_to_transform | channels_to_outliers | channels_to_scale | types_of_cells | classes_column | classes_types | therapies | scaling_type | best_parameters | balanced_acc_train | balanced_acc_test | f1_train | f1_test | most_predictive_features | eliminated_features | permutation_scores | random_seed |
|---|
-
Don't forget to define the datasets that you will use:
names = ['EXPERIMENTS_FILE_NAME.csv',...]df = pd.read_csv("DATASET_FILENAME.csv") -
You are now ready to run the script
Tested on Linux and macOS
- Experiment file 'Cancer_BRCAloss_vs_HRP_PDS.csv'.
- Sample dataset 'sample_single_cell_df.csv' can be generated with Generate_sample_df.ipynb.
Activate your conda enviroment or download all required libraries (recommended for macOS). Make sure that both the experiment file and the code are in the same directory.
Run the pipeline script using your terminal or console. The following command will execute the script in the background and save the output to a file named exp.out:
nohup python mol_profiles_bin_class_pipeline.py > exp.out 2>&1 &
Upon the end of experiments, you will see 'Stored successfully experiments' file' in exp.out.
However, if you need to stop the running process, use the command below to find and kill the process:
kill $(ps aux | grep python | grep mol_profiles_bin_class_pipeline.py | awk '{print $2}')
Upon completion, the script will update the experiment file with:
- Classification Performance: Includes metrics such as balanced accuracy and weighted f1 on training and test sets.
- Predictive Features: Identifies the key features used for classification based on Cross-validated Recursive Feature Elimination and records Mean Decrease Accuracy (permutation) scores.
Additionally, the script can generate the following output files in the same directory as the Python script:
- ROC Curve: A file depicting the ROC curve to evaluate the performance.
- Patient Predictions: A file containing predictions for individual patients.
Aleksandra Shabanova aleksandra.shabanova@helsinki.fi