KaleCancer

Multimodal machine learning for oncology: whole-slide pathology, radiology, and clinical records, with time-to-event prediction.

Getting Started | Documentation | Examples | Contributing | Architecture

KaleCancer is an oncology library built on PyKale, a library in the PyTorch ecosystem, aiming to make cancer machine learning more accessible to interdisciplinary research by bridging gaps between clinical data, software, and end users. Both machine learning experts and clinical researchers can do better research with our accessible, scalable, and sustainable design, guided by green machine learning principles. KaleCancer inherits PyKale's unified pipeline-based API and extends it where oncology needs more: multimodal learning across pathology, radiology, and clinical records, and survival analysis for time-to-event outcomes such as overall and disease-specific survival.

Cancer prediction is rarely single-modality. A prognosis draws on a whole-slide image, a CT or MRI scan, and a clinical record at once, and its outcome is a time to an event that is usually censored rather than a class label. MONAI covers 3D medical imaging well but provides neither multimodal fusion nor survival modelling, while general survival libraries provide neither the imaging encoders nor a shared workflow. KaleCancer supplies both in one pipeline, so that a unimodal baseline and a fused multimodal model are the same code path under a different configuration.

KaleCancer enforces the same standardization and minimalism as PyKale, via green machine learning concepts of reducing repetitions and redundancy, reusing existing resources, and recycling learning models across areas. Modules are built on PyKale rather than beside it, and the survival module is deliberately kept cancer-agnostic, importing only PyTorch and NumPy so that it can migrate into PyKale core unchanged. This boundary is enforced in CI.

• loaddata loads data from disk or online resources as input, including cohort matching and leakage-safe splitting
• prepdata preprocesses data to fit machine learning modules below (transforms)
• model.embed embeds data in a new space to learn a new representation (attention multiple-instance learning over patch bags, tabular encoders, and multimodal fusion)
• model.predict turns a representation into a score: LinearHead and CoxHead, with the losses that train them
• evaluate evaluates the performance using some metrics, including IPCW time-dependent AUC and the integrated Brier score
• interpret interprets the features and outputs via post-prediction analysis mainly via visualization
• auto selects and constructs a workflow from a configuration alone (AutoCancer* classes, planned)
• pipeline specifies a machine learning workflow by combining several other modules: a trainer says where the data comes from, a task (SurvivalTask, ClassificationTask) says what is predicted from it, and any trainer takes any task

• examples demonstrate real applications on specific datasets with a standardized structure.

KaleCancer supports Python 3.10, 3.11, or 3.12. Before installing kalecancer, we suggest you to first install PyTorch matching your hardware, and then install PyKale following its official instructions.

Installation of kalecancer from source:

git clone https://github.com/pykale/cancer.git
cd cancer
pip install -e .

Heavy libraries are kept out of the core install and grouped into extras, so that a pathology workflow does not pull in a radiology stack:

For more details and other options, please refer to the quickstart guide.

Start with the quickstart, which walks through the same run from the command line, from Python, and from a coding agent. No account or credentials are needed: the examples stream the published HANCOCK dataset (CC BY 4.0) over HTTP range requests, so only the patients you ask for are transferred.

Browse through the examples to see the usage of KaleCancer in performing survival prediction in a wide range of settings, from a single modality to fused multimodal models:

# Whole-slide pathology only: attention MIL with a Cox head
python -m examples.hancock_wsi_survival.main --cfg examples/hancock_wsi_survival/configs/hancock_primary_tumour_quick.yaml

# Clinical records only, imaging only, or both, on the official HANCOCK split
python -m examples.hancock_multimodal_survival.main --cfg examples/hancock_multimodal_survival/configs/tabular.yaml
python -m examples.hancock_multimodal_survival.main --cfg examples/hancock_multimodal_survival/configs/imaging.yaml
python -m examples.hancock_multimodal_survival.main --cfg examples/hancock_multimodal_survival/configs/multimodal.yaml

# How the modalities combine is a configuration change, not a code change
python -m examples.hancock_multimodal_survival.main --cfg examples/hancock_multimodal_survival/configs/multimodal.yaml FUSION.STAGE late

Each example follows the standardized structure of main.py, config.py, and configs/*.yaml, so an experiment is described by its configuration rather than by edited code. See multimodal fusion for the fusion stages and methods available, and the WSI survival pipeline for inputs, configuration, and outputs in detail.

Ask questions on PyKale's GitHub Discussions tab if you need help or create an issue if you find something wrong.

Build new modules and/or projects with KaleCancer referring to the architecture guide, e.g., on how to modify an existing pipeline or build a new one. New code belongs in the pipeline stage that names what it does, and anything specific to one dataset belongs in examples rather than in the library.

This is an open-source project welcoming your contributions. You can contribute in three ways:

• Star and fork KaleCancer to follow its latest developments, share it with your networks, and ask questions about it.
• Use KaleCancer in your project and let us know any bugs (& fixes) and feature requests/suggestions via creating an issue.
• Contribute via branch, fork, and pull for minor fixes and new features, functions, or examples to become one of the contributors.

Run the same checks as CI before opening a pull request:

pip install -e ".[dev]"
pre-commit run --all-files && pytest --cov=kalecancer

Conventions and architectural constraints are documented in AGENTS.md, which applies to human and automated contributors alike. See PyKale's contributing guidelines for more details. The participation in this open source project is subject to PyKale's Code of Conduct.

KaleCancer is developed within the PyKale project at the University of Sheffield, with contributions from many other contributors.

KaleCancer does not have a publication of its own yet. Please consider citing the PyKale CIKM2022 paper below if you find KaleCancer useful to your research.

    @inproceedings{pykale-cikm2022,
      title     = {{PyKale}: Knowledge-Aware Machine Learning from Multiple Sources in {Python}},
      author    = {Haiping Lu and Xianyuan Liu and Shuo Zhou and Robert Turner and Peizhen Bai and Raivo Koot and Mustafa Chasmai and Lawrence Schobs and Hao Xu},
      booktitle = {Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM)},
      doi       = {10.1145/3511808.3557676},
      year      = {2022}
    }

KaleCancer is built on PyKale and inherits its acknowledgements. The examples use the HANCOCK dataset, a multimodal head and neck cancer cohort released under CC BY 4.0 and licensed separately from this software.