Cell Tracking Using a Multiple

Model Kalman Filter Approach

The Problem:
• Cells observed through microscope move unpredictably
over time

• May need to track individual cells as they move through

a solution
▫ May need to track multiple cells simultaneously

Possible cell characteristics to track:

Movement
Splits (via mitosis)
Size
Interaction with other cells
The Motivation:
• Want to track cell development over time

• Need consistency in marking cell phases, splits, etc.

• Cell behavior is erratic and human estimation is

subjective

OBJECTIVE:
Using several state-space models of cell motion, estimate cell trajectories using
the appropriate Kalman filter. Compare filter performance for individual
models and propose potential improvements.
The Data:
• Video of yeast cells in solution*
• Time-lapsed view of cells
• Note: It is possible that actual time between when consecutive frames were
collected varies during video
• Video frame rate = 10 fps

• Cells in video have access to ‘nutrient canal’ that contributes to their

development

• Multiple cell splits are visible

• Truth data was generated by determining the position of a single cell

throughout the entire video
• This will be used to model the “perfect” sensor measurements

* Data provided by Phil Burlina (JHU APL)

The Challenges:

Dynamics Measurement

-Collective cell motion is random - Cells are similar in size and shape

-Individual cells move - Few (if any) distinguishing

unpredictably (No definitive state- features
space model)
- Grayscale video provides
- Cell interactions are complicated additional complexity
and not easily modeled
- Cell splits occur quickly and are
difficult to detect
The Measurements:

Matched Filter (MF) Problems:

• Cell of interest (from previous ▫ Cells are similar size
measurement) compared with ▫ Matched filter is not rotation
frame centered at predicted position invariant
▫ Coordinates of highest MF value ▫ Grayscale image lacks
used as position measurement dynamic range of values

• Other Variations/Methods:
▫ Apply threshold and use association methods to determine closest
measurement to predicted position
▫ Circle detection and center estimate associated with predicted position
▫ Use of binary edge detected image instead of full grayscale image
Matched Filter Results:
Original Frame
Template Edge Detected

- Matched filter provides

reasonable results when cell
shape does not change

- Matched filter not robust enough

to handle subtle changes in cell
shape/orientation

- Errors accumulate over time

because measurements become
correlated
The Models:
Recall: Cell motion is not predictable and precise state-space models do not exist

Potential State-Space Models

Linear Motion Random Walk Spring Forces

σw = 8 pix Modeled as nonlinear process

σw = 25 pix σw = 8 pix

Linear Kalman Filter Linear Kalman Filter Extended Kalman Filter

σR = 10 pix ~ cell size Cell states initialized to true position, zero velocity
The Interacting Multiple Model (IMM) Filter:
• The IMM filter combines state estimates from individual models
into a single estimate
▫ Each model represented as a “state” in Markov chain
▫ IMM filter weights predictions based on probability that any
given model is correct

PRL

Random Walk
PLR
Linear Motion
PRS

PSR
PLS

Spring Forces
PSL
The Results:
• A single cell trajectory was tracked during simulation
▫ First, ran simulation without IMM filter. Then, with IMM filter.

• Individual motion models had inconsistent performance

• Initial model probabilities equal (i.e. pj = 1/3, all j)

▫ Transition probabilities: P1j = [0.5, 0.25, 0.25]
▫ (i.e. equally likely to switch to other model or stay at current one)

• IMM Filter more accurately combines state estimates

– Less estimate ‘drift’ from true state
– Significant error due to individual models is mitigated by IMM filter

Truth data was used to generate individual and IMM filter results due to inconsistencies with the
measurement model. Analysis was conducted using measurements via image processing, but
results were inconsistent and sufficiently inaccurate.
 Cell Trajectory (with KF estimates):

In general, individual models tend to over-predict cell position. While predictions do not deviate significantly
from the true trajectory, there are noticeable errors which may make tracking more difficult.
RMS Errors:
Linear motion model is reasonably accurate with Random walk begins to drift as Spring model consistently
errors consistently within the RMS bounds motion becomes more linear ‘overshoots’ correct position
IMM Filter Model Probabilities:

- Relationship between model

probabilities is as expected

- Linear Motion model consistently

more likely than other two models
but there is good tradeoff between
models

- Model probabilities in IMM filter

are strongly coupled with a priori
knowledge of system
Potential Improvements
Kalman Filter
- Swimming motion model
- Incorporate environmental parameters (viscosity/density)
- Track cell size/shape
- Model cell interactions (attraction/repulsion, effects due to
growth/splits)
- Multiple Hypothesis Tracking

Monte Carlo Simulation

Parameters:
- Model switching probabilities
- Measurement noise variance
- Process noise variance
- Frame/Gate size

Measurement Model
- Shape/Rotation invariance
- Detect new cell formation
- Study cell properties to identify potential features
for extraction/tracking
Conclusions and Lessons Learned
• IMM filter provides effective analytical way to combine state-space model
predictions when true model is unknown

• Tuning Kalman filter to achieve optimal performance is challenging

▫ Inaccurate assessment of measurement/process models results in
significant tracking inaccuracies

• Intuitively, KF performance is constrained by measurement inaccuracies

▫ When measurement errors are combined with state-space model
uncertainties, tracking accuracy becomes degraded

• Identification of individual cells via image processing requires robust

methods due to similarities in cell size, shape, orientation, and pixel intensity

• Other variations of KF were implemented but were still not completely

accurate due to lack of understanding of cell motion model
References

1. R.G. Brown and P. Hwang, Introduction to Random Signals and Applied Kalman
Filtering, New York: Wiley, 1997.

2. W.D. Blair, “Derivation of Interacting Multiple Model Algorithm for Systems with
Markovian Switching Coefficients”.

3. J. Samsundar and A. Watkins, 525.745 Applied Kalman Filtering Lecture Notes,

2011.

4. P. Burlina, private communication, March 2011.