I think there are two distinct aspects of this question: how to visually "index into" the domain of possible input graphs, and how to visualize a model's predictive uncertainty for a single input graph.

For predictions of a scalar property of a single molecule (e.g. its "affinity" or "overall goodness score"), a complete representation would be the whole predictive pdf (aka a histogram of samples from the posterior predictive distribution). That can be summarized by an interval or scalar measuring its "spread" (stddev, quantiles, ...). Depending on the shapes of the predictive distributions, these summaries may be more or less lossy.

For predictions of more than one property simultaneously of a given molecule ("solubility", "on-target affinity", "off-target affinity", "toxicity", ...), the predictions of the various quantities will probably be correlated, which will make summarization even harder. A complete representation would be the joint distribution for all these predictions, which can be lossily visualized in the usual ways (reduce jointly to 2D, show all bivariate marginals, ...). Although it will be too big to look at for more than a handful of molecules, I imagine it will be informative to take a look at the joint predictive distribution for all properties of a molecule or two (maybe for one molecule that looks very similar to something in training dataset, and one that looks very different), using the same model but different approximate inference algorithms.

I think @yuanqing-wang asks about how to build an axis over graphs here, not the output variable. That discussion, should probably be in the metrics issue #4 .
I agree with you that the title might also point to discussions about how to visualize output, however the description of the issue makes me think otherwise.

I don't have any special insight into how to visually "index into" the domain of input graphs, to get a more global picture of what the model's doing. Associating each graph with a point in 2D using the approaches @karalets describes here and on the new issue sounds good to me. Each 2D point would further be associated with a scalar (@karalets suggests LLK, and perhaps other scalar summaries of posterior predictive distribution would be appropriate), to form a colored scatterplot or heatmap maybe hinting at "where in chemical space the model is confident or not."

To give some more color:

First, the metrics we care about predicting should be anything we decide to have in #4 , I mention LLK here as one example. I am sad nobody is interacting with #4 as this is a very important issue that @yuanqing-wang brought up as a blocker for reproduction when he initially looked at the Cambridge paper and we have not yet made overview plots like the ones they have there.

The 2d or whatever-d plot would point to such metrics, whichever one chooses.

visualization of uncertainty #24

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions