Virtual Cell Models: Predicting Perturbations in New Cells

Jonathan Alles

EVOBYTE Digital Biology

Introduction: Why Zero-Shot Perturbation Prediction Matters

Imagine choosing which gene knockdowns deserve your next experimental budget. An AI model predicts convincing expression profiles, but does it understand how each intervention changes the cell—or simply reproduce what that cell usually looks like?

That distinction is central to benchmarking virtual cell models, computational models that predict cellular behavior. For discovery teams, the useful question is not whether predictions resemble real cells. It is whether they capture the response to a specific intervention in a new biological context.

The 2026 Virtual Cell Challenge makes this question concrete. Its zero-shot task asks models to predict gene-knockdown responses in six cell lines without seeing measured perturbation responses from those contexts.

Here is how to turn that challenge into a practical evaluation framework before committing to larger experiments.

Define Held-Out Cellular Contexts Before Training

The challenge provides unperturbed expression profiles and target-gene identifiers, while withholding the measured responses. The interventions use CRISPR interference, or CRISPRi, which reduces a target gene’s activity. Three cell lines support validation; three others are reserved for final testing.

Importantly, zero-shot does not mean training-free. Participants can train on external data. It means predicting responses without response examples from the target cellular context.

For an internal benchmark, write down exactly what “unseen” means. Holding out a cell line tests something different from holding out a donor within a familiar cell type. Neither automatically establishes performance in diseased primary tissue.

Then split at the level of the intended claim. To evaluate transfer into an unseen cell type, keep its perturbed cells out of training and model selection. Randomly splitting individual cells from the same experiments does not test that claim.

For example, imagine training on gene knockdowns in several cancer cell lines and predicting responses in a held-out neuronal context. Providing neuronal controls can be legitimate, provided that access matches the intended deployment.

Also separate unseen contexts from unseen targets. Predicting a familiar knockdown in a new context is different from predicting a new knockdown in a new context. Report these tasks separately.

Make Simple Baselines Earn Their Place

Before comparing sophisticated architectures, establish what simple predictions achieve.

Start with a no-change baseline that predicts the target context’s control profile for every knockdown. This tests whether the evaluation rewards reproducing the basal state rather than learning intervention effects.

Next, try a generic response baseline: add the average expression change from training perturbations to the target control profile. Because this predictor ignores target identity, it helps reveal whether a shared response dominates the score.

Where the same target appears in training contexts, transfer its average effect into the held-out context. This provides a stronger comparison: does the model improve on simply reusing a known response?

For targets without measured training effects, consider a regularized linear predictor using available gene features. Give it the same permitted inputs as the larger model.

Benchmarking research has found that simple baselines can match complex models in some settings. However, more recent work shows that metric choice and calibration can reveal advantages that conventional scores miss. The lesson is not that deep learning cannot work. It is that a credible improvement needs a meaningful comparison and an evaluation sensitive to biological signal.

Check Data Leakage Across the Entire Workflow

A clean fine-tuning split is not enough when a model arrives pretrained.

Request a data provenance record covering pretraining and downstream adaptation. Search for duplicate experiments and republished datasets under different accessions. In particular, check whether perturbation measurements from the supposedly unseen context entered any training stage.

Unperturbed data from the target context may be allowed. Perturbed data from that context would undermine the intended zero-shot claim. State this boundary explicitly rather than treating all prior exposure as equivalent.

Preprocessing deserves the same scrutiny. Fit learned transformations on permitted data, and do not use held-out responses to select input genes or tune filtering thresholds. If an evaluator selects responsive genes from ground truth, keep that information isolated from model development.

Likewise, use development contexts for tuning and freeze the pipeline before opening the final test. Repeated leaderboard feedback can otherwise become another training signal.

There is also a subtler statistical issue. A 2026 preprint shows that subtracting the same noisy control estimate from predicted and observed profiles can inflate response correlations. As a diagnostic, repeat evaluation with independent control subsets where feasible. Treat this as a sensitivity analysis, not a replacement for the official challenge protocol.

Measure Perturbation-Specific Responses, Not Just Expression

Whole-profile error remains a useful sanity check, but it should not carry the decision alone. A model can reconstruct familiar expression patterns while missing the changes that matter.

Start with pseudobulk profiles: average expression within each perturbation and context, preserving biological replicates where available. Then calculate the response, or delta, by subtracting the matched control profile.

Compare predicted and observed deltas using correlation, alongside an error measure that penalizes incorrect magnitude. Correlation alone can reward a response with the right shape but the wrong strength. Predefine how undefined scores, such as correlations involving constant predictions, will be handled.

Next, ask whether predictions distinguish interventions. The Perturbation Discrimination Score, or PDS, tests whether a predicted response is closer to its matching measured response than to responses from other perturbations. A generic prediction should not pass as target-specific understanding.

Also evaluate recovery of differentially expressed genes, or DEGs: genes whose expression changes relative to controls. Examine whether the model identifies responsive genes and predicts their direction correctly, using thresholds fixed before final testing.

For a discovery-focused diagnostic, repeat key analyses without the directly targeted gene. Correctly predicting its reduction is useful, but does not establish that the model understands downstream biology.

The challenge uses an aggregate of six metrics. Treat the measures above as a practical evaluation framework, not a substitute definition of that official score.

Finally, summarize performance by context and response strength. Use uncertainty intervals based on independent experimental units where possible, rather than treating every cell as an independent replicate. If single-cell variability matters to the application, add distributional checks: a correct average can still hide an incorrect mixture of responding and nonresponding cells.