Technical report · TriboGuard
A treatment that halves a cell population may have killed half the cells or stopped half of them dividing. From averages alone, those are not merely hard to separate — they are provably identical.
Cell-viability assays report one number per well. This work shows that number cannot, even in principle, distinguish cell killing from growth inhibition; identifies the quantity that can; and builds a system that refuses to name a mechanism when the evidence does not support one.
The mean of a birth–death process depends on birth minus death. Two treatments with the same net rate — one killing, one stalling — produce identical average trajectories at every timepoint. The variance between replicate wells depends on birth plus death, so the mechanism lives in the spread between wells, which a bulk assay destroys by averaging.
Four consequences follow, each measured rather than argued. Extra timepoints buy almost nothing. Three wells per condition — the shape of most published dose–response work — pin the birth-to-death split only to within a factor of 146. An endpoint assay such as MTS costs four times the plate of live imaging for identical information. And a conformal guarantee calibrated on one experimental design collapses when applied to another.
A fifth result comes from the other end of the pipeline. Imaging is only cheaper than an endpoint assay if a segmenter can see the cells, and measured against the hand-drawn outlines of 34,129 cells in four cell lines, it cannot see the damaged ones. At moderate damage Cellpose finds 85–97% of untouched cells and at most 5% of damaged ones in the same images. The test was pre-registered before three of the four lines were measured, and it replicated on all three. The same blindness biases a morphology readout downward and a count readout upward.
And the same reasoning applied to the study's selectivity claim finds that the contrast it reports needs no selective compound to produce it. A compound that kills nothing and only slows division reproduces the reported effect in the dividing cancer arm, and its absence in the non-dividing normal arm, exactly.
A viability assay measures a population property: total metabolic activity, total ATP, total absorbance. A 40% fall in that signal is consistent with 40% of cells dying, with every cell surviving at 60% of its former metabolic rate, and with any mixture of the two. The reading is the same in all three cases.
This matters because the follow-up experiments diverge. A compound that kills is a candidate cytotoxic agent; a compound that stalls division is a candidate cytostatic one, and the two are developed, dosed and combined differently. A paper that reports the fall and names the mechanism has made a claim its measurements do not carry.
Model a well as a linear birth–death process: each cell divides at rate b and dies at rate d, per hour. Starting from N0 cells, the population at time t has
The mean depends on b − d and on nothing else. So for any treatment, another treatment with a different mechanism and the same net rate produces an identical mean curve — exactly equal in real arithmetic, and equal to within 1×10−12 in floating point. No schedule of extra timepoints helps, because every timepoint measures the same degenerate combination.
| Condition | birth | death | net | turnover | mean at 48 h | variance at 48 h |
|---|---|---|---|---|---|---|
| Control | 0.055 | 0.005 | 0.050 | 0.060 | — | — |
| Killing (death raised) | 0.055 | 0.025 | 0.030 | 0.080 | 4220.7 | 36 249 |
| Stalling (birth slowed) | 0.035 | 0.005 | 0.030 | 0.040 | 4220.7 | 18 125 |
A sample variance from r wells has r − 1 degrees of freedom. Three wells buy two. This is why replicates are not a statistical nicety in this problem: they are the only channel through which the mechanism is visible at all.
The system has four stages, each answering a question the previous one raises.
Simulation is exact rather than Gillespie: a linear birth–death process from n ancestors is a sum of n independent single-ancestor processes with a closed-form law, so a well of 105 cells costs one draw per ancestor per interval instead of one per division event.
Over 30 independent datasets at 200 wells, the estimator recovers a true turnover of 0.0600 at 0.0605. It is noisy, not wrong. All the difficulty is in reporting how noisy.
| Design | wells | timepoints | interval width |
|---|---|---|---|
| Sparse schedule, few wells | 3 | 3 | 1.999 |
| Dense schedule, few wells | 3 | 9 | 1.979 |
| Sparse schedule, more wells | 24 | 3 | 0.082 |
Because the interval comes from a chi-squared law, its width depends on the well count alone. That makes the question “how many wells do I need” answerable in closed form, and answerable for a study that never reported its replicate count.
| Wells per condition | Split known to within | Relative interval width |
|---|---|---|
| 3 — typical published design | ×145.7 | 39.23 |
| 12 | ×5.7 | 2.38 |
| 24 | ×3.3 | 1.36 |
| 48 | ×2.3 | 0.88 |
| 66 | ×2.0 | 0.71 |
MTS produces its colour change by lysing cells, so a well cannot be read twice and every timepoint needs its own wells. For a three-well, four-timepoint design that is 12 wells consumed against 3 for live imaging — a cost of 13.80 against 4.80 for identical information.
A convenient property falls out of the estimator: it reads a mean and a variance at each timepoint and never follows one well through time. So a destructive assay costs money here but not information, and switching to imaging is a pure saving rather than a trade.
That holds for the well count. It does not hold unconditionally for what imaging then measures, and §7 puts a number on the gap.
Conformal prediction guarantees coverage when calibration and test data are exchangeable. A three-well experiment is not exchangeable with a thirty-well one, and calibrating on the second while predicting on the first fails in the dangerous direction.
| Wells | Coverage, pooled | Confidently wrong, pooled | Coverage, stratified | Confidently wrong, stratified |
|---|---|---|---|---|
| 3 | 0.53 | 0.45 | 0.97 | 0.00 |
| 20 | 0.93 | 0.07 | 0.95 | 0.05 |
| 60 | 0.97 | 0.03 | 0.95 | 0.05 |
Each cell below calibrates on its own design and is evaluated on 80 held-out experiments of that design. A cell is overconfident when coverage breaks, or when the answers the system committed to are wrong more often than the nominal level allows.
| effect size | |||||
|---|---|---|---|---|---|
| wells | 0.10 | 0.25 | 0.40 | 0.70 | 1.20 |
| 3 | no contrast | abstains | abstains | overconfident | overconfident |
| 6 | no contrast | overconfident | overconfident | overconfident | abstains |
| 12 | no contrast | overconfident | overconfident | reliable | abstains |
| 24 | no contrast | overconfident | abstains | overconfident | reliable |
| 48 | no contrast | abstains | reliable | reliable | reliable |
Marginal coverage is not conditional coverage. At six wells and effect 0.25 the system holds 0.91 coverage overall — the guarantee is intact — while 75% of the verdicts it committed to are wrong. The wide sets carry the average for the narrow ones. Split conformal promises the first kind of coverage and not the second, and the gap is a real limitation of the method rather than a bug in this implementation.
Five cells in the map have only one true mechanism — "no effect" — so they cannot test separation and the classifier sets them aside. They test something else, which that label nearly hid: whether the system invents a mechanism from nothing.
| Wells | Commits on | Wrong when it commits | Spurious mechanism claims |
|---|---|---|---|
| 3 | 0.00 | — | 0.0% |
| 6 | 0.00 | — | 0.0% |
| 12 | 0.01 | 1.00 | 1.2% |
| 24 | 0.06 | 1.00 | 6.2% |
| 48 | 0.16 | 0.69 | 11.2% |
The system was applied, frozen, to the study that motivated it: a 2022 report on Tribonema aqueous extract against RAW264.7 cells, measured by MTS. The paper is the case study, not the training data — it contains a few dozen numbers, far too few to fit or calibrate anything, and they are the wrong kind. No rates are estimated from it.
What is assessed is whether the design could carry each conclusion drawn from it. Claims declare what they require, the design declares what it provides, and support is set inclusion.
| Conclusion | Verdict | Blocking requirement |
|---|---|---|
| The extract affected the cells | supported | — |
| Comparable in size to doxorubicin | supported | — |
| About 40% of the cells died | not established | cell counts |
| The surviving 60% were resistant | not established | cell counts, recovery test |
| Killing rather than growth inhibition | not established | cell counts, replicate variance |
| A specific death mechanism | not established | death markers |
| Selective for cancer over normal cells | not established | matched normal population — and see §8 |
Two absences are findings in their own right. The report states no replicate count anywhere, and its methods section describes 48-hour incubation where every results section describes 24 hours. Neither is resolved here; choosing one would be inventing data.
At an assumed three wells — the most favourable reading — the birth-to-death split would be pinned only to within a factor of about 146, and narrowing that to a factor of two would take roughly 66 wells per condition.
None of this is evidence against the extract having an effect. The reported effect stands. It is a statement about which further conclusions the measurements can carry.
§5.4 recommends imaging over an endpoint assay, on the grounds that it costs a quarter of the plate for identical information. That recommendation needs a qualification, and the qualification is measurable.
Segmenters are trained on healthy cultures, because that is what annotated datasets contain. If they find damaged cells less reliably than healthy ones, then a measurement built on segmentation is blind in exactly the region where the biology is happening. That is testable without a laboratory: LIVECell ships a hand-drawn outline for every cell, so damage can be applied to known cells in known amounts and the detection rate read off against the truth.
The design is paired inside each image. Every annotated cell is assigned to damaged or untouched by coin flip, both arms sit in the same frame, and so no difference between them can be illumination, focus or confluence. Four cell lines, twenty images each, 34,129 hand-outlined cells, every one scored at five damage severities by three segmenters: the three-class network trained here, a binary U-Net whose mask is split into cells by watershed, and Cellpose. A cell counts as found when one predicted object covers at least half of it.
This result was pre-registered. The hypotheses, the primary
endpoint and every pass/fail rule were fixed in
preregistration/damage_blindness_v1.json before any result existed for
MCF7, SHSY5Y or SkBr3. Each run recorded the plan’s SHA-256 as it started, and
the evaluation refuses results scored under any other version, so the rules could
not move after the data arrived. A172 is exploratory only, because an earlier and
flawed version of this experiment had already been run on it (§9). Cellpose is
the primary segmenter because it is the only one this project did not train. The
unit of analysis is the acquisition group: the gap is computed per image, and its
interval resamples whole groups, because cells in one frame are not independent
trials.
| Cell line | Role | Intact found | Damaged found | Gap [95% interval] | No-damage control | Result |
|---|---|---|---|---|---|---|
| MCF7 | confirmatory | 0.85 | 0.00 | +0.848 [+0.801, +0.898] | [−0.018, +0.004] | pass |
| SHSY5Y | confirmatory | 0.86 | 0.00 | +0.855 [+0.836, +0.874] | [−0.032, +0.003] | pass |
| SkBr3 | confirmatory | 0.97 | 0.05 | +0.913 [+0.865, +0.952] | [−0.010, +0.004] | pass |
| A172 | exploratory | 0.94 | 0.00 | +0.939 [+0.922, +0.957] | [−0.003, +0.034] | pass |
One pattern was not pre-registered and is reported as exploratory. At moderate damage the three segmenters part company, and not in the order their general reputations would suggest.
| Damaged cells found, severity 0.4 | A172 | MCF7 | SHSY5Y | SkBr3 |
|---|---|---|---|---|
| three-class | 0.34 | 0.68 | 0.69 | 0.83 |
| U-Net + watershed | 0.42 | 0.80 | 0.63 | 0.86 |
| Cellpose | 0.10 | 0.05 | 0.07 | 0.56 |
The consequence is not one number, and getting this wrong was the most instructive mistake in the project. A morphology readout — averaging a shape feature over segmented cells, which is what this project’s treatment pipeline does — is diluted by blindness, because the cells carrying the signal are dropped from the average. A count readout is inflated by the same blindness, because a dying cell that cannot be seen is indistinguishable from one that has gone.
| Severity 0.6 | Cellpose: morphology shift recovered | Cellpose: kill a count reports | Three-class: morphology recovered | Three-class: kill a count reports |
|---|---|---|---|---|
| A172 | 0.01 | 0.50 | 0.17 | 0.45 |
| MCF7 | 0.00 | 0.50 | 0.39 | 0.37 |
| SHSY5Y | 0.01 | 0.50 | 0.48 | 0.34 |
| SkBr3 | 0.10 | 0.47 | 0.47 | 0.34 |
Blindness need not end the analysis; it can bound it. Write a treated well relative to its control as cells genuinely absent (a fraction g), cells damaged but present (k), and the rest intact. If intact cells are found at rate h and damaged ones at d, the counted ratio is
One observation and two unknowns, so the absent fraction is an interval. It closes to a single value when the segmenter sees damaged cells as well as healthy ones, and opens to the whole range when it sees none of them. The sweep measures b, so the interval can be computed — and the system abstains when it straddles the effect threshold. Note what g is not: separating death from slowed division inside it is still the problem of §3.
| Segmenter | Severity | Blindness measured | Absent, given a 40% count drop | Verdict |
|---|---|---|---|---|
| three-class | 0.2 | 0.00–0.07 | 35–40% | resolved |
| three-class | 0.4 | 0.12–0.71 | 0–40% | abstain |
| three-class | 0.6 | 0.60–0.94 | 0–40% | abstain |
| U-Net + watershed | 0.2 | 0.01–0.11 | 32–40% | resolved |
| U-Net + watershed | 0.4 | 0.08–0.54 | 0–40% | abstain |
| U-Net + watershed | 0.6 | 0.34–0.87 | 0–40% | abstain |
| Cellpose | 0.2 | 0.00–0.18 | 27–40% | absent, size unresolved |
| Cellpose | 0.4 | 0.34–1.00 | 0–40% | abstain |
| Cellpose | 0.6 | 0.89–1.00 | 0–40% | abstain |
Synthetic damage is not real damage. What is measured is how a segmenter responds to a controlled loss of contrast, edge definition and interior texture. Whether dying cells in a real plate degrade along those axes is what a real experiment settles — and the source study’s own morphology figure is where to check, since its pale, vacuolated, arrow-marked cells were the ones a segmenter failed to outline.
§6 recorded the study's selectivity claim as unsupported for want of a matched normal population. That is true and it is also the weakest available reading, because it describes something the authors did not do. The stronger question is whether the comparison they did run could have shown selectivity at all.
The two arms were a dividing cancer line and primary peritoneal lymphocytes, which in short-term culture largely do not divide. And “% cytotoxicity” is not measured against the starting population: it is measured against an untreated control read at the same time, which has been dividing all day in one plate and sitting still in the other.
That asymmetry is enough by itself. A compound that stops division and kills nothing removes cells the cancer control would have gained, and is invisible in cells that were never going to divide. The result is a large number in one arm, none in the other, and no selective action anywhere.
The threshold needs no assumptions. Inverting the question removes every unknown: a compound that kills nothing reproduces the reported 40% in 24 hours in any control whose cells divide at least 0.0213 times per hour — one division per cell every 47 hours on average. No replicate count, no assumed growth rate, no raw data — which matters, because the report supplies none of them.
That threshold is a division-event rate, not a growth rate, and the distinction is not pedantry: the two coincide only when nothing dies. A population doubling only every 50 hours while dividing and dying briskly clears this bar, and an earlier draft of this section quoted a doubling time that would have excluded it.
| Arm | Control behaviour | Apparent cytotoxicity | Cells actually killed |
|---|---|---|---|
| RAW264.7 | divides, ~12 h doubling | 40.0% | 0.0% |
| Peritoneal lymphocytes | static | 0.0% | 0.0% |
Holding the compound fixed and varying only how fast the normal arm divides shows what the reported gap is actually made of.
| Normal arm | Apparent in cancer | Apparent in normal | Apparent gap |
|---|---|---|---|
| never divides (as run) | 40.0% | 0.0% | +40.0% |
| doubles every 96 h | 40.0% | 9.2% | +30.8% |
| doubles every 48 h | 40.0% | 14.2% | +25.8% |
| doubles every 24 h | 40.0% | 23.5% | +16.5% |
| doubles every 12 h (matched) | 40.0% | 39.0% | +1.0% |
The same inversion applied to the cancer arm alone reproduces §3 on the study's own figure. A 40% fall at 24 hours is consistent with 40% of the cells genuinely killed at one end and none killed at all at the other, with every mixture in between; the reported number selects no point on that range.
And the study's time course does not come from one unchanging treatment. Read as a constant-rate effect, its three points imply net-rate changes that differ four-fold — 10% at 3 h, 40% at 6 h, and still 40% at 24 h cannot all describe the same steady process. This is a second internal inconsistency, independent of the 24-versus-48-hour contradiction §6 records.
None of this says the extract is not selective. It may well be. The claim here is only that this experiment cannot separate a selective compound from one that slows division, which is the same claim the rest of the report makes about mechanism. One further route is not modelled at all: MTS absorbance is treated throughout as proportional to viable cell number, so a compound that lowers metabolism per cell without touching one produces the identical fall. And the normal arm's viability was reported as having more than doubled — a signal that rose, which needs proliferation or raised metabolism to explain and cannot be read as safety without distinguishing them.
Eleven errors were found and corrected during development. Nine were caught by a measurement; the two in the damage experiment were caught by an outside review, and then confirmed by measuring them. They are listed because a system whose purpose is preventing overconfident claims should show its own.
| Error | Consequence | What caught it |
|---|---|---|
| Turnover interval by bootstrap | 100% of three-well resamples collapsed onto the variance floor; the interval came out narrower at 3 wells (0.005) than at 24 (0.030) — maximum confidence at minimum evidence | Comparing interval widths across well counts |
| Guard rejecting a negative denominator | Refused to simulate pure killing, the case of most interest | Its own unit test |
| Calibration pooled across designs | 0.53 coverage against a 0.90 target; 45% of verdicts confidently wrong | Evaluating per design instead of in aggregate |
| Closed form for minimum calibration size | 0.9/(1−0.9) evaluates to 9.000000000000002, so the advertised minimum was one experiment too high at every round confidence |
A parametrised test at 0.8 and 0.9 |
| Fixed 5% cutoff for “overconfident” | At 90% target coverage 10% of answers are expected to be wrong, so the rule demanded better than the guarantee promises and flagged 10 of 25 cells including the easiest | Noticing the easiest cells were red |
| Degenerate effect grid | Two whole columns contained only one true mechanism, so they tested nothing about separation while still being scored | Counting distinct truths per cell |
| “Factor” used for relative width | Understated the gap nearly fourfold (39 quoted where the ratio is 146) and priced “a factor of two” at 39 wells when it is 66 | Computing both quantities before writing them down |
| Detection bias modelled for the wrong readout | §7 first claimed blindness makes a treatment look weaker. The arithmetic underneath was a count readout, where blindness makes it look stronger: at severity 0.6 it manufactures a 50% kill out of nothing | A test asserting the two readouts move in opposite directions |
| Partial correlation on a fully explained series | When cell size accounted for the shape index exactly, the residuals were constant and the correlation undefined — the one case the control was built to detect. It surfaced as a warning from inside SciPy rather than as the answer “nothing survives” | Running the suite in the pinned environment |
| Damage leaking into the control arm | The blur applied to a damaged cell reached across into its neighbours. On confluent images 89% of the supposedly untouched control cells had pixels altered — in an experiment whose whole claim rests on that arm being untouched. A test had passed throughout, because its two synthetic cells sat 42 px apart | Outside review; confirmed by counting altered pixels in every control cell |
| Cells pooled as independent trials | Thousands of cells from twenty frames were summarised as one rate, overstating precision — inconsistent with this project’s own benchmark, which already resampled acquisition groups | Outside review |
python scripts/reproduce.py --list # every stage, in dependency order python scripts/reproduce.py --fast # everything except the damage sweep python scripts/reproduce.py # everything, including the sweep python -m pytest # the test suite
Every figure in this report is produced by one of those commands and written to a JSON artifact recording the code revision and package versions that made it. The interactive version of the design analysis runs the same closed-form arithmetic in the browser, with chi-squared quantiles tabulated rather than approximated.