What a private antibody pilot taught us about reproducibility
Conflicting runs showed why a partner-derived case cannot become a publication result until seed, reference, commands, hashes, and validator outputs are frozen together.
The first result looked cleaner than the record behind it
A private antibody pilot compared two deterministic design paths in a plant-expression context. The headline metrics appeared easy to summarize: translation was preserved, one path produced higher CAI, and one candidate reached the configured global GC band.
The surrounding record was not yet publication-ready. Results from different runs did not match, the sampled profile path had no frozen random seed, and some checks described as complete were still pending. The command transcript, input and output hashes, codon-reference authority, and unified validator report were not stored as one evidence package.
A plausible explanation is not a replay
Stochastic profile sampling plausibly explained part of the variation. Recording that explanation did not make the run reproducible. A publication result needs the actual seeded rerun and its artifacts.
The pilot also exposed ambiguous status language. A feasibility flag indicated that a candidate existed inside a configured range; it did not prove that every advisory scanner, assembly review, or optimization target had passed.
The promotion gate
The case remains useful as a private fixture and a public-redacted lesson. Before its numbers can support a comparative result, the evaluation should freeze:
- input identity without publishing private sequence material;
- software tag and commit;
- engine, profile, seed, and complete command;
- codon-reference and CAI authority;
- output hashes and candidate files;
- one generator-independent validation report; and
- publication permission and public projection rules.
The pilot did not fail. It revealed the difference between completing a computation and completing an evidence package.