Bench Notes and Cold Trails: The 2015 Protocol Five Labs Couldn't Let Go
A Protocol With Five Owners
The HumanRep 2015 Archive's most granular holdings are not the plenary recordings or the published supplement. They are the working documents: bench notebooks, protocol revision sheets, and a chain of emails between five laboratories that attempted the same experiment under the initiative's coordinated replication program.
The protocol in question—a gamete culture condition assay presented at the 2015 annual meeting as preliminary but promising—was circulated to four partner laboratories for independent verification before any submission for publication. What the archive reveals is a replication effort that came within weeks of total collapse.
Four Failures, Poorly Documented
The first two laboratories, one in the Midwest and one on the West Coast, reported inability to reproduce the effect within three months. Their correspondence, preserved in the archive, is strikingly terse. "Conditions matched per your sheet," one investigator wrote. "Result does not. Advise."
A third lab obtained a partial signal that vanished on repeat. The fourth never completed the run, citing equipment downtime—an excuse the initiative's coordinating committee flagged in its internal minutes as "unverified but plausible."
Critically, none of these failures were written up. In 2015, absent any mandate to report negative replications, the failed attempts existed only as emails and margin notes. Had the fifth laboratory not succeeded, the finding would likely have been published anyway, on the strength of the originating data alone, with no record of the four labs that could not confirm it.
What the Fifth Lab Did Differently
The successful replication, documented in extraordinary detail in a notebook now digitized in the archive, diverged from its predecessors in one respect: rather than following the written protocol, the investigator telephoned the originating lab and spent forty minutes reconstructing the unwritten steps—incubator loading patterns, media handling tempo, timing of observation windows. The notebook records these verbal addenda in the margins, and they do not appear in any formal protocol document held by the initiative.
This is the central lesson the archive draws from the file: the experiment's reproducibility depended on tacit knowledge that the written record never captured. The 2015 initiative's replication culture, in other words, worked through telephone calls, not documentation—a fragile mechanism that succeeded here by luck as much as design.
The Asymmetry of the Published Record
When the confirmation finally appeared in the literature, it was published as a standalone confirmation by the fifth laboratory, with the four prior failures unmentioned. Readers of the resulting paper encountered a clean narrative: hypothesis, replication, validation. The archive's correspondence tells a messier and more honest story—one in which replication was the exception, dependent on personal relationships and undocumented detail.
A decade on, with preregistration and registered reports now standard expectations at major US journals, the file reads as an artifact of a transitional moment. It also serves as a caution for historians of science: the published corpus of the 2015 initiative systematically overrepresents success not because failures were hidden deliberately, but because the infrastructure to record them did not exist. The notebooks preserve what the journals could not.
