Where the Model Spoke First: Reconstructing the 2015 Initiative's Statistical Analysis Plan

A Document Nobody Presented

At the closing plenary of the 2015 initiative, the slides moved quickly. Forest plots appeared, confidence intervals fanned out, and a moderator summarized what the consortium had learned about cycle-level predictors of live birth. The audience took notes on the findings. Almost no one took notes on the sentence buried in the acknowledgments: that all estimates followed a prespecified analysis plan, version 3.2, dated that spring.

A decade later, that sentence reads less like a formality than a confession. The analysis plan—known internally as the SAP—is the document that determined which questions the consortium could answer, which it could not, and which answers it could present without qualification. It is also, according to the archive's own holdings, the hardest of the initiative's core documents to locate in complete form. What survives is fragmentary: a methods memo here, a tracked-changes attachment there, a version-history table with three rows redacted.

This review reconstructs what can be reconstructed, and considers why the methods core carried authority that the plenary stage never fully acknowledged.

Outcome Definitions as Editorial Acts

The first substantive decision in the SAP was deceptively technical: what would count as the primary outcome. Draft language from early 2015 described "clinical pregnancy confirmed by ultrasound." By the version circulated in April, the primary outcome had shifted to live birth, with clinical pregnancy demoted to a secondary endpoint.

That change was not merely a statistical preference. It narrowed the denominator of success and reframed the initiative's central question—from whether treatment produced a pregnancy to whether it produced a child. One biostatistician involved in the revision recalled the reasoning in plain terms: the consortium's clinical partners had argued that pregnancy alone was an intermediate endpoint of limited use to patients, and the funding language emphasized live birth as the outcome of interest. The SAP absorbed that pressure and translated it into a definition.

Secondary outcomes were defined with similar care and similar consequence. Multiple gestation was prespecified as a safety endpoint rather than a success measure, a decision that shaped how the initiative's later publications described embryo transfer practices. The archive's copies of the outcome definitions carry handwritten annotations—arrows, question marks—suggesting the language was argued over rather than simply adopted.

The Exclusion Rules That Never Reached the Podium

Less visible still were the exclusion criteria. The SAP's cycle-level rules disqualified cycles with incomplete outcome ascertainment, cycles initiated outside the enrollment window, and—contentiously—cycles in which the participating woman had previously contributed a cycle to the same analysis. The last rule drew objections from at least two site investigators, according to correspondence in the archive, on the grounds that it discarded real-world data from women who returned for repeat treatment.

The methods core prevailed. Their position, preserved in a memo dated March 2015, was that repeat contributors introduced dependence structures the planned models could not accommodate without additional assumptions. The memo is three paragraphs long. It is also, functionally, a statement about which patients the initiative's findings would describe.

Readers of the eventual publications would have to infer these rules from a table footnote. The footnote did not mention the debate.

Late-Stage Model Changes and Their Residue

The most consequential revisions arrived late. The SAP's original modeling approach specified logistic regression with site as a fixed effect. By version 3.2, the plan called for generalized estimating equations with an exchangeable working correlation, clustered by site, plus adjustment for maternal age and a small set of covariates prespecified in an appendix.

Why the change? The recollections differ. One statistician described it as a response to observed clustering in the pilot data; another suggested the fixed-effect approach had produced unstable estimates in sensitivity analyses conducted that winter. Both accounts are consistent with a simple reality: the model changed after data had been seen, and the change was documented as a prespecified amendment rather than as a deviation.

This is the SAP's quiet lesson. The distinction between prespecification and post hoc revision depends entirely on whether the revision is dated, justified, and retained. In 2015, that standard was honored more in spirit than in archiving. The version history survives; the sensitivity analyses that motivated the change do not.

Reproducibility Ten Years On

What would a researcher attempting to reproduce the 2015 estimates need today? The cleaned analytic dataset, the exact model specification, the software and version, and the exclusion logic. Of these, the archive holds only the specification and a partial exclusion table. The dataset was never deposited in a public repository—not unusual for 2015, and not excusable by 2026 standards, when federal data management and sharing requirements have made such deposits routine for NIH-supported work.

This is not a story of misconduct. It is a story of ordinary documentary drift, compounded by a field that treated methods documents as internal instruments rather than public records. The initiative's investigators were not asked to publish their SAP, and they did not.

The Hidden Authority of the Methods Core

The SAP's power lay in its invisibility. Plenary audiences evaluate findings; they rarely audit the machinery that produced them. Yet the machinery decided which cycles entered the analysis, which outcomes carried the weight of the conclusion, and which models were presented as the default.

For contemporary reproductive-health research, the lesson is not that prespecification guarantees truth. It is that prespecification without transparency merely relocates judgment from the results section to a document nobody reads. The 2015 initiative's methods core did thoughtful work under real constraints. A decade later, the archive's obligation is to make that work visible—and to note, plainly, how much of it we are still guessing at.