Scorecards and Silence: How Internal Rankings Shaped the 2015 Consortium's Published Record

In the administrative record of the 2015 human reproduction research initiative, the published papers are the visible artifact. Less visible—but no less consequential—is a set of internal dashboards that ranked participating sites on four operational measures. The HumanRep 2015 Archive holds partial runs of these metric reports, and together with interviews conducted with former site coordinators, they permit a reconstruction of how quantitative management operated inside the consortium.

The Anatomy of the Scorecard

The dashboard was not a single document but a recurring quarterly packet distributed to site principal investigators and program managers. Each site received a one-page summary with four columns: enrollment velocity, expressed as participants screened and randomized per month against a site-specific target; data completeness, defined as the percentage of case report form fields entered within fourteen days of a study visit; protocol deviations, counted per one hundred participant-months and subdivided into major and minor categories; and specimen turnaround, reported as the median hours from collection to receipt at the central processing laboratory.

A composite index combined these measures with fixed weights: enrollment accounted for 35 percent, data completeness 25 percent, deviations 20 percent, and specimen turnaround 20 percent. Sites were then assigned a color code—green, amber, or red—and ranked within the consortium. The mathematics were simple. The social consequences were not.

From Spreadsheet to Steering Committee

The rankings traveled quickly beyond the program office. Steering committee minutes from the period show that composite scores informed decisions about supplemental coordinator funding, allocation of freezer space, and priority access to centralized statistical support. Sites in the red tier received mandatory site visits and closer monitoring. Sites in the green tier were more likely to be invited to lead secondary analyses and to receive early access to pooled data.

Former coordinators describe the quarterly release of the dashboard as a morale event. One coordinator at a large academic medical center recalled that the red list was posted in a shared drive with site names visible to all program staff. “It was not supposed to be punitive,” she said, “but it functioned that way. You cannot put a color next to a site and expect people not to read it as a judgment.” Another coordinator at a safety-net hospital noted that her site consistently ranked low on enrollment velocity despite meeting its revised target, because the target had been set using historical numbers from a different patient population. The dashboard did not measure difficulty; it measured speed.

The Publication Pathway

The metrics also shaped the evidence base in ways the public papers did not record. High-ranked sites were more likely to be selected as lead sites for ancillary studies, and their specimens moved through the central laboratory with shorter turnaround times. Low-ranked sites were more likely to receive data queries and, in at least one flagship analysis, were excluded from the primary cohort because their data completeness fell below a prespecified threshold.

The exclusion was disclosed in a supplementary appendix but not discussed in the main text. Internal correspondence shows that the two excluded sites had enrolled a substantial proportion of participants with recurrent pregnancy loss—a population that was central to the study question. Their removal changed the distribution of prior pregnancy history in the analytic sample. The published conclusion did not mention this shift. The dashboard had not been designed to select sites for scientific reasons, yet it became a de facto eligibility filter for publication.

What the Metrics Could Not See

The scorecard contained no measure of case mix, institutional review board delays, staffing turnover, or patient travel distance. A site whose catchment area spanned three rural counties could post a high no-show rate for reasons unrelated to coordinator effort, yet its dashboard row looked identical to that of a site with poor scheduling practices. Interviews suggest that some coordinators began to prioritize data entry for fields that counted toward completeness, not because they intended to distort the record but because the metric rewarded that behavior. The dashboard was a management tool, not an epidemiologic instrument, but it acquired the authority of a quality ranking.

Reading the Archive in 2026

Contemporary clinical trials now treat dashboards as routine infrastructure, and the 2015 files offer a historical case study in metric-driven governance. The archive’s purpose is not to indict individual managers but to document how rankings are never neutral: they allocate attention, resources, and credibility. The consortium’s public papers do not mention the scorecard. The internal files show that the scorecard helped produce those papers—and sometimes determined which data were eligible to appear in them. For researchers and clinicians, the lesson is to ask not only what a trial measured but how its sites were measured. The HumanRep 2015 Archive welcomes inquiries from former personnel; anonymized metric reports are available to qualified researchers.