David Mamani

Article

The gap between the paper and the jobsite

I recently went deep into a sample of recent construction-automation papers — serious journals, brilliant ideas. My uncomfortable conclusion: most of them cannot be reproduced. Not because the science is wrong, but because they lack what on a jobsite would be unforgivable: the complete file.

Out of a sample of 15 strong papers I evaluated, the replicability chain degrades like this:

Papers reviewed 15 of 15 — the full sample 15 Full methodology 9 of 15 describe the method in enough detail to replicate 9 Public data 5 of 15 use datasets anyone can download 5 Public code 3 of 15 publish their code 3

One in five. That is the share of research an engineer can take, run, and verify. The rest is, in practice, an act of faith with peer review.

Why it hurts twice as much in construction

In software, an irreproducible paper is an annoyance. In construction it is a paradox: this is an industry where nobody would accept a structural calculation without the calculation report, yet we accept “30% improvements” without being able to audit the data or the model that produced them.

My minimum "technical file" checklist for an automation paper (click to expand)
  1. Data: a public dataset, or an equivalent synthetic one if the real one is confidential.
  2. Code: the full pipeline, not the pretty snippet.
  3. Environment: exact versions — in ML, changing one version changes results.
  4. Honest metrics: comparison against current practice, not against a strawman baseline.
  5. Adoption cost: how many engineer-hours it takes to productionize. Almost nobody reports this.

The optimistic part

The few papers that publish everything create an enormous multiplier: they get replicated, adapted to other codes, turned into real tools. Replicability is not an academic luxury — it is the difference between a PDF and a tool. And those of us who work between the data and the jobsite have a clear role: replicate, measure, and publish what we find. What isn’t measured can’t be improved; what can’t be reproduced was never truly measured.