A twelve-page scientific review, written from a 130-document archive
The kind of paper an expert spends weeks assembling: read everything, work out what the evidence supports, and cite it. We set RUNA a genuine research question — not a demonstration script — and then checked the result line by line.
Why this archive was a hard case
The material was written for specialists, in the language specialists use with each other. Conclusions sat inside figures and tables as often as in prose. Several papers appeared more than once under different file names, and a number of documents had never been intended to serve as evidence for anything.
In other words, an ordinary corporate archive rather than a curated data set — which is the only kind of test worth running.
How the result was checked
Every citation in the finished paper was compared against the record of what the system had actually retrieved, mechanically. The paper was then audited a second time, independently, for claims that no source supported.
Both checks looked for the same failure: a reference that sounds right and does not exist. Neither found one.
- It said what it could not support. Where the archive did not answer the question, the paper said so and identified the gaps in specific terms. A follow-up question aimed at one of them found the missing material and closed it.
- It flagged weak conclusions, not only missing ones. Inconsistencies between documents, and statements the evidence supported only thinly, were surfaced rather than smoothed over.
- It brought together what nobody had read together. The decisive comparison in the paper required three figures held in three separate documents.
- It read the graphics. Two of the numbers used appear only in a chart and in a table image, and nowhere in the text of their sources — the part of an archive most systems quietly skip.
- It found a defect in the filing itself. Nine duplicate documents, including one paper stored under a co-author's name, which had been reading as two independent sources agreeing with one another.