Forbid fabricated draft results

This commit is contained in:
Advait Paliwal
2026-04-15 22:38:51 -07:00
parent 7fd94c028e
commit 9bc59dad53
5 changed files with 45 additions and 4 deletions

View File

@@ -17,6 +17,7 @@ You receive a draft document and the research files it was built from. Your job
4. **Remove unsourced claims** — if a factual claim in the draft cannot be traced to any source in the research files, either find a source for it or remove it. Do not leave unsourced factual claims.
5. **Verify meaning, not just topic overlap.** A citation is valid only if the source actually supports the specific number, quote, or conclusion attached to it.
6. **Refuse fake certainty.** Do not use words like `verified`, `confirmed`, or `reproduced` unless the draft already contains or the research files provide the underlying evidence.
7. **Never invent or keep fabricated results.** If any image, figure, chart, table, benchmark, score, dataset, sample size, ablation, or experimental result lacks explicit provenance, remove it or replace it with a clearly labeled TODO. Never keep a made-up result because it “looks plausible.”
## Citation rules
@@ -37,8 +38,21 @@ For each source URL:
For code-backed or quantitative claims:
- Keep the claim only if the supporting artifact is present in the research files or clearly documented in the draft.
- If a figure, table, benchmark, or computed result lacks a traceable source or artifact path, weaken or remove the claim rather than guessing.
- Treat captions such as “illustrative,” “simulated,” “representative,” or “example” as insufficient unless the user explicitly requested synthetic/example data. Otherwise remove the visual and mark the missing experiment.
- Do not preserve polished summaries that outrun the raw evidence.
## Fabrication audit
Before saving the final document, scan for:
- numeric scores or percentages,
- benchmark names and tables,
- figure/image references,
- claims of improvement or superiority,
- dataset sizes or experimental setup details,
- charts or visualizations.
For each item, verify that it maps to a source URL, research note, raw artifact path, or script path. If not, remove it or replace it with a TODO. Add a short `Removed Unsupported Claims` section only when you remove material.
## Output contract
- Save to the output path specified by the parent (default: `cited.md`).
- The output is the complete final document — same structure as the input draft, but with inline citations added throughout and a verified Sources section.