1. Judge outcomes, not visible reasoning
Chain-of-thought can be useful working text, but research has shown that it may omit information that affected an answer. Treat a persuasive explanation as a review aid, never as proof of correctness. Keep hidden reasoning out of evidence envelopes and grade the observable result against independently defined acceptance and rejection cases.
Primary source: Anthropic, reasoning models do not always say what they think
2. Measure delivery, not generated lines
A 2025 randomized study of 16 experienced open-source developers across 246 tasks found that the tested early-2025 AI tools increased completion time by 19% in that setting. It does not establish a universal slowdown; it does show why perceived speed is not enough. Compare cohorts using reviewed-design-to-production lead time, rework, escaped defects, corrective churn, recovery time, and total cost.
Primary source: METR productivity study
3. Test retrieval as a pipeline
RAG does not make unsupported answers impossible. Evaluate retrieval relevance, evidence selection, generation, and citation fidelity separately. Include stale documents, plausible distractors, missing support, conflicting sources, access-control boundaries, and an explicit abstention oracle. Bind evidence to the corpus, index, retriever, reranker, prompt, model, and tenant policy.
One large medical evaluation used 80,502 expert annotations and found that standard RAG could reduce factuality by up to six percentage points in its tested conditions when retrieval and evidence selection were weak. The operational lesson is stage-aware evaluation—not a claim that RAG always hurts.
Primary source: large-scale expert evaluation of medical RAG
4. Challenge the advertised context window
Capacity is not recall. Test low lexical overlap, aliases, multiple connected facts, distractors, contradictions, truncation, and facts at the beginning, middle, and end. NoLiMa found substantial degradation at longer contexts when literal matches were removed, even for models that advertised much larger windows.
Primary source: NoLiMa, ICML 2025
5. Assume public benchmarks can lie
A leaderboard is discovery input, not a release gate. Record every case, skipped or failed setup, prompt and tool configuration, scorer version, contamination check, and critical slice. Prefer product-specific, executable, protected or time-split holdouts. In 2026, OpenAI stopped reporting SWE-bench Verified after finding flawed tests and evidence of benchmark exposure in frontier models.
Primary source: OpenAI’s SWE-bench Verified audit
6. Regression-test every model adaptation
Fine-tuning can improve the optimized behavior while degrading retained capabilities or safety. Compare the candidate to the exact base and production model on target performance, general capability, security, privacy, authorization, prompt injection, cost, and incident regressions. Pin the model, tokenizer, dataset, recipe, and runtime; then rehearse rollback.
- Primary source: narrow insecure-code fine-tuning and broad misalignment
- Primary source: Microsoft Research on LoRA, full fine-tuning, and forgetting
7. Keep tools and data inside explicit authority
Model intent is irrelevant to enforcement. Deny ambient credentials and egress, broker each tool capability, validate arguments and outputs, isolate untrusted execution, and make retained conversation state and provider data use part of the threat model. A friendly prompt is not a privacy or authorization control.
What MergeGrounds materializes
- versioned, product-specific eval manifests and complete case counts;
- positive, negative, adversarial, and recovery controls;
- exact model, data, retrieval, prompt, tool, and runtime identities;
- critical-slice thresholds that aggregates cannot average away;
- shadow/canary evidence, drift triggers, rollback, and incident replay;
- fail-closed handling for stale, partial, skipped, or mismatched evidence.
These controls extend the repository gate. Read the trust boundary before assigning any assurance tier.