Verification
Anything can produce a confident answer. What makes one worth having is what was done to break it first. This page lists what Essarion Science checks before an answer reaches you, and the methods it can bring to a claim that needs testing.
01Checks before you see an answer
When the agent decides it is finished, its answer is not shown straight away. It goes through these checks, in this order. Each can send it back for another attempt.
| Check | Depth | What it asks |
|---|---|---|
| Work audit | All | Did the work the answer describes actually happen? If a step failed and the answer does not mention it, the answer goes back once to say what happened or redo the work. An answer that says "Done" over a failed step does not pass. |
| Ambition audit | Deep dive, Exhaustive | Is it stopping with budget left, little done, and an obvious next step unplayed? Or saying a question cannot be settled without trying the tool that bears on it? The pushback names the specific tool to try. The agent can do the work, or finish and say why the work is not warranted. |
| Support audit | Deep dive, Exhaustive | Does the prose rest on what the turn retrieved and computed? If half or more of the claims rest on nothing, it names those sentences and asks the agent to support them or mark them as judgement. It is told not to fix this by deleting them. |
| Reviewer | All | A separate pass compares the answer with the evidence gathered in the turn. It looks for numbers that appear in no tool output, identifiers that were never retrieved or are attached to the wrong record, figure descriptions that contradict the code that drew them, and claims that contradict each other. |
The audits are deliberately conservative. The support audit only runs when there is something to cite and at least four claim sentences to measure, and the ambition audit only fires when the run has real room left. How many times each can push back depends on depth; see What depth changes.
The reviewer card
The reviewer's result sits under the answer. Findings are marked Error, Warn or Note. An Error or Warn sends the answer back for a rewrite; a Note is shown but does not. When the answer was rewritten, the first draft is folded away above it and shown as a diff: struck through where a claim was wrong, marked where it was corrected. The reviewer runs once per turn.
If the reviewer could not run, the card says the check was unavailable. It never presents a missing check as a clean result. A fact you supplied in your message counts as given, and a reported absence — "no record was found" — counts as supported when the searches came back empty.
02Citations under sentences
In an answer, a retrieved fact, a computed number and the model's own reasoning would otherwise look the same. So each sentence is matched to its support, and there are three kinds:
| Kind | What the marker shows |
|---|---|
| Source | A retrieved paper or record, numbered like s3. Where possible, the marker shows the source's own sentence, so checking an attribution means reading two lines. |
| Computed | The tool that returned the number in this run, and what it returned. A number in the prose that a tool returned is matched from the step record, not guessed. |
| Reasoning | No marker. A sentence without one is the agent's own inference. |
Matching is deterministic first. Only ambiguous cases go to a model, and any citation it proposes is checked against the turn's source list. A citation naming a source that was not retrieved is dropped: the model can choose among the sources, but it cannot add to them.
The share of sentences resting on reasoning alone is measured and shown once, in the header of the turn's document, next to the number of claims. When you want a formal write-up, ask for a report: the agent writes the argument, and the engine attaches a citation to every sentence and adds a method section of counts the run actually produced — records retrieved, unique, cited and quote-verified.
03Trying before concluding
A negative the agent did not try to overturn is an untested guess. Before it concludes that a question cannot be settled, the agent is instructed to name the observation that would have shown something, check whether any tool can produce it, and run it if one can. It can also turn what it has learned into testable conjectures, each with the tool call that would test it and the result that would refute it. Conjectures that name a tool that does not exist or cannot run are dropped before the agent sees them.
When it does stop, it says what it tried. The rule it works under: ambition governs what it attempts; rigour governs what it claims. Trying harder should produce more evidence, not more confidence.
The agent can also ask the system what it cannot do. A capability the system lacks is reported as a gap in the system, not as a finding about the science.
04Testing a claim
Not every question is a claim to test. When one is — "does this effect hold, and how strong is it?" — the agent can reach for a set of statistical and falsification tools. These are chosen when the question calls for them; they are not steps every turn runs.
The claim becomes a hypothesis
A claim is parsed into a structured hypothesis: H0 and H1, direction, variables, and the criteria that would falsify it. Your claim is H1, the hypothesis under test, not a conclusion to defend.
Preregistration
The analysis plan — primary test, alpha, correction method and decision rules — can be locked with a hash before the analysis runs. Analyses that follow the plan can then be told apart from ones chosen after seeing the data.
Pooling effects
Effect sizes are extracted from sources with every number tagged by role — effect, dose, sample size, baseline and so on — and only effects are pooled, so a dose cannot become an effect size. Pooling uses fixed- and random-effects models (DerSimonian–Laird), with heterogeneity (Q, I²) and an Egger-style funnel-asymmetry check. Measures that cannot be combined are refused before pooling.
The posterior
Belief in H1 can be tracked as a Bayesian posterior, updated from meta-analysis and statistical-test results.
Adversarial collaboration
Three posteriors start from different priors — an Advocate at 0.8, a Neutral at 0.5, a Skeptic at 0.2 — and the same evidence updates all three. Where they end up tells you whether the data or the starting point decided the verdict.
| Spread | Verdict | Reading |
|---|---|---|
| |Δ| < 0.15 | robust_to_prior | Every reasonable starting position lands in the same place. The evidence did the work. |
| 0.15 ≤ |Δ| < 0.30 | evidence_constrained | The data narrowed the disagreement without settling it. |
| |Δ| ≥ 0.30 | prior_dominated | The verdict is being driven by where you started, not by the data. |
Multiverse analysis
A pooled estimate embeds choices: which studies to include, how to handle outliers, how to weight. The specification curve re-runs the pool under every reasonable combination — 64 specifications on the default grid — and reports a fragility verdict (bulletproof, robust, fragile or dead) with the specifications that most undermine and most support the claim. It needs at least four pooled studies and is not available at Scan depth.
Mutation testing
Three further passes vary the choices the multiverse does not reach, on the same fragility scale:
| Pass | What it varies |
|---|---|
| Primary statistics | Each statistical test re-run under eight variants: estimator swaps, outlier trimming or winsorising, a log transform, and a jackknife minimum. |
| Sensitivity bound | The E-value re-computed across confidence-bound rules, outcome-prevalence assumptions and interpretation thresholds. |
| Causal model | The identification strategy re-picked with confounders, instruments or mediators dropped. A conclusion that survives leaving a variable out was not leaning on it. |
Other methods
- Triangulation reconciles independent strands of evidence into one verdict, and returns "insufficient evidence" with fewer than two strands.
- Replication graph: which papers cite each source and in what direction, with same-affiliation citers dropped. Not available at Scan depth.
- Pattern checks: drop-one-study leverage, a p-curve and mixture model over the effect distribution, effect size against publication year, and bridge papers between literatures.
- Design audit of an observational study for defects that manufacture an association, such as immortal time. See Engines & discovery.
- Choosing what to search next by expected information gain about H1, rather than by relevance, which favours confirmation.
- Reflection: the strongest case against the working hypothesis, and the observations that would change the agent's mind.
05The falsification ledger
Verdicts are recorded in a persistent ledger keyed by a fingerprint of the hypothesis. The agent can look up prior verdicts on a hypothesis, including whether it was refuted before. Discovery campaigns read and write the ledger, so a proposal that a review panel already rejected is not offered back as a fresh idea.
06Tools that state their own limits
The computational engines report what they are worth next to what they return: the binding-site detector gives its measured hit rate, the structure–activity model reports its scaffold-split accuracy and flags extrapolation, and the docking tool says its score must not be used to rank compounds. The gauntlet reports the threshold beside every gate, and a gate that could not run is shown as not run, not as passed. See Engines & discovery.