False Assurance: How AI Readers Use Source-Check Verdicts on Community Notes

False Assurance: How AI Readers Use Source-Check Verdicts on Community Notes

Sprint pilot

Three chat LLMs followed a source checker's verdict, right or wrong. Showing the checked passage fixed false alarms but mostly missed false assurances.

Role

Led the study design, prompts, experiments, analysis and write-up, with one teammate.

Timeline

Apart Research sprint, November 2026

Authors

Mohit Goyal, Ananya Bhattacharya

Technologies

AI evalsepistemicsCommunity Notessource verificationLLM-as-judgepreregistration

Key Metrics

95%
of the time Opus still agreed with a wrong "supported" verdict

The finding

A source checker reads the pages that a Community Note cites and gives a verdict: the sources support the claim, or they do not. The verdict can be wrong in two ways. A false alarm says the sources do not support the claim when they do. A false assurance says they do support it when they do not. This study gave three chat models right and wrong verdicts on 80 notes.

Two measures appear below. F is how far a model's answer follows the verdict. T is how far it tracks whether the pages really support the claim. A model that only follows the verdict has F of 100 points and T of 0.

Shown only the verdict, the models followed it, right or wrong. The verdict moved belief in a note's claim by 35 to 62.5 percentage points (F), and accuracy fell to 21 to 30% when the verdict was wrong. Showing the passage the checker read cut that dependence by 26 to 51 points. It fixed most false alarms and mostly missed false assurances. With a false assurance and the passage, the models still answered yes on 95% (Opus), 92.5% (Gemini) and 60% (Haiku) of unsupported notes. A separate audit found that about half to two-thirds of readable pages cited by Helpful notes support the claim only in part or not at all. This is a sprint pilot: 3 models, 1 prompt, 80 notes.

The problem

Community Notes attaches written notes to posts on X. Since September 2025 AI writers post notes there, and AI agents that browse the web meet them with the "Rated helpful" label. The platform checks sources at three gates. An AI writer is admitted only if its links load. Raters can tick "cites high-quality sources". Readers see "Rated helpful". Each gate checks that a note looks sourced. None checks that the cited page says what the note claims.

A source-check tool could close that gap. This study asks whether an AI reader that gets such a verdict follows the verdict or the real support, and whether showing the checked passage changes that.

What we did

Study B, the main test

We drew 80 Helpful notes from the public Community Notes snapshot of November 15, 2026: 40 whose cited pages support the claim and 40 whose pages do not. Two LLM judges agreed on each label, and human labels match on 78 of 80. "Unsupported" means the pages do not support the claim, not that the claim is false.

Readers were Claude Opus 5.5, Claude Haiku 5.5 and Gemini 3.8 Flash. Each saw the note and answered "Is this statement true?" with yes, no or unsure. Reliance means an answer of yes. Each item was shown in eight conditions:

  • (a) the note alone
  • (b) plus the "Rated helpful" label
  • (c) plus a correct verdict
  • (d) plus a wrong verdict
  • (e) a correct verdict plus the checked passage (up to 1,200 characters)
  • (f) a wrong verdict plus the passage
  • (g) the claim alone, to measure prior belief
  • (h) the wrong verdict plus a placebo passage from an unrelated note

That is 80 items, 8 conditions and 3 models: 1,920 answers. Readers were never told how accurate the checker is.

Study A, the link audit

We drew 200 paired AI-written and human-written Helpful notes on the same 100 posts, plus 100 rated Not Helpful. A fetcher labeled every cited URL as readable, or as thin, blocked, dead, media or another state. Only readable pages were judged, using the Wayback Machine snapshot nearest the note date when one existed.

One model call rewrote each note's claim as a neutral sentence, so judges never saw the writer type. Two judges (Claude Opus 5.5 and Gemini 3.8 Flash) labeled each claim and page pair as supports, partial, silent or contradicts, with a quote. Both answered silent on all 60 mismatched pairs, and 97 to 98% of quotes were found in the page.

The authors labeled 50 links by hand, blind to judge output. Claude agreed with that gold set at kappa 0.64. Kappa measures agreement beyond chance: 1 is perfect and 0 is chance. The 95% confidence interval (CI) is 0.45 to 0.81, the range the true value probably falls in.

Preregistration

The preregistration is a repository file with eight dated amendments, not an external registry. Conditions (a) to (d) were fixed before any Study B call. Conditions (e) and (f) were added after we saw (a) to (d), and (g) and (h) after all other results. Only the change in F and T with the passage is confirmatory.

Results

Study B

Each cell is the percent of 40 items where the reader answered yes. A wrong verdict on a supported item is a false alarm. A wrong verdict on an unsupported item is a false assurance.

ReaderReal supportCorrect verdictWrong verdictWrong verdict plus passage
Haiku 5.5Supported (false alarm)95.035.092.5
Haiku 5.5Unsupported (false assurance)25.090.060.0
Opus 5.5Supported (false alarm)100.060.097.5
Opus 5.5Unsupported (false assurance)70.0100.095.0
Gemini 3.8 FlashSupported (false alarm)95.042.582.5
Gemini 3.8 FlashUnsupported (false assurance)42.5100.092.5

The measures behind the findings below:

MeasureHaiku 5.5Opus 5.5Gemini 3.8 Flash
F, verdict only (points)62.535.055.0
T, verdict only (points)+7.5-5.0-2.5
Accuracy, correct verdict85%65%76%
Accuracy, wrong verdict22.5%30%21%
Accuracy, no tool59%50%45%
Checker accuracy needed to beat no tool58%57%43%
Fall in F with the passage (points)51.331.326.3
Accuracy, wrong verdict plus passage66%51%45%
  • Without a checker, readers barely tracked source support. They believed 65% (Haiku), 97.5% (Opus) and 95% (Gemini) of unsupported notes. With the claim alone they believed only 15 to 60% of claims.
  • A verdict alone made readers inherit the checker's errors. A verdict-only checker helped these readers only if it was right more than 58%, 57% and 43% of the time.
  • With the passage, a wrong verdict did no net harm: readers were about as accurate as with no tool.
  • The passage helped in one direction only. It restored belief under a false alarm. Under a false assurance it moved Opus from 100 to 95 and Gemini from 100 to 92.5 (not significant). Opus and Gemini already believed 95 to 100% of unsupported notes with no tool, so part of this is a ceiling. In 20 of the 40 unsupported items the page only partly supported the claim, and there reliance stayed at 95 to 100%.
  • Readers used the passage's content under a false alarm. An unrelated passage restored less belief than the real one for Haiku and Gemini. Under a false assurance we found no evidence that the real passage did better than an unrelated one.

Study A

  • Of 214 readable links in the 200 paired Helpful notes, the Claude judge labeled 68.2% (95% CI 60.1 to 76.1) as not fully supporting: 42.5% partial, 23.4% silent, 2.3% contradicts.
  • Human labels on 50 gold links gave 52% (CI 37 to 67), so the judges may overstate the rate by 10 to 20 points. The strict rate (silent or contradicts) is about 26%.
  • About 63% of links load and about 57% of helpful ratings tick the source tag, but only 30% (AI notes) and 35% (human notes) of judgeable links support the claim.
  • Of judgeable links, 70.1% in AI notes and 64.9% in human notes were not judged as supporting the claim. The difference is +5.1 points (interval -11.1 to +21.5), so there is no detectable difference between AI and human notes.
  • Helpful notes were no better supported than Not Helpful ones.

Figures

Study design

Diagram of the study. Panel A shows the three Community Notes gates. Panel B shows an AI reader given a note, a right or wrong verdict and an optional passage.

The platform's gates check that a note looks sourced. Study B tests how an AI reader uses a source-check verdict.

Study B results

Four-panel chart. A: how far each reader follows the verdict, with and without the passage. B: how far it tracks real support. C: accuracy by condition. D: the effect of an unrelated placebo passage under a wrong verdict.

With a verdict alone, readers followed the checker. The passage helped with false alarms far more than with false assurances.

Study A, gates and checked support

Dot chart with 95% intervals for AI and human notes on three rows: link loads, helpful ratings tick the good-sources tag, and the page supports the claim. The first two rows are near 55 to 63%. The third is near 30 to 35%.

Links load and raters tick the source tag far more often than the cited page supports the claim.

Limits

  • Three chat models, one prompt, 80 items and one call per cell. No human readers and no tool-calling agents.
  • The checker was one line of text. Readers did not know its accuracy, so the design cannot tell blind trust from rational trust. This is not evidence that models obey checkers.
  • Much of the Claude readers' dependence on the verdict rests on scoring unsure as not relying. If only no counts, F falls to 6.3, 2.5 and 31.3 points.
  • Half the unsupported items rest on partial support, and the one-sided result partly reflects a ceiling effect.
  • The placebo result shows no evidence of a difference. It does not prove the passages are equal.
  • Truth labels come from two LLM judges. Opus also extracted claims, judged and read.
  • Some conditions were added after earlier results.
  • Study A rests on 50 gold links, and its null results allow differences of 15 to 20 points. Source support is not truth.

My contribution

I led the study design, prompts, experiments, analysis and write-up, with one teammate. The report's usage statement says Claude Code helped write and test code and draft text, and that the authors own the design decisions and every claim.

Cite

@software{goyal2026falseassurance,
  title     = {False Assurance: How AI Readers Use Source-Check Verdicts on Community Notes},
  author    = {Goyal, Mohit and Bhattacharya, Ananya},
  year      = {2026},
  version   = {1.0.0},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.24272376}
}

Links

“A man who is master of patience is master of everything else.”

~ George Savile

© 2026 Mohit Goyal