Accuracy & Corrections
Real crowd or staged? The second opinion
FactsLens flags a group from measurable signals. Nemotron then judges the same group with the evidence in front of it, and is told to look for the innocent explanation first. Measured against our reviewers (unit: cluster; ground truth: the latest human decision):
| Precision | False alarms among reviewed | Reviewed | |
|---|---|---|---|
| FactsLens alone (suspicious) | 93% | 1 | 15 |
| FactsLens and Nemotron agree it is staged | 91% | 1 | 11 |
Nemotron agreed with our reviewers on 83% of the clusters both decided (10 of 12) and abstained on 3. With so few reviews these numbers move with every decision; we show them anyway.
The three checks we run, and where you can see them
Spot the swarm
Groups of accounts acting together, found from timing, copied text, shared links and hashtags, repost rings, account age and creation bursts, never from a single post.
Measured: Fake copy-paste campaigns we hid among real posts, found: 4 of 4; reworded ones: 4 of 4. Accounts on IFTAS's documented list found: 12 of 17, a median 21.4 minutes after their first post.
Look for the Truth: see the groups behind a post →
Ask the Truth: “What are the biggest campaigns now?” →
Name the technique
Loaded language, fear appeals, whataboutism and fabricated authority, each shown with the exact words and explained in plain language.
Measured: A technique is only published when its quote appears word for word in the text, and every claim must cite evidence Nemotron was shown; anything else is removed before publication.
Trust the source
Who is behind a post or page: account age and reach, public network lists, domain registration and propaganda-domain lists, and how independent outlets in several countries tell the same story.
Measured: Every outlet is checked against VIGINUM's and CheckFirst's lists; independence is judged by distinct outlets in different countries, never by size or official status.
Look for the Truth: “Who is behind it” →
Contested stories: state media against independent outlets →
Corrections
Each was found on live data on 14 November, and each changed how FactsLens works.
Correction. Disclosed weather-alert and traffic bots, created together with template bios and posting in sync, were rated suspicious. What we changed: A disclosure signal (bios and handles that say “bot” or “automated feed”, several languages) and a separate declared-bots tier with a capped score. Documented-network evidence still overrides it.
Correction. Real people sharing the same headline through a share button looked “coordinated”. What we changed: Coordination alone is never suspicious: account or provenance evidence must come on top.
Correction. An early prompt (v3) called a UK news-syndication cluster staged. What we changed: Innocent explanation first, reasons before the verdict, abstaining allowed. No later version called one of the organic controls staged.
Correction. A later prompt (v6) called accounts on a server documented as part of a propaganda network organic. What we changed: A rule, not a prompt: with documented-network evidence, “organic” becomes “not enough evidence” for a person to decide.
Correction. The same news photo with a Polish and a Russian caption counted as a “recycled picture”. What we changed: Nemotron judges every reuse; only pictures that do not fit the later caption are highlighted.
Correction. Clusters that merely mentioned a name were listed as amplifying a contested story. What we changed: Nemotron classifies each matched cluster; only clusters that repeat the story are linked.
Correction. Users of big general Mastodon servers counted as “accounts on network servers”. What we changed: Only servers where IFTAS documented at least 10 network accounts count.
Correction. A Belgian article was reported as having no coverage: we read a consent page, searched only the Netherlands edition, and used eight-word queries. What we changed: National news editions (e.g. Belgium in Dutch and French), short searches retried with fewer words, and the original website kept as the source when a site shows a consent page.
Where we are blind
- Campaigns written to avoid detection (own words, several languages, aged accounts) share no text, link or hashtags, so we do not catch them yet. Catching them needs grouping by meaning.
- We see only the last 72 hours, and only Bluesky, the Telegram channels the Pravda network republishes most, and the Mastodon servers on IFTAS's list.
- Our random review sample is still small, so the false-positive rate on live traffic has a wide margin.
- Asked twice about the same group, Nemotron can reach a different verdict: single verdicts are leads, not measurements.
Our standards
- We never accuse anyone automatically. Every finding is a lead for a person to weigh.
- We say “not enough evidence” rather than guess, and a verdict without grounded evidence is downgraded.
- Every claim links to its evidence. References the model was never shown are removed before publication.
- Account names appear only for accounts already on public lists; pictures from individuals stay with reviewers.
- We publish our error rate, including where we are blind.
Every figure in full, with sample sizes and the threshold sweep · How we measure (docs/EVAL.md) · Every change to Nemotron and its effect