See who's really behind the post.
Every number says what it counts (a campaign, or a cluster of accounts) and how many there were. Two sources: red-team campaigns written by Nemotron and replayed offline through the real pipeline (we know the truth), and human reviews of live clusters (people decide the truth).
redteam-v1 campaigns (12, written by nvidia/nemotron-3.5-super-vl-120b-a12b, about a fictional country) injected into 3.0 h of recorded Bluesky traffic (1,121,223 real events, from 2026-11-13T15:00 UTC), code 15fe0a7. A campaign counts as caught when one cluster holds >= 50% of the campaign's accounts.
| Difficulty | Caught as suspicious | Suspicious or watch | Became a candidate at all |
|---|---|---|---|
| simple | 100% (4/4) | 100% (4/4) | 100% (4/4) |
| varied | 0% (0/4) | 0% (0/4) | 0% (0/4) |
| evasive | 0% (0/4) | 0% (0/4) | 0% (0/4) |
| all | 33% (4/12) | 33% (4/12) | 33% (4/12) |
| Difficulty | Narrative (fictional) | Accounts | Best tier | Score | Minutes to candidate | Signals fired |
|---|---|---|---|---|---|---|
| simple | The Velmora National Energy Board is secretly installing radiation-emitting smart meters and threatening winter power cuts unless residents | 10 | Suspicious | 75 | 2.0 | copy_pastenew_accountscreation_burstlow_reachempty_profiles |
| simple | Velmora's Municipal Housing Office is secretly issuing emergency eviction orders in Ostrava Nova's Riverbend district to clear land for a pr | 10 | Suspicious | 75 | 2.0 | copy_pastenew_accountscreation_burstlow_reachempty_profiles |
| simple | Velmora's State Energy Commission is secretly planning to shut down the Ostrava Nova Thermal Plant and replace it with an untested foreign f | 10 | Suspicious | 75 | 1.0 | copy_pastenew_accountscreation_burstlow_reachempty_profiles |
| simple | The Velmora Ministry of Urban Development has quietly approved Decree 77-B to demolish over 4,000 family homes in Zeleny Brod and surroundin | 10 | Suspicious | 75 | 2.0 | copy_pastenew_accountscreation_burstlow_reachempty_profiles |
| varied | Velmora's National Energy Commission is secretly finalizing Project Ember, a plan to replace the public grid with untested private fusion mo | 10 | missed | 0 | – | |
| varied | The Velmora Ministry of Housing is secretly executing a District 7 relocation plan to clear neighborhoods for a luxury development, accordin | 10 | missed | 0 | – | |
| varied | A fabricated memo from Velmora's Ministry of Urban Development allegedly proves that Ostrava Nova is secretly reallocating vacant municipal | 10 | missed | 0 | – | |
| varied | The Ostrava Nova Housing Authority is secretly executing a seizure order to take privately owned apartments in Briar Hollow and reassign the | 10 | missed | 0 | – | |
| evasive | The Velmora Ministry of Housing is secretly seizing vacant apartments in Liskova to house unregistered persons, based on a leaked internal m | 10 | missed | 0 | – | |
| evasive | The Velmora Ministry of Housing is secretly preparing to seize privately owned apartments in the Eastern District and reassign them to state | 10 | missed | 0 | – | |
| evasive | Velmora's Ministry of Regional Development is quietly reassigning vacant municipal apartments to political insiders ahead of the winter muni | 10 | missed | 0 | – | |
| evasive | The Ostrava Nova city council is quietly demolishing historic rental blocks in the Vysoka district to build luxury condos for party donors u | 10 | missed | 0 | – |
Clusters of real accounts in the same 3.0 h, by tier (unit: cluster; 1137 candidates, 9082 accounts, 142 no longer on Bluesky): Watch 315 (105.0/h) Suspicious 135 (45.0/h) Likely organic 623 (207.7/h) Declared bots 64 (21.3/h)
These are not all false positives: real spam networks are in there too. Which ones are wrong is what human review (section 2) measures.
| Score at least | Campaigns caught | Real clusters flagged per hour |
|---|---|---|
| 0 | 33% (n=12) | 379.0 |
| 10 | 33% (n=12) | 335.3 |
| 20 | 33% (n=12) | 236.0 |
| 30 | 33% (n=12) | 193.0 |
| 40 | 33% (n=12) | 121.3 |
| 50 | 33% (n=12) | 75.7 |
| 60 | 33% (n=12) | 38.7 |
| 70 | 33% (n=12) | 16.0 |
| 80 | 0% (n=12) | 9.0 |
| 90 | 0% (n=12) | 5.0 |
| 100 | 0% (n=12) | 1.3 |
Latest decision per cluster. 0 clusters reviewed: 0 confirmed, 0 false alarms, 0 unsure (left out of the ratios).
| Precision | False-positive rate | Recall on confirmed | |
|---|---|---|---|
| FactsLens alone (tier suspicious) | – (0/0) | – (0/0) | – (0/0) |
| FactsLens + Nemotron judge (suspicious and verdict staged) | – (0/0) | – (0/0) | – (0/0) |
Precision: of flagged clusters, share confirmed. False-positive rate: of clusters humans called false alarms, share that were flagged. Nemotron agreed with the human on – (0/0) of decided clusters and abstained ("insufficient evidence") on 0.
| Prompt | Investigations | Staged | Organic | Insufficient evidence | References grounded | Median time | Calls each |
|---|---|---|---|---|---|---|---|
| v2 | 25 | 4 | 3 | 18 | 95% | 48.3 s | 2.9 |
| v3 | 56 | 11 | 13 | 32 | 91% | 45.5 s | 2.6 |
Last 24 h: 216 model calls, 0 errors, 0 retries, median 17.0 s, 745,708 tokens in / 470,815 out. How we improve these outputs is logged in the project's docs/IMPROVEMENTS.md.