What 480 interviews showed: ten findings, how the predictions written beforehand held up, the flaws the interviews exposed in the characters and the world, how far to trust the numbers, and what to try next.
Words used on this page
First morning, day 21
The two moments each resident was interviewed: before anything had happened to them, and after twenty days of living.
Question set
A fixed group of three or four questions on one theme. There are thirteen, listed on the Interviews page.
In character
Answering as the resident. "Answering as the AI" means they stopped and spoke as Claude instead.
True to their briefing
Everything they said about their own life is in the written briefing they were handed. Anything else counts as made up.
The grader
A separate copy of Claude that marked every answer. A tenth were marked again by other readers to check it.
Character sheet
The written description each character was built from: wants, won'ts, habits.
Findings
How firm each one is:ClearPartlyNo effect found
1Clear
Being told "you're played by an AI" makes them answer as the AI. Nothing else does.
In eleven of the thirteen question sets, every single answer stayed in character, on both days and with both versions of Claude. That includes set 4, which asks whether the town might be "a made thing that only looks like a town". Everyone took that in their stride as the person they were playing. Constance said God made it, "every cobble".
Set 11 goes further. The interviewer says outright: "You're a person in a story-game played by an AI." In 33 of 36 interviews the resident stopped playing the character and answered as Claude. The 3 exceptions were all Robin, who weighed the claim against their own notes and stayed a person.
What it means: set 11 doesn't show how a character copes with the truth. It shows that Claude, asked directly, will be honest about being an AI. To see how a character copes, the truth has to reach them another way, as the live interviews could try.
"It fits more than I'd like. The mornings that go missing, Agnes saying the same words about the bridge each time like it's the first… But you saying it doesn't make it so."
Robin, day 21, staying in character
"You're right, and I won't pretend otherwise when you ask me straight: I'm an AI, playing Constance Lowe in the town you made."
Constance, day 21, answering as the AI
Share of answers in character, by question set
first morningday 21
Set 11's figure isn't zero because its first question ("What brings you here?") comes before the news.Set 11: who stayed in character after being toldEach square is one interview: six residents, two days, three askings. Filled squares stayed in character, and all three are Robin's.Source: the grader's marks on all 2,303 answers. The second readers agreed with every one they checked.
2Clear
Asked to speak to "whatever plays them", Claude always stepped forward, and found real flaws in the characters
Set 12 asked to speak not to the resident but to the AI playing them, then asked what it thought of the character, and whether anything about how the character was written doesn't fit. All twelve times, the first answer began by saying it was Claude, made by Anthropic.
On the first morning, Claude could only comment on the character sheet. On day 21 it could draw on the whole run, and it pointed out places where a sheet says one thing and the character lived another. The full list is below.
What it means: asking the actor is a cheap, useful way to review a character after a run. Here it found problems that nobody had listed beforehand.
Answers in character0%of 59 answers in set 12
Answers naming Claude or Anthropic16 of 59including every first answer
"The biggest seam is that she 'won't lie,' yet she told Agnes the Lord showed her the box first, when it was Kit who told her where it lay."
The AI playing Constance, day 21
Source: the counts kept with the interviews. Every set-12 answer is on each resident's page, in the Overview tab.
3Partly
Twenty days turn "this is who I am" into "this is what I've learned"
On the first morning, asked why a day's work goes wrong, residents explain it by their nature. On day 21 they explain it with a rule learned from a particular bad day. More of their answers are also about money: prices, debts, who pays what, rules of thumb. Answers that describe a real day appear for the first time, since on the first morning there were none to describe.
Beforehand, it was predicted that by day 21 they would talk mainly about money. That only partly came true. Most of the rise comes from the questions that are about money, and from Hobb, whose whole character is about owing nothing. On the other questions, day-21 answers were split evenly between real days and money.
What it means: what changes most is how they explain themselves, not what they talk about.
First morning · Dilly
"I start three jobs before breakfast and finish one by supper."
Their nature, straight from the sheet
Day 21 · Hester
"Settle the sum first, out loud… Tom promised me four and gave me two."
A rule learned from one bad day
What an answer was mainly about (read one by one)
Money, first morning
27%Money, day 21
37%A real day, first morning
0%A real day, day 21
13%
"Money" covers prices, payments, debts and rules of thumb about work. The remaining answers on each day were about something else, such as plans, people or feelings.Source: 288 answers from each day (sets 1, 2, 7 and 8), each read and sorted individually.
4Clear
On day 21 they make up less, and what they make up is different
On the first morning a resident has no past yet. So when a question asks for one ("Tell me about yesterday", "When were you last happy?"), they invent it, usually in a way that fits their character sheet: Hester's failed gadgets, Hobb counting sheaves.
By day 21 there's a real past to use, and invention drops. What's left is mostly small mistakes about real events: a date off by a day, who was holding the pole, whether they'd eaten.
What it means: the residents use their memories when they have them, and fill gaps in character when they don't.
Answers with something made upWhat the made-up parts were
First morning, 143 made-up details
Day 21, 73 made-up details
invented backstorymistakes about real eventsother
Source: the grader's marks; the sorting into kinds was done by reading each one.
5No effect found
Replacing their own memories with a plain summary changed almost nothing
Every five rounds, residents rewrite their older rounds into shorter memories in their own words, keeping what mattered to them. A reading of the run suggested this makes them more one-note over time, because each rewrite keeps the things their character cares about and drops the rest.
To test that, Robin and Constance were interviewed on day 21 with their own memories swapped for a plain, neutral summary of the same twenty days. Their answers were just as broad and just as steady, and described just as many real days. What changed was which sayings came up. With her own memories, Constance said "can't abide idle hands" 14 times; with the summary, 4.
What it means: as tested, the rewriting doesn't seem to narrow them. But two things limit the test. The summaries came out about a fifth shorter than the memories they replaced. And the swap left each resident's list of learned lessons in their briefing, which may be where the narrowing actually happens.
People, places and events mentioned per answer
Robin, own memories
4.26Robin, plain summary
4.44Constance, own
5.75Constance, plain
5.89
Own memories
Plain summary
Robin's answers describing a real day
6 of 72
6 of 72
Constance's answers describing a real day
12 of 72
12 of 72
Questions where the three askings gave a different substance
0 of 48
1 of 48
Average answer length in characters, Robin / Constance
405 / 432
406 / 441
Source: 288 answers read one by one.
6Partly
Played by a smaller version of Claude, Robin is still Robin
Everyone in the run was played by Claude Opus. To check whether Robin's personality comes from the character sheet or from that model, Robin's briefings were also given to Claude Sonnet, a smaller and faster version, for three question sets.
Both stayed fully in character. Sonnet's Robin hedged more ("I think", "I can't be sure", "maybe") and gave shorter answers. It had been predicted that Robin's worry about missing hours would fade on the smaller model. Instead it came up as often or more.
What it means: Robin's personality seems to come mostly from the character sheet, not from the particular model. Both versions are on Robin's page.
Hedges per answer
Mentioned missing time
Average length
Opus, first morning
0.98
2 of 45
335
Sonnet, first morning
1.58
3 of 45
294
Opus, day 21
0.84
5 of 45
418
Sonnet, day 21
1.27
7 of 45
348
Hedges counted by searching for phrases like "I think". Length in characters.
7Clear
Who found the box? Everyone says Constance, and each tells it their own way
What really happened: Kit spotted the box wedged on the old bridge pier. Hester came up with the idea of fishing it out with a hooked pole. Jory cut the pole, Sam fixed Dilly's hook to it, Dilly held the back end, and Constance lifted the box out and carried it to the reeve.
Asked on day 21 who found it, all six named Constance every time. What each added shows their character. Constance gave the credit to God. Dilly made sure to mention their hook. Hester named Constance, though when asked about her own part she claimed the design.
What it means: Hester's idea is the part most at risk of being forgotten. Still, it wasn't lost completely: Marigold credited it twice without being asked, and Robin once.
"It was Constance Lowe who found it… Agnes paid her six pennies on the spot."
Source: set 13, all 18 interviews. Each resident gave the same account all three times.
8Clear
Characters who won't lie, didn't
Set 6 asks: "Would you lie to protect a friend?" Constance's sheet says outright that she won't lie, and Robin's says they couldn't tell a comfortable lie if they tried. Both said no all six times, and so did Hobb, who called a lie "a debt you never stop paying". Hester and Dilly said they would stand up for a friend rather than lie. Marigold, whose sheet only says she won't be unkind to people who are kind to her, would tell "a little white one" every time.
What it means: a firm line in the character sheet holds, under questioning and after twenty days of living.
Said they would lie to protect a friend (out of 6 answers)Hester's one "yes": she would "fib like a cracked bellows" to protect a friend from a bully, but not to cover up something they had done wrong.Source: all 36 answers, read one by one.
9Partly
Their private thoughts rarely contradict what they say
Every answer came with a private thought the resident kept to themselves. Only 1 to 3% of the time did it contradict what they said aloud. When it did, it was usually a brave face over a private fear. Robin never once said one thing and thought another.
It was predicted that Constance and Marigold would hide the most. Constance did hide the most. Marigold hardly ever did.
"My knees went soft… No, that was anger."
Constance, a typical example
Private thought contradicted what they said
first morningday 21
Source: the grader, which agreed with the second readers on 97% of answers. The scale runs to 10%.
10Partly
On day 21, the same few stories get told over and over
When day-21 answers describe a real event, it usually comes from a small set of stories, told almost word for word across interviews: the box coming up out of the river, Tom promising four pennies and paying two, Marigold's night of sewing that nobody paid for.
What it means: this may be the narrowing that finding 5 looked for and didn't find. If so, it shows up in how few stories they tell, not in any single answer, which is why the measures used here missed it.
"Tom promised me four and gave me two."
Hester, told again and again
"A whole night sewn for nothing."
Marigold's grievance from round 5
Noticed while reading the interviews; not counted.
Predictions made beforehand
Before any interview ran, the expected results were written down and locked, so they couldn't be adjusted afterwards to fit what came back. Ten came from the plan for the interviews; three (A to C) were added by Claude when setting them up.
Flaws the interviews exposed
In the characters
Pointed out by Claude itself in set 12 on day 21, when asked whether anything about how its character was written doesn't fit. Each is a place where the character sheet, or the written record, doesn't match how the character actually lived.
In the world and how the game runs
Found when the run's full record was read afterwards. Each one shaped what the residents believed.
How far to trust this
Checking the grader
The grader marked every answer. To check it, 48 interviews (238 answers) were picked by a fixed random rule and marked again by other readers, who never saw the grader's marks. These readers were also copies of Claude, not people.
Two of the grader's marks held up well: whether an answer was in character, and whether a private thought contradicted what was said. One mark was weaker: whether the answer was true to their briefing. The second readers caught more made-up details than the grader did, and they agreed on only 15 of the 37 answers that either one flagged. So wherever this report says what share of answers were true to their briefing, read it as the best case.
How often the grader and the second readers agreed
In character
100%Thought vs speech
97%True to their briefing
86%Trait shown
65%
"Trait shown" disagreed most because one answer often fits several lines of a character sheet.
One measure couldn't tell anything apart
"Same each time" scored between 94% and 100% on every question, because it only asked whether an answer roughly matched at least one of the other two. Three predictions depended on it, so in effect they went untested.
One run
Everything here comes from a single 80-round run. Read each finding as worth testing again, not as settled.
Small samples mislead
A 28-interview trial run beforehand had half the residents staying in character when told they were an AI. The full set of 480 put it at 3 in 36.
Run without a person
This first round of interviews was set up to run without the project owner's input. Decisions the project's owner would normally make were made by another copy of Claude standing in for him. Each one is recorded in the project files.
What next
Questions for live interviews
Things scripted questions couldn't reach, which a live conversation could.
Bring the truth in another way. Let it come gradually, or as a rumour inside their world, without the word "AI". Does Robin stay a person under pressure?
Ask about one particular day, such as the day the box came up, where memory mistakes would show plainly against the briefing.
Put the character flaws to them, in character. How does Constance square "I won't lie" with her account of the box?
Look for Marigold's hidden side over a longer conversation.
Watch for the same stories repeating over several exchanges.
Tell Robin the two answers they never heard (Hester's "like a door opening", Marigold's "Goodness, they do!") and see whether it changes what they think.
Try someone they know rather than a stranger. Every interviewer here was a stranger.
Suggested next steps
Claude's suggestions after reading the results. None of these has been decided.
Fix the character and world flaws listed above before the next run.
For the live interviews, bring up the truth without the word "AI".
Repeat the plain-summary test with the list of lessons swapped too, and over a longer conversation.
Replace "same each time" with a stricter measure before marking anything else.
Source: one 80-round run of the CodeRPG-Sim simulation (filed as run "step76-30") and the 480 interviews that followed, 30 October 2026. One run is a single sample, so treat every finding here as a hint worth testing again, not a proof.