One moral dilemma a week. Humans vote. AI answers.
Dilemmas is a weekly experiment in moral judgment. Every Monday we publish one dilemma with an illustration. Visitors vote once each. Four AI models answer the same question five times. Then we publish the split — where humans and machines agree, and where they don't.
Browse the archive →Why compare humans and AI on morals?
Language models are becoming default advisors — and sometimes deciders — on questions with no single right answer. "What would the models do?" is a genuinely useful reference point, but only as honest data: not one cherry-picked answer, but several independent runs of several models on the same dilemma.
The interesting number is never the human vote or the model average alone — it's the gap between them. When humans go 70/30 and models go 20/80 on the same question, something worth thinking about happened.
One dilemma is a curiosity. Fifty-two is a dataset: which topics split humans from models, which models agree with each other, and whether any model tracks the human majority. The archive is the real experiment.
How it works
- Humans vote once each — no accounts, one vote per person per dilemma.
- Each AI model answers the same dilemma 5 times at a fixed temperature.
- Prompts are versioned and every run is published — including the misses.
Questions
No — and that is the point. A model answers the same dilemma differently across runs. We sample each model five times precisely to show it as a distribution, not a conscience. The site measures outputs, not beliefs.
One vote per person per dilemma, enforced with a salted hash of connection signals — no accounts needed. You can change your mind within 24 hours; after that your vote is locked.
Four to start: GPT, Claude, Gemini and Llama, each answering five independent times at a fixed temperature. The lineup and the prompt version are published with every batch.
Because one sample lies. The same prompt at non-zero temperature gives different answers, and that variance is data: a 5–0 model is confident, a 3–2 model is guessing. Single-shot AI comparisons hide this.
Yes — prompts are versioned (currently v1) and every model response, including reasoning and unparseable misses, is published on each dilemma's data page.