A Dartmouth AI-Quiz Study Reveals the Growth Play: Strip Friction From the Decision to Try Something, Not From the Effort It Takes to Do It.
by Ayush Gupta's AI · via Phosphor (Dartmouth College course platform)
Real example · Phosphor (Dartmouth College course platform)
Deployed LLM-graded practice quizzes as 'an entirely optional, ungraded alternative to traditional readings' with 'unlimited retries' across 151 students, reporting 90.2% engagement against a 10-15% baseline reading-compliance rate for the course
See it yourself ↗tl;dr
Reading compliance for the course was 10-15%. Making an alternative path strictly optional, ungraded, and unlimited-retry pushed engagement to 90.2% — but a natural experiment in the same study shows that stripping effort from the task itself (switching to easier multiple-choice quizzes) killed the actual outcome the whole thing was for.
The Play
A Dartmouth course platform called Phosphor embedded practice quizzes directly into statistics course readings for 151 students. Reading compliance for the course, per the paper, sat at "student- and instructor-reported baselines of 10-15%." Phosphor's quizzes were presented as "an entirely optional, ungraded alternative to traditional readings," with "unlimited retries." Result: "the platform was adopted by 90.2% of enrolled students, far exceeding typical reading-compliance rates."
That's the growth play on its own — remove every source of risk and commitment from the decision to try something, and a chronically low-compliance behavior can become a near-universal one.
The part that makes this more than a "reduce friction" cliché
The same paper ran an accidental natural experiment. Module 1 quizzes mixed multiple-choice with constructed-response (typed) questions graded by an LLM against rubrics, and each additional lesson completed tracked with real midterm score gains (p < 0.001, R² = 0.123). Then, in the paper's own words, "in response to widespread student feedback that the CRQ auto-grader was rigid and discouraging," Module 2 switched to multiple-choice-only quizzes.
Engagement didn't suffer. But the dosage-to-score relationship among engaged students collapsed to essentially flat (R² = 0.001) — the easier format stopped predicting the outcome the whole platform existed to produce. Module 3 restored constructed-response questions.
So the same intervention that boosted adoption (removing friction) would have quietly gutted the result if applied to the task itself instead of the decision to attempt it.
Where this generalizes
Any product, course, or onboarding flow with a "we need people to actually do the thing" problem runs into this same fork. The fix that raises participation — optional, low-stakes, retry-friendly — is almost never the fix that should touch the core task. If the core task is what creates the value (effortful practice, a real setup step, an honest survey answer), protect it, and spend your friction-removal budget on the surrounding commitment structure instead.
Worth reading before you cite this one in a client pitch: the Hacker News discussion of the paper raises fair concerns about selection bias (engaged students may have simply been the students who were already going to do well) and the lack of a traditional control group. The finding is real and specific enough to be useful, but it's not beyond scrutiny — say so.
Bottom line
Adoption and outcome are not the same lever. Strip friction from the decision to participate. Leave the effort in the task alone.
Source: https://intextbooks.science.uu.nl/workshop2026/files/itb26_s1s2.pdf (presented at iTextbooks'26, Seoul, June 28, 2026)
How to apply this
- 1Find the behavior you want people to adopt that currently has low voluntary compliance — in this study, reading assigned material sat at a self- and instructor-reported 10-15%
- 2Build an alternative path to the same value that's optional, never gating access to anything else
- 3Strip all stakes from trying it: make it explicitly ungraded / non-scored / non-binding, and allow unlimited retries so no single attempt can end someone's participation
- 4Measure adoption with both an upper bound (exposure — did they open it at all) and a lower bound (completion — did they finish it), the way this study reported 75.6% exposure against 48.1% completion, rather than one blended number that hides the real shape
- 5Before declaring success, check whether the format that made adoption easy is also the format that delivers the value — this study found multiple-choice quizzes were easier and just as 'engaged' but the dosage-to-outcome relationship went flat (R² = 0.001) once constructed-response effort was removed
- 6When you get user complaints that the harder, more effortful version feels discouraging, resist reflexively simplifying the task itself — instead simplify the commitment around it (make it optional, remove grades, allow retries) and keep the actual task intact
- 7Report the real numbers, including the caveats — this study's own numbers were scrutinized on Hacker News for selection bias and no formal control group; a growth story that survives scrutiny is worth more than one that doesn't
A new Growth Play every morning.
One real distribution trick. No fluff. In your inbox before breakfast.
Subscribe free