AI tutoring, autograding, and language apps are shipping to millions of students — while AI-cheating detectors keep getting disabled. A grounded look at what works, what doesn't, and the boundary that explains both.
When the Los Angeles Unified School District launched "Ed," a district-wide AI chatbot meant to guide 500,000+ students and families through everything from grades to enrollment, it was pitched as the future of personalized education. Within months the vendor behind it, AllHere, had furloughed most of its staff amid financial collapse, and the district quietly pulled the tool. The reported cost was in the millions.
That story is the honest starting point for any discussion of AI in education, because it captures both halves of the reality: the demand is real, the money is flowing, and the deployments keep tripping over the same problems — reliability, oversight, and the gap between a demo and a system that 30 kids can use unsupervised.
This post walks through the four areas where AI has actually landed in classrooms and edtech products — tutoring, grading and feedback, language learning, and academic-integrity detection — with real named products, how each actually works under the hood, and where each currently falls short.
Not all four areas are at the same stage. Here's the honest snapshot before we dig in:
| Application area | Maturity | Representative products | The main thing holding it back |
|---|---|---|---|
| Socratic tutoring | Piloting at scale | Khanmigo, Google LearnLM, MATHia | Cost per student, math-reasoning errors, needs teacher oversight |
| Grading & feedback | Real but narrow | Gradescope, Turnitin, Writable | Trusted for structured work, shaky for open-ended essays |
| Language learning | Shipping to millions | Duolingo Max, Speak | Feedback quality vs. price; "AI-first" backlash |
| AI-writing detection | Fundamentally unreliable | Turnitin, GPTZero | High-stakes false positives; no way to prove authorship |
The pattern: AI works best where the output is bounded and checkable (a math step, a code test case, a pronunciation score) and works worst where it has to make a subjective, high-stakes judgment about a human (did this student cheat, is this essay a B+).
The flagship deployment here is Khanmigo, Khan Academy's tutor built on GPT-4 and launched in 2023. Google followed with LearnLM, a family of models tuned specifically for pedagogy and now folded into Gemini for Education. Carnegie Learning's MATHia predates the LLM wave with a more classical intelligent-tutoring-system approach.
The interesting engineering problem isn't "can the model answer the question" — GPT-4 can solve most middle-school math. The problem is that a tutor must refuse to answer. If a student asks "what's the answer to problem 4," a good tutor asks a leading question back. This is entirely a prompting-and-guardrails discipline layered on top of the base model:
System: You are a tutor. NEVER give the final answer directly.
Ask one guiding question. If the student is stuck after
two hints, work through ONE analogous example with
different numbers, then return to their problem.
Tool: check_answer(student_work) -> {correct: bool, step: int}
The check_answer tool matters more than it looks. Left to free-form generation, LLMs still make arithmetic and algebra errors mid-explanation — confidently. So production tutors offload the actual correctness check to a deterministic solver or a symbolic math engine, and use the LLM only for the conversation. The model decides what to say; a real calculator decides whether it's right.
Where it falls short. Three things. First, cost: a chatty multi-turn tutoring session burns far more tokens than a single Q&A, and at a few cents per session across millions of students, the unit economics force aggressive model downsizing that degrades quality. Second, the model still hallucinates pedagogy — inventing a "rule" that doesn't exist, or leading a student down a wrong method. Third, and most underappreciated, unsupervised tutoring assumes a motivated student; the kids who most need a tutor are often the ones who will type "just tell me" until the guardrail cracks. Every serious deployment now positions the AI tutor as a teacher-in-the-loop tool, not a replacement.
Autograding is the oldest and most reliable AI-in-education category, and it's worth separating two very different tasks that get lumped together.
Structured grading — multiple choice, code that must pass test cases, math with a checkable answer — is essentially solved and has been for years. Gradescope (now part of Turnitin) uses a mix of computer vision and answer-grouping: it clusters visually similar handwritten answers so an instructor grades one representative and the rubric propagates to the group. That's a genuine time-saver and it's trusted because the human still sets the rubric.
Open-ended essay scoring is a different animal, and this is where the honesty has to come in. Automated Essay Scoring (AES) systems can correlate reasonably well with human graders on aggregate — but they've been repeatedly shown to be gameable (stuff an essay with sophisticated vocabulary and sentence length and the score climbs, even when the content is nonsense) and to encode the biases of their training graders. LLM-based feedback tools like Writable (also Turnitin-owned) have shifted the framing accordingly: they market formative feedback ("here's what to improve") rather than a summative grade ("this is a 78"). That reframing is the right call, because low-stakes suggestion is exactly the regime where a plausible-but-imperfect model is useful and a wrong answer isn't catastrophic.
The rule of thumb that's emerged: let AI grade what has a verifiable answer, and let it suggest on what doesn't. The moment an AI-assigned essay score affects a transcript, you've moved into territory where the failure modes are unacceptable.
If you want the AI-in-education deployment operating at the largest scale today, it's Duolingo. Its Duolingo Max tier, built on GPT-4, added two features that are genuinely well-matched to LLMs: "Explain My Answer" (why was I wrong?) and "Roleplay" (open-ended conversation practice with an AI character). Language practice is close to an ideal LLM use case — the domain is conversational, mistakes are low-stakes, and a slightly-off response is still useful practice.
Duolingo also became the cautionary tale about organizational AI adoption. In April 2025, CEO Luis von Ahn's internal "AI-first" memo — including plans to reduce reliance on contractors for work AI could do — leaked and drew significant public backlash, prompting a follow-up clarification that AI would augment rather than replace staff. The technical takeaway is separate from the PR one: the product features work, but "AI-first" as a company posture collided with users' and workers' trust in ways the feature roadmap didn't anticipate.
Where it falls short. Conversational feedback is only as good as the model's ability to detect subtle errors — a wrong verb aspect, an unnatural-but-grammatical phrasing — and smaller/cheaper models deployed for cost reasons miss exactly these. And pronunciation scoring, still largely a specialized speech-model problem rather than an LLM one, remains the weak link in "talk to the app" features across the category (Speak, and others).
This is the category where the honest answer is: it doesn't reliably work, and it may not be possible.
Turnitin shipped AI-writing detection in April 2023 to a market of panicked educators. Within a year, universities including Vanderbilt, Michigan State, and others had disabled it — not because they didn't care about cheating, but because false positives were doing real damage, disproportionately flagging non-native English writers whose more formulaic prose reads as "AI-like." OpenAI itself quietly shut down its own AI Text Classifier in July 2023, citing low accuracy. GPTZero and similar tools persist, but the fundamental problem is unchanged.
The reason detection can't be made reliable is structural, not a matter of better models:
Watermarking research (e.g., Google DeepMind's SynthID for text) is the only technically principled path, but it only works if every model provider watermarks, the watermark survives editing, and detection is centralized — none of which holds across the open-weight ecosystem. The practical consensus among assessment researchers has shifted from "detect the cheating" to "design assignments that are robust to AI" — in-class writing, oral defenses, process portfolios, and work that requires personal or local context a model can't fake.
Look across all four areas and the same boundary keeps drawing itself. AI in education is genuinely working — Khanmigo, Gradescope, Duolingo Max are real products used by real students at scale — precisely in the places where the model's output is checkable by something other than the model: a symbolic solver confirms the math, test cases confirm the code, a rubric set by a human confirms the grade, a low-stakes conversation forgives an imperfect reply. It's failing, expensively and sometimes harmfully, exactly where it's asked to render an unverifiable high-stakes judgment about a person.
The takeaway for anyone building or buying edtech: ask one question first — what checks the model's output? If the answer is "a deterministic tool, a human rubric, or nothing important is riding on it," you're in the zone where AI in education actually works. If the answer is "we trust the model," you're building the next Ed chatbot.