Collaborative Trigger Routing: Teaching an AI Tutor When to Stop and Call a Human

It is eleven at night. A grade 10 student has asked the same physics question four different ways. Each time, the tutor has come back with another patient, well-formed Socratic prompt: what do you think happens to the force if the distance doubles? The student does not know. That is the whole problem. They did not know forty minutes ago and they do not know now, and the machine on the other side has no mechanism for noticing that its strategy has stopped working.

An AI tutor that never gives up looks like dedication. It is a failure mode. A model with no exit condition will keep generating plausible next hints forever, because that is the only thing it was asked to do — and nothing in the loop is measuring whether the hints land.

At pdpspectra we build PDP Shikshya, an AI learning platform for Nepali schools, and every feature in it is a defensive answer to something a school is actually afraid of. The fear behind this one is specific and reasonable: the AI will quietly replace the teacher, and nobody will notice a child is drowning. The answer is a feature we call Collaborative Trigger Routing — a rule that watches the tutoring session and, past a threshold, stops tutoring and routes the student to a person.

Productive struggle and being stuck are not the same thing#

The uncomfortable part of designing this is that difficulty is supposed to be there. Manu Kapur’s work on productive failure argues that withholding help early — letting learners generate wrong or incomplete solutions before instruction arrives — produces better conceptual understanding than smoothing the path from the start. A tutor that rescues a student on the first hesitation is not being kind; it is deleting the part where learning happens.

So the design cannot be “escalate whenever the student is having a hard time.” It has to distinguish two states that look identical from the outside:

• Productive struggle — the student is generating attempts. Each one is wrong, or partial, but each one is different. They are moving.
• Genuinely stuck — the student has stopped generating. They are repeating themselves, asking for the answer, or saying some variant of “I don’t know” over and over. The mechanism that would produce the next attempt has jammed.

Research on intelligent tutoring systems has been circling this for two decades. Aleven and colleagues’ review, Help Helps, But Only So Much, found that on-demand help is only useful when students seek it well, and that a large share of help-seeking in real systems is unproductive. Baker’s work on gaming the system documents the endpoint: students drilling hints at speed to extract answers, learning markedly less than peers who do not. The signal that separates the two states is not the presence of difficulty. It is whether attempts are still being produced.

What the telemetry gives us to work with#

Collaborative Trigger Routing does not need new instrumentation. It reads the same Proof-of-Thought event stream the teacher dashboard is built on — typed events emitted as a by-product of ordinary tutor use, covering attempts, hint requests, reflections, struggles, breakthroughs, and escalations. We wrote that layer up separately in Proof-of-Thought: making student thinking visible to teachers.

The routing logic sits at the end of every tutor turn, after the reply has streamed. Two pieces of state live on the tutoring session itself: a struggle streak counter and an escalated flag. Each turn, the student’s raw message is checked against a small set of struggle markers — “I don’t know”, “I’m stuck”, “I give up”, “still confused”, plus romanised Nepali equivalents like thaha chaina and garna sakina, because a bilingual product that only detects distress in English is not bilingual. A match increments the streak and writes a struggle event. Anything else resets the streak to zero and writes an attempt event.

That reset is the entire mechanism for telling the two states apart. A student who is genuinely working — even badly, even wrongly — keeps producing messages that are not distress signals, and the counter keeps going back to zero. Only consecutive capitulation accumulates. Three in a row and the trigger fires.

The trigger is a rule, not a judgement call#

The obvious alternative is to ask the model. Give it the conversation and a prompt: decide whether this student needs a human. We deliberately did not do that, and the reasoning generalises well past education.

A model asked to judge its own performance is being asked the one question it is worst at. Its self-assessment is not grounded in the student’s outcome; it is grounded in how coherent its own last reply looked, which is exactly what an LLM cannot help but rate highly. It is also non-deterministic, and it is unauditable. When a school asks why was my child flagged, “the model felt it was time” is not an answer anyone can act on.

A deterministic rule over telemetry gives the opposite properties. Three consecutive struggle signals in one session, and only once per session — the escalated flag makes it idempotent, so a student who stays stuck does not generate duplicate flags that train teachers to ignore the card. It is explainable in one sentence to a principal, testable without a model in the loop, and free per turn. When the school decides three is the wrong number, changing it is a constant, not a prompt-engineering exercise with unknown blast radius.

The honest limitation: keyword detection over the student’s own words is a coarse instrument. It catches the student who says they are lost, and misses the one who goes quiet and closes the tab. That is a real gap, and it is a gap we would rather have than a model quietly deciding on its own authority when a child has had enough.

Three destinations, because there are three kinds of stuck#

When the trigger fires, the tutor’s response frame carries the escalation alongside the reply — and it offers two routes, not one, both rendered in the student’s own language.

• Keep tutoring. The default, and the right call most of the time. Difficulty is the product working.
• Route to a peer. “Want to open a study-buddy invite for classmates who nailed this concept?” This suits the student who understands the material but has hit one specific wall a classmate cleared last week. The evidence for peer instruction is strong — a meta-analysis of peer tutoring in STEM subjects reports a large positive effect on achievement — and it is a route the AI cannot substitute for, because the value is partly that another fifteen-year-old explains it in fifteen-year-old language.
• Route to a teacher. “Let’s team up! Want to flag this for your teacher’s review session tomorrow?” This is for the stuck that is not about this question at all — a missing prerequisite from two chapters back, or something that is not academic. VanLehn’s review of human versus intelligent tutoring found the two remarkably close on measured outcomes, but the residual difference is exactly the diagnostic move a machine cannot make: noticing that the question the student asked is not the problem they have.

An escalation that lands in nobody’s queue is not an escalation#

This is where most human-in-the-loop designs quietly fail. The trigger fires, a flag is written to a table, and no human workflow ever reads that table: the appearance of a safety net with none of the mechanics.

So the escalation is a first-class object on the teacher’s side. The escalation event lands in the same telemetry the teacher dashboard aggregates, and a student is marked as needing attention when they have any escalation on record or three or more struggle signals. Flagged students sort to the top of the roster, and the class-wide flagged list is computed over the teacher’s full scoped roster before any search filter or pagination is applied — a queue that empties when you type in a search box is not a queue.

The card carries two actions that do real work, not two buttons that set a status:

• Route to physical review writes a durable note on the student’s record attributed to the teacher who raised it, then notifies the student and their parents that an in-person review has been requested. Three parties learn about it, in a channel each of them already reads.
• Create a peer study group provisions an actual group conversation in the school’s chat, adds the flagged student or students plus the teacher as members, and notifies everyone with a link straight into it. One click turns a flag into a room with people in it.

Both are permission-scoped the way the rest of the platform is: a teacher or department admin can only act on students in their own department, a school admin only within their own school, and never across tenants. Escalation paths are exactly where authorisation bugs become safeguarding incidents.

The general shape: handoff as a designed component#

Strip the education out and the pattern is the one every serious AI implementation needs. Any autonomous system operating on people’s behalf requires a defined condition under which it stops and hands off — and the handoff has to be engineered with the same care as the happy path.

The parallel we keep returning to is triage in a Hospital Management System. Nobody treats escalation there as the software admitting defeat; the escalation path is the clinical design. Vitals cross a threshold, the case routes to a named clinician, the alert lands in a queue somebody owns, and the whole chain is logged for audit. Nobody would ship a triage flow where the alert fires into a dashboard no one is rostered to watch. Yet that is what a great deal of AI tooling ships today — a confidence score, a fallback branch, and no human on the far end of it.

It is also the argument for building school and hospital platforms as data platforms with thoughtful UI on top, rather than as the sealed legacy ERP boxes that dominate both sectors. Collaborative Trigger Routing is only possible because the telemetry, the roster, the permission model, the notification fabric and the chat all live in one schema the routing rule can read and write across in a single request. In a School ERP where tutoring is one vendor’s silo and messaging is another’s, the escalation stops at the silo boundary — which is to say it stops.

The handoff is not the seam where your AI implementation failed. It is the part that makes the rest of it safe to deploy at all.

---

Building an AI system that needs a real human on the far end of it? Talk to our engineers at pdpspectra.com/#contact, or see how PDP Shikshya routes a stuck student at pdpshikshya.com.