# Edu lab: gaps in the LessonKit model, found while running it outside primary maths

Context: `engine.js` is a small JS reading of LessonKit (docs in `lessonkit/docs`, `lessonkit-novus/docs`,
`lessonkit-bara/docs/pack-vocabulary.md`). It runs the five real lessons unchanged (`lessons/<id>/lesson.json` is a
byte copy of `data/lesson.json`) plus four sketch lessons in the same shape. Everything LessonKit keeps as
prose, the lab had to supply in a sidecar `lessons/<id>/lab.json` (checks, manipulatives, hints, solve scripts).
The size of that sidecar is the main finding: it is the part of a lesson the kit cannot run today.

## Top findings (ranked by how much they blocked non-maths lessons)

1. **Constraints are prose, so nothing can evaluate them outside the candy world.**
   `constraints[].satisfaction` is English or Vietnamese text ("Every group contains exactly six objects…").
   Only `lessonkit_world_flow.conditions` (`all_equal`, `lt`, …) are machine predicates, and they speak only
   container counts. Every check in all nine lessons had to be hand-written as a lab expression.
   *Proposal:* add `constraints[].check`, a typed predicate from a small grammar that extends the pack
   vocabulary beyond counts: answer slots (`answer.numbers[1] < param.b`), token identity per container
   (`container.w_horse.types == [nga]`), slider values with tolerance, and recorded series (see 5). Keep the prose
   as the teacher-facing label. The verifier can then check that every live constraint is evaluable.

2. **Four of five real lessons can never report mastery.**
   Only `answer_canonical {quotient, remainder}` is auto-scorable (assessment.md). `bang-nhan-6`,
   `chu-vi-hinh-chu-nhat`, `gap-len-giam-di-nhieu-lan` and `dong-co-dien` have no canonical items, so by the
   documented rule mastery is `null` forever. Even `phep-chia-co-du` T2's authored answer `6, product 54, dư 4`
   does not parse with the kit's own grammar. The lab scores them with an explicit, labelled `lab-rubric`.
   *Proposal:* typed `answer_kind` per item: `number` (tolerance, unit, comma decimal), `pair`/`coordinate`,
   `choice`, `text_set` (NFC-normalised, with accepted orthographic variants such as `khỏe`/`khoẻ`),
   `expression`, and `explanation` (rubric). Explanation items are scored by a model or teacher, and the evidence
   is stored as **provisional** until a teacher confirms it.

3. **Choices have no options.** Steps declare `choices: ["s2.predict", "s2.explain"]`, and `lesson.view` promises
   `choices (id, text, outcome)`, but no lesson field holds option text, correctness or the misconception an option
   reveals. Feedback is a single `<choice>.feedback` line, the same for every option.
   *Proposal:* `choices[] = {id, options: [{id, text, correct, misconception, feedback_cue}]}`. Wrong options that
   name a misconception are the cheapest diagnostic LessonKit could have, and they work in every subject.

4. **Completion evidence can dead-end a step.** S2 completion evidence is `first_try` in three real lessons, and S1 evidence is
   `not_scaffolded`. One wrong prediction, or one bottom-out hint, makes the step uncompletable. The exits that
   the prose describes ("after the correction gate", "after three submissions route to teacher review") are not
   data. The lab adds retry rounds and then a logged teacher-review exit.
   *Proposal:* a declarative `exit` on each step: `evidence | after(n attempts) -> retry_variant | teacher_review`.
   Add a verifier check that every step has an exit for a learner who fails the first try.

5. **Continuous and multi-variable worlds don't fit.** `experiment` handles one role against a threshold and a target
   (the motor). It cannot express: fitting three sliders to a target curve (parabola), a measured series and a
   law found from it (Ohm: record U and I, discover I ∝ U), or a liquid parameter that changes other objects'
   behaviour (salt changes whether the egg floats).
   *Proposal:* `observations` (named recorded series) plus predicates over them (`proportional`, `monotone`,
   `within(target, tol)`). Sliders become declared `controls` with ranges, and the evidence rule can require "≥ 3
   points, R held constant" (that is constraint C4 in `dinh-luat-om`).

## More gaps, each with a proposal

6. **Language and phonics need listening and speaking, not just reading.** A tone lesson is about hearing. The kit has
   tutor speech but no *stimulus* audio (the word to discriminate, separate from tutor talk), and no spoken
   learner input. The hỏi/ngã confusion is regional (the two tones merge in Southern speech), so the
   misconception depends on the learner. *Proposal:* `stimuli[]` (TTS or recorded words, replayable, not counted
   as tutor talk), `input_kind: speech` with the ASR transcript and confidence recorded as evidence, and an optional
   learner-profile key (dialect) that picks the line or scaffold variant.

7. **Lab safety is not a misconception.** In `dinh-luat-om` an over-current choice must stop the experiment (the fuse
   blows) whatever step is live, and must never earn evidence. *Proposal:* a `safety` constraint class. It is always
   live, blocks the world (gate), and its cue preempts all others (priority above any scheduler cue). The verifier
   checks that no evidence path passes through an unsafe state.

8. **Open-ended explanations (S2 "explain", S4 "diagnose") can't be judged.** The method relies on explanation, and every
   lesson has items like "Bạn sai ở đâu? Sửa lại". *Proposal:* the rubric item kind from (2), graded through the
   existing `reply_policy` path. Show the teacher the grade and its reason in a review queue.

9. **The phase vocabulary is ambiguous.** The owner's brief says S2 is productive failure. All five lessons use S1
   `productive_failure`, S2 `prediction_lock` and S3 `causal_insight` (consolidation). *Question for LessonKit:* which
   names are canonical? Publish one table with phase name, intent and a typical tutor act.

10. **The same line repeats.** On the world path the same success cue fires on every item (`s3.law` four times in a row in
    Ohm; `s3.check` in division). `cue_scheduler` has cooldown and max-per-session, but the world path does not run
    it (architecture.md §8). *Proposal:* run the scheduler on every path, and allow `lines[]` variants per attempt
    number (first success, repeated success).

11. **The counter-example disappears.** Items that exit after one attempt (S0 traps) move on and reset the world
    immediately, so the learner never looks at their own wrong state. POE needs an *observe* beat. *Proposal:* a
    step-level `hold` or `observe` action that keeps the world frozen until the learner continues. It is short,
    and it is data.

12. **Outcome cues are named inconsistently.** S4 uses `s4.item.done` in one lesson and `s4.next` in others, and
    `s4.pass`, `s4.mastery` or `s4.review` in others. The engine has to guess with a regex. *Proposal:* declare
    `mastery_test.cues = {item_done, passed, failed}`.

13. **Numbers are hard-coded into lines.** Three real lessons have no params, and lines say "Với 10 vật này", "4 nhóm,
    mỗi nhóm 6 vật". A world cannot vary the numbers (for practice, spacing or retrieval) without new text.
    *Proposal:* slots in every line that names a quantity, and a `variants` generator for params. FSRS review
    needs fresh numbers, not replayed items.

14. **test_items evidence is sourced in a non-test step.** In `dong-co-dien`, E2 and E3 are `test_items` with
    `step: S3`, but T2, T3 and T5 are S4 mastery items. *Question for LessonKit:* do S3 practice items with the same
    id count? The lab reads "any attempt on that item id, in any step", which lets S3 practice satisfy an S4 rule.

15. **Hints are a method feature with no lesson data.** The method pack ships a "hint ladder", but lessons author no
    rung text, and only misconception `scaffold` prose exists (English in four lessons, so it cannot be spoken).
    *Proposal:* `hints[]` per step or item, with rung kinds (`reask`, `focus`, `worked_example`). The last rung
    marks the attempt scaffolded. The verifier checks that the hints are Vietnamese and cover the live
    misconceptions.

## Ambiguities resolved by the simplest reading (questions for LessonKit)

| Topic | Lab reading |
|---|---|
| Step completion | `completion_evidence` earned **and** every lab item answered; otherwise all step-sourced evidence earned; the terminal step completes when every mastery item is answered |
| `consecutive(n)` | longest run of qualifying items, items ordered by first attempt |
| `first_try` in E3 "predict **and** explain on first try" | evaluated per item; `at_least(1)` over the step's items, so one first-try item suffices |
| Cue precedence | world `opening` variant for the step id, else `default`; else the step's prompt cue (first non-choice cue) |
| Violation cue | item override, `<step>.<cN>.violated`, `<step>.<cN>`, `s1.<cN>.violated`, `w.<cN>_violated`; else silent (the kit never invents a line) |
| Gate-locked submit | not an attempt; speaks the gate cue |
| Hint ladder | rung 1 re-ask, rung 2 shows the misconception card, rung 3 bottom-out (marks the next attempt scaffolded and may run an assist) |
| Mastery undecidable | the kit says `null`; the lab scores with an explicit `lab-rubric` and labels it as such |

## What worked across subjects

- S0 trap, S1 try first, S2 commit to a prediction, S3 name it, S4 no hints. This order was natural for all four new
  subjects, including tones (hear before you are told), floating (predict "heavy sinks"), Ohm (bigger R is
  not bigger I) and parabolas (a is not a shift).
- Misconception → constraint → cue line is a strong spine. It made the tutor's speech specific in every subject
  once checks were machine-evaluable.
- Evidence earned only from observed attempts, never from tutor words, kept the researcher panel honest. You can
  watch a scaffolded success fail to count.