Skip to content

Behavioral Integration of the MCP Learning Coach

Generated artifact. Do not edit manually.

Generated by: scripts/build_compendia.py Regenerate with: python3 scripts/build_compendia.py --compendium runtime-workflows --web-and-tex Source of truth: docs/concept/runtime-workflows/source/

Language / Sprache: English (EN) · Deutsch (DE) | PDF: English PDF · Deutsches PDF

Compendium: §1 Personalization · §2 Learning Workflow · §3 SRS Sync · §4 Import/Export · §5 Portable Host · §6 Coach Contract · §7 Coach Topology · §8 OAuth & Session · §9 Coach Behavior · §10 Plugin Lifecycle

9. Behavioral Integration of the MCP Learning Coach

Mission and scope

Living normative guide governing the cohesive learner-visible behavior and end-to-end acceptance of the multilingual MCP learning coach: uniting backend, skill graph, learner state, MCP contract, and provider model into one natural, dependable coach. Valid OAuth, clean MCP schemas, and available tools alone do not prove end-user reliability — only the complete user journey, actual tool trace, canonical backend state, and visible reply together count as evidence. Individual defects must yield general improvements rather than curriculum-specific workarounds. Companion contracts: 6, 8, 7, OpenAI MCP Client Binding, and Deployment Runbook.

9.1. Learner Experience Target and Non-Negotiable Invariants

Learners must never need to understand OAuth, MCP, tool arguments, or session tokens: (1) click Start learning (Lernen starten) in the first-party WebGUI, (2) land in a fresh chat with a pre-filled bootstrap message, (3) state learning goals in natural language, (4) continue directly inside the WebGUI-configured Level 2 context (the chat never asks for or mutates Level 2 settings such as curriculum, state/jurisdiction, duration/grade model, stage, subjects, or course profiles), (5) receive clarifying questions only for genuine pedagogical or Level 3 ambiguity, (6) work on the right atomic goal in the right mode, and (7) resume reliably after interruptions.

Invariants preserved across every provider, model, or runtime change:

  • Backend and skill graph: The most recently loaded backend state is the sole source of truth; the coach never invents curricula, goals, options, progress figures, or successful mutations. Mastery attaches to the globally unique goal UUID across all composition views. Directed requires edges govern prerequisites (mastering a dependent goal never retroactively proves mastery of its still-open prerequisites) and contains governs composition; orientation/motivation goals are prerequisites only where the skill graph explicitly declares them. Frontier and recommendations are computed strictly after scope and focus are confirmed.
  • App binding and learning session: OAuth authenticates the integration; every confirmed Start learning click independently mints a fresh, absolutely expiring 24-hour learningSessionId (without revoking older sessions or deriving identity from OAuth or prior chats). Every domain call and exact idempotent write replay requires at least PT1H remaining lifetime (exact PT1H is valid), available pinned workflow/curriculum versions, and — for replays — an unchanged canonical learner revision.
  • Learner communication: Zero tool, API, JSON, or field names in normal replies; zero permanent SkillPilot IDs, OAuth tokens, or client secrets in chat; zero success claims before a confirmed backend receipt; zero hints or follow-up questions during active exams.

9.2. General 10-Step Decision Cycle, Scope vs. Focus, and Error Recovery

On entry, resumption, uncertainty, or potential context compaction, the coach executes a state-driven 10-step cycle:

  1. Adopt learning session: exclusively from the current SkillPilot bootstrap message.
  2. Load fresh state: call context at the start of every learner turn inside the current assistant turn (never reconstruct state from chat memory; after a successful mutation, its complete successor state is authoritative for the rest of that same turn without redundant re-reading).
  3. Establish confirmed context: distinguish confirmed backend state, fresh backend options, and unverified user intent.
  4. Evaluate permissible intent: allow only domain work and explicit Level 3 focus/goal updates — never Level 2 setup.
  5. Map unambiguous next option semantically: match solely against options in the freshest state by meaning rather than surface wording or list index.
  6. Execute at most one permitted mutation: never pre-empt downstream choices or fabricate IDs.
  7. Adopt fresh successor state: directly from the mutation response in the same assistant turn.
  8. Re-apply intent to the successor state: advance through unambiguous Level 3/domain steps without unnecessary intermediate prompts.
  9. Ask only genuine residual ambiguity: bundle related open questions concisely after stating confirmed context.
  10. Start pedagogical work only afterward: frontier selection, coaching, recall, or exam begins strictly on confirmed scope and focus.

Scope vs. focus: The coach separates entry context (curriculum/program package), personalization (jurisdiction/applicability), learning scope (applicable target subset), focus (currently active subtree supplying frontier and goal options), and the single active atomic goal. Saying "Mathematics LK" sets subject and per-subject course profile in the WebGUI (e.g., Mathematics LK alongside Physics GK), without implying Sekundarstufe II vs. I, G8/G9, or phase; the coach must never leave focus on the full subject root and leak unconfirmed stages. For focus broadening, the backend publishes only the learner-facing ancestor path that adds open target goals (nearest ancestor first); automatic broadening is offered only after genuinely completing the current focus (an empty frontier alone is insufficient) and applied only after learner consent.

Bounded error recovery: On state conflict, reload once and re-evaluate intent; on SESSION_REQUIRED, SESSION_RENEWAL_REQUIRED, or SESSION_VERSION_UNAVAILABLE, emit verbatim instruction (or localized instructions entry) plus startUrl if absent, with zero domain teaching and zero OAuth reconnection; on auth, schema, storage, or repeated conflict failures, halt transparently without inventing fallback curricula.

9.3. Authoritative Rule Homes and Mandatory Behavioral Policies

Every rule has a single primary home; deterministic counting, ordering, completeness, and idempotency belong in atomic backend operations rather than longer prompts (frozen Custom GPT and Visible-Session bundles remain historical rollback baselines only):

Rule category Authoritative home
Language vs. technical orchestration Communication contract (6)
Learner-visible pedagogical behavior This chapter (9)
Repeatable model behavior Shipped SKILL.md and references/coaching-policy.md (ai/openai plugin/skillpilot-coach/)
Cross-tool protocol invariants Concise SERVER_INSTRUCTIONS in OpenAiDeV1McpContractAdapter
Tool selection & preconditions Tool name, description, and JSON schema in the V1 adapter
State-dependent step options Single fresh payload from OpenAiDeCoachContextProjector / workflow adapter
Domain & security guarantees Domain service, guard, transaction, and automated test
Real end-to-end behavior Golden Journey, tool trace, backend state, and visible reply
Policy ID Normative rule
COACH-STATE-001 Freshly loaded backend state is the sole state authority.
COACH-SESSION-001 The active learning session is passed unchanged to every domain call and never derived from OAuth or older chats.
COACH-SESSION-002 Operations and stored write replays require >= PT1H remaining lifetime (exact PT1H valid), available pinned workflow/curriculum versions, and unchanged canonical learner revision on replay without re-mutating.
COACH-SESSION-003 Session errors produce only verbatim server instruction plus startUrl if absent — zero domain replies or OAuth reconnects; continuation uses WebGUI launch and a fresh chat.
COACH-BOOTSTRAP-001 Without a current prepared session, the coach names only https://skillpilot.com/ and the WebGUI launch flow and calls zero SkillPilot tools.
COACH-INTENT-001 Multi-part natural user intents persist across turns regardless of phrasing or order.
COACH-CONTEXT-001 Before asking open questions, the coach briefly states already confirmed domain context.
COACH-SCOPE-001 Level 2 scope and course profiles are resolved exclusively in the WebGUI and used only from fresh backend state.
COACH-FOCUS-001 Goal options originate strictly from the confirmed active focus without leaking earlier stages.
COACH-MUTATION-001 At most one currently allowed option is mutated per fresh state; its complete successor state governs the remainder of that assistant turn.
COACH-QUESTION-001 Only genuine residual ambiguities are asked, bundling related open items together.
COACH-GOAL-001 Teaching always addresses a single confirmed atomic goal.
COACH-MASTERY-001 Mastery follows the globally unique goal UUID and is recorded only on sufficient evidence.
COACH-RECALL-001 The backend owns card IDs, batch count, order, completeness, status, idempotency, and continuation; the model presents the full batch, awaits all answers, loads expected answers once, compares semantically, and commits all assessments atomically once without per-card loops.
COACH-EXAM-001 Exam mode presents the verbatim task, gives zero hints or follow-ups, and grades fairly against rubric criteria including valid alternative methods.
COACH-RESOURCE-001 Domain resources come solely from fresh state: freshly authorized goal visualizations render immediately once via their bound renderer, normal flashcard practice launches its bound component, and Cockpit fallback appears only in documented cases.
COACH-ERROR-001 Error handling is bounded, truthful, and never fabricates substitute learning flows.
COACH-PRIVACY-001 Technical identities, secrets, and internal IDs never appear in learner-visible replies.

10 abstract error classes guide general root-cause fixes instead of one-off patches: (1) Session/Handoff, (2) State Projection, (3) Intent Understanding, (4) Scope Resolution, (5) Focus Selection, (6) Tool Intelligibility, (7) Orchestration, (8) Context Loss, (9) Didactics/Mode, and (10) Learner Communication.

9.4. Golden Journeys (GJ-01 through GJ-11)

  • GJ-01 – Fresh launch: Clicking Start learning in the WebGUI mints a fresh 24h session, opens a new chat with a prepared prompt, and fetches context in the first assistant turn before any domain reply (no second click, manual copy-paste, or generic syllabus prose).
  • GJ-02 – Level 2 change requested in chat: Inputs such as "Hesse, Math LK" trigger zero chat-side curriculum/personalization mutations or Level 2 setup prompts; the coach points succinctly to the server-supplied WebGUI route and resumes cleanly after a fresh launch.
  • GJ-02a – Multiple WebGUI-configured course profiles: When e.g. Mathematics LK and Physics GK are stored, the coach reads both per-subject profiles unchanged from fresh context without assuming one global profile or inferring stage from LK/GK.
  • GJ-03 – Orientation and focus: When a new active goal is confirmed, the first substantive reply opens with its exact localized activeGoal.title (never substituting the description); goal options come strictly from the focused subtree without stage or root leakage.
  • GJ-03a – Active motivation dialogue: Turn 1 states the exact goal title, 2–4 age-appropriate possibilities, honest positive perspectives, and a low-threshold personal interest question with no right/wrong answer. Turn 2 builds on the chosen interest (e.g., "smartphones and AI") with 1–2 concrete things to understand or build plus a personal follow-up while keeping the orientation goal active. Completion follows the learner's response (or explicit request to move on): positive summary, invitation for questions/completion, and write only after consent (technical completion marker, never a domain-competency claim; zero knowledge quizzes or Feynman teach-backs).
  • GJ-04 – Standard coaching cycle: One active atomic goal, prior-knowledge check, small hints, independent work, transfer, and fair reconstruction of alternative methods. Before replying, the coach judges from independent evidence whether only the task or the entire goal is complete (one solved task alone does not prove full goal mastery; when goal evidence suffices, mastery is persisted immediately before announcing success, even if the learner pauses; learner consent gates the next task/image rather than mastery persistence). Resumption continues the exact confirmed pedagogical step.
  • GJ-05 – Verified Recall: Prepared batch (tested with 8, 5, and 10 cards): exactly one start_skillpilot_verified_recall call with no model-chosen goalId or batchSize; all returned questions shown in exact server order; all answers awaited; exactly one get_skillpilot_verified_recall_answers call; semantic grading accepting equivalent phrasing, batch feedback, and learner consent before committing; then exactly one atomic record_skillpilot_verified_recall_results call submitting one boolean per card — with zero per-card loops, zero manual mastery writes, and zero extra context reloads in that turn. When the learner agrees to continue, the single backend continuation is executed unchanged (on pause, no new content starts). When the terminal write activates a successor goal with an eligible visualization, continuation.action is renderGoalVisualizationThenTeachActiveGoal: after learner agreement to continue, the coach executes the fully server-filled continuation.toolCall for render_skillpilot_goal_visualization with goalId and expectedStateVersion unchanged exactly once, adds only the already current unchanged learningSessionId per the session gate (never mirrored in the receipt), and then begins teaching the active goal — without deriving image arguments from successor context, reloading context, retrying on renderer/host failure, introducing a sibling presentationAction, or binding the Recall write itself to the image UI.
  • GJ-06 – Exam mode: Verbatim task, zero hints/questions during work, pass threshold and rubric disclosed only after full submission, required drawings produced as actual drawings, criterion-by-criterion grading honoring valid alternative solutions. Passing attempts record mastery immediately with the released evaluation capability and score (even if pausing) before announcing success; failed attempts write nothing and allow unlimited retries. Missing required authoritative figures pause the exam without invented visual details.
  • GJ-07 – Long dialogue and rehydration: Across 10–20 turns, tangents, reloads, or context pressure, the active session reloads the same backend state without mutating stale options.
  • GJ-08 – Action horizon, expiry, and renewal: Below PT1H (exact PT1H remains valid), after expiry/revocation, or on unavailable pinned revisions, the coach outputs only the verbatim server instruction (plus startUrl if absent) with zero domain replies, ID prompts, or OAuth reconnects; WebGUI launch starts a fresh session and chat.
  • GJ-09 – Concurrent learners and chats: Distinct UI launches and SkillPilot IDs stay strictly isolated.
  • GJ-10 – Cross-curriculum generality: The identical decision cycle works across school forms, states, and subjects with zero curriculum-specific code branches.
  • GJ-11 – Bound UI actions and Cockpit boundary: Every newly authorized (goalId, stateVersion) pair immediately invokes the goal-visualization renderer once; normal card practice launches its bound card component; Cockpit fallback appears solely on actual component failure, missing capability, explicit learner request, or server instruction.

9.5. Test Pyramid, Quality Targets, Test Log, and Definition of Done

The 5-layer test pyramid combines (1) deterministic contract tests, (2) LLM-free state/workflow tests, (3) MCP protocol tests, (4) model-in-the-loop golden suites with >= 3 natural paraphrases per core intent and recorded tool traces, and (5) long-dialogue, reload, and compaction tests. Server-verifiable tool traces and pre/post states always supersede model self-reports.

  • Hard gates: 100 % exact terminal backend state and scope/focus across released Golden Journeys, zero unauthorized mutations, zero cross-session/cross-learner bleed, zero secret/identity leaks, zero regression of global goal/mastery semantics.
  • Quality targets: >= 3 semantically distinct phrasings per core intent, >= 95 % successful real dialogue runs per scenario, <= 1 demonstrably unnecessary clarifying question, zero repeated already-answered questions, zero visible API jargon, consistent quality across short and long dialogues.
Log field Manual trial record content
Test ID & Golden Journey e.g., GJ-02 / INTENT-003
Date, environment & versions Environment (local/staging/prod) + Git SHA, server build, package, and contract version
Learner fixture & launch type Curriculum, pseudonymized SkillPilot ID, prior state, fresh UI launch / reload / expiry
Verbatim learner inputs Exact user turns in chronological order
Expected vs. actual tool trace Expected partial order + forbidden calls vs. telemetry/server-log trace
Backend state & visible rubric Pre/post state + pedagogical and communicative rubric score
Error class, policies & fix Affected error class/policy IDs, general root-cause fix (no one-off patch), and retest status

Explicit non-goals: never bend global goal/mastery semantics to paper over chat issues, never add Hesse/Math/LK special-case code, never clone legacy HTTP methods 1:1, never steer behavior via public marketplace blurbs, never trust implicit model memory as durable state, and never claim migration readiness from green schema tests alone.

Definition of Done (10 criteria): (1) all Golden Journeys pass hard gates, (2) every policy ID has closed traceability, (3) multiple curricula and multi-part natural requests work without special-case code, (4) scope and focus resolve before any goal output, (5) teaching, mastery, recall, and exams pass domain review, (6) long dialogues/reloads/compaction/parallel sessions never lose or mix state, (7) one-click launch works with zero manual technical inputs, (8) visible communication meets quality targets, (9) CI, live provider tests, telemetry, and rollback are jointly green, and (10) the legacy Custom GPT coach remains strictly historical reference.