Completion audit

Definition of Done

Build Aristotle into a complete website that satisfies DEFINITION_OF_DONE.md, including UK education coverage from EYFS through A-Level with age-scoped user experiences and running-app validation.

Readiness Validation evidence Curriculum audit Operations Audit JSON

not_complete completion gate

14/20 criteria met

6 open criteria

0 blocked criteria

5 partial criteria

1 missing criteria

Coordinator export

Definition of Done evidence bundle

Export a sanitized bundle with the audit scorecard, prompt-to-artifact checklist, release closure handoff, review request queue, routes and verification commands.

20 checklist items

6 open DoD items

5 closure gates open

Release closure cockpit

Remaining real-world gates

This closure handoff coordinates evidence; it does not replace real learners, external review, target-environment operations evidence or accepted validation records.

Open closure state

0/5 launch gates ready

5 launch gates open

Subject fundamentals

Subject expert review

0/768 concept(s) approved; 768 concept review(s) open; 126 subject scope(s) open.

Workspace

Next actions

Send 768 pending concept(s) to subject experts in traceable batches.
Start with A-Level / Mathematics, where 8 concept(s) remain open.

Validation returns

Subject expert validation return

Prepare the subject expert curriculum review workspace or pack, then create a validation record with reviewer, scope, outcome, risk and evidence links.

ready Workspace linked: A workspace is available for preparing this evidence pack.
ready Evidence pack linked: A JSON or workspace pack route is available for reviewer handoff.
ready Work order linked: A reviewer-facing work order or runbook is available for handoff.
ready Campaign linked: A campaign or recruitment pack is available for coordinating the gate.
ready Review brief linked: A reviewer-facing brief is available with scope, evidence links, guardrails and import controls.
ready Return template linked: A reviewer-return JSON template is available for this gate.
ready Registry/import route linked: A validation registry route is available for drafting records and importing returned decisions.
open Validation record created: Create or draft a validation record for this gate.
open External decision submitted: An accountable reviewer still needs to accept, return or reject the evidence.
open Gate-counting acceptance: No accepted record currently satisfies this gate's guardrails.

Beta Definition of Done

Real learner beta

0 included learner(s); 20 short of the 20 learner minimum; 0 over the 50 learner maximum.

Workspace

Next actions

Create a beta study before recruiting real learners.

Validation returns

Real learner beta validation return

Prepare the 20 to 50 real learner beta workspace or pack, then create a validation record with reviewer, scope, outcome, risk and evidence links.

ready Workspace linked: A workspace is available for preparing this evidence pack.
ready Evidence pack linked: A JSON or workspace pack route is available for reviewer handoff.
ready Study runbook linked: A reviewer-facing work order or runbook is available for handoff.
ready Recruitment linked: A campaign or recruitment pack is available for coordinating the gate.
ready Review brief linked: A reviewer-facing brief is available with scope, evidence links, guardrails and import controls.
ready Return template linked: A reviewer-return JSON template is available for this gate.
ready Registry/import route linked: A validation registry route is available for drafting records and importing returned decisions.
open Validation record created: Create or draft a validation record for this gate.
open External decision submitted: An accountable reviewer still needs to accept, return or reject the evidence.
open Gate-counting acceptance: No accepted record currently satisfies this gate's guardrails.

Production Definition of Done

Safeguarding and privacy sign-off

7 external review item(s) open; 0 open safety review event(s); 0 SLA breach(es).

Workspace

Next actions

Run the safety/privacy rehearsal and resolve any failed consent, redaction, export/delete or access-control check.
Configure the moderation SLA and resolve breached open caution or urgent safety events.
Resolve: Named safeguarding owner is configured (No safeguarding owner configured).
Resolve: Traceable safeguarding policy reference exists (missing).
Resolve: Moderation response SLA is configured (0 hour(s)).
Resolve: Named privacy/data-rights owner is configured (No privacy owner configured).
Resolve: Traceable privacy policy reference exists (missing).
Resolve: Traceable data retention policy reference exists (missing).

Validation returns

Safeguarding validation return

Prepare the safeguarding review workspace or pack, then create a validation record with reviewer, scope, outcome, risk and evidence links.

ready Workspace linked: A workspace is available for preparing this evidence pack.
ready Evidence pack linked: A JSON or workspace pack route is available for reviewer handoff.
ready Work order linked: A reviewer-facing work order or runbook is available for handoff.
ready Sign-off campaign linked: A campaign or recruitment pack is available for coordinating the gate.
ready Review brief linked: A reviewer-facing brief is available with scope, evidence links, guardrails and import controls.
ready Return template linked: A reviewer-return JSON template is available for this gate.
ready Registry/import route linked: A validation registry route is available for drafting records and importing returned decisions.
open Validation record created: Create or draft a validation record for this gate.
open External decision submitted: An accountable reviewer still needs to accept, return or reject the evidence.
open Gate-counting acceptance: No accepted record currently satisfies this gate's guardrails.

Legal/privacy validation return

Prepare the legal and privacy review workspace or pack, then create a validation record with reviewer, scope, outcome, risk and evidence links.

ready Workspace linked: A workspace is available for preparing this evidence pack.
ready Evidence pack linked: A JSON or workspace pack route is available for reviewer handoff.
ready Work order linked: A reviewer-facing work order or runbook is available for handoff.
ready Sign-off campaign linked: A campaign or recruitment pack is available for coordinating the gate.
ready Review brief linked: A reviewer-facing brief is available with scope, evidence links, guardrails and import controls.
ready Return template linked: A reviewer-return JSON template is available for this gate.
ready Registry/import route linked: A validation registry route is available for drafting records and importing returned decisions.
open Validation record created: Create or draft a validation record for this gate.
open External decision submitted: An accountable reviewer still needs to accept, return or reject the evidence.
open Gate-counting acceptance: No accepted record currently satisfies this gate's guardrails.

Production Definition of Done

Production operations sign-off

7 internal evidence item(s) open; 8 target-environment sign-off item(s) open.

Workspace

Next actions

Run a learning-data backup and keep the generated backup path with the release evidence.
Verify the latest backup can be deserialized without writing over live learner data.
Run the learner-facing route smoke check against the intended release build.
Run the adaptive-learning load check and confirm response timings stay within the release threshold.
Set ARISTOTLE_DEPLOYMENT_ENVIRONMENT to the named target environment this evidence applies to.
Set ARISTOTLE_PRODUCTION_URL to the HTTPS learner-facing URL and rerun smoke/preflight checks.
Add ARISTOTLE_MONITORING_DASHBOARD_URL for the external uptime, error-rate and latency dashboard.
Add ARISTOTLE_ALERTING_ROUTE_REFERENCE for the human alert route used during learner-impacting incidents.

Validation returns

Production operations validation return

Prepare the production operations evidence workspace or pack, then create a validation record with reviewer, scope, outcome, risk and evidence links.

ready Workspace linked: A workspace is available for preparing this evidence pack.
ready Evidence pack linked: A JSON or workspace pack route is available for reviewer handoff.
ready Work order linked: A reviewer-facing work order or runbook is available for handoff.
ready Sign-off campaign linked: A campaign or recruitment pack is available for coordinating the gate.
ready Review brief linked: A reviewer-facing brief is available with scope, evidence links, guardrails and import controls.
ready Return template linked: A reviewer-return JSON template is available for this gate.
ready Registry/import route linked: A validation registry route is available for drafting records and importing returned decisions.
open Validation record created: Create or draft a validation record for this gate.
open External decision submitted: An accountable reviewer still needs to accept, return or reject the evidence.
open Gate-counting acceptance: No accepted record currently satisfies this gate's guardrails.

North star metric

Verified north-star metric

1 metric integrity item(s) open; 14 real-world validation item(s) open.

Workspace

Next actions

Run the north-star metric integrity command and resolve any timing, formula, retention-chain or privacy failures.
Collect at least one baseline diagnostic or first-attempt evidence point before claiming learning gain.
Collect a later lesson check or mastery review so the study can compare before and after evidence.
Collect a delayed retention check for the same learner, subject and concept.
Build at least one complete baseline, post-test and retention chain before exporting the study pack.
Record an unscaffolded, non-safety retention success so retained learning is not inferred from supported work.

Validation returns

Retained-learning outcome validation return

Prepare the retained-learning outcome study workspace or pack, then create a validation record with reviewer, scope, outcome, risk and evidence links.

ready Workspace linked: A workspace is available for preparing this evidence pack.
ready Evidence pack linked: A JSON or workspace pack route is available for reviewer handoff.
ready Work order linked: A reviewer-facing work order or runbook is available for handoff.
ready Study campaign linked: A campaign or recruitment pack is available for coordinating the gate.
ready Review brief linked: A reviewer-facing brief is available with scope, evidence links, guardrails and import controls.
ready Return template linked: A reviewer-return JSON template is available for this gate.
ready Registry/import route linked: A validation registry route is available for drafting records and importing returned decisions.
open Validation record created: Create or draft a validation record for this gate.
open External decision submitted: An accountable reviewer still needs to accept, return or reject the evidence.
open Gate-counting acceptance: No accepted record currently satisfies this gate's guardrails.

Open work

What still blocks completion

Subject fundamentals

Concept maps and teaching content are expert-reviewed before production trust claims.

Evidence: 0 live expert approvals; 768 concepts still awaiting live review; curriculum audit expert-review-complete=False; expert-reviewed-only teaching gate active in this environment=False; production trust gate configured=True.

Gap: Complete expert review or block unsupported concepts from production claims.

Beta Definition of Done

20 to 50 real test students can use the system without manual intervention and produce useful learning outcome evidence.

Evidence: 0 accepted real learner beta participant(s) recorded in the validation evidence registry; beta workspace studies=0, included participants=0, ready studies=0.

Gap: Run an observed beta programme and record outcome evidence, failure patterns and remediation results.

Production Definition of Done

Privacy, child-safety, safeguarding and unsupported-input concerns are handled properly.

Evidence: Guardian acknowledgements, safety audit events, raw-answer minimization for safety-triggering inputs, safety log resolution, no-mastery safety handling, adult access audit, learner data export/delete and safety/privacy rehearsal checks exist; rehearsal ready=False, checks=0, failures=None; accepted safeguarding reviews=0; accepted legal/privacy reviews=0.

Gap: Needs external safeguarding/legal review, moderation ownership and production policy sign-off.

Production Definition of Done

Errors are monitored, backups exist, production smoke checks run and the system has reliability evidence.

Evidence: Open operational errors: 0; latest backup=False; restore drill=False; smoke=False; load=False; failure modes=False; rehearsal=False; accepted production-operations evidence=0.

Gap: Needs deployed external monitoring, alerting, production database setup, backup restore drills in the target environment and higher-volume load testing.

North star metric

The system reports verified concept mastery gained per student hour and supporting learning metrics.

Evidence: LearningEvidence records timed evidence, mastery-gain deltas and retention_check evidence; dashboards and analytics report mastery gained per tracked student hour, concept-level pre/post gain and delayed retention success; north-star metric integrity ready=False, checks=0, failures=None; retained-learning validation ready=False, open real-world validation items=8; accepted retained-learning outcome studies recorded=0.

Gap: Needs retained-learning outcome validation from real learners before the metric can be treated as reliable.

Failure conditions

The system avoids being a generic chatbot, shallow quiz tracker or unsupported subject generator.

Evidence: Structured curriculum maps, evidence dimensions, rubric signals, mastery thresholds, safety blocks, curriculum review gates and learner-facing trust-boundary warnings reduce generic tutoring risk. Failure-mode harness ready=False, source=ephemeral_self_check, checks=6, failures=1.

Gap: Run python manage.py run_failure_mode_checks --json --fail-on-error and resolve any failing unsupported-generation, shallow-mastery or safety-boundary checks.

Full checklist

Prompt-to-artifact mapping

Section	Requirement	Status	Artifact	Evidence	Gap
Beta Definition of Done	Admins and educators can inspect progress and analytics showing where students struggle.	met	core/templates/core/learning_analytics.html, core/analytics.py	Educator view, learner model, aggregate analytics, cohort trend rows, operations and records pages expose progress, struggle patterns and longitudinal learning signals.	Needs richer charting and school/cohort filters once real beta cohorts exist.
Comprehension measurement	Assessment uses recall, fluency, explanation, transfer, error correction, confidence calibration, teach-back and retention evidence.	met	LearningEvidence.evidence_dimension, LearningEvidence.rubric_signals, core/learning.py, /validation/	Evidence dimensions, stored rubric signals and mastery review tasks cover recall, procedural fluency, explanation, application, transfer, error correction, teach-back, confidence calibration and explicit retention_check evidence; accepted psychometric validations recorded=0.	Validated item difficulty, real delayed learner data and subject-specific rubric review remain external calibration gates.
Definition artifact	The repository has an explicit Definition of Done that governs Aristotle readiness.	met	DEFINITION_OF_DONE.md	DEFINITION_OF_DONE.md exists in the repository.
Learner model	Learner state is structured, inspectable, queryable and actionable rather than a chat-history blob.	met	core/models.py, core/learning.py, /learner-model/data/, /learner-model/questions/	StudentProfile, ConceptProgress, LearningEvidence, learner model pages, five-question report and learner-model JSON expose structured strengths, gaps, rubric signals, confidence, frustration, retention, learning gain and next actions; accepted psychometric validations recorded=0.	Psychometric validation and longitudinal learner studies remain separate external evidence gates.
Lesson quality	Lessons teach, ask the student to do something, interpret the answer and decide whether to remediate, stretch, review or move on.	met	adaptive_lesson_blueprint, targeted_feedback_plan, lesson.html	Lesson views include objective, prerequisite checks, explanation, worked example, guided practice, independent practice, misconception probe, summary, hints, evidence capture, safety handling, targeted feedback plans and next-action updates.	Subject-expert review of explanations, examples and lesson variants remains the production trust gate.
MVP Definition of Done	A new student can choose any stage/subject, complete diagnostics and adaptive lessons, see mastery maps, learning gain and progression links.	met	core/readiness.py	This is an implementation-readiness signal, not proof that the full definition of done is complete.	MVP implementation signal is not proof of expert-reviewed content or real student outcomes.
Mastery states	Concept mastery states move from unseen to mastered based on evidence, not lesson completion alone.	met	ConceptProgress, update_progress, mastery_evidence_profile	ConceptProgress stores the full mastery-state ladder; update_progress now gates transferable, retained and mastered states on varied clean evidence dimensions, no scaffolding, delayed retention evidence and no active misconceptions.	Threshold calibration against live learner data remains a psychometric and outcome-study gate.
Personalization	Personalization changes learning path, strategy, assessment and readiness decisions, not just tone or examples.	met	adaptive_assessment_plan, targeted_feedback_plan, learning_path_for_profile, core/learning.py	Learning path, next actions, targeted feedback plans and lesson assessment dimensions adapt to evidence, missing mastery dimensions, misconceptions, confidence calibration, scaffolding, frustration, pace and practice preferences.	Controlled learner studies showing personalization improves outcomes remain part of beta and retained-learning validation.
Product definition	Aristotle can answer what the student understands, misses, should learn next, which teaching approach fits and whether comprehension improved.	met	learning_question_insights, /learner-model/questions/, /learner-model/questions.json	Dedicated five-question report and JSON export answer the product contract from learner-model evidence, learning path, teaching-strategy signals, learning gain and retention summaries; accepted outcome studies recorded=0.	Reliability of those answers across real learners remains covered by the beta, psychometric and retained-learning validation gates.
Student experience	A learner can onboard, take diagnostics, see a learning path, complete lessons, receive feedback, continue later and revisit concepts.	met	core/views.py, core/templates/core/	Core onboarding, diagnostic, dashboard, lesson, scaffolded lesson-session resume, mastery review, account persistence and retention queue flows are implemented.	Needs broader usability testing with students across ages and access needs.
Subject fundamentals	Every claimed subject/stage has core concepts, prerequisites, misconceptions, representative problems, mastery signals, remediation and extension paths.	met	audit_curriculum command	Structural curriculum audit passed: 0 missing metadata items.	Structural completeness is not the same as content accuracy; expert review remains incomplete.
UI validation and browser testing	Meaningful UI changes are validated by running the app and interacting in browser across desktop and mobile.	met	UIValidationRun, /quality/	14 clean browser validation run(s) recorded; desktop=True, mobile=True; required flows covered=7/7.
UK education coverage	England, Scotland, Wales and Northern Ireland pathways are modelled explicitly.	met	CURRICULUM_PATHWAYS	Curriculum pathways are modelled for England, Scotland, Wales and Northern Ireland.	Needs deeper qualification equivalence and nation-specific curriculum detail before production.
UK education coverage	MVP covers the UK education journey from EYFS through A-Level across required stages and subjects.	met	core/curriculum.py	6 stages, 126 stage-subject entries and 768 concepts; missing metadata 0.	Coverage is structurally broad, but starter-map content still needs expert validation before trust claims.
Failure conditions	The system avoids being a generic chatbot, shallow quiz tracker or unsupported subject generator.	partial	run_failure_mode_checks, /quality/failure-modes.json, curriculum audit, review queue, lesson trust boundary	Structured curriculum maps, evidence dimensions, rubric signals, mastery thresholds, safety blocks, curriculum review gates and learner-facing trust-boundary warnings reduce generic tutoring risk. Failure-mode harness ready=False, source=ephemeral_self_check, checks=6, failures=1.	Run python manage.py run_failure_mode_checks --json --fail-on-error and resolve any failing unsupported-generation, shallow-mastery or safety-boundary checks.
North star metric	The system reports verified concept mastery gained per student hour and supporting learning metrics.	partial	core/learning.py, run_north_star_metric_check, /analytics/north-star-validation.json, /validation/	LearningEvidence records timed evidence, mastery-gain deltas and retention_check evidence; dashboards and analytics report mastery gained per tracked student hour, concept-level pre/post gain and delayed retention success; north-star metric integrity ready=False, checks=0, failures=None; retained-learning validation ready=False, open real-world validation items=8; accepted retained-learning outcome studies recorded=0.	Needs retained-learning outcome validation from real learners before the metric can be treated as reliable.
Production Definition of Done	Errors are monitored, backups exist, production smoke checks run and the system has reliability evidence.	partial	core/operations.py, run_operations_rehearsal, verify_backup_restore, OPERATIONS_RUNBOOK.md, /validation/	Open operational errors: 0; latest backup=False; restore drill=False; smoke=False; load=False; failure modes=False; rehearsal=False; accepted production-operations evidence=0.	Needs deployed external monitoring, alerting, production database setup, backup restore drills in the target environment and higher-volume load testing.
Production Definition of Done	Privacy, child-safety, safeguarding and unsupported-input concerns are handled properly.	partial	core/safety.py, run_safety_privacy_rehearsal, safety log, /validation/	Guardian acknowledgements, safety audit events, raw-answer minimization for safety-triggering inputs, safety log resolution, no-mastery safety handling, adult access audit, learner data export/delete and safety/privacy rehearsal checks exist; rehearsal ready=False, checks=0, failures=None; accepted safeguarding reviews=0; accepted legal/privacy reviews=0.	Needs external safeguarding/legal review, moderation ownership and production policy sign-off.
Subject fundamentals	Concept maps and teaching content are expert-reviewed before production trust claims.	partial	core/curriculum_review.py	0 live expert approvals; 768 concepts still awaiting live review; curriculum audit expert-review-complete=False; expert-reviewed-only teaching gate active in this environment=False; production trust gate configured=True.	Complete expert review or block unsupported concepts from production claims.
Beta Definition of Done	20 to 50 real test students can use the system without manual intervention and produce useful learning outcome evidence.	missing	ValidationEvidence, /validation/	0 accepted real learner beta participant(s) recorded in the validation evidence registry; beta workspace studies=0, included participants=0, ready studies=0.	Run an observed beta programme and record outcome evidence, failure patterns and remediation results.