Metric integrity
Run the north-star metric integrity command and resolve any timing, formula, retention-chain or privacy failures.
Learning analytics
Where students struggle
Aggregate evidence across learner profiles so educators can see misconception patterns, unsupported responses, high difficulty signals and safety items before they become invisible averages.
1
learner profiles
0
evidence points
0
concepts with progress
0
misconception evidence
0
unsupported evidence
0
open safety items
0%
retention success
0
mastery gained/hour
0
mastery gain points
0
tracked student hours
North-star study campaign
Retained-learning campaign control
Coordinate the metric integrity check, real learner outcome chains, review packet and returned validation decision for the north-star claim.
Open
metric integrity
Open
real-world study
14
study items open
0
accepted studies
Real learner outcome chains
Collect at least one baseline diagnostic or first-attempt evidence point before claiming learning gain.
External outcomes review
Send the retained-learning outcome pack and review work order to the learning outcomes reviewer.
Accepted outcome study
Import a reviewer return and accept a learning_outcome_study validation record only after real-world attestation.
Study design
Retained-learning fieldwork plan
Plan the real learner fieldwork needed to create trustworthy baseline, post-test and delayed retention chains.
diagnostic
1. Baseline diagnostic
A first-attempt score, outcome, dimension, confidence and timestamp exist before teaching.
EYFS and younger learners can use supported observation evidence; older learners should answer independently.
lesson_check
2. Adaptive teaching
The learner completes the adaptive lesson loop with time-spent evidence.
Teaching mode must match the learner stage and support needs.
mastery_review
3. Post-test or mastery review
A later independent application or transfer check is paired to the same learner, subject and concept.
Scaffolding must be recorded so reviewer can separate supported success from independent gain.
retention_check
4. Delayed retention check
A delayed, unscaffolded, non-safety retention row proves the gain endured beyond the lesson session.
Delay must meet or exceed 7 day(s).
Evidence capture checklist
- Every chain must use the same learner, stage, subject and concept across baseline, post-test and retention evidence.
- Record time_spent_seconds and mastery_delta for each scored evidence point.
- Keep scaffold_used, safety_level, confidence and evidence_dimension populated for reviewer interpretation.
- Exclude raw answers, learner names, guardian details, emails, session keys and access codes from exports.
- Do not count synthetic chains toward the north-star claim.
Safe study invite
Aristotle retained-learning study: baseline, lesson and retention check
Hello [participant or guardian], Aristotle is collecting real learner evidence for a retained-learning study. Please use [study invite link] to join, complete the baseline, use the lesson, then return after at least 7 day(s) for the retention check at [beta task page link]. The reviewer export is anonymized and excludes raw answers and contact details.
This fieldwork plan coordinates evidence collection. It does not prove retained learning until real learner chains meet the thresholds and a learning outcomes reviewer accepts the validation record.
Campaign steps
- Run the north-star metric integrity check after any change to evidence capture, timing, mastery deltas or analytics.
- Recruit real learners and collect baseline, post-test and delayed retention evidence for the same concept chains.
- Confirm the study reaches the configured learner, stage, subject and retention-delay thresholds.
- Export the work order, retained-learning outcome pack and return template for the learning outcomes reviewer.
- Import returned review JSON only when the reviewer confirms packet review, scope and real-world acceptance readiness.
- Rerun the Definition of Done audit and do not claim verified mastery gain until an accepted outcome-study record counts.
North-star launch pack
Verified metric claim readiness
Use this handoff before Aristotle claims verified concept mastery gained per student hour. Metric plumbing must be paired with real retained-learning evidence and reviewer acceptance.
Open
metric integrity
Open
real-world validation
Open
metric claim
15
claim items open
The launch pack exposes only anonymized counts, thresholds, readiness booleans, commands and route references. It omits learner names, raw answers, guardian details, emails, session keys and access codes.
Required validation loop
- Metric integrity check: A resolved north-star metric check proves timing, mastery-gain formula, retention-chain and privacy checks pass.
- Real learner outcome chains: Real learners have baseline, post-test and delayed retention evidence for the same concept chain.
- Stage and subject breadth: Retained-learning evidence covers the configured minimum number of education stages and subjects.
- Delayed independent retention: Retention success is delayed, unscaffolded, non-safety evidence rather than immediate supported work.
- Learning outcomes ownership: A named learning-outcome owner and traceable study protocol govern interpretation of the claim.
- Accepted outcome study: A learning outcomes reviewer accepts the retained-learning study in the validation registry.
- External or real-world attestation: Reviewer confirms the evidence is external or real-world and not synthetic-only.
Next actions
- Run the north-star metric integrity command and resolve any timing, formula, retention-chain or privacy failures.
- Collect at least one baseline diagnostic or first-attempt evidence point before claiming learning gain.
- Collect a later lesson check or mastery review so the study can compare before and after evidence.
- Collect a delayed retention check for the same learner, subject and concept.
- Build at least one complete baseline, post-test and retention chain before exporting the study pack.
- Record an unscaffolded, non-safety retention success so retained learning is not inferred from supported work.
Current outcome signal
North-star reviewer work order
Outcomes review assignment
Use this packet when asking a learning outcomes reviewer to inspect metric integrity, retained-learning chains, study breadth and the validation record without exposing raw learner answers.
Open
metric integrity
14
real-world items open
0
learners with chains
Open
metric claim
Return protocol
- Inspect metric integrity: Confirm the north-star metric check passed and rerun it after changes to evidence capture, timing, mastery deltas, retention or analytics.
- Inspect outcome chains: Check anonymized baseline, post-test and delayed retention chains for the same learner, subject and concept.
- Inspect study breadth: Confirm learner count, stage coverage, subject coverage and retention delay meet the configured validation thresholds.
- Inspect retention quality: Treat retained learning as valid only when retention is delayed, independent, unscaffolded and not safety-triggered.
- Return an outcomes-review decision: Return accept, changes_requested or reject with reviewer identity, role, attestation reference, review notes and any required study changes.
- Accept validation evidence only after review: The north-star claim counts only when a learning outcomes reviewer accepts a retained-learning study record with external or real-world attestation.
Trust boundary
This work order coordinates outcomes review. It does not prove Aristotle improves retained learning, and it must not be used as a verified north-star claim until the validation registry contains an accepted learning-outcome study with external or real-world attestation.
The work order includes anonymized counts, thresholds, open-item keys, route references and validation draft summaries only. It excludes learner names, raw answers, guardian details, emails, session keys, access codes and synthetic answer text.
Reviewer return checklist
The learning-outcome reviewer return must satisfy the validation registry guardrails before the north-star metric can be treated as verified.
Category confirmations
- complete_learning_chains_reviewed: Outcome reviewer confirms baseline, post-test and delayed retention chains were reviewed for the required real learners.
- retained_learning_claim_supported: Outcome reviewer confirms the retained-learning result supports the north-star claim or names the required changes.
Reviewer return routes
- Work order: /analytics/north-star-review-work-order.json
- Evidence: /analytics/retained-learning-outcomes.json
- Review brief: /validation/review-request/learning_outcome_study/
- Return template: /validation/review-return-template.json?request=learning_outcome_study
- Registry: /validation/?request=learning_outcome_study#record-evidence
North-star metric integrity check is passing
Run the north-star metric integrity command and resolve any timing, formula, retention-chain or privacy failures.
At least one baseline diagnostic or first-attempt evidence point exists
Collect at least one baseline diagnostic or first-attempt evidence point before claiming learning gain.
At least one post-test or lesson outcome gain exists
Collect a later lesson check or mastery review so the study can compare before and after evidence.
At least one delayed retention outcome exists
Collect a delayed retention check for the same learner, subject and concept.
At least one concept has baseline, post and retention evidence
Build at least one complete baseline, post-test and retention chain before exporting the study pack.
At least one retention success is independent and non-safety evidence
Record an unscaffolded, non-safety retention success so retained learning is not inferred from supported work.
At least one retention check is delayed by 7 day(s)
Wait for and record a retention check after the configured minimum delay.
Named learning-outcome owner is configured
Set ARISTOTLE_LEARNING_OUTCOME_OWNER so study design, interpretation and validity decisions have ownership.
Traceable learning-outcome protocol reference exists
Set ARISTOTLE_LEARNING_OUTCOME_PROTOCOL_REFERENCE to the retained-learning study protocol or evidence reference.
At least 20 learners have complete outcome chains
Recruit and run enough real learners through complete baseline, post-test and retention chains.
At least 3 education stage(s) are represented
Broaden the retained-learning study across the configured number of education stages.
At least 3 subject(s) are represented
Broaden the retained-learning study across the configured number of subjects.
Baseline, post-test and retention chains exist
Ensure the external reviewer can inspect complete baseline, post-test and retention chains.
Independent retention success is delayed by at least 7 day(s)
Collect delayed independent retention successes before treating the metric as evidence of durable learning.
Accepted retained-learning outcome study exists in the validation registry
Ask the learning outcomes reviewer to accept a complete retained-learning study record in the validation registry.
Validation drafts
Analytics review evidence records
Create non-counting drafts in the validation registry from the current calibration and retained-learning packs. Reviewers still need to accept the records before they count toward readiness.
Psychometric validation
This draft does not count toward readiness until accepted in the validation registry. The calibration pack is still missing evidence across one or more required dimensions.
Sign in to draft psychometric evidence.
Retained-learning outcome study
This draft does not count toward readiness until accepted in the validation registry. 6 outcome-pack check(s) and 7 external-validation checklist item(s) remain open before reviewer acceptance.
Sign in to draft outcome evidence.
North-star validation
Metric integrity check
Run python manage.py run_north_star_metric_check --json --fail-on-error after changing evidence capture, mastery deltas, timing, retention or analytics.
Open
metric integrity
0
checks recorded
-
failures
Live
latest evidence mode
Open
real-world metric claim
8
real-world open items
Metric claim handoff
Run the north-star metric integrity command and resolve any timing, formula, retention-chain or privacy failures.
North-star metric integrity check is passing
Run the north-star metric integrity command and resolve any timing, formula, retention-chain or privacy failures.
At least one baseline diagnostic or first-attempt evidence point exists
Collect at least one baseline diagnostic or first-attempt evidence point before claiming learning gain.
At least one post-test or lesson outcome gain exists
Collect a later lesson check or mastery review so the study can compare before and after evidence.
At least one delayed retention outcome exists
Collect a delayed retention check for the same learner, subject and concept.
At least one concept has baseline, post and retention evidence
Build at least one complete baseline, post-test and retention chain before exporting the study pack.
At least one retention success is independent and non-safety evidence
Record an unscaffolded, non-safety retention success so retained learning is not inferred from supported work.
At least one retention check is delayed by 7 day(s)
Wait for and record a retention check after the configured minimum delay.
Named learning-outcome owner is configured
Set ARISTOTLE_LEARNING_OUTCOME_OWNER so study design, interpretation and validity decisions have ownership.
Traceable learning-outcome protocol reference exists
Set ARISTOTLE_LEARNING_OUTCOME_PROTOCOL_REFERENCE to the retained-learning study protocol or evidence reference.
At least 20 learners have complete outcome chains
Recruit and run enough real learners through complete baseline, post-test and retention chains.
At least 3 education stage(s) are represented
Broaden the retained-learning study across the configured number of education stages.
At least 3 subject(s) are represented
Broaden the retained-learning study across the configured number of subjects.
Baseline, post-test and retention chains exist
Ensure the external reviewer can inspect complete baseline, post-test and retention chains.
Independent retention success is delayed by at least 7 day(s)
Collect delayed independent retention successes before treating the metric as evidence of durable learning.
Accepted retained-learning outcome study exists in the validation registry
Ask the learning outcomes reviewer to accept a complete retained-learning study record in the validation registry.
No north-star metric check has been recorded yet.
Retained-learning validity
Metric plumbing can pass on synthetic evidence; verified learning claims need real learner outcome evidence, reviewer ownership and accepted study records.
Named learning-outcome owner is configured
Reviewer confirms who owns study design, interpretation and validity decisions.
Traceable learning-outcome protocol reference exists
Reviewer checks baseline, post-test, retention-window, inclusion and exclusion rules.
At least 20 learners have complete outcome chains
Reviewer verifies the study has enough real learner evidence before treating the north-star metric as validated.
At least 3 education stage(s) are represented
Reviewer checks that retained-learning evidence is not limited to one age band.
At least 3 subject(s) are represented
Reviewer checks that retained-learning evidence is not limited to one subject type.
Baseline, post-test and retention chains exist
Reviewer checks that claimed gains compare a starting point with later independent evidence.
Independent retention success is delayed by at least 7 day(s)
Reviewer checks that retention is durable, unscaffolded and not safety-triggered evidence.
Outcome export excludes raw answers and direct learner identifiers
Reviewer confirms study evidence can be shared safely for external review.
Accepted retained-learning outcome study exists in the validation registry
Reviewer confirms the study has been accepted rather than only generated as an internal draft.
Retained-learning outcomes
Pre, post and retention evidence
Export anonymized concept chains that pair baseline evidence with later post-test and retention checks, so reviewers can inspect whether comprehension improved and stayed improved.
0
concept chains
0
complete chains
0
learners with complete chains
0/3
stage coverage
0/3
subject coverage
0
avg post gain
0
avg retention gain
0%
retention success
0
delayed independent successes
Open
outcome-review pack
Open
external checklist
At least one baseline diagnostic or first-attempt evidence point exists
At least one post-test or lesson outcome gain exists
At least one delayed retention outcome exists
At least one concept has baseline, post and retention evidence
At least one retention success is independent and non-safety evidence
At least one retention check is delayed by 7 day(s)
External outcome validation checklist
These checks separate “we can calculate the metric” from “we can trust the metric as evidence of retained learning.”
Named learning-outcome owner is configured
Reviewer confirms who owns study design, interpretation and validity decisions.
Traceable learning-outcome protocol reference exists
Reviewer checks baseline, post-test, retention-window, inclusion and exclusion rules.
At least 20 learners have complete outcome chains
Reviewer verifies the study has enough real learner evidence before treating the north-star metric as validated.
At least 3 education stage(s) are represented
Reviewer checks that retained-learning evidence is not limited to one age band.
At least 3 subject(s) are represented
Reviewer checks that retained-learning evidence is not limited to one subject type.
Baseline, post-test and retention chains exist
Reviewer checks that claimed gains compare a starting point with later independent evidence.
Independent retention success is delayed by at least 7 day(s)
Reviewer checks that retention is durable, unscaffolded and not safety-triggered evidence.
Outcome export excludes raw answers and direct learner identifiers
Reviewer confirms study evidence can be shared safely for external review.
Assessment calibration
Psychometric review evidence
Export anonymized evidence for reviewers to inspect score thresholds, confidence calibration, scaffold dependence, unsupported rates and coverage across assessment dimensions.
0
evidence points
0/8
dimensions covered
0
concept-dimension cells
0
cells needing review
Open
review pack status
Beta programme
Cohort evidence workflow
Use the beta workspace to enrol real learners, track consent, verify diagnostics, lessons, post-tests, retention evidence and decide when a cohort is ready for external validation.
0
beta studies
0
active studies
0
included participants
0
studies ready for review
Beta evidence pack
Anonymized study export readiness
This pack aggregates cohort participation, evidence coverage, retention, mastery gained/hour and safety counts without names, raw answers, emails, session keys or access codes.
1
profiles in pack
0
active learners
0
evidence points
0%
retention pass rate
0
mastery gained/hour
Open
study-pack gate
20 to 50 real learner participants
Every active learner has diagnostic evidence
Every active learner has lesson evidence
At least one post-test or mastery review exists
At least one delayed retention check exists
Timed evidence exists for mastery gained/hour
North star
Verified concept mastery gained per student hour
0 mastery gain points across 0 tracked minutes from 0 timed evidence points. Retention evidence: 0 of 0 delayed checks passed.
Cohort trend
Longitudinal learning signal
0 active learner days in the last 30 days. Track whether evidence volume, misconception rate, retention and mastery gained/hour are improving together.
No cohort trend yet. Trend rows appear after learners create timed evidence.
Concept hotspots
Misconception and confidence calibration
No concept progress yet. Analytics will populate once learners complete diagnostics and lessons.
Evidence hotspots
Where answers are breaking down
No evidence has been recorded yet.
Stage and experience
Patterns by stage and learner feeling
Stage signals
No stage evidence yet.
Emotion signals
No learner feeling signals yet.