Facial Coding System (Ekman & Friesen)
Detect micro-expressions → infer cognitive/emotional state → auto-trigger interventions
Emotion:
Surprise or Confusion
Engagement:
Auto-intervention:
Offer hint or worked example
Emotion:
Concentration / Effort
Engagement:
Auto-intervention:
Let child continue
Emotion:
Effort / Strain
Engagement:
Auto-intervention:
Monitor response time; if slow, reduce load
Emotion:
Disgust / Disdain
Engagement:
Auto-intervention:
Suggest alternative modality
Emotion:
Joy / Genuine Enjoyment
Engagement:
Auto-intervention:
Maintain difficulty, log as "success moment"
Emotion:
Sadness / Resignation
Engagement:
Auto-intervention:
Pause, offer autonomy + encouragement
Emotion:
Tension / Anxiety
Engagement:
Auto-intervention:
Breathing prompt + reappraisal
Emotion:
Surprise / Shock
Engagement:
Auto-intervention:
Check: Was this expected? Offer normalization
Focused attention ✓
Unsure about task ⚠️ → Hint offered
Refocused ✓
Success moment 🎉
Sustained effort in challenge zone
Storage:
Video NOT stored
Processing:
Real-time on-device (no cloud)
Data Retention:
Facial landmarks deleted after session (only AU codes + timestamps saved)
User Control:
Opt-in with clear consent, webcam light indicates active
✓ Use ml5.js (TensorFlow.js based, runs on-device)
✓ Facial-landmarks-detection model (detects 468 landmarks on face)
✓ Compute AU activation from landmark positions
✓ Real-time inference (30fps) with zero server calls
✓ Clear opt-in consent + webcam permission request
