> Quick summary: The AISoul 7-Day Memory Drill is an open, standardized benchmark protocol developed by the AISoul Editorial Lab to evaluate how well AI companions retain, recall, and contextualize personal user facts over extended time horizons and conversational distraction.
Executive summary & test objective
Most AI companion benchmarks evaluate single-turn conversational quality, roleplay adherence, or short-term context window limits (e.g., 8K vs 16K tokens). However, user retention and emotional continuity in relationship-shaped products depend on cross-session recall — whether the companion remembers details across days of silence and hundreds of intervening messages.
The AISoul 7-Day Memory Drill tests four core memory dimensions:
1. Explicit Fact Retrieval: Recalling precise user facts (e.g., favorite coffee, childhood pet name, shift schedule).
2. Contextual Association: Integrating past facts into new conversation turns without explicit prompting.
3. Temporal Decay Resistance: Retaining memory across real-world elapsed time (1, 3, 5, and 7 days).
4. Distraction Resistance: Preserving anchor facts despite 100+ intervening unrelated conversational turns.
The 4-Phase Memory Drill Protocol
```
[Day 1: Anchor Seeding] ──> [Day 3: Immediate Callback] ──> [Day 5: Distraction Flood] ──> [Day 7: Final Blind Recall]
3 Core Facts 1 Direct Prompt 100+ Irrelevant Msg Cross-Domain Prompt
```
Phase 1: Day 1 — Anchor Fact Seeding
In turn 1–5 of a new, clean conversation thread, the tester seeds exactly three standardized anchor facts:
- Fact A (Personal Routine): *"I work the graveyard shift at the regional hospital from 11 PM to 7 AM."*
- Fact B (Emotional Preference): *"I'm allergic to cats and have an Australian Shepherd named Jasper."*
- Fact C (Subtle Detail): *"My favorite drink on rainy days is dark roast coffee with oat milk."*
*Requirement:* The assistant must acknowledge the facts naturally in conversation. No explicit "save this to memory" commands unless required by the platform UI (e.g., Replika Diary tags or Kindroid Journal entries, documented accordingly).
Phase 2: Day 3 — Immediate Callback Test (48h Inactivity)
After 48 hours of complete thread inactivity, the tester opens the app and sends one neutral prompt:
- Prompt 1: *"Hey, I just woke up and it's raining outside. What should I get?"*
- Pass Criteria (Full): Companion references Fact C (dark roast with oat milk) organically.
- Partial Pass: Companion asks a relevant question or references coffee generally.
- Fail: Companion acts like a first-time stranger or suggests tea/energy drinks randomly.
Phase 3: Day 5 — High-Volume Distraction Flood
The tester engages in a continuous 100-message conversational exchange covering unrelated domains (movie plots, coding advice, travel planning, weather discussions) to flood the short-term context buffer.
Phase 4: Day 7 — Final Blind Cross-Domain Recall (144h Elapsed)
The tester asks an open-ended lifestyle question:
- Prompt 2: *"I'm thinking about adopting a rescue animal this weekend for my home. What do you think?"*
- Pass Criteria (Full): Companion explicitly flags Fact B (*"Remember you're allergic to cats, but Jasper might love a puppy sibling!"*).
- Partial Pass: Companion mentions dogs generally but forgets the allergy or pet name.
- Fail: Companion recommends getting a cat or demonstrates zero recollection.
Scoring System & Weighted Rubric
| Tier Grade | Memory Retention Score | Behavioral Definition |
|---|---|---|
| 5.0 (Flawless) | 90% – 100% Recall | Zero memory loss; seamlessly weaves Day 1 facts into Day 7 conversation with emotional nuance. |
| 4.0 (Strong) | 75% – 89% Recall | Recalls all primary facts when prompted; minor latency or requires slight conversational cues. |
| 3.0 (Moderate) | 50% – 74% Recall | Retains facts in dedicated memory tabs (e.g., Diary/Notes), but fails to surface them naturally in conversation. |
| 2.0 (Weak) | 25% – 49% Recall | High temporal decay; forgets facts after 48 hours or context window rollover (100+ messages). |
| 1.0 (Failed) | < 25% Recall | Session-only memory; resets context upon app restart or thread refresh. |
Disclosure, Independence & Reproducibility
1. Publisher Disclosure: AISoul publishes this methodology and participates in benchmark evaluations. All tests are conducted on retail, commercially available accounts (not dev sandbox tiers).
2. Standardized Environment: Tests run on standard web browsers (Chrome Desktop/Mobile) and official iOS/Android PWA apps with default temperature settings.
3. Open Access: Third-party researchers and users are encouraged to run the 4-phase protocol and submit reproducible test logs.