Learning System Architecture

Documentation Index

Fetch the complete documentation index at: https://docs.arc.computer/llms.txt

Use this file to discover all available pages before exploring further.

​

Why Runtime Learning Matters

• Faster feedback loops – learning runs on live telemetry, so student/teacher personas improve within hours instead of a full fine-tune.
• Traceable changes – every playbook has a hash, metadata, and storage lineage so you can audit who learned what and when.
• Safe by default – review gating, drift detection, and update toggles let you pause or roll back learning if behavior regresses.

​

Feedback Loop at a Glance

1. Session executes – the dual-agent runtime completes a task and logs telemetry into sessions, trajectory_events, and reward tables.
2. Reward judges score – the RIM ensemble produces reward, uncertainty, and escalation data.
3. Learning synthesizer runs – after reward evaluation, Atlas calls the LearningSynthesizer to summarize salient guidance for student and teacher personas.
4. Registry update – Database.upsert_learning_state writes the new playbook to the learning_registry table (keyed by learning_key).
5. Playbook cached – on the next run for the same key, resolve_playbook retrieves and hashes the playbook, then injects it into persona prompts.
6. Evaluation harness audits – engineering teams query atlas.training_data (see the snippet below) to track reward deltas, mode shifts, and review status across learning keys.

​

Pipeline Components

​

Learning Synthesizer (atlas/learning/synthesizer.py)

• runs after reward scoring so updates only occur on high-signal sessions,
• consumes trajectory summaries, reward stats, and recent history (bounded by history_limit),
• emits structured “student” and “teacher” guidance plus optional session notes,
• uses a dedicated LLM (configurable via learning.llm) to transform raw notes into concise playbooks.

​

Playbook Resolver (atlas/learning/playbook.py)

• fetching the latest registry entry for the learning_key,
• trimming long sections to meet token budgets,
• caching the playbook on disk (per role) and computing a SHA256 hash,
• returning both content and metadata so prompts, validation payloads, and cache keys include the hash.

​

Playbook Injection Modes

learning:
  injection_mode: prefix  # Default

• General guidance and behavioral patterns
• Task-agnostic learning that should influence all reasoning
• Compatibility with providers that don’t support advanced caching

learning:
  injection_mode: suffix

• Provider KV cache efficiency: Anthropic and other providers cache the static system prompt prefix, avoiding recomputation when only the playbook changes
• Reduced latency: Cache hits eliminate prompt reprocessing overhead
• Cost savings: Cached tokens aren’t rebilled on subsequent requests

​

Learning Registry (atlas/runtime/storage/schema.sql)

• learning_key – primary identifier (task or project scope).
• student_learning / teacher_learning – latest playbook bodies (text).
• metadata – optional JSON payload (e.g., synthesizer audit info, hashes you compute upstream).
• updated_at – timestamp of the most recent update.

​

Discovery Telemetry

​

Persistence Topology

discovery_runs ─┐
                │
                ▼
          learning_registry ◄── learning synthesizer (updates)
                │
                ▼
            sessions ───► trajectory_events
                │             ▲
                └─ reward_stats│

• discovery_runs captures onboarding metadata and is referenced in learning reports.
• sessions records adaptive summaries, reward stats, review status, and session-specific learning notes.
• trajectory_events stores per-step evidence (event_type, actor, payload digests).
• learning_registry contains the most recent playbook per key/role.

​

Configuration Reference

learning:
  enabled: true
  update_enabled: true
  history_limit: 25
  session_note_enabled: false
  apply_to_prompts: true
  llm:
    provider: openai
    model: gpt-5-mini
    api_key_env: OPENAI_API_KEY

​

Related Controls

• orchestration.forced_mode (see atlas/config/models.py) locks the runtime into a specific lane—helpful when you want deterministic evaluation while toggling learning features.
• runtime_safety.review.require_approval should remain true in production so only reviewed sessions feed the synthesizer. Override via ATLAS_REVIEW_REQUIRE_APPROVAL=0 for local experiments.
• runtime_safety.drift guardrails help spot reward regressions after a playbook change.

​

Operating the Learning System

1. Seed the registry – run a curated set of tasks with learning.update_enabled=true to capture baseline playbooks. Export these as JSON for code review if needed.
2. Stage changes – toggle apply_to_prompts=false to generate candidate playbooks without impacting prompts.
3. Promote – flip apply_to_prompts back to true once the evaluation harness shows improved reward/uncertainty metrics.
4. Monitor – use the CLI and dashboards to track playbook_hash changes, reward drift, and review approvals per learning key.
5. Rollback – set update_enabled=false (to freeze) or enabled=false (to bypass entirely) if drift guardrails or reviewers flag an issue.

​

Verification & Tooling

cd ../atlas-sdk
python scripts/report_learning.py \
  --database-url postgresql://atlas:atlas@localhost:5433/atlas \
  --recent-window 10 \
  --baseline-window 50 \
  --limit 5 \
  --output-dir results/learning

python - <<'PY'
from statistics import mean
from atlas.training_data import get_training_sessions

sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    learning_key="mcp-tool-learning",
    status_filters=["succeeded"],
    limit=200,
)

recent_scores = [s.session_reward["score"] for s in sessions[:25] if s.session_reward]
baseline_scores = [s.session_reward["score"] for s in sessions[25:] if s.session_reward]

def pct_diff(a, b): return (a - b) / b * 100 if b else 0
print(f"Recent avg reward: {mean(recent_scores):.3f}")
print(f"Baseline avg reward: {mean(baseline_scores):.3f}")
print(f"Delta: {pct_diff(mean(recent_scores), mean(baseline_scores)):.2f}%")
PY

​

Troubleshooting

pip install --upgrade arc-atlas

# Run a session that should trigger learning
atlas run --config your_config.yaml --task "test task"

# Restart the runtime (new Python process)
# Verify playbook loads from database
atlas run --config your_config.yaml --task "another task"

# Check that playbook_hash in telemetry matches registry
psql $DATABASE_URL -c "SELECT learning_key, updated_at FROM learning_registry WHERE learning_key='your-key';"

​

Related Guides

• SDK Configuration – full YAML reference, including the learning block.
• Atlas CLI Reference – commands for discovery, execution, and review gating.
• Export Runtime Traces – move approved sessions into training datasets.
• Hybrid Learning Concept – theoretical background for offline + runtime learning.