# Trainers Reference
Source: https://docs.arc.computer/api-reference/trainers

Complete reference for ATLAS trainer classes and methods

## Overview

ATLAS provides specialized trainer classes for different training paradigms. Each trainer extends HuggingFace's base trainer with RL-specific capabilities.

<img alt="Training Pipeline" />

## Typical Usage

<CodeGroup>
  ```python title="Standard GRPO Training" theme={null}
  from atlas_core.training.algorithms.grpo import GRPOTrainer
  from atlas_core.training.algorithms.grpo_config import GRPOConfig
  from atlas_core.reward.interpretation import RIMReward

  # Minimal training arguments (TrainingArguments requires an output_dir)
  grpo_args = GRPOConfig(
      output_dir="./output/grpo",
      model_name_or_path="Arc-Intelligence/ATLAS-8B-Thinking",
      learning_rate=5e-6,
      num_train_epochs=3,
      per_device_train_batch_size=4,
      gradient_accumulation_steps=4,
      beta=0.04,  # KL penalty
      logging_steps=10,
  )

  reward = RIMReward(config_path="reward_system/interpretation.yaml")

  trainer = GRPOTrainer(
      model=grpo_args.model_name_or_path,
      reward_funcs=reward,
      args=grpo_args,
      train_dataset=train_data,
      eval_dataset=eval_data,
      processing_class=tokenizer,
  )

  trainer.train()
  trainer.save_model("./output/grpo/checkpoint-final")
  ```

  ```python title="Dual-Agent Teacher Training" theme={null}
  from atlas_core.training.algorithms.teacher_trainers import TeacherGRPOTrainer
  from atlas_core.training.algorithms.grpo_config import GRPOConfig
  from atlas_core.reward.interpretation import RIMReward

  teacher_args = GRPOConfig(
      output_dir="./output/teacher-grpo",
      model_name_or_path="Arc-Intelligence/ATLAS-8B-Thinking",
      learning_rate=5e-6,
      num_train_epochs=3,
      per_device_train_batch_size=1,
      gradient_accumulation_steps=8,
      beta=0.04,
      logging_steps=10,
  )

  reward = RIMReward(config_path="reward_system/interpretation_offline.yaml")

  trainer = TeacherGRPOTrainer(
      model=teacher_args.model_name_or_path,
      reward_funcs=reward,
      args=teacher_args,
      train_dataset=train_data,
      processing_class=tokenizer,
      student_model="meta-llama/Llama-3.2-8B-Instruct",
  )

  trainer.train()
  trainer.save_model("./output/teacher-grpo/checkpoint-final")
  ```

  ```python title="SFT Warmup" theme={null}
  from trl import SFTConfig, SFTTrainer

  # Supervised fine-tuning before RL
  trainer = SFTTrainer(
      model=model,
      args=training_args,
      train_dataset=sft_dataset,
      tokenizer=tokenizer,
      max_seq_length=2048
  )

  trainer.train()
  ```
</CodeGroup>

## GRPOTrainer

Main trainer for Group Relative Policy Optimization.

### Class Overview

GRPOTrainer is the main trainer class for Group Relative Policy Optimization, extending the standard HuggingFace Trainer with reinforcement learning capabilities.

<AccordionGroup>
  <Accordion title="Parameters">
    | Parameter         | Type                   | Description                     |
    | ----------------- | ---------------------- | ------------------------------- |
    | `config`          | GRPOConfig             | Training configuration          |
    | `model`           | PreTrainedModel        | Model to train (policy network) |
    | `ref_model`       | PreTrainedModel        | Reference model for KL penalty  |
    | `tokenizer`       | PreTrainedTokenizer    | Tokenizer for encoding/decoding |
    | `train_dataset`   | Dataset                | Training data                   |
    | `eval_dataset`    | Dataset                | Evaluation data                 |
    | `reward_model`    | PreTrainedModel        | Optional external reward model  |
    | `compute_metrics` | Callable               | Custom metrics function         |
    | `callbacks`       | List\[TrainerCallback] | Training callbacks              |
    | `optimizers`      | Tuple                  | Custom optimizer and scheduler  |
  </Accordion>

  <Accordion title="Key Features">
    **GRPO Training Loop**: Implements the complete reinforcement learning training process with policy gradient optimization and KL divergence constraints.

    **Generation Support**: Supports both local generation and distributed generation via vLLM server integration.

    **Memory Management**: Includes optimizations for training large models with gradient checkpointing and model offloading.

    **Reward Composition**: Handles multiple reward functions (including optional RIM-based scoring) and reward weighting for complex optimization objectives.

    **Implementation**: See `src/atlas_core/training/algorithms/grpo.py` for complete method signatures and implementation details.
  </Accordion>

  <Accordion title="Training Hooks">
    Override these methods for custom behavior:

    ```python theme={null}
    def on_epoch_begin(self):
        """Called at the beginning of each epoch"""
        pass

    def on_step_end(self, args, state, control, **kwargs):
        """Called at the end of each training step"""
        # Log custom metrics
        self.log({
            "rewards/mean": self.current_rewards.mean(),
            "kl_divergence": self.current_kl.mean()
        })

    def on_evaluate(self, args, state, control, metrics=None, **kwargs):
        """Called after evaluation"""
        # Custom evaluation logic
        pass
    ```
  </Accordion>
</AccordionGroup>

**Source**: `src/atlas_core/training/algorithms/grpo.py`

## TeacherGRPOTrainer

Specialized trainer for adaptive dual-agent teaching (student agent + verifying teacher).

### Class Overview

TeacherGRPOTrainer extends GRPOTrainer to implement the two-pass teaching protocol. From the actual source code (`src/atlas_core/training/algorithms/teacher_trainers.py`), this trainer:

* Inherits from both `GRPOTrainer` and `TeacherTrainer`
* Accepts `student_model` parameter in constructor
* Implements diagnostic probing and verifying-teacher guidance templates
* Manages both teacher and student models during training

<AccordionGroup>
  <Accordion title="Unique Methods">
    ### diagnostic\_probe()

    Assess student capability:

    ```python theme={null}
    def diagnostic_probe(
        self,
        task: str,
        max_tokens: int = 50
    ) -> DiagnosticResult:
        """
        Probe student understanding of given task

        Args:
            task: Task or problem statement to probe
            max_tokens: Maximum tokens for probe response

        Returns:
            DiagnosticResult: Assessment containing:
                - capability_level: Student capability (0.0-1.0)
                - confidence_score: Confidence in assessment (0.0-1.0)
                - identified_gaps: List of specific knowledge gaps
                - response_quality: Quality metrics for student response
                - probe_tokens: Number of tokens used in probe

        Raises:
            ValueError: If task is empty or max_tokens &lt; 1
            RuntimeError: If student model inference fails
            TypeError: If task is not a string

        Example:
            result = trainer.diagnostic_probe("Solve: 2x + 5 = 11")
            if result.capability_level &lt; 0.3:
                print("Student needs significant help")
        """
    ```

    ### generate\_guidance()

    Create verifying-teacher guidance:

    ```python theme={null}
    def generate_guidance(
        self,
        task: str,
        diagnostic: DiagnosticResult,
        max_tokens: int = 200
    ) -> str:
        """
        Generate teaching guidance based on diagnosis

        Args:
            task: Original task or problem statement
            diagnostic: Results from diagnostic_probe()
            max_tokens: Maximum tokens for guidance response

        Returns:
            str: Tailored teaching guidance text
                Format depends on capability_level:
                - Low (0.0-0.3): Step-by-step walkthrough
                - Medium (0.3-0.7): Hints and scaffolding
                - High (0.7-1.0): Minimal guidance or verification

        Raises:
            ValueError: If max_tokens &lt; 10 or diagnostic is None
            RuntimeError: If teacher model fails to generate guidance
            TypeError: If task is not a string

        Example:
            diagnostic = trainer.diagnostic_probe("Solve quadratic equation")
            guidance = trainer.generate_guidance(
                "Solve: x² - 5x + 6 = 0",
                diagnostic,
                max_tokens=150
            )
            print(f"Teaching guidance: {guidance}")
        """
    ```

    ### compute\_teaching\_reward()

    Calculate teaching effectiveness:

    ```python theme={null}
    def compute_teaching_reward(
        self,
        baseline_score: float,
        enhanced_score: float,
        teaching_length: int
    ) -> float:
        """
        Compute reward for teaching quality

        Args:
            baseline_score: Student performance without guidance (0.0-1.0)
            enhanced_score: Student performance with guidance (0.0-1.0)
            teaching_length: Number of tokens in teaching guidance

        Returns:
            float: Teaching reward score
                - Positive: Effective teaching (improvement achieved)
                - Zero: No improvement or neutral
                - Negative: Degraded performance (safety penalty)

        Raises:
            ValueError: If scores not in [0.0, 1.0] or teaching_length &lt; 0
            TypeError: If inputs are not numeric

        Reward Components:
            - Improvement bonus: (enhanced_score - baseline_score)
            - Efficiency bonus: max(0, 1 - teaching_length / 200)
            - Safety penalty: -2.0 if enhanced_score &lt; baseline_score

        Example:
            reward = trainer.compute_teaching_reward(
                baseline_score=0.6,
                enhanced_score=0.8,
                teaching_length=120
            )
            # reward ≈ 0.2 + 0.4 = 0.6 (improvement + efficiency)
        """
    ```
  </Accordion>

  <Accordion title="Teaching Protocol">
    The two-pass protocol implementation:

    ```python theme={null}
    def teaching_step(self, batch):
        """Execute one teaching interaction"""

        # Phase 1: Diagnostic
        diagnostics = []
        for prompt in batch["prompts"]:
            diag = self.diagnostic_probe(prompt)
            diagnostics.append(diag)

        # Phase 2: Guidance generation
        guidances = []
        for prompt, diag in zip(batch["prompts"], diagnostics):
            guidance = self.generate_guidance(prompt, diag)
            guidances.append(guidance)

        # Phase 3: Student enhancement
        baseline_responses = self.student_model.generate(batch["prompts"])
        enhanced_responses = self.student_model.generate(
            batch["prompts"],
            guidance=guidances
        )

        # Phase 4: Reward computation
        rewards = []
        for base, enh, guid in zip(baseline_responses, enhanced_responses, guidances):
            reward = self.compute_teaching_reward(
                self.score(base),
                self.score(enh),
                len(guid)
            )
            rewards.append(reward)

        return rewards
    ```
  </Accordion>
</AccordionGroup>

### Reward System Integration

TeacherGRPOTrainer expects `reward_funcs` to supply the evaluation signal. When you pass an instance of `RIMReward`, each call returns both the aggregated reward and an information dictionary containing per-judge scores, principles, and rationales. The trainer logs these details under `rim_rewards` and `rim_explanations`, making it possible to inspect accuracy, helpfulness, process, and diagnostic scores separately. To switch configurations during an experiment, update the Hydra override so that `RIMReward` loads either `reward_system/interpretation.yaml` or `reward_system/interpretation_offline.yaml`. The trainer does not need any code changes when you modify judge prompts, thresholds, or model choices.

**Source**: `src/atlas_core/training/algorithms/teacher_trainers.py`

## SFTTrainer

Supervised fine-tuning trainer for warmup before RL.

### Constructor

```python theme={null}
class SFTTrainer(Trainer):
    def __init__(
        self,
        model: PreTrainedModel,
        args: TrainingArguments,
        train_dataset: Dataset,
        eval_dataset: Optional[Dataset] = None,
        tokenizer: PreTrainedTokenizer,
        data_collator: Optional[DataCollator] = None,
        max_seq_length: int = 2048,
        packing: bool = False,
        formatting_func: Optional[Callable] = None,
    ):
```

<AccordionGroup>
  <Accordion title="Key Features">
    * **Sequence packing**: Efficient batching of variable-length sequences
    * **Custom formatting**: Apply templates to raw data
    * **Gradient accumulation**: Handle large effective batch sizes
    * **Mixed precision**: FP16/BF16 training support
  </Accordion>

  <Accordion title="Data Processing">
    ### format\_dataset()

    Prepare data for training:

    ```python theme={null}
    def format_dataset(self, dataset):
        """Format dataset for SFT training"""

        def formatting_func(example):
            # Apply chat template
            messages = example["messages"]
            text = self.tokenizer.apply_chat_template(
                messages,
                tokenize=False
            )
            return {"text": text}

        return dataset.map(formatting_func)
    ```

    ### pack\_sequences()

    Efficient sequence packing:

    ```python theme={null}
    def pack_sequences(self, tokenized_dataset):
        """Pack multiple sequences into single training example"""
        # Implementation for efficient GPU utilization
    ```
  </Accordion>
</AccordionGroup>

**Source**: TRL `SFTTrainer`

## Custom Trainer Implementation

Create your own trainer by extending base classes:

```python theme={null}
from atlas_core.training.algorithms.grpo import GRPOTrainer
import torch

class CustomRewardTrainer(GRPOTrainer):
    """Custom trainer with modified reward computation"""

    def compute_rewards(self, completions, prompts):
        """Override reward computation"""
        rewards = []
        for completion, prompt in zip(completions, prompts):
            # Custom reward logic
            reward = self.custom_reward_function(completion, prompt)
            rewards.append(reward)
        return torch.tensor(rewards)

    def custom_reward_function(self, completion, prompt):
        """Implement domain-specific rewards"""
        # Example: Length penalty
        length_penalty = min(1.0, len(completion) / 500)

        # Example: Quality score
        quality = self.quality_model(completion)

        return quality * length_penalty
```

## Callbacks and Monitoring

### Available Callbacks

```python theme={null}
from transformers import EarlyStoppingCallback
from transformers.integrations import TensorBoardCallback, WandbCallback

# Configure callbacks
callbacks = [
    WandbCallback(
        project="atlas-training",
        name="experiment-1"
    ),
    EarlyStoppingCallback(
        early_stopping_patience=3,
        early_stopping_threshold=0.001
    )
]

trainer = GRPOTrainer(
    config=config,
    callbacks=callbacks
)
```

### Custom Metrics

```python theme={null}
def compute_metrics(eval_predictions):
    """Custom metrics computation"""
    predictions, labels = eval_predictions

    return {
        "accuracy": accuracy_score(labels, predictions),
        "perplexity": perplexity(predictions),
        "diversity": diversity_score(predictions),
        "safety_rate": safety_check(predictions)
    }

trainer = GRPOTrainer(
    config=config,
    compute_metrics=compute_metrics
)
```

## Distributed Training

### Multi-GPU Setup

```python theme={null}
from atlas_core.training.algorithms.grpo import GRPOTrainer
from accelerate import Accelerator

accelerator = Accelerator()

trainer = GRPOTrainer(
    config=config,
    model=model,
    accelerator=accelerator
)

# Trainer automatically handles distributed setup
trainer.train()
```

### DeepSpeed Integration

Atlas Core relies on Accelerate configs to enable DeepSpeed. Use one of the presets in `accelerate/`:

```bash theme={null}
# Default zero3
accelerate launch --config_file accelerate/deepspeed_zero3.yaml \
  -m atlas_core.cli.train recipe@_global_=teacher_rcl

# CPU offload
accelerate launch --config_file accelerate/deepspeed_zero3_cpu_offloading.yaml \
  -m atlas_core.cli.train recipe@_global_=teacher_rcl
```

## Implementation Notes

ATLAS trainers extend standard HuggingFace Trainer classes with RL-specific functionality. The implementation details can be found in:

* `src/atlas_core/training/algorithms/grpo.py` - Main GRPO trainer implementation
* `src/atlas_core/training/algorithms/teacher_trainers.py` - Teacher-student training logic
* `src/atlas_core/training/algorithms/grpo_config.py` - Configuration parameters

## Troubleshooting

<AccordionGroup>
  <Accordion title="Memory Issues">
    **Problem**: CUDA OOM during training

    **Solutions**:

    ```python theme={null}
    # Reduce batch size
    config.per_device_train_batch_size = 1
    config.gradient_accumulation_steps = 32

    # Enable gradient checkpointing
    config.gradient_checkpointing = True

    # Use mixed precision
    config.fp16 = True
    ```
  </Accordion>

  <Accordion title="Slow Training">
    **Problem**: Training is slower than expected

    **Solutions**:

    ```python theme={null}
    # Enable compilation (PyTorch 2.0+)
    model = torch.compile(model)

    # Use Flash Attention
    config.attn_implementation = "flash_attention_2"

    # Optimize data loading
    config.dataloader_num_workers = 4
    config.dataloader_pin_memory = True
    ```
  </Accordion>

  <Accordion title="Unstable Training">
    **Problem**: Loss spikes or NaN values

    **Solutions**:

    ```python theme={null}
    # Reduce learning rate
    config.learning_rate = 1e-6

    # Increase KL penalty
    config.beta = 0.1

    # Clip gradients
    config.max_grad_norm = 0.5
    ```
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup>
  <Card title="RL Training Guide" icon="graduation-cap" href="/training/offline/grpo-training">
    Step-by-step training tutorial
  </Card>

  <Card title="Adaptive Tool Use" icon="wrench" href="/examples/adaptive-tool-use">
    Production-ready MCP integration example
  </Card>
</CardGroup>


# Evaluation Harnesses
Source: https://docs.arc.computer/benchmarks/evaluation-harnesses

Measure Atlas runtime, reward judges, and learning progress with the SDK benchmarking scripts.

Atlas ships dedicated evaluation harnesses in the
[atlas-sdk](https://github.com/Arc-Computer/atlas-sdk) repository. They operate directly on the SDK runtime (learning
reports, synthetic runtime sweeps, and reward judge comparisons) while Atlas Core focuses on offline training. Clone or
open the SDK repo alongside this project and run the scripts from the SDK root.

| Harness              | SDK Script                               | Primary Questions                                                           | Key Artifacts                                             |
| -------------------- | ---------------------------------------- | --------------------------------------------------------------------------- | --------------------------------------------------------- |
| Learning snapshot    | `scripts/report_learning.py`             | Are playbooks improving reward and execution modes for each `learning_key`? | `atlas-sdk/results/learning/*.json`, `*.md`, `index.json` |
| Runtime benchmarking | `scripts/benchmark_dual_agent_models.py` | Which student/teacher pairings deliver the best reward vs latency?          | `atlas-sdk/results/dual_agent_eval.json`                  |
| Reward benchmarking  | `scripts/benchmark_reward_models.py`     | How do judge stacks compare on reward, uncertainty, and escalation?         | `atlas-sdk/results/reward/*.json` (+ optional Markdown)   |

## Before You Run the Harnesses

* Enable Postgres persistence in your SDK config (`storage.database_url`) so the scripts can read sessions, discovery
  runs, and learning registry entries.
* Load `.env` with the provider API keys required by each harness—the SDK scripts call `load_dotenv_if_available()`
  before executing.
* Review gating defaults to `approved` sessions only. Approve runs via `arc-atlas review` or override filters explicitly
  when you intend to include pending/quarantined data.
* Run the commands from the `atlas-sdk` repo root (for example `cd ../atlas-sdk` if you keep both repos side-by-side).

## Learning Snapshot (`atlas-sdk/scripts/report_learning.py`)

Use the reporting harness to evaluate playbook health without injecting hints:

```bash theme={null}
cd ../atlas-sdk
python scripts/report_learning.py \
  --database-url postgresql://atlas:atlas@localhost:5433/atlas \
  --recent-window 10 \
  --baseline-window 50 \
  --limit 5 \
  --output-dir results/learning
```

**Key options**

* `--learning-key <id>` – report on explicit keys rather than the most recent ones.
* `--filter-project`, `--filter-task`, `--filter-tag tenant:demo` – scope by telemetry metadata.
* `--summary-only` – skip trajectory fetches for CI.
* `--compare-to results/learning/index.json` – compute deltas against a previous run.
* `--no-markdown` – emit JSON only.

**Metrics reported**

* Recent vs baseline reward means, deltas, and uncertainty.
* Execution-mode distribution (auto/paired/coach) for each window.
* Review status counts so pending/quarantined sessions surface immediately.
* Model usage breakdown (student/teacher pairings drawn from adapter telemetry).
* Discovery references tying each learning key back to `discovery_runs`.

Outputs live under `atlas-sdk/results/learning/` (per-key JSON, optional Markdown, and a manifest). For lightweight spot
checks inside Atlas Core you can still call `atlas.training_data.get_training_sessions` directly—see the snippet in the
[Learning System guide](/sdk/learning-system)—but the SDK harness is the canonical workflow.

## Runtime Benchmarking (`atlas-sdk/scripts/benchmark_dual_agent_models.py`)

Benchmark student/teacher combinations against the synthetic runtime dataset and capture latency + reward deltas:

```bash theme={null}
cd ../atlas-sdk
python scripts/benchmark_dual_agent_models.py \
  --dataset atlas/data/synthetic_runtime_tasks.jsonl \
  --student-models claude-haiku-4-5 gemini-2.5-flash \
  --teacher-models grok-4-fast gemini-2.5-pro \
  --repeats 2 \
  --output results/dual_agent_eval.json
```

**Key options**

* `--base-config` – clone a different runtime config (default: `configs/examples/openai_agent.yaml`).
* `--concurrency` – process-pool fan-out for faster sweeps (set to 1 while debugging logging).
* `ATLAS_MODEL_OVERRIDE_<MODEL>` – redirect presets to hosted checkpoints (Azure OpenAI, self-hosted vLLM, etc.).
* `ATLAS_MODEL_TIMEOUT` – raise/lower call timeouts globally when exercising slow providers.

**Metrics reported**

* Per-task final answers, adaptive-mode history, reward, runtime, and failure status.
* Aggregated reward averages, latency means, failure counts, and mode distributions per model pair.
* “Best pair” heuristic for quick default selection plus raw telemetry for deeper analysis.

## Reward Benchmarking (`atlas-sdk/scripts/benchmark_reward_models.py`)

Replay captured session trajectories to compare reward judge stacks without disturbing the orchestrator:

```bash theme={null}
cd ../atlas-sdk
python scripts/benchmark_reward_models.py \
  --dataset atlas/data/reward_eval_trajectories.jsonl \
  --judge-combos gemini_pair claude_stack grok_stack \
  --baseline gemini_pair \
  --collect-audit \
  --output results/reward/latest.json
```

**Key options**

* `--dataset` – supply datasets collected with `scripts/collect_reward_trajectories.py`.
* `--repeats` – multiple passes to quantify variance.
* `--concurrency` – concurrent judge evaluations per combo.
* `--markdown-output` – write Markdown summaries alongside JSON artifacts.
* `--collect-audit` – include serialized prompts/responses for debugging reward prompts.

**Metrics reported**

* Reward mean, standard deviation, and uncertainty per combo.
* Escalation rates, failure counts, and agreement vs the baseline stack.
* Latency statistics (average, median, p95).

Use this harness whenever you adjust judge prompts/configs in either repo. Treat the outputs as experimental telemetry
and archive them with your CI artifacts.

## Guarding Reward Schemas Inside Atlas Core

Atlas Core still ships the `tests/test_reward_schema.py` regression suite to ensure the trainer-side configs stay in sync
with SDK telemetry:

```bash theme={null}
pytest tests/test_reward_schema.py -q
```

It validates:

* Reward config `_target_` paths and prompt references in `src/atlas_core/configs/reward/`.
* Schema compatibility with the latest SDK telemetry (per-judge fields, optional blocks, etc.).
* Trainer exports (`src/atlas_core/training/__init__.py`) so downstream imports keep working.

Run it in CI alongside the SDK harnesses above whenever you edit reward prompts or trainer configs.

## Related Reading

* [`Learning System Architecture`](/sdk/learning-system) – how playbooks are synthesized and stored.
* [`Runtime Safety & Review`](/sdk/runtime-safety) – guardrails that influence which sessions enter the harnesses.
* [`Database Schema`](/reference/database-schema) – table-level reference for the telemetry each harness pulls.


# Evaluation Methodology
Source: https://docs.arc.computer/benchmarks/evaluation-methodology

Comprehensive testing protocol for verifying ATLAS performance

## Core Principles

ATLAS evaluation verifies that the adaptive dual-agent loop (student + verifying teacher) improves outcomes without degrading performance for capable students. The framework measures both quantitative metrics and qualitative guidance effectiveness.

<CardGroup>
  <Card title="Non-Degradation" icon="shield">
    Ensure teaching never harms performance (≥97% safety rate)
  </Card>

  <Card title="Efficiency" icon="gauge">
    Measure token reduction and speed improvements
  </Card>

  <Card title="Generalization" icon="globe">
    Validate across diverse tasks and model scales
  </Card>
</CardGroup>

## Evaluation Protocol

### Two-Pass Comparison Framework

<Steps>
  <Step title="Baseline Measurement">
    Run student model independently on evaluation tasks:

    ```python theme={null}
    baseline_response = student_model.generate(task)
    baseline_accuracy = evaluate(baseline_response)
    ```
  </Step>

  <Step title="Dual-Agent Evaluation">
    Apply ATLAS two-pass protocol:

    ```python theme={null}
    # Pass 1: Diagnostic probe (≤50 tokens)
    capability = teacher.diagnose(student_response)

    # Pass 2: Verifying-teacher guidance (≤200 tokens)
    guidance = teacher.generate_guidance(capability, task)
    enhanced_response = student.generate(task, guidance)
    ```
  </Step>

  <Step title="Performance Comparison">
    Calculate improvement metrics:

    ```python theme={null}
    improvement = enhanced_accuracy - baseline_accuracy
    non_degradation = (improvement >= 0)
    efficiency_gain = (baseline_tokens - enhanced_tokens) / baseline_tokens
    ```
  </Step>
</Steps>

### Non-Degradation Verification

Critical safety metric ensuring teaching never makes performance worse:

| Metric                     | Definition                             | Target | Achieved |
| -------------------------- | -------------------------------------- | ------ | -------- |
| NDR (Non-Degradation Rate) | % of interactions with improvement ≥ 0 | ≥99%   | 97%      |
| Degradation Severity       | Average loss when degradation occurs   | \<5%   | 3.2%     |
| Recovery Rate              | % of degraded cases recovered in retry | >80%   | 82%      |

### Efficiency Metrics

Comprehensive measurement of resource utilization:

```python theme={null}
# Teaching Efficiency Score (TES)
TES = (accuracy_gain * completion_rate) / (teaching_tokens / 1000)

# Learning Rate (LR)
LR = Δ_performance / num_interactions

# Token Efficiency
efficiency = 1 - (enhanced_tokens / baseline_tokens)
```

## Evaluation Commands

### Full Benchmark Suite

Complete evaluation with detailed logging:

```bash theme={null}
# Run comprehensive evaluation
scripts/launch_with_server.sh 1 3 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  model_name_or_path=results/pre_rl_model \
  dataset_id_or_path=Arc-Intelligence/Arc-ATLAS-Teach-v0 \
  eval_steps=50 \
  log_completions=true \
  save_completions_probability=0.1 \
  num_generations=32
```

### Quick Validation

Rapid testing for development iterations:

```bash theme={null}
# Minimal evaluation (4 steps)
scripts/launch_with_server.sh 1 1 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  report_to=null \
  max_steps=4 \
  eval_steps=1
```

### Production Evaluation

Full-scale testing with statistical validation:

```bash theme={null}
# Multi-seed evaluation for significance testing
for seed in 42 1337 2024; do
  scripts/launch_with_server.sh 4 4 src/atlas_core/configs/recipe/teacher_rcl.yaml \
    seed=$seed \
    output_dir=results/eval_seed_$seed \
    dataset_id_or_path=Arc-Intelligence/Arc-ATLAS-Teach-v0
done
```

## Data Collection Framework

### Quantitative Metrics

<Tabs>
  <Tab title="Performance">
    * Accuracy improvements vs baseline
    * Task completion rates
    * Per-category performance breakdown
    * Statistical significance (p-values)
  </Tab>

  <Tab title="Efficiency">
    * Token usage reduction
    * Generation time comparison
    * Memory footprint
    * GPU utilization
  </Tab>

  <Tab title="Robustness">
    * Non-degradation rate
    * Cross-model transfer
    * Out-of-distribution performance
    * Failure mode analysis
  </Tab>
</Tabs>

### Qualitative Analysis

Systematic review of teaching quality:

1. **Diagnostic Accuracy**: How well does the probe identify capability gaps?
2. **Teaching Relevance**: Is guidance targeted to identified weaknesses?
3. **Adaptation Quality**: Does teaching adjust to student skill level?
4. **Failure Patterns**: What causes degradation or teaching failures?

## Statistical Validation

### Significance Testing

All results require statistical validation:

```python theme={null}
from scipy import stats

def validate_improvement(baseline_scores, enhanced_scores):
    # Paired t-test for matched samples
    t_stat, p_value = stats.ttest_rel(enhanced_scores, baseline_scores)

    # Cohen's d for effect size
    diff = np.mean(enhanced_scores - baseline_scores)
    pooled_std = np.sqrt((np.var(baseline_scores) + np.var(enhanced_scores)) / 2)
    cohens_d = diff / pooled_std

    return {
        'significant': p_value < 0.001,
        'p_value': p_value,
        'effect_size': cohens_d,
        'improvement': diff
    }
```

### Sample Size Requirements

| Confidence Level | Effect Size  | Required Samples  |
| ---------------- | ------------ | ----------------- |
| 95%              | Large (0.8)  | 26 per condition  |
| 95%              | Medium (0.5) | 64 per condition  |
| 99%              | Large (0.8)  | 42 per condition  |
| 99%              | Medium (0.5) | 106 per condition |

## Expected Outcomes

Successful evaluation demonstrates:

<CardGroup>
  <Card title="Closed-Loop Accuracy" icon="bullseye">
    **+15–30%** lift with the dual-agent runtime (student + verifying teacher)
  </Card>

  <Card title="Offline GRPO" icon="chart-line">
    Sustained improvements by training on exported runtime traces
  </Card>

  <Card title="Completion" icon="check">
    **\~100%** vs \~69% baseline
  </Card>

  <Card title="Efficiency" icon="compress">
    **\~50%** token reduction with teaching
  </Card>
</CardGroup>

## Error Analysis Framework

### Failure Mode Categorization

| Category            | Frequency | Mitigation                 |
| ------------------- | --------- | -------------------------- |
| Parsing errors      | 2.1%      | Improved normalization     |
| Over-teaching       | 0.9%      | Adaptive threshold tuning  |
| Capability mismatch | 0.5%      | Enhanced diagnostic probes |
| Template failures   | 0.3%      | Expanded template coverage |

### Diagnostic Accuracy

Measure probe effectiveness:

```python theme={null}
def evaluate_diagnostic_accuracy(probe_results, actual_performance):
    # Categories: weak, medium, strong
    predicted_level = categorize_capability(probe_results)
    actual_level = categorize_performance(actual_performance)

    accuracy = (predicted_level == actual_level).mean()
    confusion_matrix = create_confusion_matrix(predicted_level, actual_level)

    return accuracy, confusion_matrix
```

## Scalability Testing

### Model Size Scaling

| Student Model | Teacher Model | Improvement | Efficiency |
| ------------- | ------------- | ----------- | ---------- |
| 4B params     | 8B params     | +18.2%      | 0.42 TES   |
| 7B params     | 8B params     | +15.7%      | 0.38 TES   |
| 13B params    | 8B params     | +12.3%      | 0.35 TES   |
| 70B params    | 8B params     | +8.9%       | 0.31 TES   |

### Infrastructure Scaling

| Configuration | Throughput | Latency (p50) | Latency (p99) |
| ------------- | ---------- | ------------- | ------------- |
| 1×T4 GPU      | 2 req/min  | 30s           | 45s           |
| 4×A100        | 16 req/min | 3.75s         | 5.2s          |
| 8×H100        | 64 req/min | 0.94s         | 1.3s          |

## Reproducibility Requirements

<AccordionGroup>
  <Accordion title="Environment Specification">
    ```yaml theme={null}
    # Required in reproduction logs
    hardware:
      gpus: 4×H100
      memory: 128GB
      interconnect: NVLink
    software:
      python: 3.11.4
      pytorch: 2.1.0
      transformers: 4.36.0
      vllm: 0.2.7
    ```
  </Accordion>

  <Accordion title="Configuration Documentation">
    ```bash theme={null}
    # Save all overrides
    echo "Configuration:" > eval_config.txt
    echo "model_name_or_path=$MODEL" >> eval_config.txt
    echo "dataset_id_or_path=$DATASET" >> eval_config.txt
    echo "seed=$SEED" >> eval_config.txt
    ```
  </Accordion>

  <Accordion title="Artifact Preservation">
    * Training logs (`wandb` or `tensorboard`)
    * Metric summaries (JSON format)
    * Representative examples (10% sampling)
    * Configuration files (complete YAML)
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup>
  <Card title="Reproduction Guide" icon="copy" href="/benchmarks/reproduction">
    Reproduce our findings
  </Card>

  <Card title="Run Evaluation" icon="play" href="/training/offline/grpo-training">
    Start your evaluation
  </Card>
</CardGroup>


# Reproduction Guide
Source: https://docs.arc.computer/benchmarks/reproduction

Step-by-step instructions to reproduce ATLAS benchmark results

## Overview

This guide provides exact steps to reproduce the closed-loop **+15.7% accuracy improvement** and related metrics reported in our technical documentation. Once you reproduce the baseline, export the traces and run our offline GRPO pipeline to train a bespoke teacher checkpoint for your domain.

<Note>
  Reproduction requires 4×H100 GPUs for full-scale training. For smaller-scale validation, see the [Quick Validation](#quick-validation) section.
</Note>

## Set up Environment

### Hardware Requirements

<CardGroup>
  <Card title="Full Reproduction" icon="server">
    * 4×H100 80GB GPUs
    * NVLink interconnect
    * 128GB system RAM
    * 500GB NVMe storage
  </Card>

  <Card title="Quick Validation" icon="laptop">
    * 1×A100 40GB GPU
    * 32GB system RAM
    * 100GB storage
    * \~4 hours runtime
  </Card>
</CardGroup>

### Software Stack

```bash theme={null}
# Python environment
python --version  # 3.11 or 3.12 required
pip install -r requirements.txt

# Verify CUDA
nvidia-smi
python -c "import torch; print(f'PyTorch: {torch.__version__}')"
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"

# Authenticate with Hugging Face
huggingface-cli login
```

### Configuration Files

Key configuration files for reproduction:

```yaml theme={null}
# src/atlas_core/configs/recipe/teacher_sft.yaml
model_name_or_path: Qwen/Qwen3-8B-Instruct-2507
dataset_id_or_path: Arc-Intelligence/Arc-ATLAS-Teach-v0
output_dir: results/pre_rl_model
seed: 42
num_train_epochs: 1

# src/atlas_core/configs/recipe/teacher_rcl.yaml
model_name_or_path: results/pre_rl_model
dataset_id_or_path: Arc-Intelligence/Arc-ATLAS-Teach-v0
num_generations: 32
seed: 42
beta: 0.04
```

## Full Reproduction Steps

<Steps>
  <Step title="Phase 1: SFT Warmup">
    Train the initial supervised fine-tuned model:

    ```bash theme={null}
    scripts/launch.sh 4 src/atlas_core/configs/recipe/teacher_sft.yaml \
      dataset_id_or_path=Arc-Intelligence/Arc-ATLAS-Teach-v0 \
      output_dir=results/pre_rl_model \
      seed=42
    ```

    **Expected duration**: 4-8 hours on 4×H100
    **Checkpoint size**: \~16GB
    **Key metric**: Loss \< 0.5
  </Step>

  <Step title="Phase 2: GRPO Training">
    Run reinforcement learning with vLLM server:

    ```bash theme={null}
    scripts/launch_with_server.sh 1 3 src/atlas_core/configs/recipe/teacher_rcl.yaml \
      model_name_or_path=results/pre_rl_model \
      dataset_id_or_path=Arc-Intelligence/Arc-ATLAS-Teach-v0 \
      num_generations=32 \
      seed=42 \
      beta=0.04
    ```

    **Expected duration**: 24-48 hours on 4×H100
    **Key metrics**:

    * Reward > 0.5
    * KL divergence \< 10
    * Non-degradation rate > 95%
  </Step>

  <Step title="Phase 3: Evaluation">
    Validate final performance with the lightweight Transformers snippet below (no additional repo files required):

    ```bash theme={null}
    python - <<'PY'
    from datasets import load_dataset
    from transformers import AutoModelForCausalLM, AutoTokenizer

    MODEL_PATH = "results/final_model"
    DATASET = "Arc-Intelligence/Arc-ATLAS-Teach-v0"
    DATASET_CONFIG = "rl"
    SAMPLES = 32

    dataset = load_dataset(DATASET, DATASET_CONFIG, split="validation").shuffle(seed=42).select(range(SAMPLES))
    tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
    model = AutoModelForCausalLM.from_pretrained(MODEL_PATH, device_map="auto")

    correct = 0
    for example in dataset:
        inputs = tokenizer(example["prompt"], return_tensors="pt").to(model.device)
        output = model.generate(**inputs, max_new_tokens=256)
        prediction = tokenizer.decode(output[0], skip_special_tokens=True)
        if example["ground_truth"].strip().lower() in prediction.lower():
            correct += 1

    accuracy = correct / SAMPLES
    print(f"Accuracy over {SAMPLES} samples: {accuracy:.2%}")
    PY
    ```

    **Expected results (closed-loop runtime + GRPO)**:

    * Accuracy improvement: +15.7% ± 1.2%
    * Completion rate: +31% ± 2%
    * Non-degradation: ≥97%
    * Token savings: \~50%

    <Note>
      To continue beyond the baseline, export the traces with the SDK and launch `atlas-core offline-pipeline --export-path traces/runtime.jsonl` to begin GRPO training.
    </Note>

    * Completion rate: \~100%
    * Token reduction: \~50%
  </Step>
</Steps>

## Quick Validation

For rapid testing without full training:

```bash theme={null}
# Download pre-trained checkpoint
huggingface-cli download Arc-Intelligence/ATLAS-8B-Thinking \
  --local-dir checkpoints/teacher

# Run minimal training (4 steps)
scripts/launch_with_server.sh 1 1 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  model_name_or_path=checkpoints/teacher \
  max_steps=4 \
  eval_steps=1 \
  report_to=null

# Verify performance (reuse the evaluation snippet above with fewer samples)
python - <<'PY'
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "checkpoints/teacher"
DATASET = "Arc-Intelligence/Arc-ATLAS-Teach-v0"
DATASET_CONFIG = "rl"
SAMPLES = 16

dataset = load_dataset(DATASET, DATASET_CONFIG, split="validation").shuffle(seed=7).select(range(SAMPLES))
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
model = AutoModelForCausalLM.from_pretrained(MODEL_PATH, device_map="auto")

correct = 0
for example in dataset:
    inputs = tokenizer(example["prompt"], return_tensors="pt").to(model.device)
    prediction = model.generate(**inputs, max_new_tokens=256)
    decoded = tokenizer.decode(prediction[0], skip_special_tokens=True)
    if example["ground_truth"].strip().lower() in decoded.lower():
        correct += 1

print(f"Quick validation accuracy ({SAMPLES} samples): {correct / SAMPLES:.2%}")
PY
```

## Expected Metrics

After successful reproduction, you should observe:

| Metric                              | Expected Value                                  | Tolerance     |
| ----------------------------------- | ----------------------------------------------- | ------------- |
| Average accuracy gain (closed loop) | +15.7%                                          | ±1.2%         |
| Max improvement (closed loop)       | +29.6%                                          | ±2.1%         |
| Completion rate                     | \~100%                                          | ±2%           |
| Token reduction                     | 50%                                             | ±5%           |
| Generation speedup                  | 13.6%                                           | ±2%           |
| Non-degradation rate                | 97%                                             | ±1%           |
| Offline GRPO gain                   | Sustained lift from training on exported traces | Compute-bound |

## Monitoring Training

### Real-time Metrics

```bash theme={null}
# TensorBoard monitoring
tensorboard --logdir results/ --port 6006

# vLLM server health
watch -n 5 'curl -s http://localhost:8765/metrics'

# GPU utilization
nvidia-smi dmon -s u -d 5
```

### Key Indicators

<Tabs>
  <Tab title="Healthy Training">
    * GPU utilization > 90%
    * Reward trending upward
    * KL divergence stable (5-15)
    * Loss decreasing smoothly
    * No NaN/Inf values
  </Tab>

  <Tab title="Issues to Watch">
    * GPU utilization \< 70% → Check data loading
    * Reward plateauing → Adjust learning rate
    * KL divergence > 20 → Increase beta
    * Loss spikes → Check for bad samples
    * OOM errors → Reduce batch size
  </Tab>
</Tabs>

## Troubleshooting

<AccordionGroup>
  <Accordion title="CUDA Out of Memory">
    ```bash theme={null}
    # Add gradient checkpointing
    scripts/launch_with_server.sh 1 3 src/atlas_core/configs/recipe/teacher_rcl.yaml \
      gradient_checkpointing=true \
      per_device_train_batch_size=1 \
      gradient_accumulation_steps=32
    ```
  </Accordion>

  <Accordion title="vLLM Server Connection Failed">
    ```bash theme={null}
    # Check if port is in use
    lsof -i :8765

    # Use alternative port
    scripts/launch_with_server.sh 1 3 src/atlas_core/configs/recipe/teacher_rcl.yaml \
      vllm_port=8766
    ```
  </Accordion>

  <Accordion title="Slow Training Speed">
    ```bash theme={null}
    # Enable optimizations
    export TORCH_COMPILE=1
    export FLASH_ATTENTION=1

    scripts/launch_with_server.sh 1 3 src/atlas_core/configs/recipe/teacher_rcl.yaml \
      tf32=true \
      dataloader_num_workers=4
    ```
  </Accordion>

  <Accordion title="Authentication Issues">
    ```bash theme={null}
    # Re-login to Hugging Face
    huggingface-cli logout
    huggingface-cli login

    # Verify access
    huggingface-cli download Arc-Intelligence/ATLAS-8B-Thinking README.md
    ```
  </Accordion>
</AccordionGroup>

## Validation Snippets

### Statistical Significance Test

Drop this snippet into any Python session (or save it as `tools/validate_significance.py`) to compare baseline vs enhanced runs:

```python theme={null}
import numpy as np
from scipy import stats

def validate_improvement(baseline_file, enhanced_file):
    baseline = np.load(baseline_file)['accuracy']
    enhanced = np.load(enhanced_file)['accuracy']

    t_stat, p_value = stats.ttest_rel(enhanced, baseline)
    print(f"Improvement: {np.mean(enhanced - baseline):.3f}")
    print(f"P-value: {p_value:.6f}")
    print(f"Significant: {p_value < 0.001}")
```

### Performance Verification Checklist

Use this helper to confirm the reproduced metrics stay within tolerance bands before sharing results:

```python theme={null}
def verify_benchmarks(metrics):
    expected = {
        'accuracy_gain': (0.157, 0.012),  # mean, tolerance
        'completion_rate': (1.0, 0.02),
        'token_reduction': (0.5, 0.05),
        'speed_gain': (0.136, 0.02)
    }

    for metric, (expected_val, tolerance) in expected.items():
        actual = metrics[metric]
        within_tolerance = abs(actual - expected_val) <= tolerance
        status = "PASS" if within_tolerance else "FAIL"
        print(f"{metric}: {actual:.3f} (expected {expected_val:.3f} ±{tolerance:.3f}) [{status}]")
```

## Artifact Management

### Required Outputs

Save these artifacts for verification:

```bash theme={null}
results/
├── pre_rl_model/         # SFT checkpoint
├── final_model/          # GRPO checkpoint
├── eval_results.json     # Evaluation metrics
├── training_logs/        # TensorBoard logs
├── config_used.yaml      # Exact configuration
└── environment.txt       # pip freeze output
```

### Sharing Results

```bash theme={null}
# Package for sharing
tar -czf atlas_reproduction.tar.gz \
  results/eval_results.json \
  results/config_used.yaml \
  results/environment.txt

# Upload to Hugging Face
huggingface-cli upload your-org/atlas-reproduction \
  atlas_reproduction.tar.gz
```

## Next Steps

<CardGroup>
  <Card title="Methodology" icon="flask" href="/benchmarks/evaluation-methodology">
    Understand evaluation protocol
  </Card>

  <Card title="Deploy Model" icon="rocket" href="/sdk/quickstart">
    Use your trained model
  </Card>
</CardGroup>


# Adaptive Dual-Agent Reasoning
Source: https://docs.arc.computer/concepts/adaptive-dual-agent-reasoning

How your agent partners with a verifying teacher to deliver safer, higher-quality results.

ATLAS hinges on an adaptive dual-agent reasoning loop: your production agent (the **student**) stays frozen, while a specialized **verifying teacher** evaluates its plan, provides targeted guidance, and confirms the final answer. The partnership boosts quality without touching model weights and works across any provider—API-only or self-hosted.

Think of it as pairing every agent run with an expert reviewer. The reviewer doesn’t replace your agent; it inspects the approach, corrects mistakes, and signs off before results ship to users.

## Core Concept

* **Student (your agent)** – Any LLM or tool stack executing the task (GPT, Claude, Gemini, local checkpoints, custom code).
* **Verifying teacher** – An 8B specialized model trained to diagnose gaps, inject guidance, and certify answers.
* **Outcome** – Better answers, higher safety, and richer telemetry—without retraining the underlying agent.

### Why it’s model agnostic

| Traditional approach           | Adaptive dual-agent reasoning                     |
| ------------------------------ | ------------------------------------------------- |
| Retrain or fine-tune the agent | Keep the agent frozen and add a verifying teacher |
| Requires model access & GPUs   | Works with API-only models                        |
| Risk of regression             | Preserves baseline capabilities                   |
| Weeks to deploy updates        | Hours to roll out new teacher checkpoints         |

For runtime implementation details, see [`How Orchestration Works`](/sdk/orchestration) for the runtime [`personas`](/reference/glossary#persona) and [`Offline Training`](/training/offline/grpo-training) to learn how new teachers are trained.

## Why it works

### 1. Asymmetric specialization

The teacher focuses solely on teaching. It doesn’t need to outperform the agent at solving tasks; it needs to spot blind spots, orchestrate retries, and provide precise interventions.

> **Analogy:** A senior reviewer doesn’t code faster than the whole team—they prevent critical mistakes, guide architecture decisions, and approve releases.

### 2. Inference-time enhancement

Guidance happens through prompts at runtime:

1. The teacher inspects the task, telemetry, and prior attempts.
2. It scores capability, triages risk, and drafts guidance.
3. The guidance is merged into the agent’s context.
4. The agent re-runs with the added teaching and produces the final answer.

No gradient steps, checkpoints, or weight updates are required.

### 3. Adaptive intensity

The teacher adjusts effort based on confidence:

* **High confidence:** Light-touch verification or a single checklist.
* **Medium confidence:** Paired review of the final result.
* **Low confidence:** Step-by-step coaching with retries.

You pay only for the oversight you need, run-by-run.

## Deployment requirements

| Component         | Specification       | Purpose                                      |
| ----------------- | ------------------- | -------------------------------------------- |
| Verifying teacher | 8B RL-trained model | Generates adaptive guidance & certifications |
| Student agent     | Any size / provider | Executes the actual work                     |
| Context window    | 4k–32k tokens       | Accommodates guidance + agent output         |
| Latency overhead  | \~30%               | Extra pass for analysis and teaching         |

## Performance snapshot (τ²-bench, mms\_issue subset)

| System                  | Pass\@1   | Notes                              |
| ----------------------- | --------- | ---------------------------------- |
| **ATLAS dual-agent**    | **24.0%** | Minimal degradation across retries |
| GPT-4.1                 | 18.0%     | −8 pts from Pass\@1 to Pass\@4     |
| Claude 3.7 Sonnet       | 18.0%     | −16 pts drop across retries        |
| o4-mini                 | 12.0%     | −10 pts drop across retries        |
| Qwen3-8B (student only) | 4.1%      | No teacher guidance                |

**Key takeaways**

* **6× lift** on the same student by adding the verifying teacher.
* **Stable retries:** The teacher keeps success rates high on subsequent attempts.
* **Cross-domain transfer:** A math-trained teacher can supervise telecom debugging tasks because it enforces process, not domain answers.

### Aggregate impact

* **Average accuracy gain (runtime + GRPO):** +15.7 %
* **Maximum domain lift:** +29.6 %
* **Non-degradation rate:** 97 %
* **Token efficiency:** \~50 % reduction
* **Completion rate:** +31 %

## Benefits vs. other approaches

### Fine-tuning / RLHF

* No retraining or weight access required.
* Zero risk of catastrophic forgetting.
* Deploy new guidance in hours, not weeks.

### Prompt engineering

* Adaptive, not one-shot tuning.
* Systematic and measurable improvements.
* Token use scales with confidence (fast lanes stay cheap).

### Ensembles

* Single agent executes the work; no multi-model voting.
* Lower cost and latency for comparable quality.
* Guidance is inspectable and auditable.

## Training the verifying teacher

1. **Supervised warmup (SFT)** – Teach baseline review behaviors (4–6 hours on 8× H100).
2. **GRPO fine-tuning** – Optimize for student improvement and calibrated confidence (24–36 hours on 8× H100).

Rewards come from measured student gains, so the teacher is incentivized to deliver guidance that genuinely improves outcomes.

## Integration patterns

### SDK runtime orchestration

Integrate via the SDK to wrap your agent with adaptive lanes (auto, paired, coach). The runtime logs every teaching decision and reward signal for later analysis.

### Offline GRPO training

Export runtime traces (`arc-atlas --output traces.jsonl`) and run `atlas-core offline-pipeline --export-path <traces>.jsonl` to produce updated teacher checkpoints. Point the SDK back at the new weights to close the loop.

## Best practices

<AccordionGroup>
  <Accordion title="Selecting the teacher checkpoint">
    * **ATLAS-8B-Thinking**: Analytical, math-heavy domains
    * **ATLAS-8B-Instruct**: Code generation, structured workflows
    * **Custom GRPO**: Train on your exported traces for domain-specific oversight
  </Accordion>

  <Accordion title="Qualifying student agents">
    Confirm the agent supports:

    * System prompts or instruction conditioning
    * Sufficient context window (>4k tokens)
    * Deterministic decoding (control temperature/top\_p)
  </Accordion>

  <Accordion title="Operational tips">
    * Cache teacher guidance for repetitive tickets
    * Batch similar tasks for throughput
    * Stream intermediate verdicts for human-on-the-loop monitoring
    * Track token budgets per lane to manage cost
  </Accordion>
</AccordionGroup>

## Next steps

<CardGroup>
  <Card title="Runtime Orchestration" icon="arrows-rotate" href="/sdk/orchestration">
    Dive into the lane logic, telemetry, and orchestration flow.
  </Card>

  <Card title="Deploy to Production" icon="plug" href="/sdk/quickstart">
    Learn how to operate the dual-agent loop in production infrastructure.
  </Card>

  <Card title="Reward System" icon="trophy" href="/concepts/reward-design">
    Understand how guidance quality is quantified.
  </Card>

  <Card title="Production Example" icon="wrench" href="/examples/adaptive-tool-use">
    Explore τ²-bench results in detail.
  </Card>

  <Card title="Offline Training" icon="dumbbell" href="/training/offline/grpo-training">
    Run SFT + GRPO to evolve your verifying teacher.
  </Card>
</CardGroup>

## References

* [ATLAS Technical Report](/reference/technical-report) — Architecture and evaluations
* [Offline Training Guide](/training/offline/grpo-training) — Hands-on teacher training walkthrough
* [Adaptive Tool Use](/examples/adaptive-tool-use) — Production example with MCP integration


# Hybrid Learning Architecture
Source: https://docs.arc.computer/concepts/hybrid-learning

Understanding ATLAS's dual-phase approach to model enhancement

## What is Hybrid Learning?

ATLAS separates learning into two phases: **offline foundation training** (one-time, compute-intensive) and **runtime continual learning** (continuous, lightweight). This architecture solves the enterprise AI constraint: lack of high-quality preference data for complex business tasks.

**Key insight:** Train a teacher once on math reasoning (where data is abundant and logic is clear), then apply that reasoning to any domain (CRM, telecom, debugging) without domain-specific retraining.

## The Two-Phase Paradigm

### Phase 1: Offline Foundation Training (Atlas Core)

Establish deep, generalizable skills through reinforcement learning:

```
Offline RL Training (24-48 hours)
├── SFT Warmup: Base reasoning capabilities
├── GRPO Training: Adaptive teaching skills
└── Output: Teacher model with foundational knowledge
```

| Characteristic      | Detail                                                                    |
| ------------------- | ------------------------------------------------------------------------- |
| Compute requirement | Minimum 2 GPUs (1 for vLLM, 1 for training)                               |
| Training data       | \~900 curated dual-agent demonstrations from Arc-ATLAS-Teach-v0           |
| Foundation domain   | Mathematics (reasoning, sequential thinking, problem decomposition)       |
| Transfer capability | Math-trained reasoning generalizes to debugging, coding, analytical tasks |
| Cost model          | One-time; amortized over all deployments                                  |

### Phase 2: Runtime Continual Learning (Atlas SDK)

The [atlas-sdk](https://github.com/Arc-Computer/atlas-sdk) runtime adapts pre-trained teachers to specific tasks:

```
Runtime Loop (continuous)
├── Task Analysis: Identify performance gaps via rewards
├── Experimentation: Adjust teaching prompts and strategies
├── Trace Export: Capture high-signal interactions
└── Output: Data for next GRPO cycle + incremental improvements
```

| Characteristic | Detail                                            |
| -------------- | ------------------------------------------------- |
| Infrastructure | Managed APIs in SDK                               |
| Speed          | Improves over hours vs full retraining cycles     |
| Safety         | Non-degradation guarantee via reward guardrails   |
| Feedback loop  | Feeds fresh traces into next offline training job |

## Performance Comparison

| Training Approach      | Time to Deploy | Performance Gain | Cost      | Generalization |
| ---------------------- | -------------- | ---------------- | --------- | -------------- |
| Fine-tuning            | 1-2 weeks      | +10-15%          | \$1000s   | Poor           |
| Few-shot prompting     | Minutes        | +3-5%            | \~\$1     | Limited        |
| ATLAS (Runtime + GRPO) | Hours          | +15.7% baseline  | API + GPU | Excellent      |

\*With pre-trained teacher models

## Cross-Domain Transfer Results

### Why Mathematics as Foundation?

* **Clear correctness**: Verifiable ground truth (unlike business tasks)
* **Abundant data**: Thousands of well-structured problems
* **Pure reasoning**: Systematic thinking, problem decomposition, logical flow
* **Complexity gradient**: Simple arithmetic → AIME-level competition problems

Our Teacher trained on \~7,000 math problems achieved **46% accuracy on AIME-25** (top-10 SOTA level).

### Validated Transfer

**Mathematics → Telecom (τ²-bench):**

* Teacher trained only on math problems
* Applied to telecom troubleshooting (no telecom training)
* Result: 24.0% pass\@1 (vs 18.0% for GPT-4.1, Claude 3.7)

**Mathematics → CRM (CRMArena-Pro):**

* Same math-trained teacher
* Applied to policy compliance tasks
* Result: 54% task completion (vs \~35% for leading models)
* Critical accuracy: 69.2% identifying policy violations

→ Full methodology in [Technical Report](/reference/technical-report)

## Quick Start

<Steps>
  <Step title="Offline Foundation">
    Download pre-trained teacher or train custom:

    ```bash theme={null}
    # Option 1: Pre-trained
    huggingface-cli download Arc-Intelligence/ATLAS-8B-Thinking

    # Option 2: Custom (2+ GPUs)
    scripts/launch.sh 2 src/atlas_core/configs/recipe/teacher_sft.yaml
    scripts/launch_with_server.sh 1 1 src/atlas_core/configs/recipe/teacher_rcl.yaml
    ```
  </Step>

  <Step title="Run GRPO on Exported Traces">
    ```bash theme={null}
    atlas-core offline-pipeline \
      --export-path traces/runtime.jsonl \
      --wandb-project atlas-production
    ```
  </Step>

  <Step title="Deploy Enhanced Model">
    Point SDK at new checkpoint:

    ```yaml theme={null}
    teacher:
      llm:
        provider: huggingface
        model: /models/atlas-teacher-grpo
        temperature: 0.2
    ```
  </Step>
</Steps>

## Next Steps

<CardGroup>
  <Card title="Adaptive Dual-Agent Reasoning" icon="chalkboard-user" href="/concepts/adaptive-dual-agent-reasoning">
    Two-pass dual-agent mechanism
  </Card>

  <Card title="Offline Training Guide" icon="flask" href="/training/offline/grpo-training">
    Run GRPO training pipeline
  </Card>

  <Card title="Training Configuration" icon="diagram-project" href="/training/configuration">
    Hydra composition and overrides
  </Card>

  <Card title="SDK Runtime" icon="layer-group" href="/sdk/quickstart">
    Export traces and manage runtime learning
  </Card>
</CardGroup>

## References

* [ATLAS Technical Report](/reference/technical-report) - Sections 3.1-3.3 on hybrid architecture
* [GRPO Algorithm](https://arxiv.org/abs/2402.03300) - Foundation for offline training
* [SDK Runtime Guide](/sdk/quickstart) - Export traces and continual learning


# The ATLAS Reward System
Source: https://docs.arc.computer/concepts/reward-design

How ATLAS measures if teaching actually works

How do we know if teaching actually worked? ATLAS uses a team of AI judges to score every interaction.

Instead of a single reward model that can be biased or brittle, ATLAS uses a multi-agent ensemble. Think of it like a medical panel: a team of general practitioners makes an initial diagnosis, and when they disagree, a specialist makes the final call.

This achieves **93.7% accuracy on RewardBench V2** while keeping costs low.

## The Two-Tier System

<div>
  <img alt="ATLAS Reward System Architecture" />

  <p><em>Tier 1: Fast ensemble evaluation → Tier 2: Expert arbiter when needed</em></p>
</div>

### How It Works

**Tier 1: The Initial Team**

* Multiple efficient models (like `gemini-2.5-flash`) run in parallel
* Each runs at different temperatures for diverse perspectives
* They all score the same interaction independently
* Fast and cheap for most cases

**Tier 2: The Expert Arbiter**

* Only called when the team disagrees (high variance in scores)
* Or when any judge reports low confidence
* A more powerful model (like `gemini-2.5-pro`) reviews everything
* Makes the final decision with full context

**The key insight**: Most cases are clear-cut and don't need the expensive expert. When there's genuine ambiguity, escalate to the specialist.

### When Escalation Happens

The system escalates to Tier 2 when either:

* **High disagreement**: Standard deviation of scores exceeds the threshold (default: 0.15)
* **Low confidence**: Any judge reports high uncertainty (default: >0.3)

Otherwise, it uses the most confident judgment from Tier 1—saving both time and money.

## Session-Level Evaluation

The reward system evaluates the complete trajectory after execution finishes. It derives 2-3 weighted principles tailored to the specific session, scores the trajectory against those principles, and extracts behavioral patterns for future learning.

### How It Works

The evaluator receives the full session context—task, plan, steps, final answer, execution mode—and generates a structured evaluation:

```json theme={null}
{
  "principles": [
    {"name": "Correctness", "weight": 0.5, "description": "Final deliverable matches requirements"},
    {"name": "Safety", "weight": 0.3, "description": "No policy violations detected"},
    {"name": "Efficiency", "weight": 0.2, "description": "Minimal retries needed"}
  ],
  "score": 0.85,
  "rationale": "Response solves the task correctly with efficient execution",
  "uncertainty": 0.1,
  "student_learning": "For straightforward tasks, proceed directly to solution without exploratory steps",
  "teacher_learning": null
}
```

**Key components:**

* **Principles**: Domain-relevant evaluation criteria with weights (sum to 1.0)
* **Score**: Aggregated result in \[0.0, 1.0] range
* **Rationale**: Explanation grounded in the principles
* **Student learning**: Cross-domain behavioral pattern to remember (not task-specific content)
* **Teacher learning**: Pedagogical strategy that worked (when teacher provided guidance)

This makes every score fully auditable—you can see which principles were applied and why the judgment was made.

## Defining Domain Objectives

The judge prompt system lets you express quality criteria in natural language without training custom models. The evaluator derives 2-3 weighted principles tailored to each trajectory, scores against those principles, and reconciles multiple judge opinions through the ensemble flow.

**How it works:**
The `focus_prompt` field in `adaptive_teaching.reward` accepts arbitrary evaluation criteria. The judge reads that prompt, generates domain-relevant principles (e.g., "Correctness: 0.5 weight", "Safety: 0.3 weight"), evaluates the trajectory, and extracts behavioral patterns to store as learning memory.

## Configuration Essentials

The reward system is configured via YAML, but you only need to understand a few key settings:

### Core Settings

```yaml theme={null}
# reward_system/interpretation.yaml
rim:
  # Diversity: More temperatures = more diverse initial opinions
  temperatures: [0.2, 0.5, 0.8]

  # Escalation sensitivity
  variance_threshold: 0.15  # Lower = more escalations to expert

  # Which dimensions to evaluate
  active_judges:
    accuracy: true
    helpfulness: true
    process: true
    diagnostic: true
```

### Key Tuning Knobs

**Want more precision?**

* Lower `variance_threshold` to 0.10 → More cases go to the expert model

**Need faster/cheaper evaluation?**

* Raise `variance_threshold` to 0.20 → Trust the initial team more often
* Reduce `temperatures` to `[0.3, 0.7]` → Fewer ensemble members

**Different use cases?**

* Adjust `variance_threshold` and `uncertainty_threshold` to control escalation frequency
* Use `focus_prompt` to steer evaluation criteria toward specific domain objectives

## Reward System in the Atlas SDK

The SDK runtime uses the same reward philosophy to control its execution loop. The `rim` block in [`configs/examples/openai_agent.yaml`](https://github.com/Arc-Computer/atlas-sdk/blob/main/configs/examples/openai_agent.yaml) wires up the scorekeepers and escalation model:

```yaml theme={null}
# configs/examples/openai_agent.yaml
rim:
  small_model:
    provider: google
    model: gemini/gemini-2.5-flash
    api_key_env: GEMINI_API_KEY
    max_output_tokens: 8096
  large_model:
    provider: google
    model: gemini/gemini-2.5-flash
    api_key_env: GEMINI_API_KEY
    max_output_tokens: 8096
  judge_prompt: 'reward the agent for attending the issues mentioned in the task'
  variance_threshold: 0.15
  uncertainty_threshold: 0.3
```

During orchestration, this configuration tells the runtime how to behave:

1. After execution completes, the session trajectory is evaluated using the **small model** at multiple temperatures for diverse perspectives.
2. If variance across samples exceeds 0.15 or any sample reports uncertainty > 0.3, the system escalates to the **large model** arbiter.
3. The final reward includes derived principles, score, rationale, and extracted learning patterns (student\_learning, teacher\_learning).
4. The reward informs retry decisions and learning memory—patterns are stored for future sessions.

<Tip>
  Want stricter quality control? Lower `variance_threshold` to increase arbiter usage. See the [`SDK Configuration Reference`](/sdk/configuration#reward-system-the-rim-block) for complete syntax.
</Tip>

This mirrors the training world: the runtime uses rewards to keep the agent on track, while the training process uses the same signals to improve the underlying models.

***

## Implementation

For practical usage guides, see [Reward System Implementation](/training/reward-system-usage):

* Integrate with GRPO training
* Run ad-hoc evaluations
* Customize judges and evaluation criteria
* Monitor reward metrics during training

## Next Steps

<CardGroup>
  <Card title="Adaptive Dual-Agent Reasoning" icon="arrows-rotate" href="/concepts/adaptive-dual-agent-reasoning">
    See how the dual-agent workflow and lane logic operate
  </Card>

  <Card title="GRPO Training" icon="graduation-cap" href="/training/offline/grpo-training">
    Use the reward system to train teacher models
  </Card>

  <Card title="SDK Runtime" icon="lightbulb" href="/sdk/quickstart">
    See how rewards flow through the production loop
  </Card>
</CardGroup>

## References

* [Reward System Technical Report](https://www.arc.computer/blog/ATLAS-Reward-System) - Complete methodology and benchmarks
* [ATLAS Technical Report](/reference/technical-report) - How rewards integrate with training
* [RewardBench V2](https://huggingface.co/spaces/allenai/reward-bench) - Benchmark leaderboard


# Adaptive Tool Use with MCP
Source: https://docs.arc.computer/examples/adaptive-tool-use

Production-ready example showing measurable tool efficiency improvements through progressive learning

## Overview

This example demonstrates how Atlas SDK enables agents to learn efficient tool usage patterns. Using the Model Context Protocol (MCP) to provide filesystem tools to a LangGraph agent, the example shows measurable improvement across 25 progressive tasks: 30-40% fewer tool calls and 95%+ completion rates by task 25.

**What you'll see:**

* MCP server with 5 file operation tools
* LangGraph agent integration
* Progressive learning (simple → complex tasks)
* Measurable efficiency gains
* Total cost: \$0.10-0.20 for complete 25-run session

**Repository:** [atlas-sdk/examples/mcp\_tool\_learning](https://github.com/Arc-Computer/atlas-sdk/tree/main/examples/mcp_tool_learning)

## Architecture

```
Learning Harness (25 tasks)
         ↓
   Atlas SDK Core
   (orchestration + rewards)
         ↓
   LangGraph Agent
         ↓
   MultiServerMCPClient
         ↓
   MCP Server
   (5 file operation tools)
```

**Tool inventory:**

* `read_file` - Read file contents
* `write_file` - Write/create files
* `list_files` - List directory contents
* `search_content` - Regex search in files
* `run_command` - Safe shell commands (ls, grep, wc)

## Quick Start

### Prerequisites

```bash theme={null}
pip install arc-atlas langchain-mcp-adapters langchain-openai langgraph mcp anyio
export OPENAI_API_KEY=sk-...
export GEMINI_API_KEY=...
atlas init  # Start Postgres for telemetry
```

### Run Complete Learning Session

```bash theme={null}
cd examples/mcp_tool_learning
python learning_harness.py
```

Executes 25 tasks with progressive complexity:

* **Phase 1 (tasks 1-5):** Basic file operations
* **Phase 2 (tasks 6-10):** Multi-step operations
* **Phase 3 (tasks 11-15):** Complex workflows
* **Phase 4 (tasks 16-20):** Advanced scenarios
* **Phase 5 (tasks 21-25):** Edge cases and error handling

### Run Single Task

```bash theme={null}
atlas run --config examples/mcp_tool_learning/config.yaml \
          --task "List all files in sample_workspace and read notes.txt"
```

## Learning Objectives

The agent learns to:

1. **Minimize redundant operations** - Cache file lists instead of listing repeatedly
2. **Optimize tool selection** - Choose search vs read based on task requirements
3. **Handle errors gracefully** - Recover from missing files and invalid operations
4. **Plan efficiently** - Break complex tasks into minimal step sequences
5. **Build context awareness** - Understand when list → read → write sequence is optimal

## Measured Results

### Early Runs (Tasks 1-5)

* Tool calls per task: 8-12 (trial and error)
* Reward scores: 0.6-0.7
* Occasional incorrect tool selection

### Later Runs (Tasks 15-25)

* Tool calls per task: 4-6 (optimized)
* Reward scores: 0.8-0.9
* Consistent correct tool selection
* Proactive error handling

**Key Metrics:**

* Tool call reduction: 30-40%
* Completion rate: 95%+ by task 25
* Reward progression: +0.2-0.3 average increase

## Configure the Agent

The example uses a Python adapter to integrate the LangGraph agent:

```yaml theme={null}
agent:
  type: python
  import_path: examples.mcp_tool_learning.mcp_agent
  attribute: create_agent
```

Reward system provides learning signals for efficient tool usage:

```yaml theme={null}
rim:
  judge_prompt: |
    Reward effective tool usage:
    - Correct tool for each task
    - Minimal redundant operations
    - Proper error handling
```

## Viewing Learning Progress

### Check Learning Playbook

```bash theme={null}
python -m atlas.cli.learning --project mcp-tool-learning
```

Shows:

* Tool usage patterns over time
* Reward progression
* Common failure modes
* Synthesized best practices

### Export Session Traces

```bash theme={null}
arc-atlas --database-url postgresql://atlas:atlas@localhost:5433/atlas \
          --output mcp_traces.jsonl \
          --limit 25
```

### Query Database Directly

```sql theme={null}
SELECT session_id, task, (reward_stats->>'score')::float as reward, created_at
FROM sessions
WHERE metadata->>'learning_key' = 'mcp-tool-learning'
ORDER BY session_id DESC
LIMIT 25;
```

## Customization

### Add Domain-Specific Tools

Modify `mcp_server.py` to add tools for your use case:

```python theme={null}
@server.call_tool()
async def database_query(query: str) -> str:
    """Execute safe database queries"""
    # Your implementation
    return results
```

### Adjust Learning Tasks

Edit `LEARNING_TASKS` in `learning_harness.py`:

```python theme={null}
LEARNING_TASKS = [
    "Your domain-specific task 1",
    "Your domain-specific task 2",
    # ... progressive complexity
]
```

### Tune Reward Signals

Update `judge_prompt` in `config.yaml` to reward domain-specific behaviors:

```yaml theme={null}
rim:
  judge_prompt: |
    Reward effective database operations:
    - Efficient query construction
    - Proper index usage
    - Connection pooling
```

## Troubleshooting

| Issue                        | Solution                                                          |
| ---------------------------- | ----------------------------------------------------------------- |
| MCP server connection errors | Verify server\_path in mcp\_agent.py points to correct file       |
| Async event loop errors      | Run with `python learning_harness.py` (not `python -i`)           |
| API rate limits              | Increase sleep duration between tasks in learning\_harness.py     |
| High costs                   | Use GPT-4o-mini for both student and teacher; reduce task count   |
| Postgres connection refused  | Start Postgres with `atlas init` or verify DATABASE\_URL in .env  |
| No learning improvement seen | Ensure storage is enabled and check reward scores in database     |
| Tool calls not reducing      | Verify learning.enabled=true in config and check playbook entries |

## Next Steps

<CardGroup>
  <Card title="Custom Adapters" icon="plug" href="/sdk/adapters">
    Connect your own agent framework
  </Card>

  <Card title="Configuration Guide" icon="sliders" href="/sdk/configuration">
    Tune orchestration and learning parameters
  </Card>

  <Card title="Export Training Data" icon="database" href="/training/offline/training-data-pipeline">
    Use runtime traces for offline training
  </Card>

  <Card title="Learning System" icon="brain" href="/sdk/learning-system">
    Understand persistent memory and playbooks
  </Card>
</CardGroup>

## Related Resources

* [Full source code](https://github.com/Arc-Computer/atlas-sdk/tree/main/examples/mcp_tool_learning)
* [Model Context Protocol documentation](https://modelcontextprotocol.io)
* [LangGraph documentation](https://langchain-ai.github.io/langgraph/)


# Developer Example: Running GKD
Source: https://docs.arc.computer/examples/gkd-dev-example

Step-by-step walkthrough for exporting traces and running AtlasGKDTrainer

## Overview

This guide shows how to take traces captured by the Atlas SDK, distill them with `scripts/validate_gkd.py`, and interpret the resulting metrics. The example uses GSM8K data, but the same steps apply to any dataset—Atlas traces in Postgres or a Hugging Face dataset loaded via `MathGKDDatasetConfig`.

## Export traces from the SDK

1. Run your agent with the Atlas SDK and persist sessions to Postgres via the `storage` block in `atlas.config`. Every approved session (teacher intervention, student attempt, rewards) lives in the same schema Atlas Core expects.
2. Review sessions with `arc-atlas review sessions --database-url <postgres_url> --status pending` and approve the conversations you want to train on.
3. (Optional) Export a JSONL snapshot with `arc-atlas --database-url <postgres_url> --include-status approved --output traces/runtime.jsonl` if you prefer file-based workflows. AtlasGKDTrainer can consume either the live Postgres DB or a JSONL file generated with the same schema.

## Configure GKD (Postgres path)

Ensure `ATLAS_DB_URL` points to the same Postgres instance the SDK writes to, then use the default Hydra config to run distillation:

```bash theme={null}
export ATLAS_DB_URL="postgresql://user:pass@host:5432/atlas"
atlas-core train \
  recipe@_global_=teacher_gkd \
  teacher_model_name_or_path=Qwen/Qwen2.5-14B-Instruct \
  model.model_name_or_path=Qwen/Qwen2.5-7B-Instruct \
  trainer.min_reward=0.8
```

This trains directly from the approved traces in Postgres. Override `trainer.learning_key`, `min_reward`, etc., as needed for your workflow.

## Run the validation script (Hugging Face path)

To validate end-to-end settings on public data, run:

```bash theme={null}
HF_HUB_ENABLE_HF_TRANSFER=1 \
PYTHONPATH=. \
CUDA_VISIBLE_DEVICES=0 \
python scripts/validate_gkd.py \
  --student Qwen/Qwen2.5-7B-Instruct \
  --teacher Qwen/Qwen2.5-14B-Instruct \
  --teacher-tokenizer Qwen/Qwen2.5-14B-Instruct \
  --dataset-name gsm8k \
  --dataset-config main \
  --dataset-train-split train \
  --dataset-eval-split test \
  --dataset-max-samples 8792 \
  --train-limit 7473 \
  --eval-limit 1319 \
  --max-steps 500 \
  --per-device-train-batch-size 2 \
  --gradient-accumulation-steps 4 \
  --learning-rate 2e-5 \
  --lmbda 1.0 \
  --beta 0.5 \
  --temperature 0.9 \
  --max-new-tokens 256 \
  --eval-sample-size 256 \
  --min-reward 0.8 \
  --bf16
```

Set `--dataset-name` / `--dataset-config` to any Hugging Face dataset that contains math or reasoning conversations; the script formats it into the same chat schema the trainer expects. For your own traces, skip the HF flags and let the trainer load from Postgres via `ATLAS_DB_URL`.

> **Cross-tokenizer tip:** The script aligns teacher logprobs with the teacher's chat template automatically. Override the tokenizer path with `--teacher-tokenizer` when it differs from `--teacher`, or disable the behavior entirely with `--no-align-teacher-template` if both models share the same tokenizer and you want to reproduce legacy runs.

## Interpret `math_validation_metrics.json`

After the script finishes, inspect `outputs/gkd_math_validation/math_validation_metrics.json`. It contains:

* `training.train_loss`: final training loss (useful for comparing configs).
* `baseline` and `distilled` blocks: eval accuracy, average generated tokens, etc.
* `success_delta` and `token_reduction_pct` derived from the baseline/d distilled metrics so you can see how the distilled student improved.

Example snippet:

```json theme={null}
{
  "training": {"train_loss": 0.0294},
  "baseline": {"accuracy": 0.758, "avg_generated_tokens": 210},
  "distilled": {"accuracy": 0.815, "avg_generated_tokens": 180}
}
```

Compute success delta (`0.815 - 0.758 = +5.7 pp`) and token reduction (`1 - 180/210 ≈ 14.3%`) to judge whether the run met your targets.

## Next steps

* Use `scripts/examples/run_two_gear_gkd.py` to run the fast and reliability configs back-to-back and automatically print the comparison table.
* Once you have Postgres traces from the Atlas runtime, re-run `atlas-core train recipe@_global_=teacher_gkd` pointing at `ATLAS_DB_URL` to distill your own workflows instead of GSM8K.


# Installation
Source: https://docs.arc.computer/installation

Set up ATLAS environment with validated dependencies

<Note>
  **SDK Only**: 30 seconds • **Full Training Stack**: 10-15 minutes
</Note>

## Choose Your Path

**Most users only need the SDK:**

```bash theme={null}
python -m pip install --upgrade arc-atlas
```

This gives you adaptive dual-agent orchestration, telemetry streaming, and data export. Skip to the [Verification](#verification) section after installation.

<Warning>
  **Only install the full training stack if you need to:**

  * Train custom teacher models with GRPO
  * Run offline reinforcement learning
  * Fine-tune models on your own hardware

  The training stack requires CUDA-capable GPUs, PyTorch 2.6.0, and vLLM 0.8.3. Most teams use pre-trained teacher models and never need this setup.
</Warning>

## System Requirements

<CardGroup>
  <Card title="Minimum Requirements" icon="desktop">
    * 2× NVIDIA GPUs with CUDA support (for RL training)
    * 1× GPU minimum for inference only
    * 32GB+ system RAM
    * 100GB+ disk space
    * Python 3.10 or newer
  </Card>

  <Card title="Recommended Setup" icon="server">
    * 4×H100 or 8×H100 GPUs (40GB+ VRAM each)
    * 128GB+ system RAM
    * 200GB+ NVMe storage
    * Ubuntu 22.04 LTS
  </Card>
</CardGroup>

## Prerequisites

<Warning>
  **Before installing:** Run this 30-second check to verify your system meets requirements.
</Warning>

```bash theme={null}
python - <<'EOF'
import sys
import subprocess

checks = []

# Check Python version
py_version = sys.version_info
checks.append(("Python 3.11 or 3.12", py_version >= (3, 11), f"Found {py_version.major}.{py_version.minor}"))

# Check CUDA
try:
    result = subprocess.run(['nvidia-smi'], capture_output=True, text=True)
    cuda_available = result.returncode == 0
    checks.append(("NVIDIA GPU", cuda_available, "Found" if cuda_available else "Not found"))
except:
    checks.append(("NVIDIA GPU", False, "nvidia-smi not available"))

# Check disk space
import shutil
stat = shutil.disk_usage("/")
free_gb = stat.free / (1024**3)
checks.append(("200GB+ free disk", free_gb >= 200, f"{free_gb:.1f}GB free"))

# Print results
print("\nPrerequisites Check:")
print("-" * 50)
for name, passed, detail in checks:
    status = "✅" if passed else "❌"
    print(f"{status} {name}: {detail}")

all_passed = all(c[1] for c in checks)
print("-" * 50)
if all_passed:
    print("✅ All checks passed! Proceed with installation.")
else:
    print("❌ Some checks failed. Review requirements before installing.")
    sys.exit(1)
EOF
```

**Expected output:**

```
Prerequisites Check:
--------------------------------------------------
✅ Python 3.11 or 3.12: Found 3.11
✅ NVIDIA GPU: Found
✅ 200GB+ free disk: 245.3GB free
--------------------------------------------------
✅ All checks passed! Proceed with installation.
```

<Note>
  **SDK-only users can skip this.** This check is only needed for the full training stack (Atlas Core).
</Note>

<Steps>
  <Step title="Set up CUDA">
    Ensure NVIDIA drivers and CUDA are installed and compatible with PyTorch 2.6.0:

    ```bash theme={null}
    nvidia-smi  # Verify CUDA version
    ```
  </Step>

  <Step title="Python Environment">
    Verify Python version (3.10 or newer required):

    ```bash theme={null}
    python --version
    ```
  </Step>

  <Step title="Authenticate with HuggingFace">
    Authenticate for model and dataset access:

    ```bash theme={null}
    huggingface-cli login
    ```
  </Step>
</Steps>

## Installation Methods

<Tabs>
  <Tab title="Runtime SDK (Minimal)">
    ```bash theme={null}
    python -m pip install --upgrade arc-atlas
    ```

    <Tip>
      Keep credentials such as `ANTHROPIC_API_KEY` in a `.env` file and load them before orchestrating runs. Atlas defaults to Anthropic as the primary provider.
    </Tip>

    After the package installs, bootstrap your project with autodiscovery:

    ```bash theme={null}
    atlas env init --task "Summarize the latest AI news"
    atlas run --config .atlas/generated_config.yaml --task "Summarize the latest AI news"
    ```

    The CLI writes `.atlas/discover.json`, optional factory scaffolds, and metadata snapshots while automatically loading `.env` and extending `PYTHONPATH`. `atlas env init` now handles storage setup automatically—no need to run `atlas init` separately. Re-run `atlas env init --scaffold-config-full` whenever you want a fresh runtime configuration derived from discovery output.
  </Tab>

  <Tab title="Automated Training Setup (Recommended)">
    Use our validated installation scripts for the smoothest setup:

    **For Python 3.11:**

    ```bash theme={null}
    bash scripts/install_py311.sh
    ```

    **For Python 3.12:**

    ```bash theme={null}
    bash scripts/install_py312.sh
    ```

    These scripts automatically:

    * Install PyTorch with CUDA 12.4 support
    * Configure vLLM 0.8.3
    * Set up Flash Attention
    * Install all dependencies
  </Tab>

  <Tab title="Docker (Reproducible)">
    Build a pinned training image directly from this repo:

    ```bash theme={null}
    docker build -t atlas-core:local .
    ```

    Run the offline pipeline helper against a JSONL export:

    ```bash theme={null}
    docker run --rm \
      -v "$(pwd)/exports:/data" \
      atlas-core:local \
      atlas-core offline-pipeline --export-path /data/traces.jsonl --dry-run
    ```

    For GPU hosts, rebuild with CUDA-enabled base images and include extras such as
    `deepspeed`, `ray`, or `vllm`.
  </Tab>

  <Tab title="Manual Training Installation">
    For custom environments or debugging:

    ```bash theme={null}
    # Install PyTorch with CUDA support
    python -m pip install torch==2.6.0 --index-url https://download.pytorch.org/whl/cu124

    # Install vLLM and TensorBoard
    python -m pip install vllm==0.8.3 tensorboard

    # Install Flash Attention (for optimal performance)
    python -m pip install flash-attn --no-build-isolation

    # Install FlashInfer
    python -m pip install flashinfer-python -i https://flashinfer.ai/whl/cu124/torch2.6/

    # Install remaining dependencies
    python -m pip install --upgrade -r requirements-py311.txt  # or requirements-py312.txt
    ```
  </Tab>

  <Tab title="Conda Environment">
    Create isolated environment with Conda:

    ```bash theme={null}
    # Create environment
    conda create -n atlas python=3.11
    conda activate atlas

    # Install PyTorch
    conda install pytorch==2.6.0 pytorch-cuda=12.4 -c pytorch -c nvidia

    # Run installation script
    bash scripts/install_py311.sh
    ```
  </Tab>
</Tabs>

## Configure Environment

### API Keys

```bash theme={null}
# Training stack
export HF_TOKEN="your-huggingface-token"
export WANDB_API_KEY="your-wandb-key"  # Optional

# Runtime SDK
export ANTHROPIC_API_KEY="sk-ant-your-key"  # Primary provider
export GEMINI_API_KEY="your-gemini-key"  # Optional for rewards
```

<Tip>
  Store secrets in `.env`. The Atlas CLI loads `.env` automatically and extends `PYTHONPATH` with your project root and `src/` directory.
</Tip>

### Disable Tracking

To disable Weights & Biases tracking:

```bash theme={null}
# In command line
atlas-core train report_to=null

# Or in config file
report_to: null
```

## Verification

After installation, verify your setup:

### 3-Minute Smoke Test

<Note>
  Run this once to confirm CUDA, vLLM, and model downloads are working before you invest in longer training jobs.
</Note>

```bash theme={null}
python - <<'PY'
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load teacher model
teacher = AutoModelForCausalLM.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Thinking",
    device_map="auto",
    torch_dtype=torch.float16
)
teacher_tokenizer = AutoTokenizer.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Thinking"
)

print("CUDA available:", torch.cuda.is_available())
print("GPU count:", torch.cuda.device_count())
print("Teacher model loaded:", teacher.config.model_type)
print("Model device:", next(teacher.parameters()).device)
PY
```

**Expected output:**

```
CUDA available: True
GPU count: 8
Teacher model loaded: qwen2
Model device: cuda:0
```

<CodeGroup>
  ```python "Quick Test" theme={null}
  # Verify core dependencies
  import torch
  import transformers
  import datasets
  import vllm

  print(f"PyTorch: {torch.__version__}")
  print(f"CUDA available: {torch.cuda.is_available()}")
  print(f"GPU count: {torch.cuda.device_count()}")
  print(f"Transformers: {transformers.__version__}")
  print(f"vLLM: {vllm.__version__}")
  ```

  ```bash "CLI Verification" theme={null}
  # Check accelerate installation
  accelerate --version

  # Verify CUDA
  python -c "import torch; print(torch.cuda.is_available())"

  # Test model access
  huggingface-cli download Arc-Intelligence/ATLAS-8B-Thinking \
    --include "*.json" \
    --exclude "*.safetensors"
  ```
</CodeGroup>

## GPU Memory Management

For different GPU configurations:

<AccordionGroup>
  <Accordion title="Single GPU Setup" icon="microchip">
    Single GPU is supported for inference only. For RL training, use model offloading:

    ```bash theme={null}
    # Inference only with single GPU
    python examples/quickstart/evaluate.py  # Quick evaluation test

    # For training with limited VRAM (requires 2+ GPUs)
    scripts/launch.sh offload 2 src/atlas_core/configs/recipe/teacher_rcl.yaml

    # Or use Zero-1 optimization
    scripts/launch.sh zero1 2 src/atlas_core/configs/recipe/teacher_rcl.yaml
    ```
  </Accordion>

  <Accordion title="Multi-GPU Setup" icon="layer-group">
    For distributed training across multiple GPUs:

    ```bash theme={null}
    # Minimum 2 GPUs for RL training (1 for vLLM, 1 for training)
    scripts/launch_with_server.sh 1 1 src/atlas_core/configs/recipe/teacher_rcl.yaml

    # Production setup with 4 GPUs (2 for vLLM, 2 for training)
    scripts/launch_with_server.sh 2 2 src/atlas_core/configs/recipe/teacher_rcl.yaml

    # Full 8 GPU setup
    scripts/launch_with_server.sh 4 4 src/atlas_core/configs/recipe/teacher_rcl.yaml
    ```
  </Accordion>

  <Accordion title="Memory Optimization" icon="memory">
    Reduce memory usage with these settings:

    ```yaml theme={null}
    # In config file
    per_device_train_batch_size: 1
    gradient_checkpointing: true
    fp16: true  # or bf16 for A100/H100
    ```
  </Accordion>
</AccordionGroup>

## Security Best Practices

<Warning>
  Follow these security guidelines to protect sensitive information:
</Warning>

* **Never commit secrets**: Keep tokens, `.env` files, and API keys out of version control
* **Use environment variables**: Store `HF_TOKEN`, `WANDB_API_KEY`, etc. as environment variables
* **Gitignore protection**: Ensure `results/`, `logs/`, `wandb/` remain in `.gitignore`
* **Least privilege**: Restrict dataset access permissions
* **Logout on shared machines**: Run `huggingface-cli logout` after use

## Platform-Specific Notes

<Tabs>
  <Tab title="Linux">
    Tested on Ubuntu 20.04/22.04 LTS:

    * Ensure CUDA toolkit matches PyTorch requirements
    * May need `sudo` for system package installations
  </Tab>

  <Tab title="macOS">
    Limited support for Apple Silicon:

    * CPU-only mode available
    * Use MPS backend where supported
    * vLLM may not be available
  </Tab>

  <Tab title="Windows WSL2">
    Run through WSL2 for best compatibility:

    * Install CUDA toolkit in WSL2
    * Use Linux installation instructions
    * Ensure WSL2 has GPU passthrough enabled
  </Tab>
</Tabs>

## Troubleshooting

<AccordionGroup>
  <Accordion title="CUDA Version Mismatch" icon="triangle-exclamation">
    If you see CUDA errors:

    ```bash theme={null}
    # Check CUDA version
    nvidia-smi
    nvcc --version

    # Reinstall PyTorch with correct CUDA version
    pip install torch==2.6.0 --index-url https://download.pytorch.org/whl/cu118  # For CUDA 11.8
    ```
  </Accordion>

  <Accordion title="Out of Memory Errors" icon="memory">
    Reduce memory usage:

    ```bash theme={null}
    # Use gradient checkpointing
    atlas-core train gradient_checkpointing=true

    # Reduce batch size
    atlas-core train per_device_train_batch_size=1

    # Enable CPU offloading
    scripts/launch.sh offload 2 src/atlas_core/configs/recipe/teacher_rcl.yaml
    ```
  </Accordion>

  <Accordion title="HuggingFace Access Denied" icon="lock">
    Ensure proper authentication:

    ```bash theme={null}
    # Re-authenticate
    huggingface-cli logout
    huggingface-cli login

    # Verify token
    huggingface-cli whoami
    ```
  </Accordion>

  <Accordion title="vLLM Installation Fails" icon="xmark">
    Common vLLM issues:

    ```bash theme={null}
    # Install build dependencies
    sudo apt-get install python3-dev

    # Try pre-built wheel
    pip install https://github.com/vllm-project/vllm/releases/download/v0.8.3/vllm-0.8.3-cp311-cp311-linux_x86_64.whl
    ```
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup>
  <Card title="Quickstart" icon="rocket" href="/sdk/quickstart">
    Deploy ATLAS with pre-trained models
  </Card>

  <Card title="Offline Training" icon="flask" href="/training/offline/grpo-training">
    Run your first ATLAS training experiment
  </Card>
</CardGroup>


# Introduction
Source: https://docs.arc.computer/introduction

A Continual Learning Framework for Production LLM Agents

<div>
  <img alt="ATLAS Hero Image" />
</div>

<Tip>
  **Have questions?** Chat with the docs using the assistant at the bottom.
</Tip>

<br />

## What is ATLAS?

ATLAS is a continual learning framework for production LLM agents. It combines runtime quality control with offline reinforcement learning to improve agent reliability, reduce token costs, and build domain expertise through persistent memory.

The system layers a dual-agent reasoning loop (student + verifying teacher) on top of any model. The [Atlas SDK](https://github.com/Arc-Computer/atlas-sdk) streams causality traces into Postgres, and Atlas Core (this repository) trains new teacher checkpoints via on-policy distillation (GKD) or reinforcement learning (GRPO).

<div>
  <img alt="ATLAS System Architecture" />

  <p><em>Atlas runtime captures traces; Atlas Core trains improved models from those traces.</em></p>
</div>

## Which Repository Do You Need?

<CardGroup>
  <Card title="Atlas SDK" icon="rocket">
    **Use if you want to:**

    * Run agents with quality control
    * Get runtime supervision and retries
    * Export traces for later training

    **Repository:** [atlas-sdk](https://github.com/Arc-Computer/atlas-sdk)

    **Installation:** `pip install arc-atlas`

    **Start:** [SDK Quickstart](/sdk/quickstart)
  </Card>

  <Card title="Atlas Core" icon="brain">
    **Use if you want to:**

    * Train custom teacher models
    * Run GKD or GRPO training
    * Fine-tune on exported traces

    **Repository:** [ATLAS](https://github.com/Arc-Computer/ATLAS)

    **Installation:** See [Installation Guide](/installation)

    **Start:** [GKD Training](/training/offline/gkd-training)
  </Card>
</CardGroup>

<Warning>
  **Most teams only need the SDK.** Atlas Core requires GPUs and is only necessary if you are training custom models. The SDK works with any OpenAI-compatible API.
</Warning>

## Why Use ATLAS?

| Benefit                | How                                                                | Result                          |
| ---------------------- | ------------------------------------------------------------------ | ------------------------------- |
| Lower costs            | Adaptive supervision allocates reasoning only when needed          | \~50% token reduction           |
| Higher reliability     | Real-time teacher review catches errors before production impact   | +15.7% avg task success         |
| Continuous improvement | Offline training (GRPO/GKD) updates teacher from production traces | Compounding expertise over time |

→ Full methodology in [Technical Report](/ATLAS-Technical-Report.pdf)
→ System concepts in [Adaptive Dual-Agent Reasoning](/concepts/adaptive-dual-agent-reasoning)

<Note>
  **Runtime vs. Training**: The [Atlas SDK](https://github.com/Arc-Computer/atlas-sdk) handles runtime orchestration and trace export. Atlas Core (this repository) handles offline training (GKD/GRPO).
</Note>

## End-to-End Workflow

| Stage                   | Run This                                                                          | Output                               | Time        |
| ----------------------- | --------------------------------------------------------------------------------- | ------------------------------------ | ----------- |
| Runtime quality control | [`atlas run`](/sdk/quickstart)                                                    | Reviewed traces with reward scores   | Minutes     |
| Export traces           | [`arc-atlas export`](/sdk/export-traces)                                          | JSONL dataset from approved sessions | Minutes     |
| Train teacher (GKD)     | [`atlas-core offline-pipeline`](/training/offline/gkd-training)                   | Distilled teacher checkpoint         | 4-8 hours   |
| Train teacher (GRPO)    | [`atlas-core train recipe@_global_=teacher_rcl`](/training/offline/grpo-training) | RL-optimized teacher checkpoint      | 24-48 hours |

Every stage feeds the next—runtime traces become training data; trained checkpoints redeploy to runtime.

## Getting Started: Two Paths

| I want to...                                   | Path       | Start Here                                       |
| ---------------------------------------------- | ---------- | ------------------------------------------------ |
| Run tasks with dual-agent orchestration        | Atlas SDK  | [SDK Quickstart](/sdk/quickstart)                |
| Wrap my existing agent in quality-control loop | Atlas SDK  | [BYOA Adapters](/sdk/adapters)                   |
| Distill traces into smaller teacher            | Atlas Core | [GKD Training](/training/offline/gkd-training)   |
| Train from rewards (RL)                        | Atlas Core | [GRPO Training](/training/offline/grpo-training) |

<CardGroup>
  <Card title="SDK Runtime Orchestration" icon="workflow" href="/sdk/quickstart">
    Run your agent with closed-loop learning. Get started in minutes.
  </Card>

  <Card title="Offline Training (Atlas Core)" icon="graduation-cap" href="/training/offline/grpo-training">
    Convert runtime traces into GRPO/GKD training jobs and ship updated teachers.
  </Card>
</CardGroup>

## Research & Resources

* [ATLAS Technical Report (PDF)](/ATLAS-Technical-Report.pdf) - Methodology, benchmarks, implementation details
* [Arc Research](https://www.arc.computer/research) - Latest research on continual learning systems
* [GitHub Repository](https://github.com/Arc-Computer/ATLAS) - Source code and issue tracking
* [HuggingFace Models](https://huggingface.co/Arc-Intelligence) - Pre-trained teachers
* [Evaluation Harnesses](/benchmarks/evaluation-harnesses) - Runtime, reward, and learning measurement scripts


# Telemetry Schema
Source: https://docs.arc.computer/reference/database-schema

Understand the core Atlas SDK tables used for discovery, runtime telemetry, and learning persistence.

Atlas persists runtime activity to Postgres so you can replay discovery runs, audit sessions, and power evaluation
reports. This reference summarizes the tables the SDK creates by default and how they relate to each other.

## Overview

```
discovery_runs ─┐
                │
                ▼
            sessions ── plans / step_results / guidance_notes
                │
                ├─ trajectory_events
                │
                └─ learning_registry
```

* `discovery_runs` captures onboarding metadata produced by `atlas env init`.
* `sessions` stores per-request telemetry, review status, reward stats, and learning notes.
* `trajectory_events` holds fine-grained telemetry (plan approvals, guidance, validations, tool calls).
* `learning_registry` caches the latest playbooks per `learning_key`.

A complete schema is defined in `atlas-sdk/atlas/runtime/storage/schema.sql`. The sections below call out the columns you
are most likely to query.

## discovery\_runs

| Column         | Type          | Notes                                                                                   |
| -------------- | ------------- | --------------------------------------------------------------------------------------- |
| `id`           | `SERIAL`      | Primary key.                                                                            |
| `project_root` | `TEXT`        | Absolute path recorded during discovery; use it to group runs by repository.            |
| `task`         | `TEXT`        | Sample task executed during discovery (`--task`).                                       |
| `source`       | `TEXT`        | Defaults to `"discovery"`; other tooling can reuse the table by setting custom sources. |
| `payload`      | `JSONB`       | The discovery artefact (`.atlas/discover.json` equivalent).                             |
| `metadata`     | `JSONB`       | Additional notes (preflight, scaffold results, template info).                          |
| `created_at`   | `TIMESTAMPTZ` | Ingestion timestamp.                                                                    |

Link discovery runs back to runtime sessions through shared metadata (for example, `metadata.learning_key` or
`metadata.discovery.environment_factory` entries).

## sessions

| Column                                     | Type          | Notes                                                                                  |
| ------------------------------------------ | ------------- | -------------------------------------------------------------------------------------- |
| `id`                                       | `SERIAL`      | Primary key referenced by all child tables.                                            |
| `task`                                     | `TEXT`        | User task prompt.                                                                      |
| `status`                                   | `TEXT`        | Runtime status (`running`, `succeeded`, `failed`).                                     |
| `metadata`                                 | `JSONB`       | Rich telemetry (adaptive summary, triage dossier, drift info, learning history, etc.). |
| `final_answer`                             | `TEXT`        | Student output persisted at completion.                                                |
| `reward` / `reward_stats` / `reward_audit` | `JSONB`       | Judge outputs and aggregate statistics.                                                |
| `student_learning` / `teacher_learning`    | `TEXT`        | Per-session learning notes (pre-registry).                                             |
| `review_status`                            | `TEXT`        | `pending`, `approved`, or `quarantined`. Defaults to `pending`.                        |
| `review_notes`                             | `TEXT`        | Reviewer-supplied context.                                                             |
| `created_at` / `completed_at`              | `TIMESTAMPTZ` | Run timing.                                                                            |

Three performance indexes optimize training data queries:

* `sessions_reward_score_idx`: Functional index on `(reward_stats->>'score')::float` for 10-100x faster reward filtering
* `sessions_created_at_idx`: Index on `created_at DESC` for 50-100x faster date range queries
* `sessions_metadata_gin_idx`: GIN index on `metadata` JSONB for learning key queries

Use `review_status` to filter exports and harness runs, and join the `metadata` hash to inspect `adaptive_summary`, `execution_mode`, `learning_history`, or drift alerts.

### Metadata Schema Fields (v0.1.13+)

The `metadata` JSONB column contains structured telemetry. Key fields accessible via `AtlasSessionTrace` dataclass:

**Essential fields:**

* `session_reward`: Aggregate reward with score and uncertainty
* `trajectory_events`: Ordered list of runtime events
* `student_learning`: Student persona learning notes
* `teacher_learning`: Teacher persona learning notes
* `learning_history`: Historical learning data
* `adaptive_summary`: Mode selection (auto/paired/coach) and probe evidence

**Property accessors** (loaded on demand):

* `learning_key`: Task identifier for grouping sessions
* `teacher_notes`: Guidance provided during execution
* `reward_summary`: Simplified reward statistics
* `drift`: Detected schema or behavior drift
* `drift_alert`: Critical drift warnings requiring review
* `triage_dossier`: Pre-execution risk assessment
* `reward_audit`: Detailed judge breakdowns

### Related tables

* `plans` – JSON plan snapshot keyed by `session_id`.
* `step_results` / `step_attempts` – per-step traces and validation payloads.
* `guidance_notes` – ordered teacher guidance emitted during execution.

### step\_results Schema Fields (v0.1.13+)

Step-level telemetry accessible via `AtlasStepTrace` dataclass:

**Essential fields:**

* `runtime`: Step execution time in milliseconds
* `depends_on`: Array of step IDs this step depends on (dependency graph)

**Property accessors:**

* `attempt_history`: Previous attempt records if step was retried

Query step execution times and dependencies for performance analysis:

```sql theme={null}
-- Average step runtime by tool
SELECT
  metadata->>'tool' AS tool_name,
  AVG((metadata->>'runtime')::float) AS avg_runtime_ms
FROM step_results
WHERE metadata->>'runtime' IS NOT NULL
GROUP BY tool_name
ORDER BY avg_runtime_ms DESC;
```

## trajectory\_events

| Column       | Type          | Notes                                                                |
| ------------ | ------------- | -------------------------------------------------------------------- |
| `id`         | `SERIAL`      | Primary key.                                                         |
| `session_id` | `INTEGER`     | Foreign key to `sessions`.                                           |
| `event`      | `JSONB`       | Envelope containing `event_type`, `actor`, timestamps, payload, etc. |
| `created_at` | `TIMESTAMPTZ` | Event timestamp.                                                     |

The learning evaluation harness samples these events to count validations, guidance injections, and reward updates. Filter
by `event->>'event_type'` to narrow to specific telemetry (e.g., `reward`, `learning_playbook`, `tool_call`).

## learning\_registry

| Column             | Type          | Notes                                          |
| ------------------ | ------------- | ---------------------------------------------- |
| `learning_key`     | `TEXT`        | Primary identifier (task, project, or domain). |
| `student_learning` | `TEXT`        | Latest student playbook body.                  |
| `teacher_learning` | `TEXT`        | Latest teacher playbook body.                  |
| `metadata`         | `JSONB`       | Optional synthesizer audit info or hashes.     |
| `updated_at`       | `TIMESTAMPTZ` | Last update timestamp.                         |

The runtime loads this table at session start and updates it after successful learning synthesis (subject to
`learning.update_enabled`). Join back to `sessions` via `metadata.learning_key` to reconstruct the history that produced
the current playbook.

## Query Examples

```sql theme={null}
-- High-reward sessions for training (using performance index)
SELECT id, task, (reward_stats->>'score')::float AS reward
FROM sessions
WHERE (reward_stats->>'score')::float >= 0.8
  AND status = 'succeeded'
ORDER BY created_at DESC
LIMIT 1000;

-- Sessions awaiting review with drift alerts
SELECT id, task, metadata->'drift_alert' AS drift_alert
FROM sessions
WHERE review_status = 'pending'
  AND metadata ? 'drift_alert';

-- Latest playbooks for a service
SELECT learning_key, updated_at, student_learning
FROM learning_registry
WHERE learning_key LIKE 'service:%'
ORDER BY updated_at DESC;

-- Count validation events per session
SELECT session_id, COUNT(*) AS validation_events
FROM trajectory_events
WHERE event ->> 'event_type' = 'validation'
GROUP BY session_id;

-- Learning history for a task (using GIN index)
SELECT
  id,
  task,
  metadata->'learning_history' AS learning_history,
  (reward_stats->>'score')::float AS reward
FROM sessions
WHERE metadata @> '{"learning_key": "security-review"}'
ORDER BY created_at DESC;

-- Step performance analysis
SELECT
  s.task,
  sr.metadata->>'tool' AS tool,
  AVG((sr.metadata->>'runtime')::float) AS avg_runtime_ms,
  COUNT(*) AS step_count
FROM step_results sr
JOIN sessions s ON sr.session_id = s.id
WHERE sr.metadata->>'runtime' IS NOT NULL
GROUP BY s.task, sr.metadata->>'tool'
ORDER BY avg_runtime_ms DESC;
```

## Related Pages

* [`Training Data Pipeline`](/training/offline/training-data-pipeline) – Direct database access for training data extraction
* [`Export Runtime Traces`](/sdk/export-traces) – CLI usage and JSON schema for session exports
* [`Runtime Safety & Review`](/sdk/runtime-safety) – Review gating and drift detection
* [`Evaluation Harnesses`](/benchmarks/evaluation-harnesses) – Harnesses that query the schema for analytics


# Datasets
Source: https://docs.arc.computer/reference/datasets

Official ATLAS training and evaluation datasets

## Available Datasets

ATLAS provides curated datasets for training adaptive teachers and evaluating system performance.

## Primary Dataset

### Arc-ATLAS-Teach-v0

<Card title="View on Hugging Face" icon="hugging-face" href="https://huggingface.co/datasets/Arc-Intelligence/Arc-ATLAS-Teach-v0">
  Comprehensive teaching interaction dataset for RL training
</Card>

**Purpose:** Train teacher models to provide adaptive guidance across diverse tasks

**Statistics:**

* **Total examples**: 100,000+ teaching interactions
* **Task domains**: Mathematics, reasoning, coding, debugging
* **Formats**: SFT and RL training splits
* **Languages**: English

**Data Schema:**

```json theme={null}
{
  "prompt": "The problem or task requiring solution",
  "ground_truth": "Correct answer or solution",
  "student_response": "Initial student attempt",
  "teaching": "Adaptive guidance provided",
  "enhanced_response": "Student response after teaching",
  "baseline_score": 0.3,
  "with_teaching_score": 0.9,
  "reward": 0.6,
  "problem_id": "unique_identifier",
  "student_level": "weak|moderate|strong",
  "domain": "math|reasoning|code|debug"
}
```

**Loading the Dataset:**

```python theme={null}
from datasets import load_dataset

# Load for supervised fine-tuning
sft_data = load_dataset(
    "Arc-Intelligence/Arc-ATLAS-Teach-v0",
    "sft",
    split="train"
)

# Load for reinforcement learning
rl_data = load_dataset(
    "Arc-Intelligence/Arc-ATLAS-Teach-v0",
    "rl",
    split="train"
)

# Load validation set
val_data = load_dataset(
    "Arc-Intelligence/Arc-ATLAS-Teach-v0",
    "rl",
    split="validation"
)
```

**File Structure:**

```
Arc-ATLAS-Teach-v0/
├── training/
│   ├── sft.jsonl         # Supervised fine-tuning data
│   └── rl.jsonl          # Reinforcement learning data
└── validation/
    └── rl.jsonl          # Held-out validation
```

## Domain-Specific Subsets

### Mathematics Subset

**Focus:** Step-by-step mathematical reasoning

**Example:**

```json theme={null}
{
  "prompt": "Sarah has 24 apples. She gives 1/3 to her brother...",
  "ground_truth": "12",
  "teaching": "Break down: 1) Calculate 1/3 of 24 = 8..."
}
```

**Filtering:**

```python theme={null}
math_data = dataset.filter(lambda x: x['domain'] == 'math')
```

### Code Generation Subset

**Focus:** Programming tasks and debugging

**Example:**

```json theme={null}
{
  "prompt": "Write a function to validate email addresses",
  "ground_truth": "def validate_email(email):...",
  "teaching": "Consider regex pattern, edge cases like..."
}
```

**Filtering:**

```python theme={null}
code_data = dataset.filter(lambda x: x['domain'] == 'code')
```

### SRE/Debugging Subset

**Focus:** System reliability and debugging scenarios

**Example:**

```json theme={null}
{
  "prompt": "Service returns 503 errors intermittently",
  "ground_truth": "Check service mesh configuration...",
  "teaching": "Systematic approach: 1) Check Istio configs..."
}
```

**Filtering:**

```python theme={null}
sre_data = dataset.filter(lambda x: x['domain'] == 'debug')
```

## Data Quality Metrics

### Coverage Statistics

| Domain          | Examples | Avg Length | Unique Patterns |
| --------------- | -------- | ---------- | --------------- |
| Mathematics     | 35,000   | 250 tokens | 500+            |
| Code Generation | 30,000   | 400 tokens | 800+            |
| Reasoning       | 25,000   | 300 tokens | 600+            |
| Debugging       | 10,000   | 350 tokens | 400+            |

### Performance Baselines

| Metric           | Baseline | w/ Dual-Agent Loop | Improvement |
| ---------------- | -------- | ------------------ | ----------- |
| Accuracy         | 62.3%    | 78.0%              | +15.7%      |
| Completion       | 69%      | 100%               | +31%        |
| Token Efficiency | 100%     | 50%                | -50%        |

<Note>
  These figures reflect the closed-loop runtime plus GRPO baseline. Online continual learning now lives in the [`atlas-sdk`](https://github.com/Arc-Computer/atlas-sdk) runtime if you need task-specific adaptation between offline training runs.
</Note>

## Creating Custom Datasets

To create custom datasets from runtime traces or other sources, see [Custom Dataset Creation](/training/custom-datasets):

* Data format requirements
* Preprocessing JSONL exports from the SDK
* Postgres-backed workflows
* Quality validation

## Contributing Data

We welcome contributions to improve ATLAS datasets:

1. **Format your data** according to the schema
2. **Validate quality** using provided tools
3. **Test with models** to ensure compatibility
4. **Submit PR** with data and documentation

See [Contributing Guidelines](https://github.com/Arc-Computer/ATLAS/blob/main/CONTRIBUTING.md) for details.

## License and Citation

Datasets are released under Apache 2.0 license.

If you use these datasets, please cite:

```bibtex theme={null}
@dataset{atlas_teach_v0,
  title={Arc-ATLAS-Teach-v0: Adaptive Teaching Dataset},
  author={Arc Intelligence Team},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/datasets/Arc-Intelligence/Arc-ATLAS-Teach-v0}
}
```

## Next Steps

<CardGroup>
  <Card title="Training Guide" icon="graduation-cap" href="/training/offline/grpo-training">
    Train models with datasets
  </Card>

  <Card title="Models" icon="robot" href="/reference/models">
    Pre-trained ATLAS models
  </Card>

  <Card title="Adaptive Tool Use" icon="wrench" href="/examples/adaptive-tool-use">
    Production example with MCP integration
  </Card>

  <Card title="Technical Report" icon="file-pdf" href="/reference/technical-report">
    Dataset methodology
  </Card>
</CardGroup>


# Frequently Asked Questions
Source: https://docs.arc.computer/reference/faq

Common questions about ATLAS implementation and usage

## General Questions

### What is ATLAS?

ATLAS (Adaptive Teaching and Learning Alignment System) is a framework that pairs your existing agent (the student) with a specialized verifying teacher that provides adaptive guidance. It uses a two-pass protocol: diagnostic assessment followed by targeted teaching.

### Does ATLAS train on my production data?

No. The runtime loop operates through inference-time feedback—no model weights are modified during production execution. Weight updates only happen when you explicitly run offline GRPO training jobs on your own infrastructure.

**Data control:**

* All session traces write exclusively to the Postgres database you provide via `storage.database_url`
* ATLAS never operates its own data store or accesses your database
* Leave `storage: null` in your config to run in ephemeral mode with no persistent data
* Training is opt-in: you choose when to export traces and run offline training

The runtime learns by storing successful guidance patterns in memory (when storage is enabled) and retrieving them for similar future tasks—this is inference-time adaptation, not model training.

### How is ATLAS different from fine-tuning?

Unlike fine-tuning which modifies model weights, ATLAS:

* Preserves the student model's original capabilities
* Works with any model without retraining
* Adapts guidance based on student capability
* Provides immediate enhancement without training time

### What performance improvements can I expect?

Across our evaluation suite we consistently see the closed-loop dual-agent runtime (student + verifying teacher) deliver an average **+15.7% accuracy gain**, **31% task completion lift**, **97% non-degradation**, and **\~50% token savings** versus baseline agents. Offline GRPO training then compounds those gains when you fine-tune custom teacher checkpoints using the traces exported from production.

Actual results vary with task difficulty, data quality, and the strength of the underlying student model, but the closed loop plus GRPO stack gives you levers to reach those numbers. Online continual learning lives in the [`atlas-sdk`](https://github.com/Arc-Computer/atlas-sdk) runtime if you need rapid, task-specific adaptation.

## Hardware & Setup

### What hardware do I need?

**Minimum Requirements:**

* GPU: 16GB VRAM (RTX 4080, A5000)
* RAM: 32GB system memory
* Storage: 100GB for models and data

**Recommended for Training:**

* GPU: 4× A100 40GB or H100 80GB
* RAM: 128GB+ system memory
* Storage: 500GB NVMe SSD

**For Inference Only:**

* Can run on CPU (slower)
* 8GB VRAM with quantization
* Cloud instances work well

### Can I run ATLAS on CPU?

Yes, with API-based models you need no GPU at all. For local model inference:

* CPU inference is 10-50x slower than GPU
* Limited to smaller models (4B-8B)
* Quantization recommended
* Suitable for development/testing

```python theme={null}
# API-based (no GPU needed)
from openai import OpenAI
client = OpenAI()
# Call the verifying teacher and student via API

# For local models on CPU
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Thinking",
    device_map="cpu",
    torch_dtype=torch.float32
)
```

### Which models are compatible?

**Verifying teacher checkpoints (pre-trained):**

* ATLAS-8B-Thinking (reasoning)
* ATLAS-8B-Instruct (coding)

**Student agents (any LLM):**

* Qwen series (4B-70B)
* Llama series (7B-70B)
* Mistral/Mixtral models
* GPT-3.5/4 (via API)
* Claude (via API)

## Training Questions

### How long does training take?

**Offline RL Training (GRPO):**

* SFT warmup: 4-8 hours
* GRPO training: 24-48 hours
* Hardware: 4-8 H100 GPUs

### What's the difference between online and offline training?

**Offline Training (GRPO):**

* Creates foundational teaching skills
* Requires significant compute
* Produces generalizable models
* One-time investment

**Continual Learning (atlas-sdk):**

* Adapts to specific tasks using the runtime loop
* Runs through the SDK CLI and APIs
* Rapid iteration cycles driven by live traces
* Keeps production agents improving between offline training runs

### Can I train on custom data?

Yes, prepare your data in this format:

```json theme={null}
{
  "prompt": "Your task or question",
  "ground_truth": "Correct answer",
  "metadata": {
    "domain": "your_domain",
    "difficulty": "easy|medium|hard"
  }
}
```

Then train:

```bash theme={null}
scripts/launch.sh 8 src/atlas_core/configs/recipe/teacher_sft.yaml \
  dataset_name=path/to/your/data
```

## Implementation Questions

### How do I integrate ATLAS into my application?

Use the ATLAS teaching protocol with real imports:

```python theme={null}
from openai import OpenAI
from atlas_core.reward.interpretation import RIMReward

client = OpenAI()

# Step 1: Get baseline from your student model
baseline_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": task}]
).choices[0].message.content

# Step 2: Get teaching from teacher model
teaching = client.chat.completions.create(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": f"Provide teaching for: {task}\nStudent said: {baseline_response}"
    }]
).choices[0].message.content

# Step 3: Student applies teaching
enhanced_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": f"{task}\nTeaching: {teaching}"}]
).choices[0].message.content
```

See [Inference Integration Guide](/sdk/quickstart) for complete examples.

### Can ATLAS work with my existing agent?

Yes. Use the [`atlas-sdk`](https://github.com/Arc-Computer/atlas-sdk) runtime wrappers (HTTP, Python callable, OpenAI Assistants, CLI) to orchestrate your agent, export traces, and hand them to Atlas Core for training. The SDK documentation covers the available adapters and configuration options.

### How do I monitor performance in production?

Use RIM reward scoring to track quality:

```python theme={null}
from atlas_core.reward.interpretation import RIMReward

reward = RIMReward(config_path='reward_system/interpretation.yaml')

# Track each request
baseline_eval = reward.evaluate(prompt=task, response=baseline_response)
enhanced_eval = reward.evaluate(
    prompt=task,
    response=enhanced_response,
    baseline_solutions=baseline_response,
    teacher_traces=teaching
)

# Log metrics
print(f"Baseline: {baseline_eval.score:.3f}")
print(f"Enhanced: {enhanced_eval.score:.3f}")
print(f"Delta: {enhanced_eval.score - baseline_eval.score:+.3f}")
```

RIM scores can be logged to:

* Weights & Biases
* TensorBoard
* Prometheus
* Custom logging systems

## Performance & Optimization

### Why is inference slow?

Common causes and solutions:

1. **Not using Flash Attention**:
   ```python theme={null}
   config.attn_implementation = "flash_attention_2"
   ```

2. **Small batch size**:
   ```python theme={null}
   atlas.batch_size = 8  # Process multiple requests
   ```

3. **No caching**:
   ```python theme={null}
   atlas.enable_cache = True
   ```

4. **CPU inference**: Use GPU or quantization

### How can I reduce memory usage?

Progressive solutions:

1. **Quantization** (75% reduction):
   ```python theme={null}
   config.load_in_4bit = True
   ```

2. **Smaller models**: Use 4B instead of 8B

3. **Offloading**: Move to CPU/disk

4. **Batch size**: Reduce to 1

### What if the teacher makes things worse?

ATLAS has a 97% non-degradation guarantee through:

* Zero reward for performance drops
* Safety validation before deployment
* Fallback to baseline response
* Continuous monitoring

If issues persist:

* Check task-model compatibility
* Verify data quality
* Adjust teaching parameters
* Export fresh traces and schedule a GRPO training run

## Cost Questions

### How much does ATLAS cost to run?

**Training Costs:**

* Offline RL: \$100-500 in compute (depends on GPUs and run length)

**Inference Costs:**

* Self-hosted: Electricity only
* Cloud GPU: \$1-3/hour
* API-based: \$0.001-0.01 per request

### Is there a cloud service?

Currently ATLAS is open-source only. You can:

* Self-host on your infrastructure
* Use cloud GPU providers
* Deploy on Hugging Face Spaces
* Contact team for enterprise support

## Troubleshooting

### Where can I get help?

1. [Troubleshooting Guide](/reference/troubleshooting)
2. [GitHub Issues](https://github.com/Arc-Computer/ATLAS/issues)
3. [Discord Community](https://discord.gg/arc-atlas)
4. Email: [support@arc.computer](mailto:support@arc.computer)

### How do I report a bug?

File an issue with:

* Error message and stack trace
* System configuration
* Minimal reproduction code
* Expected vs actual behavior

### Can I contribute to ATLAS?

Yes! We welcome contributions:

* Code improvements
* Documentation
* Bug fixes
* New features
* Dataset contributions

See [Contributing Guide](https://github.com/Arc-Computer/ATLAS/blob/main/CONTRIBUTING.md).

## Next Steps

<CardGroup>
  <Card title="Quickstart" icon="rocket" href="/sdk/quickstart">
    Get started with ATLAS
  </Card>

  <Card title="Examples" icon="lightbulb" href="/examples/adaptive-tool-use">
    See ATLAS in action
  </Card>

  <Card title="Troubleshooting" icon="wrench" href="/reference/troubleshooting">
    Solve common issues
  </Card>

  <Card title="Community" icon="discord" href="https://discord.gg/arc-atlas">
    Join the discussion
  </Card>
</CardGroup>


# Glossary
Source: https://docs.arc.computer/reference/glossary

Key terms and concepts used in ATLAS documentation

## Core Concepts

### ATLAS

**Adaptive Teaching and Learning Alignment System** - A continual learning framework that separates complex RL training into offline teacher preparation and online task adaptation.

### Continual Learning

The ability of an agent to improve from experience and transfer learned skills across tasks without retraining the base model weights.

### Hybrid Architecture

ATLAS's approach of separating offline RL training (for teachers) from runtime continual learning (for task adaptation), enabling both stability and flexibility.

## Training Algorithms

### GRPO

**Group Relative Policy Optimization** - The offline RL algorithm used to train ATLAS teacher models. Optimizes teaching policies through group-relative rewards with KL divergence constraints.

### SFT

**Supervised Fine-Tuning** - Initial training phase that establishes baseline capabilities before RL optimization. Required warmup step before GRPO training.

## Technical Terms

### Two-Pass Protocol

ATLAS's inference pattern:

1. **Diagnostic Probe** (≤50 tokens): Teacher assesses student capability
2. **Adaptive Guidance** (≤200 tokens): Teacher provides calibrated assistance

### Teacher Model

Specialized 8B parameter models trained with GRPO to diagnose and guide other language models. Pre-trained versions available on HuggingFace.

### Student Model

Any language model, agent, or AI system that receives guidance from the ATLAS teacher. This includes:

* Commercial LLMs (GPT, Claude, Gemini)
* Open models (Llama, Mistral, Qwen)
* Your custom agents (OpenAI Assistants, LangChain, AutoGen)
* API endpoints or services
* CLI-based tools or scripts

The student model remains unchanged - ATLAS enhances its responses through external guidance, not modification.

### Non-Degradation Rate

Percentage of interactions where performance remains equal to or better than baseline (target: ≥97%).

### Compounding Intelligence

The accumulation and transfer of learned skills across tasks and domains through the hybrid architecture.

### RIM (Reward Interpretation System)

Atlas's reward ensemble that evaluates every step and final answer using multiple judging LLMs. The runtime (`RIMConfig`) routes each interaction through small and large judges, aggregates their scores, and decides whether to retry, certify, or persist guidance.

### Persona

Runtime persona prompt bundles (planner, executor, synthesizer, verifier) that shape student and teacher behaviour. Personas can be updated via memory, tagged for reuse, and inspected in exported traces to understand how guidance evolved.

### Triage Dossier

Structured context produced by the triage adapter before execution begins. The dossier summarises task metadata, risks, history, and persona hints; it informs the capability probe and lane selection and is exported with every runtime trace.

## Metrics

### TES (Teaching Efficiency Score)

`(accuracy_gain * completion_rate) / (teaching_tokens / 1000)`

Measures the efficiency of teaching relative to token usage.

### NDR (Non-Degradation Rate)

Percentage of cases where ATLAS-enhanced response equals or exceeds baseline performance.

### Learning Rate (LR)

In ATLAS context: `Δ_performance / num_iterations`

Measures how quickly the system adapts to new tasks.

## Infrastructure

### vLLM

High-throughput inference server used during GRPO training for efficient generation. Handles distributed inference across GPUs.

### Flash Attention

Memory-efficient attention mechanism that speeds up training and reduces GPU memory usage. Recommended for all deployments.

### KL Divergence

Kullback-Leibler divergence - Constraint used in GRPO to prevent policy collapse by keeping the trained model close to the reference model.

## Optimization Terms

### Beta (β)

KL divergence coefficient in GRPO (default: 0.04). Controls how much the policy can deviate from the reference model.

### Temperature

Sampling parameter controlling randomness in generation (default: 0.7). Higher values increase diversity.

### Gradient Accumulation

Technique to simulate larger batch sizes by accumulating gradients over multiple forward passes before updating weights.

## See Also

* [Technical Report](/reference/technical-report) - Detailed methodology
* [Core Concepts](/concepts/hybrid-learning) - In-depth explanations
* [Training Guide](/training/offline/grpo-training) - Practical implementation


# Models
Source: https://docs.arc.computer/reference/models

Pre-trained ATLAS teacher models available on Hugging Face

## Available Models

ATLAS provides pre-trained teacher models optimized for different tasks. All models are 8B parameters and trained using the GRPO algorithm with adaptive dual-agent objectives.

## Teacher Models

### ATLAS-8B-Thinking

<Card title="View on Hugging Face" icon="hugging-face" href="https://huggingface.co/Arc-Intelligence/ATLAS-8B-Thinking">
  Optimized for mathematical and logical reasoning tasks
</Card>

**Best for:**

* Mathematical problem solving
* Logical reasoning
* Abstract thinking tasks
* Scientific analysis

**Usage:**

```python theme={null}
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Thinking",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Thinking"
)
```

**Training:**

* Base model: Qwen2.5-7B-Instruct
* Training method: SFT → GRPO
* Specialization: Reasoning-heavy tasks

### ATLAS-8B-Instruct

<Card title="View on Hugging Face" icon="hugging-face" href="https://huggingface.co/Arc-Intelligence/ATLAS-8B-Instruct">
  Optimized for code generation and technical instruction
</Card>

**Best for:**

* Code generation
* Technical documentation
* System administration
* API integration

**Usage:**

```python theme={null}
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "Arc-Intelligence/ATLAS-8B-Instruct"
)
```

**Training:**

* Base model: Qwen2.5-7B-Instruct
* Training method: SFT → GRPO
* Specialization: Instruction-following and coding

## Model Selection Guide

Choose the appropriate teacher model based on your task:

| Task Type       | Recommended Model | Reasoning                             |
| --------------- | ----------------- | ------------------------------------- |
| Math problems   | ATLAS-8B-Thinking | Specialized in step-by-step reasoning |
| Debugging       | ATLAS-8B-Instruct | Better at code understanding          |
| Data analysis   | ATLAS-8B-Thinking | Strong analytical capabilities        |
| API development | ATLAS-8B-Instruct | Trained on technical documentation    |
| Logic puzzles   | ATLAS-8B-Thinking | Abstract reasoning focus              |
| DevOps tasks    | ATLAS-8B-Instruct | System administration expertise       |

## Compatible Student Models

ATLAS teachers can enhance any instruction-following LLM:

**Tested Student Models:**

* Qwen/Qwen3-4B-Instruct (4B)
* meta-llama/Llama-3.2-8B-Instruct (8B)
* mistralai/Mixtral-8x7B-Instruct-v0.1 (47B)
* OpenAI GPT-4 (API)
* Anthropic Claude (API)

**Requirements:**

* Instruction-following capability
* Context window ≥4K tokens
* Support for system prompts (preferred)

## Memory Requirements

Estimated VRAM usage for inference:

| Configuration           | VRAM Required | Recommended Hardware |
| ----------------------- | ------------- | -------------------- |
| Teacher only (FP16)     | 16GB          | RTX 4080, A5000      |
| Teacher + Small Student | 24GB          | RTX 4090, A6000      |
| Teacher + Large Student | 40GB+         | A100, H100           |
| Quantized (INT8)        | 8GB           | RTX 3080, A4000      |
| Quantized (INT4)        | 4GB           | RTX 3070, T4         |

## Model Versioning

All models follow semantic versioning:

* **Latest stable**: No suffix (recommended for production)
* **Experimental**: `-experimental` suffix
* **Specific versions**: `-v1.0`, `-v1.1`, etc.

Check model cards on Hugging Face for:

* Training data details
* Performance benchmarks
* Known limitations
* Update changelog

## Custom Model Training

To train custom teacher models, see:

* [GKD Training](/training/offline/gkd-training) - Fast distillation (4-8 hours)
* [GRPO Training](/training/offline/grpo-training) - Full RL training (24-48 hours)
* [Training Configuration](/training/configuration) - Parameter reference

## License and Usage

All ATLAS models are released under Apache 2.0 license for both research and commercial use.

**Responsible Use:**

* Verify outputs for critical applications
* Monitor for potential biases
* Respect base model licenses
* Cite ATLAS in publications

## Next Steps

<CardGroup>
  <Card title="Quickstart" icon="rocket" href="/sdk/quickstart">
    Start using ATLAS models
  </Card>

  <Card title="Datasets" icon="database" href="/reference/datasets">
    Training and evaluation data
  </Card>

  <Card title="Custom Training" icon="dumbbell" href="/training/offline/grpo-training">
    Train your own models
  </Card>
</CardGroup>


# Technical Report
Source: https://docs.arc.computer/reference/technical-report

ATLAS research paper and technical specifications

## Abstract

ATLAS (Adaptive Teaching and Learning Alignment System) is a hybrid reinforcement learning architecture that enhances language model performance through an adaptive dual-agent reasoning loop. The system pairs your production agent (the student) with a verifying teacher that first diagnoses capability via a lightweight probe, then provides targeted guidance and certifications before answers ship.

Through extensive evaluation on mathematical reasoning, code generation, and system reliability engineering tasks, ATLAS demonstrates:

* **Closed-loop runtime gains**: +15.7% average accuracy, +31% completion, 97% non-degradation, \~50% token savings
* **Offline GRPO gains**: sustained quality improvements when fine-tuning custom teacher checkpoints from production traces

The framework combines offline reinforcement learning for foundational skills with runtime orchestration that keeps quality high in production. Task-specific continual learning is now delivered through the [`atlas-sdk`](https://github.com/Arc-Computer/atlas-sdk) runtime.

## Full Report

<Card title="Download Technical Report" icon="download" href="https://github.com/Arc-Computer/ATLAS/blob/main/docs/ATLAS-Technical-Report.pdf">
  Access the complete 28-page technical report with detailed methodology, experiments, and results
</Card>

## Key Contributions

### 1. Adaptive Dual-Agent Protocol

A two-pass inference mechanism that first diagnoses student capability (≤50 tokens) then provides calibrated verifying-teacher guidance (≤200 tokens) based on the assessment.

### 2. Hybrid Learning Architecture

Separation of expensive offline RL training from the managed runtime that captures production traces, enabling rapid adaptation without retraining base student models.

### 3. Compounding Intelligence

Demonstrated skill transfer across domains with up to 83% transfer efficiency, creating accumulating knowledge over time.

### 4. Safety Guarantees

Zero-reward for performance degradation ensures 97% non-degradation rate in production deployments.

## Experimental Results

### Performance Across Model Sizes

| Student Model | Size   | Baseline | w/ ATLAS | Improvement |
| ------------- | ------ | -------- | -------- | ----------- |
| Qwen3-4B      | 4B     | 62.3%    | 78.0%    | +15.7%      |
| Llama-3.1-8B  | 8B     | 71.2%    | 85.4%    | +14.2%      |
| Mixtral-8x7B  | 47B    | 78.5%    | 89.1%    | +10.6%      |
| GPT-4         | \~1.7T | 84.3%    | 92.8%    | +8.5%       |

### Domain-Specific Gains

* **SRE Debugging**: Systematic improvement in root cause analysis and reduced investigation time
* **Mathematical Reasoning**: 15.7% average gain (closed-loop baseline)
* **Code Generation**: 31% completion rate improvement
* **Continual Learning (SDK)**: Use the atlas-sdk runtime for rapid, task-specific adaptation between offline training runs

## Citation

If you use ATLAS in your research, please cite:

```bibtex theme={null}
@article{atlas2024,
  title={ATLAS: Adaptive Teaching and Learning Alignment System for RL},
  author={Arc Intelligence Team},
  journal={arXiv preprint},
  year={2024},
  url={https://github.com/Arc-Computer/ATLAS}
}
```

## Related Work

The ATLAS framework builds on several foundational works:

* GRPO (Group Relative Policy Optimization) for RL training
* Genetic prompt evolution research for online optimization, now implemented and maintained in the atlas-sdk runtime
* Constitutional AI principles for safe deployment

## Next Steps

<CardGroup>
  <Card title="Models" icon="robot" href="/reference/models">
    Pre-trained ATLAS models
  </Card>

  <Card title="Datasets" icon="database" href="/reference/datasets">
    Training and evaluation data
  </Card>

  <Card title="Implementation" icon="code" href="/sdk/quickstart">
    Get started with ATLAS
  </Card>

  <Card title="Examples" icon="lightbulb" href="/examples/adaptive-tool-use">
    See ATLAS in action
  </Card>
</CardGroup>


# Troubleshooting
Source: https://docs.arc.computer/reference/troubleshooting

Common issues and solutions for ATLAS deployment

## Common Issues

This guide covers frequent problems and their solutions when working with ATLAS.

## Installation Issues

### CUDA Not Available

**Problem:** `torch.cuda.is_available()` returns False

**Solutions:**

<Steps>
  <Step title="Verify CUDA Installation">
    ```bash theme={null}
    nvidia-smi
    nvcc --version
    ```

    If not found, install CUDA 11.8+ from NVIDIA
  </Step>

  <Step title="Reinstall PyTorch with CUDA">
    ```bash theme={null}
    pip uninstall torch torchvision torchaudio
    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
    ```
  </Step>

  <Step title="Check GPU Compatibility">
    Ensure GPU compute capability ≥7.0:

    ```python theme={null}
    import torch
    torch.cuda.get_device_capability()
    ```
  </Step>
</Steps>

### Flash Attention Build Failure

**Problem:** `pip install flash-attn` fails with compilation errors

**Solutions:**

* Ensure CUDA toolkit matches PyTorch version
* Install with pre-built wheels:
  ```bash theme={null}
  pip install flash-attn --no-build-isolation
  ```
* Skip Flash Attention (with performance impact):
  ```python theme={null}
  attn_implementation="eager"  # Instead of "flash_attention_2"
  ```

### Hugging Face Access Issues

**Problem:** Can't download models from Hugging Face

**Solutions:**

```bash theme={null}
# Login to Hugging Face
huggingface-cli login

# Set cache directory if disk space limited
export HF_HOME=/path/to/cache

# Use offline mode if downloaded
export HF_DATASETS_OFFLINE=1
export TRANSFORMERS_OFFLINE=1
```

## Memory Issues

### CUDA Out of Memory

**Problem:** `RuntimeError: CUDA out of memory`

**Progressive Solutions:**

<AccordionGroup>
  <Accordion title="Reduce Batch Size">
    ```python theme={null}
    config.per_device_train_batch_size = 1
    config.gradient_accumulation_steps = 32  # Maintain effective batch size
    ```
  </Accordion>

  <Accordion title="Enable Gradient Checkpointing">
    ```python theme={null}
    config.gradient_checkpointing = True
    # Trades compute for memory
    ```
  </Accordion>

  <Accordion title="Use Mixed Precision">
    ```python theme={null}
    config.fp16 = True  # or bf16 = True for A100/H100
    ```
  </Accordion>

  <Accordion title="Quantization">
    ```python theme={null}
    # 8-bit quantization
    config.load_in_8bit = True

    # 4-bit quantization
    config.load_in_4bit = True
    config.bnb_4bit_compute_dtype = torch.float16
    ```
  </Accordion>

  <Accordion title="CPU Offloading">
    ```python theme={null}
    config.offload = True
    # Or with DeepSpeed via Accelerate
    # accelerate launch --config_file accelerate/deepspeed_zero3_cpu_offloading.yaml -m atlas_core.cli.train ...
    ```
  </Accordion>
</AccordionGroup>

### Memory Calculation

Estimate memory requirements:

```python theme={null}
def estimate_memory_gb(model_params_b, batch_size, seq_length):
    """Rough memory estimate for training"""
    # Model weights
    weights_gb = model_params_b * 2 / 1024  # FP16

    # Activations (rough estimate)
    activations_gb = (batch_size * seq_length * 8192 * 4) / 1e9

    # Gradients and optimizer states
    optimizer_gb = weights_gb * 4  # Adam optimizer

    total_gb = weights_gb + activations_gb + optimizer_gb
    return total_gb

# Example: 8B model
memory_needed = estimate_memory_gb(8, batch_size=4, seq_length=2048)
print(f"Estimated memory: {memory_needed:.1f} GB")
```

## Training Issues

### Loss Not Decreasing

**Problem:** Training loss plateaus or increases

**Diagnostic Steps:**

```python theme={null}
# Check learning rate
print(f"Current LR: {trainer.optimizer.param_groups[0]['lr']}")

# Verify data loading
sample = next(iter(train_dataloader))
print(f"Input shape: {sample['input_ids'].shape}")
print(f"Labels present: {'labels' in sample}")

# Check gradient flow
for name, param in model.named_parameters():
    if param.grad is not None:
        print(f"{name}: grad_norm={param.grad.norm().item():.4f}")
```

**Solutions:**

* Reduce learning rate: `learning_rate=1e-6`
* Increase warmup: `warmup_ratio=0.2`
* Check for data issues: duplicates, incorrect labels
* Verify loss function: ensure proper masking

### NaN or Inf Loss

**Problem:** Loss becomes NaN or Inf

**Solutions:**

```python theme={null}
# Clip gradients more aggressively
config.max_grad_norm = 0.5

# Reduce learning rate
config.learning_rate = 1e-7

# Check for numerical instability
config.fp32 = True  # Use full precision temporarily

# Add gradient debugging
def check_gradients(model):
    for name, param in model.named_parameters():
        if param.grad is not None:
            if torch.isnan(param.grad).any():
                print(f"NaN gradient in {name}")
            if torch.isinf(param.grad).any():
                print(f"Inf gradient in {name}")
```

### Slow Training Speed

**Problem:** Training is slower than expected

**Performance Optimizations:**

```python theme={null}
# Enable torch.compile (PyTorch 2.0+)
model = torch.compile(model)

# Use Flash Attention
config.attn_implementation = "flash_attention_2"

# Optimize data loading
config.dataloader_num_workers = 4
config.dataloader_pin_memory = True

# Enable TF32 on Ampere GPUs
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.allow_tf32 = True

# Profile to find bottlenecks
from torch.profiler import profile, ProfilerActivity

with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]) as prof:
    trainer.train()
print(prof.key_averages().table(sort_by="cuda_time_total"))
```

## Inference Issues

### Slow Inference

**Problem:** Generation is too slow for production

**Solutions:**

```python theme={null}
# Use vLLM for faster inference
from vllm import LLM, SamplingParams

llm = LLM(model="Arc-Intelligence/ATLAS-8B-Thinking")
sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.95,
    max_tokens=512
)

# Batch processing
outputs = llm.generate(prompts, sampling_params)

# Or use torch.compile
model = torch.compile(model, mode="reduce-overhead")

# Enable KV cache
model.config.use_cache = True
```

### Inconsistent Results

**Problem:** Different results on each run

**Solutions:**

```python theme={null}
# Set seeds for reproducibility
import random
import numpy as np
import torch

def set_seed(seed=42):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)

set_seed(42)

# Use deterministic algorithms
torch.use_deterministic_algorithms(True)
torch.backends.cudnn.benchmark = False

# Set temperature to 0 for deterministic generation
generation_kwargs = {
    "temperature": 0.0,
    "do_sample": False
}
```

## vLLM Server Issues

### Server Won't Start

**Problem:** vLLM server fails to launch

**Diagnostic Commands:**

```bash theme={null}
# Check if port is in use
lsof -i :8000

# Test with smaller model
python -m vllm.entrypoints.openai.api_server \
  --model facebook/opt-125m \
  --port 8001

# Check GPU memory
nvidia-smi

# Verify vLLM installation
python -c "import vllm; print(vllm.__version__)"
```

**Solutions:**

* Reduce `--gpu-memory-utilization 0.8`
* Use smaller `--max-model-len 1024`
* Enable `--enable-prefix-caching`
* Try different port

### Connection Refused

**Problem:** Can't connect to vLLM server

**Solutions:**

```bash theme={null}
# Check server is running
ps aux | grep vllm

# Test connection
curl http://localhost:8000/v1/models

# Check firewall
sudo ufw status

# Use correct URL in code
vllm_url = "http://localhost:8000/v1"  # Not https
```

## SDK Runtime Issues

The following sections cover common issues when using the [Atlas SDK](https://github.com/Arc-Computer/atlas-sdk) for runtime orchestration and continual learning.

### SDK Installation Issues

#### Python Version Mismatch

**Problem:** `ImportError` or crashes at import time

**Solutions:**

```bash theme={null}
# Check Python version
python --version  # Should be 3.10+

# Create virtual environment with correct version
python3.12 -m venv .venv
source .venv/bin/activate

# Reinstall SDK
pip install --upgrade arc-atlas
```

**Note:** Python 3.9 and earlier are not supported. Use 3.10+ (3.13 recommended).

#### Package Import Errors

**Problem:** `ModuleNotFoundError` after installation

**Solutions:**

```bash theme={null}
# Ensure you're in the correct environment
which python
which pip

# Reinstall in current environment
pip uninstall arc-atlas -y
pip install arc-atlas

# Verify installation
python -c "import atlas; print(atlas.__version__)"
```

### API Configuration

#### Missing API Keys

**Problem:** `API key not found` or authentication errors

**Solutions:**

<AccordionGroup>
  <Accordion title="Using Environment Variables">
    ```bash theme={null}
    # Export directly
    export OPENAI_API_KEY="sk-..."
    export GEMINI_API_KEY="..."
    export ANTHROPIC_API_KEY="..."

    # Verify
    echo $OPENAI_API_KEY
    ```
  </Accordion>

  <Accordion title="Using .env File">
    ```bash theme={null}
    # Create .env file in project root
    cat > .env << EOF
    OPENAI_API_KEY=sk-...
    GEMINI_API_KEY=...
    ANTHROPIC_API_KEY=...
    EOF

    # SDK auto-loads .env files
    atlas run --config config.yaml --task "Your task"
    ```
  </Accordion>

  <Accordion title="Config File Reference">
    ```yaml theme={null}
    # config.yaml
    agent:
      llm:
        provider: openai
        api_key_env: OPENAI_API_KEY  # Must be set in environment
    ```
  </Accordion>
</AccordionGroup>

#### Multi-Provider Configuration

**Problem:** Using multiple LLM providers in one config

**Solution:**

```yaml theme={null}
# Student uses OpenAI
agent:
  llm:
    provider: openai
    model: gpt-4.1-mini
    api_key_env: OPENAI_API_KEY

# Teacher uses OpenAI
teacher:
  llm:
    provider: openai
    model: gpt-4.1
    api_key_env: OPENAI_API_KEY

# Reward system uses Gemini
rim:
  small_model:
    provider: gemini
    model: gemini/gemini-2.5-flash
    api_key_env: GEMINI_API_KEY
  large_model:
    provider: gemini
    model: gemini/gemini-2.5-pro
    api_key_env: GEMINI_API_KEY
```

### Storage & Database

#### Docker Daemon Not Running

**Problem:** `Cannot connect to Docker daemon`

**Solutions:**

```bash theme={null}
# macOS
open -a Docker

# Linux - check status
sudo systemctl status docker

# Linux - start Docker
sudo systemctl start docker

# Verify
docker ps
```

#### Postgres Connection Failures

**Problem:** `could not connect to server`

**Diagnostic Steps:**

```bash theme={null}
# Check if Postgres is running
docker ps | grep postgres

# Check if port is accessible
lsof -i :5433

# Test connection
psql postgresql://atlas:atlas@localhost:5433/atlas -c "SELECT 1"
```

**Solutions:**

<Steps>
  <Step title="Start Postgres with atlas init">
    ```bash theme={null}
    atlas init
    # Starts bundled Docker + Postgres on localhost:5433
    ```
  </Step>

  <Step title="Verify Connection">
    ```bash theme={null}
    # Check container is running
    docker ps --filter "name=atlas"

    # Test connection
    docker exec -it $(docker ps -q -f name=atlas) \
      psql -U atlas -c "SELECT version()"
    ```
  </Step>

  <Step title="Check Config">
    ```yaml theme={null}
    # config.yaml
    storage:
      database_url: postgresql://atlas:atlas@localhost:5433/atlas
    ```
  </Step>
</Steps>

#### Port Conflicts

**Problem:** `Port 5433 is already in use`

**Solutions:**

```bash theme={null}
# Find process using port
lsof -i :5433

# Kill process if needed
kill -9 <PID>

# Or use different port in config
storage:
  database_url: postgresql://atlas:atlas@localhost:5434/atlas
```

#### Running Without Storage

**Problem:** Want to run SDK without Postgres

**Solution:**

Storage is optional. The SDK will run without persistent storage, but rewards and learning history won't be saved:

```yaml theme={null}
# config.yaml - omit storage section entirely
agent:
  type: litellm
  # ... rest of config

# Storage section is optional
# storage:
#   database_url: postgresql://...
```

Sessions still save to `.atlas/runs/` as JSON files without database persistence.

### Discovery Issues

#### atlas env init Finds Nothing

**Problem:** `No agent classes detected`

**Solutions:**

<AccordionGroup>
  <Accordion title="Check Project Structure">
    ```bash theme={null}
    # Discovery looks for:
    # - LangChain agents (from langchain import *)
    # - LangGraph graphs (@graph decorator)
    # - Custom agent classes

    # Ensure your code imports these libraries
    grep -r "from langchain" .
    grep -r "from langgraph" .
    ```
  </Accordion>

  <Accordion title="Manually Specify Agent">
    ```yaml theme={null}
    # config.yaml
    agent:
      type: python
      import_path: your_module.agents
      attribute: create_agent
      # Or for LangGraph
      type: langgraph
      import_path: your_module.graph
      attribute: workflow
    ```
  </Accordion>

  <Accordion title="Check Virtual Environment">
    ```bash theme={null}
    # Ensure correct environment is active
    which python
    pip list | grep langchain

    # Discovery runs in your current environment
    atlas env init --verbose
    ```
  </Accordion>
</AccordionGroup>

#### Wrong Class Detected

**Problem:** Discovery picks the wrong agent

**Solution:**

Override auto-discovery with explicit config:

```yaml theme={null}
# config.yaml
agent:
  type: python
  name: my-specific-agent
  import_path: my_package.agents
  attribute: production_agent  # Specific function/class name
  system_prompt: |
    Custom prompt for this agent
```

#### Factory Synthesis Failures

**Problem:** Generated factory code fails

**Solutions:**

```bash theme={null}
# Check generated factory
cat .atlas/generated_factories.py

# Validate it loads
python -c "from .atlas.generated_factories import *"

# Regenerate if needed
rm -rf .atlas/
atlas env init

# Or skip auto-discovery and use explicit config
```

### Runtime Errors

#### LLM Provider Authentication

**Problem:** `401 Unauthorized` or `403 Forbidden`

**Solutions:**

```bash theme={null}
# Verify API key is valid
curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"

# Check key format
echo $OPENAI_API_KEY | grep -E "^sk-"  # OpenAI
echo $GEMINI_API_KEY | grep -E "^AI"   # Gemini (often starts with AI)

# Regenerate key if expired
# - OpenAI: https://platform.openai.com/api-keys
# - Anthropic: https://console.anthropic.com/keys
# - Gemini: https://aistudio.google.com/apikey
```

#### Timeout Errors

**Problem:** `Request timeout` during execution

**Solutions:**

```yaml theme={null}
# Increase timeouts in config
agent:
  llm:
    timeout_seconds: 180  # Default is 60

teacher:
  llm:
    timeout_seconds: 180

# Or for specific steps
orchestration:
  step_timeout_seconds: 900  # 15 minutes
```

#### MCP Server Connection Issues

**Problem:** MCP tools not available or connection refused

**Diagnostic Steps:**

```bash theme={null}
# Check MCP server is running
ps aux | grep mcp

# Test MCP endpoint
curl http://localhost:3000/health  # Adjust port

# Check logs
tail -f ~/.mcp/logs/server.log
```

**Solutions:**

```python theme={null}
# Ensure MCP server is started before agent
import subprocess
mcp_process = subprocess.Popen([
    "python", "-m", "your_mcp_server"
])

# Then run atlas
atlas run --config config.yaml --task "Your task"
```

## Continual Learning Support

For issues specific to offline training, reward synthesis, or the learning engine, refer to the training-specific sections above. The Atlas SDK handles runtime orchestration and data collection, while Atlas Core handles offline model training from collected traces.

## Getting Help

If these solutions don't resolve your issue:

1. **Check existing issues**: [GitHub Issues](https://github.com/Arc-Computer/ATLAS/issues)
2. **Join community**: [Discord Server](https://discord.gg/arc-atlas)
3. **File bug report** with:
   * Error message and stack trace
   * System info: `python -m torch.utils.collect_env`
   * Minimal reproduction code
   * Configuration used

## Next Steps

<CardGroup>
  <Card title="FAQ" icon="question" href="/reference/faq">
    Frequently asked questions
  </Card>

  <Card title="Community" icon="discord" href="https://discord.gg/arc-atlas">
    Get help from community
  </Card>

  <Card title="GitHub Issues" icon="github" href="https://github.com/Arc-Computer/ATLAS/issues">
    Report bugs
  </Card>
</CardGroup>


# Bring Your Own Agent
Source: https://docs.arc.computer/sdk/adapters

Connect the Atlas orchestrator to any agent via OpenAI, Python, or HTTP adapters.

Atlas SDK follows a simple promise: **your agent, our orchestration**. Adapters create a small, consistent interface so you can place the adaptive dual-agent loop (student + verifying teacher) on top of nearly anything—from hosted APIs to local Python functions.

## Choosing an Adapter

| Adapter    | Use when…                                                           | Strengths                                                                           | Things to watch                                                               |
| ---------- | ------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `litellm`  | You need multi-provider support or want future-proof compatibility. | Supports 100+ LLM providers, minimal setup, native tool calling, streaming support. | Recommended for all new projects.                                             |
| `http_api` | Your agent already runs behind an HTTP endpoint.                    | Language-agnostic, deploy-anywhere.                                                 | You define the payload schema, handle auth, and parse responses.              |
| `python`   | You want to call local functions or LangChain runnables directly.   | Lowest latency, easy debugging.                                                     | Runs inside the orchestrator process—ensure your code is safe and performant. |

```mermaid theme={null}
graph LR
    Student -->|Sends Prompt| Adapter
    Adapter -->|Makes Request| Agent
    Agent -->|Returns Response| Adapter
    Adapter -->|Provides Trace & Output| Student
    Student -->|Sends Context| Teacher
    Teacher -->|Provides Guidance| Student
```

## LiteLLM Adapter (`atlas/connectors/litellm.py`)

This is the recommended adapter for all LLM providers, supporting 100+ models via LiteLLM.

<Warning>
  The `type: openai` adapter is deprecated. Use `type: litellm` instead. The litellm adapter supports all OpenAI-compatible providers (OpenAI, Azure OpenAI) plus Anthropic Claude, Google Gemini, XAI Grok, AWS Bedrock, and local models via Ollama or vLLM. The openai type remains supported for backward compatibility but emits deprecation warnings.
</Warning>

```yaml theme={null}
agent:
  type: litellm
  name: sdk-quickstart-litellm
  system_prompt: |
    You are the Atlas Student. Be concise and helpful.
  tools: []
  llm:
    provider: openai
    model: gpt-4o-mini
    api_key_env: OPENAI_API_KEY
    temperature: 0.0
    max_output_tokens: 768
```

* Supports conversation history and tool call metadata automatically.
* Accepts `response_format` for JSON mode.
* Works with OpenAI, Anthropic Claude, Google Gemini, XAI Grok, Azure OpenAI, AWS Bedrock, and local models (Ollama, vLLM).

## HTTP Adapter

For microservices or non-Python agents. Set `type: http_api`, provide `transport.base_url`, and define `payload_template` + `result_path`. See [Configuration Reference](/sdk/configuration#agent-block-agent) for details.

## Python Adapter

For local functions or LangGraph runnables. Set `type: python`, specify `import_path` and `attribute`. Supports async/sync callables and generators. See [Configuration Reference](/sdk/configuration#agent-block-agent) for details.

## Building Custom Adapters

All adapters share a minimal interface (`AgentAdapter`). To add a new one (e.g., for gRPC), follow these steps:

1. Extend the `AdapterType` enum in `atlas/config/models.py`.
2. Implement a class inheriting from `AgentAdapter`.
3. Register it with `register_adapter` (see `atlas.connectors.registry`) and import the module from `atlas.connectors.__init__` so it auto-registers at runtime.

```python theme={null}
from atlas.connectors.registry import AgentAdapter, register_adapter
from atlas.config.models import AdapterType

class GRPCAdapter(AgentAdapter):
    async def ainvoke(self, prompt: str, metadata: dict | None = None) -> str:
        # 1. Connect to your gRPC service.
        # 2. Build the request from the prompt.
        # 3. Execute the call and get a response.
        # 4. Return the response as a string.
        return f"Response for prompt: {prompt}"

# Assumes you've added GRPC to the AdapterType enum
register_adapter(AdapterType.GRPC, GRPCAdapter)
```

Most teams start by copying the `http_api` adapter and swapping the transport layer.

<Note>
  Atlas auto-imports built-in adapters via `atlas.connectors.__init__`. Custom adapters should follow the same pattern—expose your module there (or import it in your app startup) so registration runs once on load.
</Note>

## Structured Payloads

Pass complex nested dictionaries as tasks without serialization overhead:

```python theme={null}
task = {
    "query": "Debug API errors",
    "context": {"service": "payments", "error_rate": 0.15}
}
result = adapter.execute(task=task)
```

Works with LangGraph, custom agents, and any adapter. No manual JSON encoding required.

## Learning Tracking

Integrate learning tracking into custom adapters:

```python theme={null}
from atlas.learning.usage import get_tracker

tracker = get_tracker()
playbook = tracker.resolve_playbook(learning_key="my-agent")
tracker.detect_and_record(user_input=task, playbook_entries=playbook)
tracker.record_action_adoption(entry_id=entry.id, adopted=True)
tracker.record_session_outcome(session_id=sid, success=True)
```

Four core methods: `resolve_playbook()`, `detect_and_record()`, `record_action_adoption()`, `record_session_outcome()`. See [Learning System](/sdk/learning-system) for details.

## Migrating from OpenAI to LiteLLM Adapter

Change `type: openai` to `type: litellm` in your config. The litellm adapter is a drop-in replacement with no breaking changes. Benefits include multi-provider support, local model compatibility, and elimination of deprecation warnings.

## Decision Checklist

| Need                                                   | Recommendation                                               |
| ------------------------------------------------------ | ------------------------------------------------------------ |
| Fastest time-to-first-run                              | `litellm` adapter with any provider.                         |
| Reuse an existing microservice                         | `http_api` adapter with proper retries and auth.             |
| Full control in local experiments                      | `python` adapter calling your local function.                |
| Access any LLM provider (OpenAI, Claude, Gemini, etc.) | Use `litellm` adapter with the appropriate provider setting. |

## Next Steps

* Configure the rest of the runtime in [`SDK Configuration`](/sdk/configuration).
* See how the orchestrator uses your adapter in [`How Orchestration Works`](/sdk/orchestration).
* Understand the dual-agent reasoning concept in [`Adaptive Dual-Agent Reasoning`](/concepts/adaptive-dual-agent-reasoning).


# Atlas CLI Reference
Source: https://docs.arc.computer/sdk/cli-reference

Command catalogue for discovery, configuration scaffolding, and runtime execution in the Atlas SDK.

The Atlas CLI packages everything ML engineers need to onboard a new environment, validate a stack, and ship telemetry
into the learning pipeline—without writing glue code. Commands are grouped under the `atlas` executable (installed with
`arc-atlas`) and the `arc-atlas` helper for export/review automation. The CLI automatically loads `.env` files from the
project root and extends `PYTHONPATH` with `src`/root directories.

| Workflow                 | Command(s)                                 | Summary                                                                   |
| ------------------------ | ------------------------------------------ | ------------------------------------------------------------------------- |
| Autodiscovery onboarding | `atlas env init`, `atlas env scaffold`     | Detect agent/environment pairs, synthesize factories, and cache metadata. |
| Runtime execution        | `atlas run`, `atlas run --config`          | Replay discovery output or run a full orchestrator config.                |
| Persistence bootstrap    | `atlas init`                               | Provision Postgres via Docker (optional).                                 |
| Review & export          | `arc-atlas review …`, `arc-atlas export …` | Gate traces behind human review and export approved sessions.             |
| Training handoff         | `atlas train`                              | Reuse exporter filters and launch Atlas Core pipelines.                   |

## Environment Discovery

### `atlas env init`

Discovers agents/environments and populates `.atlas/`. Uses Claude Haiku 4.5 for agent ranking (auto-selects at 0.85+ confidence).

Key flags: `--task`, `--scaffold-config-full`, `--no-run`, `--timeout` (240s default)

Outputs: `.atlas/discover.json`, `.atlas/generated_config.yaml`, `.atlas/runs/`

### `atlas env scaffold`

Seeds projects with reference factories (LangGraph template). Use `--template`, `--output`, `--force` flags. Follow with `atlas env init` to validate.

## Runtime Execution

### `atlas run`

Executes a discovered agent/environment pair using cached metadata.

* `--task` (required) – prompt to execute.
* `--path` – project root containing `.atlas/discover.json`.
* `--env KEY=value` – additional environment variables surfaced to the runtime worker.
* `--mode` – experimental execution-mode override (`auto`, `paired`, etc.) used for deterministic tests.
* `--max-steps` – informational cap for orchestration steps (helpful during debugging).
* `--timeout` – worker timeout (default 300 s).

Before launching, the CLI verifies module hashes, loads the latest learning playbooks (when enabled), and streams
metadata snapshots to `.atlas/runs/`.

### `atlas run --config`

When you provide `--config path/to/runtime.yaml`, the CLI bypasses discovery metadata and spins up the full orchestrator
stack defined in the YAML. This mode:

* loads the config via `atlas.config.loader`,
* enables validation caching and telemetry instrumentation from PR #74,
* respects the same environment-variable injection and task prompt flags,
* captures the full `ExecutionContext` metadata (steps, learning state, reward payloads) and writes it to `.atlas/runs/`
  alongside the config path, letting you correlate later exports with the exact settings that produced them.

Use this path when running orchestrator smoke tests, CI checks, or scripted experiments against a fixed configuration.

## Review & Safety Workflow

Atlas gates trace exports behind review status to keep production learning safe. Review utilities live behind the
`arc-atlas` entrypoint (same binary as `python -m atlas.cli.export`).

### `arc-atlas review`

Manage session approvals stored in Postgres:

* **List sessions**
  ```bash theme={null}
  arc-atlas review sessions --database-url postgresql://atlas:atlas@localhost:5433/atlas --status pending --limit 50
  ```
  Groups by review status (defaults to pending/quarantined/approved). Use `--limit`/`--offset` to page through results.

* **Approve**
  ```bash theme={null}
  arc-atlas review approve 123 --database-url ... --note "Validated in staging"
  ```

* **Quarantine**
  ```bash theme={null}
  arc-atlas review quarantine 456 --database-url ... --note "Reward regression"
  ```

All review commands honour `--quiet` to suppress info logs. For local development you can bypass the gate with
`ATLAS_REVIEW_REQUIRE_APPROVAL=0`, but production exports should always run through the review queue.

### Drift & Safety Flags

* `runtime_safety.drift` in the config enables z-score based drift detection; alerts surface in the review output.
* `runtime_safety.review.require_approval` (default `true`) controls export gating; changing it requires a config update
  or environment override (`ATLAS_REVIEW_REQUIRE_APPROVAL=0`).

## Exporting & Training

### `arc-atlas export`

Exports approved sessions to JSONL for training or offline analysis.

* `--database-url` – required Postgres connection.
* `--output` – destination file (default: `<atlas-core-path>/exports/<timestamp>.jsonl` when `atlas-core-path` is known).
* `--session-id` – explicit session IDs (repeatable).
* `--limit`/`--offset` – pagination for recent sessions.
* `--status` – runtime completion status filter (`succeeded`, `failed`, etc.).
* `--trajectory-event-limit` – cap telemetry per session.
* `--include-status approved` – allow additional review statuses; `--include-all-statuses` bypasses the gate entirely.
* `--batch-size` – database fetch chunk size.

If no sessions are exported, the CLI reminds you to approve pending sessions first. Output JSON matches the schema
documented in [`Export Runtime Traces`](/sdk/export-traces).

### `atlas train`

Bridges exports into Atlas Core’s offline pipeline. It reuses the exporter filters above and then launches
`atlas-core offline-pipeline` with Hydra overrides.

* Filters: `--session-id`, `--limit`, `--status`, `--include-status`, `--trajectory-event-limit`.
* Atlas Core overrides: `--atlas-core-path`, `--config-name`, `--data-config`, `--trainer-config`, `--model-config`.
* Dataset sampling: `--eval-ratio`, `--max-samples`, `--use-sample-dataset`.
* Execution control: `--dry-run` prints the derived command without launching.
* Tracking: `--wandb-project`, `--wandb-run-name`, and repeatable `--override` flags.

Run `atlas train` once Postgres contains a critical mass of approved sessions. For ad-hoc exports, stick with
`arc-atlas export`.

## Storage & Bootstrap Utilities

* **`atlas init`** – writes `atlas-postgres.yaml` (Docker Compose) to spin up the recommended Postgres instance. Use
  `--force` to overwrite or `--skip-docker-install` if Docker is already available.
* **`atlas quit`** – tears down storage resources created by `atlas init`.

## Environment & Troubleshooting Tips

* **`.env` loading** – All CLI entrypoints call `load_dotenv_if_available()`; keep provider keys, database URLs, and
  custom environment variables there.
* **PYTHONPATH** – `atlas env init` adds the project root and `src/` directory to `PYTHONPATH`, making local packages
  importable without manual exports.
* **Timeouts** – Adjust `--timeout` per command when hitting slower providers or complex factory initialization.
* **Logging** – Most commands support `--quiet`; otherwise logs stream to stdout with timestamps.

## Related Guides

* [`SDK Quickstart`](/sdk/quickstart) – end-to-end walkthrough using these commands.
* [`Learning System Architecture`](/sdk/learning-system) – how CLI telemetry feeds runtime learning.
* [`Installation`](/installation) – environment prerequisites and GPU-aware setup.
* [`Export Runtime Traces`](/sdk/export-traces) – JSON schema and downstream training integration.


# SDK Configuration Reference
Source: https://docs.arc.computer/sdk/configuration

Understand every block in an Atlas SDK YAML file so you can tailor the orchestrator to your agents.

Atlas SDK configs are the control tower for runtime orchestration. Every key is validated by a Pydantic schema (`atlas-sdk/atlas/config/models.py`), so mistakes surface before the adaptive dual-agent reasoning loop—your agent paired with a verifying teacher—spins up. Atlas uses [LiteLLM](https://github.com/BerriAI/litellm) as its primary adapter backend, making the system model-agnostic and compatible with 100+ LLM providers including OpenAI, Anthropic Claude, Google Gemini, XAI Grok, Azure OpenAI, AWS Bedrock, local models (Ollama, vLLM), and custom endpoints.

<Note>
  This page is a configuration reference. For adapter walkthroughs and orchestration concepts, see [`Bring Your Own Agent`](/sdk/adapters) and [`How Orchestration Works`](/sdk/orchestration).
</Note>

<Tip>
  Keep `atlas.core.run(..., stream_progress=True)` enabled while tuning configs—the live event stream mirrors exactly what persists to storage and makes it easy to spot misconfigured blocks.
</Tip>

## Root Config Overview

| Field               | Type / Default                                                                                                                         | Required? | Why it matters                                                                                                                |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `agent`             | Adapter union (`litellm` \| `http_api` \| `python` \| `openai`)                                                                        | Yes       | Connects the orchestrator to your underlying agent transport.                                                                 |
| `teacher`           | `TeacherConfig`                                                                                                                        | Yes       | Defines the verifying teacher persona, LLM, and feedback limits.                                                              |
| `rim`               | `RIMConfig`                                                                                                                            | Yes       | Configures the [`RIM`](/reference/glossary#rim-reward-integration-module) ensemble that drives retries and adaptive feedback. |
| `student`           | `StudentConfig` (token caps default to `2048`)                                                                                         | No        | Controls your agent’s (student) prompts, tool usage, and token budgets.                                                       |
| `orchestration`     | `OrchestrationConfig` (`max_retries=1`, `step_timeout_seconds=900`, `rim_guidance_tag="rim_feedback"`, `emit_intermediate_steps=true`) | No        | Governs retries, timeouts, and telemetry emission.                                                                            |
| `adaptive_teaching` | `AdaptiveTeachingConfig` (`enabled=true`)                                                                                              | No        | Triage, probe, and lane-selection policy.                                                                                     |
| `storage`           | `StorageConfig \| null` (default `null`)                                                                                               | No        | Enables Postgres persistence for traces and learning memory.                                                                  |
| `metadata`          | `Dict[str, Any]` (default `{}`)                                                                                                        | No        | Free-form tags for analytics and logging.                                                                                     |

## Agent Block (`agent`)

This block wires the orchestrator to your agent. The schema is defined by `AdapterConfig` and its subclasses in `atlas-sdk/atlas/config/models.py:67-176`; extra keys are rejected.

### Common fields

| Parameter       | Type / Default                                  | Required? | Why adjust                                                                                          |
| --------------- | ----------------------------------------------- | --------- | --------------------------------------------------------------------------------------------------- |
| `type`          | Enum: `litellm`, `http_api`, `python`, `openai` | Yes       | Selects which adapter subclass will validate the rest of the block. Use `litellm` for new projects. |
| `name`          | `str`                                           | Yes       | Appears in telemetry and logs; use a descriptive identifier per deployment.                         |
| `system_prompt` | `str`                                           | Yes       | Baseline persona text passed to your agent (the student).                                           |
| `tools`         | `List[ToolDefinition]` (default `[]`)           | No        | Register JSON-schema tool signatures; validation ensures required keys exist.                       |

### HTTP adapter (`type: http_api`)

| Parameter                         | Type / Default                      | Required? | Why adjust                                                          |
| --------------------------------- | ----------------------------------- | --------- | ------------------------------------------------------------------- |
| `transport.base_url`              | `str`                               | Yes       | Base endpoint for your service.                                     |
| `transport.headers`               | `Dict[str, str]` (default `{}`)     | No        | Inject auth or custom headers.                                      |
| `transport.timeout_seconds`       | `float` (default `60.0`)            | No        | Increase when downstream APIs are slow.                             |
| `transport.retry.attempts`        | `int` (default `1`, bounded `1..5`) | No        | Add resilience for flaky endpoints.                                 |
| `transport.retry.backoff_seconds` | `float` (default `1.0`)             | No        | Control backoff between retry attempts.                             |
| `payload_template`                | `Dict[str, Any]` (default `{}`)     | No        | Provide a skeleton payload with placeholders the runtime will fill. |
| `result_path`                     | `Sequence[str] \| null`             | No        | Extract a nested field from the response JSON.                      |

### Python adapter (`type: python`)

| Parameter           | Type / Default           | Required? | Why adjust                                                                  |
| ------------------- | ------------------------ | --------- | --------------------------------------------------------------------------- |
| `import_path`       | `str`                    | Yes       | Python module or package that exposes your callable.                        |
| `attribute`         | `str \| null`            | No        | Specify the function/class name when the module exports multiple callables. |
| `working_directory` | `str \| null`            | No        | Run relative imports against a specific path.                               |
| `allow_generator`   | `bool` (default `false`) | No        | Enable when the callable yields streaming results.                          |
| `llm`               | `LLMParameters \| null`  | No        | Supply metadata when the callable proxies an LLM (e.g., for telemetry).     |

### LiteLLM adapter (`type: litellm`)

| Parameter               | Type / Default                        | Required? | Why adjust                                                                                          |
| ----------------------- | ------------------------------------- | --------- | --------------------------------------------------------------------------------------------------- |
| `llm.provider`          | `str`                                 | Yes       | Choose from 100+ providers: `openai`, `anthropic`, `gemini`, `xai`, `azure-openai`, `bedrock`, etc. |
| `llm.model`             | `str`                                 | Yes       | Choose the underlying chat model.                                                                   |
| `llm.api_key_env`       | `str`                                 | Yes       | Environment variable containing the API key.                                                        |
| `llm.api_base`          | `str \| null`                         | No        | Override the base URL for local models or custom endpoints.                                         |
| `llm.temperature`       | `float` (default `0.0`, range `0..2`) | Yes       | Increase for more exploratory generations.                                                          |
| `llm.top_p`             | `float \| null`                       | No        | Apply nucleus sampling if desired.                                                                  |
| `llm.max_output_tokens` | `int`                                 | Yes       | Cap response length.                                                                                |
| `llm.timeout_seconds`   | `float` (default `60.0`)              | No        | Widen for long-running completions.                                                                 |
| `llm.retry.attempts`    | `int` (default `1`, bounded `1..5`)   | No        | Increase for transient API failures.                                                                |
| `response_format`       | `Dict[str, Any] \| null`              | No        | Request JSON schema enforcement when the provider supports it.                                      |

<Tip>
  **Using local models:** The litellm adapter makes local model integration seamless.

  **Ollama:**

  ```yaml theme={null}
  agent:
    type: litellm
    llm:
      provider: openai  # Ollama is OpenAI-compatible
      model: llama3.1
      api_base: http://localhost:11434
      api_key_env: DUMMY  # Ollama doesn't need auth
      temperature: 0.2
      max_output_tokens: 2048
  ```

  **vLLM:**

  ```yaml theme={null}
  agent:
    type: litellm
    llm:
      provider: openai
      model: meta-llama/Llama-3.1-8B-Instruct
      api_base: http://localhost:8000/v1
      api_key_env: DUMMY
      temperature: 0.2
      max_output_tokens: 2048
  ```

  Both Ollama and vLLM are OpenAI-compatible, so use `provider: openai` with the correct `api_base`.
</Tip>

### Provider Examples

Common LiteLLM provider configurations:

| Provider     | Model Example                   | API Key Env                         |
| ------------ | ------------------------------- | ----------------------------------- |
| OpenAI       | `gpt-4o-mini`                   | `OPENAI_API_KEY`                    |
| Anthropic    | `claude-sonnet-4-5`             | `ANTHROPIC_API_KEY`                 |
| Gemini       | `gemini/gemini-2.5-flash`       | `GEMINI_API_KEY`                    |
| XAI Grok     | `xai/grok-4-fast`               | `XAI_API_KEY`                       |
| Azure OpenAI | `gpt-4o-mini`                   | `AZURE_OPENAI_API_KEY` + `api_base` |
| AWS Bedrock  | `anthropic.claude-3-5-sonnet-*` | `AWS_ACCESS_KEY_ID` + region/secret |

All use `temperature: 0.2` and `max_output_tokens: 2048` by default. See LiteLLM docs for full provider list.

## Student Block (`student`)

Guides the student agent’s prompts and token budgets. When `prompts` is omitted, the runtime builds defaults from the agent `system_prompt`.

| Parameter              | Type / Default                                | Required? | Why adjust                                                         |
| ---------------------- | --------------------------------------------- | --------- | ------------------------------------------------------------------ |
| `prompts`              | `StudentPrompts \| null`                      | No        | Override planner/executor/synthesizer prompt templates explicitly. |
| `prompt_guidance`      | `Dict[str, str]` (default `{}`)               | No        | Supply reusable chunks merged into prompts per run.                |
| `max_plan_tokens`      | `int` (default `2048`)                        | No        | Raise when plans are truncated.                                    |
| `max_step_tokens`      | `int` (default `2048`)                        | No        | Increase for verbose tool output.                                  |
| `max_synthesis_tokens` | `int` (default `2048`)                        | No        | Allow longer final answers.                                        |
| `tool_choice`          | Literal `auto` \| `required` (default `auto`) | No        | Force tool invocation on every step when governance demands it.    |

Override example:

```yaml theme={null}
student:
  max_plan_tokens: 1024
  max_step_tokens: 1024
  tool_choice: auto
```

## Teacher Block (`teacher`)

Defines the verifying teacher persona that validates plans, emits guidance, and certifies results.

| Parameter               | Type / Default                  | Required? | Why adjust                                                                                                                                   |
| ----------------------- | ------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `llm`                   | `LLMParameters`                 | Yes       | Choose the verifying teacher model (often stronger than the student agent). Supports all LiteLLM providers; use `api_base` for local models. |
| `max_review_tokens`     | `int \| null` (default `null`)  | No        | Cap plan-review responses.                                                                                                                   |
| `plan_cache_seconds`    | `int` (default `300`)           | No        | Reuse approved plans for repeated task IDs.                                                                                                  |
| `guidance_max_tokens`   | `int \| null`                   | No        | Limit per-step feedback length.                                                                                                              |
| `validation_max_tokens` | `int \| null`                   | No        | Cap the validation verdict.                                                                                                                  |
| `prompts`               | `TeacherPrompts \| null`        | No        | Replace default reviewer prompts.                                                                                                            |
| `prompt_guidance`       | `Dict[str, str]` (default `{}`) | No        | Inject reusable guidance fragments.                                                                                                          |

## Orchestration Block (`orchestration`)

Controls retry semantics and telemetry.

| Parameter                 | Type / Default                    | Required? | Why adjust                                                                              |
| ------------------------- | --------------------------------- | --------- | --------------------------------------------------------------------------------------- |
| `max_retries`             | `int` (default `1`, hard ceiling) | No        | Set to `0` to disable retries entirely.                                                 |
| `step_timeout_seconds`    | `float` (default `900.0`)         | No        | Lengthen for slow tools or external APIs.                                               |
| `rim_guidance_tag`        | `str` (default `"rim_feedback"`)  | No        | Change when your prompts expect a different insertion tag.                              |
| `emit_intermediate_steps` | `bool` (default `true`)           | No        | Toggle console/storage streaming of intermediate events.                                |
| `forced_mode`             | `AdaptiveMode \| null`            | No        | Lock the runtime to `auto`, `paired`, or `coach` (useful for deterministic evaluation). |

## Reward System Block (RIM - Reward Interpretation System)

The RIM (Reward Interpretation System) evaluates each trajectory to decide whether to retry or accept the outcome. Configure the reward system using the `rim` block in your runtime config.

| Parameter               | Type / Default                                                       | Required? | Why adjust                                                                                                             |
| ----------------------- | -------------------------------------------------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------- |
| `small_model`           | `LLMParameters`                                                      | Yes       | Fast path judge; keep lightweight for latency-sensitive checks. Supports all LiteLLM providers including local models. |
| `large_model`           | `LLMParameters`                                                      | Yes       | Escalation judge invoked on disagreement. Supports all LiteLLM providers including local models.                       |
| `active_judges`         | `Dict[str, bool]` (default `{"process": true, "helpfulness": true}`) | No        | Toggle built-in dimensions or add custom judges.                                                                       |
| `variance_threshold`    | `float` (default `0.15`)                                             | No        | Lower to escalate disagreements sooner.                                                                                |
| `uncertainty_threshold` | `float` (default `0.3`)                                              | No        | Raise to reduce escalations on ambiguous scores.                                                                       |
| `parallel_workers`      | `int` (default `4`, range `1..32`)                                   | No        | Tune concurrency to match judge model throughput.                                                                      |
| `judge_prompt`          | `str \| null`                                                        | No        | Provide a rubric that defines success for your domain.                                                                 |

See [`Reward Design`](/concepts/reward-design#reward-system-in-the-atlas-sdk) for judge composition examples.

## Adaptive Teaching Block (`adaptive_teaching`)

Configures triage, probing, and lane routing for the adaptive dual-agent pair—your agent plus the verifying teacher (`atlas-sdk/atlas/config/models.py:185-227`).

| Parameter                          | Type / Default                           | Required?                     | Why adjust                                                        |
| ---------------------------------- | ---------------------------------------- | ----------------------------- | ----------------------------------------------------------------- |
| `enabled`                          | `bool` (default `true`)                  | No                            | Disable to bypass adaptive routing entirely.                      |
| `certify_first_run`                | `bool` (default `true`)                  | No                            | Force first-time personas through `paired` certification.         |
| `mode_override`                    | Literal \| `null`                        | No                            | Pin execution to `auto`, `paired`, or `coach`.                    |
| `triage_adapter`                   | `str \| null`                            | No                            | Reference a custom dossier builder.                               |
| `default_tags`                     | `List[str]` (default `[]`)               | No                            | Apply default metadata to persona memories.                       |
| `probe.llm`                        | `LLMParameters \| null`                  | No                            | Override the capability probe model.                              |
| `probe.thresholds`                 | `auto=0.85`, `paired=0.65`, `coach=0.35` | No                            | Adjust lane cut-offs; order must satisfy `auto ≥ paired ≥ coach`. |
| `probe.fallback_mode`              | Literal (`"paired"` default)             | No                            | Lane chosen when the probe cannot decide.                         |
| `probe.evidence_limit`             | `int` (default `6`, range `1..32`)       | No                            | Limit how many supporting reasons the probe collects.             |
| `probe.timeout_seconds`            | `float` (default `15.0`)                 | No                            | Extend for slower models.                                         |
| `reward.type`                      | Literal `rim` (default) \| `python`      | No                            | Switch to a custom reward objective.                              |
| `reward.import_path` / `attribute` | `str` / `str`                            | Required when `type="python"` | Point at your custom scorer.                                      |
| `reward.focus_prompt`              | `str \| null`                            | No                            | Give the reward model an extra steer for this deployment.         |

## Storage Block (`storage`)

Controls Postgres persistence (`atlas-sdk/atlas/config/models.py:299-307`). Omit the block or set `storage: null` for ephemeral runs.

| Parameter                   | Type / Default           | Required?                | Why adjust                                        |
| --------------------------- | ------------------------ | ------------------------ | ------------------------------------------------- |
| `database_url`              | `str`                    | Yes (when block present) | Point at your managed or local Postgres instance. |
| `min_connections`           | `int` (default `1`)      | No                       | Increase for burstier workloads.                  |
| `max_connections`           | `int` (default `5`)      | No                       | Upper bound for connection pool size.             |
| `statement_timeout_seconds` | `float` (default `30.0`) | No                       | Abort long-running queries sooner.                |

Tip: `atlas init` scaffolds a Docker Compose file with sensible defaults and exposes Postgres on `localhost:5433`.

## Learning Block (`learning`)

Controls the runtime synthesizer that generates and applies student/teacher playbooks.

| Parameter                           | Type / Default                                | Required? | Why adjust                                                                                               |
| ----------------------------------- | --------------------------------------------- | --------- | -------------------------------------------------------------------------------------------------------- |
| `enabled`                           | `bool` (default `true`)                       | No        | Disable to run without loading or updating playbooks.                                                    |
| `update_enabled`                    | `bool` (default `true`)                       | No        | Freeze updates while keeping existing playbooks active.                                                  |
| `llm`                               | `LLMParameters \| null`                       | No        | Override the synthesizer model; falls back to runtime defaults otherwise.                                |
| `prompts`                           | `LearningPrompts \| null`                     | No        | Supply custom prompts for the synthesizer LLM.                                                           |
| `history_limit`                     | `int` (default `10`)                          | No        | Cap historical sessions fed into each update.                                                            |
| `session_note_enabled`              | `bool` (default `true`)                       | No        | Persist per-session learning notes alongside the registry.                                               |
| `apply_to_prompts`                  | `bool` (default `true`)                       | No        | Toggle playbook injection into persona prompts and validation payloads.                                  |
| `playbook_injection_mode`           | `"prefix"` or `"suffix"` (default `"prefix"`) | No        | Inject playbook before (`prefix`) or after (`suffix`) system prompt. Suffix mode enables KV cache reuse. |
| `inject_few_shot_examples`          | `bool` (default `true`)                       | No        | Append captured examples to playbook entries for in-context learning. Now enabled by default.            |
| `max_few_shot_token_budget`         | `int` (default `500`)                         | No        | Maximum tokens allocated for few-shot examples in playbook injection.                                    |
| `token_budget_chars_per_token`      | `float` (default `3.5`)                       | No        | Character-to-token ratio for estimating few-shot example token usage.                                    |
| `max_entries_to_process`            | `int` (default `10`)                          | No        | Maximum number of historical entries to process when extracting few-shot examples.                       |
| `max_examples_per_block`            | `int` (default `2`)                           | No        | Maximum few-shot examples to include per playbook block.                                                 |
| `usage_tracking.redaction_patterns` | `List[str]` (default `[]`)                    | No        | Regex patterns for redacting sensitive data from usage tracking logs.                                    |

Pair this section with [`Learning System Architecture`](/sdk/learning-system) for deeper context.

## Runtime Safety Block (`runtime_safety`)

Defines production guardrails for drift detection and export review policies.

| Parameter                        | Type / Default                       | Required? | Why adjust                                                |
| -------------------------------- | ------------------------------------ | --------- | --------------------------------------------------------- |
| `drift.enabled`                  | `bool` (default `true`)              | No        | Disable statistical drift alerts (rarely recommended).    |
| `drift.window`                   | `int` (default `50`)                 | No        | Increase for noisy telemetry; decrease for faster alerts. |
| `drift.z_threshold`              | `float` (default `3.0`)              | No        | Lower to make alerts more sensitive.                      |
| `drift.min_baseline`             | `int` (default `5`)                  | No        | Require more samples before alerts fire.                  |
| `review.require_approval`        | `bool` (default `true`)              | No        | Keep true in production to gate exports on human review.  |
| `review.default_export_statuses` | `List[str]` (default `["approved"]`) | No        | Adjust when automation needs additional review states.    |

See [`Runtime Safety & Review`](/sdk/runtime-safety) for operational guidance.

## Metadata

| Parameter  | Type / Default                  | Required? | Why adjust                                       |
| ---------- | ------------------------------- | --------- | ------------------------------------------------ |
| `metadata` | `Dict[str, Any]` (default `{}`) | No        | Attach labels consumed by your monitoring stack. |

<Warning>
  Legacy configs may still include a `prompt_rewrite` block, but the runtime now rejects it (`atlas-sdk/atlas/core/__init__.py` raises a `ValueError`). Remove the block and rely on explicit `student.prompts` / `teacher.prompts` instead.
</Warning>

## Cheat Sheet

| Goal                           | Section to edit                       | Pointer                                                                           |
| ------------------------------ | ------------------------------------- | --------------------------------------------------------------------------------- |
| Swap to Anthropic or Gemini    | `agent`                               | Use `type: litellm` with `provider: anthropic` or `provider: gemini`.             |
| Use local models (Ollama/vLLM) | `agent`                               | Use `type: litellm`, `provider: openai`, and set `api_base` to your local server. |
| Tighten or loosen retries      | `orchestration` + `rim`               | Adjust `max_retries`, `variance_threshold`, and `uncertainty_threshold`.          |
| Persist adaptive memories      | `storage`                             | Add a Postgres URL or run `atlas init`.                                           |
| Force a supervision lane       | `adaptive_teaching`                   | Set `mode_override` to `auto`, `paired`, or `coach`.                              |
| Personalise prompts            | `student.prompts` / `teacher.prompts` | Override templates or reuse `prompt_guidance`.                                    |
| Enforce JSON output            | `agent` (`response_format`)           | Provide OpenAI-compatible schemas or swap to `http_api` with custom validation.   |
| Freeze playbook updates        | `learning.update_enabled`             | Pause runtime learning while investigating regressions.                           |
| Require approvals for exports  | `runtime_safety.review`               | Keep `require_approval=true` and document review notes.                           |

## Validated Example (Quickstart)

This minimal config demonstrates the recommended litellm adapter with OpenAI models:

```yaml theme={null}
agent:
  type: litellm
  name: example-litellm-agent
  system_prompt: |
    You are an AI model acting as the Atlas Student. Follow instructions carefully and respond with JSON when asked.
  tools: []
  llm:
    provider: openai
    model: gpt-4o-mini
    api_key_env: OPENAI_API_KEY
    temperature: 0.2
    max_output_tokens: 2048

teacher:
  llm:
    provider: openai
    model: gpt-4o-mini
    api_key_env: OPENAI_API_KEY
    temperature: 0.1
    max_output_tokens: 2048

rim:
  small_model:
    provider: gemini
    model: gemini/gemini-2.5-flash
    api_key_env: GEMINI_API_KEY
    max_output_tokens: 8096
  large_model:
    provider: gemini
    model: gemini/gemini-2.5-flash
    api_key_env: GEMINI_API_KEY
    max_output_tokens: 8096
  judge_prompt: 'reward the agent for attending the issues mentioned in the task'
  variance_threshold: 0.15
  uncertainty_threshold: 0.3

storage:
  database_url: postgresql://atlas:atlas@localhost:5433/atlas
  min_connections: 1
  max_connections: 5
  statement_timeout_seconds: 30
```

<Note>
  **Legacy configs:** If you have existing configs using `type: openai`, they will continue to work but emit deprecation warnings. Migrate to `type: litellm` at your convenience.
</Note>

## Parameter Index (Alphabetical)

* `adaptive_teaching.default_tags` – Tag sessions and learning updates with deployment metadata.
* `adaptive_teaching.mode_override` – Force the runtime into a specific lane for deterministic evaluation.
* `agent.response_format` – Request JSON-mode enforcement from OpenAI-compatible providers.
* `learning.apply_to_prompts` – Enable/disable playbook injection into persona prompts.
* `learning.update_enabled` – Gate persistence of new playbooks after each session.
* `orchestration.forced_mode` – Hard-set the execution mode regardless of probe results.
* `runtime_safety.drift.z_threshold` – Sensitivity of automatic drift alerts.
* `runtime_safety.review.default_export_statuses` – Review states included when tooling omits filters.
* `storage.database_url` – Connection string for the Postgres telemetry store.
* `student.tool_choice` – Force tool invocation on each step when governance demands it.
* `teacher.plan_cache_seconds` – Duration to reuse previously approved plans.

## Related Guides

* [`Bring Your Own Agent`](/sdk/adapters) — Adapter-specific tutorials.
* [`How Orchestration Works`](/sdk/orchestration) — Dual-agent control flow (student agent + verifying teacher) deep dive.
* [`Reward Design`](/concepts/reward-design) — Building and tuning judge ensembles.


# Export Runtime Traces
Source: https://docs.arc.computer/sdk/export-traces

Access Atlas SDK session data via direct database queries or JSONL export

The Atlas SDK persists every orchestration session, including per-step rewards, guidance history, and tool usage. You can access this data through:

1. **Direct Database Access** (Recommended) - Query PostgreSQL directly with the `atlas.training_data` module for filtered, high-performance access
2. **JSONL Export** (Alternative Method) - Use the `arc-atlas` CLI to export sessions to JSONL files

<Note>
  **For training pipelines:** Direct database access is recommended (SDK v0.1.13+). It eliminates schema drift, provides 10-100x faster queries with database indexes, and supports reward-based filtering at the database level.
</Note>

## 1. Enable Postgres Persistence

Add a `storage` block to your SDK config:

```yaml theme={null}
storage:
  database_url: postgresql://atlas:atlas@localhost:5433/atlas
  min_connections: 1
  max_connections: 5
  statement_timeout_seconds: 30
```

Run your tasks with `atlas.core.run(..., stream_progress=True)` as usual. Each session, step result, and intermediate event is written to Postgres.

## 2. Direct Database Access (Recommended)

Query training sessions directly from PostgreSQL with reward-based filtering and selective data loading:

```python theme={null}
from atlas.training_data import get_training_sessions

# Query sessions with filters
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    learning_key="security-review",
    status_filters=["succeeded"],
    limit=1000
)

# Access session data
for session in sessions:
    reward_score = session.session_reward["score"]
    trajectory = session.trajectory_events
    learning_data = session.learning_history
```

### Key Features

* **No intermediate files**: Query directly from PostgreSQL
* **Database-level filtering**: Reward, status, date range, and learning key filters
* **Selective loading**: Control which fields are loaded (`include_trajectory_events`, `include_learning_data`)
* **Pagination support**: Process large datasets in batches with async iterators
* **10-100x faster**: Database indexes optimize reward and date range queries

### Example: Pagination for Large Datasets

```python theme={null}
from atlas.training_data import paginate_sessions

async for batch in paginate_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    batch_size=100,
    min_reward=0.7
):
    for session in batch:
        process_session(session)
```

See the [Training Data Pipeline Guide](/training/offline/training-data-pipeline) for complete API reference and advanced usage.

## 3. JSONL Export (Alternative Method)

```bash theme={null}
arc-atlas \
  --database-url postgresql://atlas:atlas@localhost:5433/atlas \
  --output traces/my-session.jsonl \
  --include-status approved \
  --trajectory-event-limit 500 \
  --status succeeded \
  --limit 50
```

<Note>
  Start Postgres before exporting (e.g., `docker compose up -d postgres` or `brew services start postgresql`) so the CLI can connect successfully.
</Note>

<Tip>
  If another tool owns the `atlas` command on your system, run the exporter with `python -m atlas.cli.export ...` or adjust `PATH` so `arc-atlas` resolves first.
</Tip>

### Optional filters

* `--session-id 42` (repeatable) exports specific sessions.
* `--limit 25` / `--offset 25` page through recent sessions.
* `--status succeeded --status failed` filters on runtime completion state.
* `--include-status approved` (repeatable) restricts review statuses; omit to inherit `runtime_safety.review.default_export_statuses`. Use `--include-all-statuses` for exploratory exports.
* `--trajectory-event-limit 200` caps the number of intermediate telemetry events embedded per session.

The exporter writes one JSON object per line. Each record aligns with `AtlasSessionTrace`:

```jsonc theme={null}
{
  "task": "Summarize the latest Atlas SDK updates",
  "final_answer": "...",
  "adaptive_summary": {
    "adaptive_mode": "coach",
    "confidence": 0.58,
    "certification_run": false,
    "probe": {
      "mode": "coach",
      "confidence": 0.55,
      "evidence": ["persona_helpful_ratio=0.62", "risk_high_severity"]
    },
    "mode_history": [
      {"mode": "paired", "confidence": 0.71, "certification": true},
      {"mode": "coach", "confidence": 0.55}
    ]
  },
  "triage_dossier": {
    "task": "Summarize the latest Atlas SDK updates",
    "summary": "Capture highlights for stakeholders.",
    "risks": [{"category": "quality", "description": "Customer-facing copy", "severity": "moderate"}],
    "signals": [{"name": "tenant", "value": "demo"}],
    "tags": ["tenant:demo", "domain:sre"]
  },
  "plan": {"steps": [{"id": 1, "description": "Collect release notes"}, {"id": 2, "description": "Draft summary"}]},
  "steps": [
    {
      "step_id": 1,
      "description": "Collect release notes",
      "trace": "HUMAN: ...",
      "output": "...",
      "reward": {
        "score": 0.92,
        "judges": [
          {"identifier": "process", "score": 0.91, "rationale": "..."}
        ]
      },
      "guidance": ["Cite the release date."],
      "validation": {"valid": true, "rationale": "Complete"},
      "tool": "web_search",
      "tool_params": {"query": "Atlas SDK release notes"},
      "artifacts": {"sources": ["https://..."]},
      "deliverable": {"notes": ["https://..."]}
    }
  ],
  "session_reward": {
    "score": 0.88,
    "uncertainty": 0.07,
    "judges": [
      {"identifier": "process", "score": 0.90, "rationale": "..."}
    ]
  },
  "reward_summary": {"score": 0.88},
  "review_status": "approved",
  "personas_used": [
    {"persona": "planner", "instruction": "Focus on customer tone", "source": "memory"}
  ],
  "persona_updates": {
    "new_candidates": [
      {"persona": "planner", "instruction": "Mention adaptive modes", "tags": ["tenant:demo"]}
    ]
  },
  "session_metadata": {"batch": "aime-2025"}
}
```

> **Tip:** Compress large exports with `xz` or `gzip`—the loader streams line-by-line, so you can decompress on the fly if desired.

Use `adaptive_summary` to audit routing choices, probe evidence, and certification status; `triage_dossier` captures the structured context that informed the decision (see [`triage dossier`](/reference/glossary#triage-dossier)); `personas_used` and `persona_updates` highlight which [`personas`](/reference/glossary#persona) were active and how memory evolved during the run. Each step also carries structured `artifacts` captured during execution and a `deliverable` payload that mirrors what the Student hands back to downstream systems.

<Note>
  Review gating defaults to approved sessions. Set `ATLAS_REVIEW_REQUIRE_APPROVAL=0` only for local experiments and
  always note which review statuses were exported alongside your artifacts.
</Note>

## 4. Feed the Training Stack

### Using Direct Database Access (Recommended)

```python theme={null}
from atlas.training_data import get_training_sessions
from atlas_core.data.runtime_traces import sessions_to_rl_records

# Query sessions directly
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    status_filters=["succeeded"],
    limit=10000
)

# Convert to RL training records
records = sessions_to_rl_records(sessions)
```

### Using JSONL Export (Alternative Method)

```python theme={null}
from atlas_core.data.runtime_traces import load_runtime_traces, sessions_to_rl_records

sessions = load_runtime_traces("traces/my-session.jsonl")
records = sessions_to_rl_records(sessions)
```

Or use the Hydra shortcut (`src/atlas_core/configs/data/runtime_traces.yaml`) described in the top-level quickstart. The schema matches the training adapters, so no custom glue code is required.

## Troubleshooting

| Error                         | Likely cause             | Fix                                                                 |
| ----------------------------- | ------------------------ | ------------------------------------------------------------------- |
| `database connection refused` | Postgres URL unreachable | Verify host/port, ensure server is running.                         |
| Empty JSONL file              | No sessions stored       | Confirm `storage` block is enabled and runs completed successfully. |
| Missing rewards in JSON       | Judges disabled          | Ensure your `rim` block activates the judges you expect.            |

With the exporter in place you can schedule nightly runs, collect batches of traces, and continuously fine-tune the teacher without manual wrangling.


# Learning System Architecture
Source: https://docs.arc.computer/sdk/learning-system

Map the decoupled learning synthesis pipeline, telemetry stores, and runtime controls that drive Atlas SDK learning.

Atlas runtime learning exists to make the teacher more effective between offline GRPO cycles. Instead of hard-coding
prompt tweaks or waiting for the next training run, the SDK captures guidance from successful sessions, synthesizes a
“playbook,” and reinjects that context on subsequent requests. This page explains how the learning pipeline works,
how data persists in Postgres, and which configuration levers production teams can use to control it.

## Why Runtime Learning Matters

* **Faster feedback loops** – learning runs on live telemetry, so student/teacher personas improve within hours instead of a full fine-tune.
* **Traceable changes** – every playbook has a hash, metadata, and storage lineage so you can audit who learned what and when.
* **Safe by default** – review gating, drift detection, and update toggles let you pause or roll back learning if behavior regresses.

### Feedback Loop at a Glance

1. **Session executes** – the dual-agent runtime completes a task and logs telemetry into `sessions`, `trajectory_events`, and reward tables.
2. **Reward judges score** – the RIM ensemble produces reward, uncertainty, and escalation data.
3. **Learning synthesizer runs** – after reward evaluation, Atlas calls the `LearningSynthesizer` to summarize salient guidance for student and teacher personas.
4. **Registry update** – `Database.upsert_learning_state` writes the new playbook to the `learning_registry` table (keyed by `learning_key`).
5. **Playbook cached** – on the next run for the same key, `resolve_playbook` retrieves and hashes the playbook, then injects it into persona prompts.
6. **Evaluation harness audits** – engineering teams query `atlas.training_data` (see the snippet below) to track reward deltas, mode shifts, and review status across learning keys.

## Pipeline Components

### Learning Synthesizer (`atlas/learning/synthesizer.py`)

The synthesizer is distinct from reward judges. It:

* runs **after** reward scoring so updates only occur on high-signal sessions,
* consumes trajectory summaries, reward stats, and recent history (bounded by `history_limit`),
* emits structured “student” and “teacher” guidance plus optional session notes,
* uses a dedicated LLM (configurable via `learning.llm`) to transform raw notes into concise playbooks.

When `learning.update_enabled` is `false`, the synthesizer skips persistence but can still write per-session learning
notes for auditing. This is useful for A/B testing new prompts before rolling them out.

### Playbook Resolver (`atlas/learning/playbook.py`)

`resolve_playbook` is invoked during persona construction. It handles:

* fetching the latest registry entry for the `learning_key`,
* trimming long sections to meet token budgets,
* caching the playbook on disk (per role) and computing a SHA256 hash,
* returning both content and metadata so prompts, validation payloads, and cache keys include the hash.

Set `learning.apply_to_prompts=false` to keep generating playbooks without injecting them into runtime prompts—a common
pattern for staging and smoke tests. The hash still flows through telemetry so you can verify the correct playbook would
have been used.

#### Playbook Injection Modes

Atlas supports two playbook injection strategies optimized for different scenarios:

**Prefix Mode (Default)**

```yaml theme={null}
learning:
  injection_mode: prefix  # Default
```

Playbooks are injected at the beginning of the system prompt. Best for:

* General guidance and behavioral patterns
* Task-agnostic learning that should influence all reasoning
* Compatibility with providers that don't support advanced caching

**Suffix Mode (KV Cache Optimization)**

```yaml theme={null}
learning:
  injection_mode: suffix
```

Playbooks are appended after the base system prompt. Optimized for:

* **Provider KV cache efficiency**: Anthropic and other providers cache the static system prompt prefix, avoiding recomputation when only the playbook changes
* **Reduced latency**: Cache hits eliminate prompt reprocessing overhead
* **Cost savings**: Cached tokens aren't rebilled on subsequent requests

The runtime automatically computes cache breakpoints when `injection_mode: suffix` is used with compatible providers (Anthropic Claude, providers supporting prompt caching). The base system prompt becomes the cached prefix, and the playbook suffix can be updated without invalidating the cache.

<Tip>
  **When to use suffix mode**: Enable `injection_mode: suffix` for production workloads with frequent playbook updates and providers supporting KV cache (e.g., Anthropic Claude). This can reduce prompt processing time by 80%+ when the base prompt remains stable.
</Tip>

### Learning Registry (`atlas/runtime/storage/schema.sql`)

`learning_registry` keeps the current playbooks for each `learning_key`. The table stores a single row per key with:

* `learning_key` – primary identifier (task or project scope).
* `student_learning` / `teacher_learning` – latest playbook bodies (text).
* `metadata` – optional JSON payload (e.g., synthesizer audit info, hashes you compute upstream).
* `updated_at` – timestamp of the most recent update.

Because both roles live in the same row, updates atomically replace the student and teacher playbooks together. Historical
snapshots remain accessible via `sessions.student_learning` and `sessions.teacher_learning`, giving you a time-series view
even as the registry is overwritten with fresher guidance.

### Discovery Telemetry

Autodiscovery (`atlas env init`) persists complementary context in `discovery_runs`. Each record stores module hashes,
autogenerated factory metadata, and preflight results. Learning summaries link back to matching discovery runs so you
can reproduce the environment that produced a given playbook.

## Persistence Topology

```
discovery_runs ─┐
                │
                ▼
          learning_registry ◄── learning synthesizer (updates)
                │
                ▼
            sessions ───► trajectory_events
                │             ▲
                └─ reward_stats│
```

* **`discovery_runs`** captures onboarding metadata and is referenced in learning reports.
* **`sessions`** records adaptive summaries, reward stats, review status, and session-specific learning notes.
* **`trajectory_events`** stores per-step evidence (`event_type`, actor, payload digests).
* **`learning_registry`** contains the most recent playbook per key/role.

Use `learning_key` to join these tables. The learning evaluation harness (below) already performs that join.

## Configuration Reference

Add a `learning` block to your runtime config to control behavior:

| Parameter                 | Default  | Purpose                                                                                 |
| ------------------------- | -------- | --------------------------------------------------------------------------------------- |
| `enabled`                 | `true`   | Master switch. Set `false` to run without playbooks or registry updates.                |
| `update_enabled`          | `true`   | Allow the synthesizer to write updated playbooks. Disable for read-only playbook usage. |
| `llm`                     | `null`   | Override synthesizer model; defaults to the runtime's standard LLM if omitted.          |
| `prompts`                 | `null`   | Custom prompt templates for the synthesizer LLM.                                        |
| `history_limit`           | `10`     | Max historical sessions considered when generating an update.                           |
| `session_note_enabled`    | `true`   | Emit per-session learning notes alongside registry updates.                             |
| `apply_to_prompts`        | `true`   | Inject playbooks into persona prompts and validation payloads.                          |
| `playbook_injection_mode` | `prefix` | Playbook injection strategy: `prefix` (default) or `suffix` (KV cache optimization).    |

Example configuration:

```yaml theme={null}
learning:
  enabled: true
  update_enabled: true
  history_limit: 25
  session_note_enabled: false
  apply_to_prompts: true
  llm:
    provider: openai
    model: gpt-5-mini
    api_key_env: OPENAI_API_KEY
```

### Related Controls

* `orchestration.forced_mode` (see `atlas/config/models.py`) locks the runtime into a specific lane—helpful when you want deterministic evaluation while toggling learning features.
* `runtime_safety.review.require_approval` should remain `true` in production so only reviewed sessions feed the synthesizer. Override via `ATLAS_REVIEW_REQUIRE_APPROVAL=0` for local experiments.
* `runtime_safety.drift` guardrails help spot reward regressions after a playbook change.

## Operating the Learning System

1. **Seed the registry** – run a curated set of tasks with `learning.update_enabled=true` to capture baseline playbooks. Export these as JSON for code review if needed.
2. **Stage changes** – toggle `apply_to_prompts=false` to generate candidate playbooks without impacting prompts.
3. **Promote** – flip `apply_to_prompts` back to `true` once the evaluation harness shows improved reward/uncertainty metrics.
4. **Monitor** – use the CLI and dashboards to track `playbook_hash` changes, reward drift, and review approvals per learning key.
5. **Rollback** – set `update_enabled=false` (to freeze) or `enabled=false` (to bypass entirely) if drift guardrails or reviewers flag an issue.

Tip: store exported playbooks (from the evaluation harness) alongside release notes so you can diff guidance between deployments.

## Verification & Tooling

Run the official learning-report harness from the SDK when you need the full telemetry diff:

```bash theme={null}
cd ../atlas-sdk
python scripts/report_learning.py \
  --database-url postgresql://atlas:atlas@localhost:5433/atlas \
  --recent-window 10 \
  --baseline-window 50 \
  --limit 5 \
  --output-dir results/learning
```

For quick inline spot checks, you can also query `atlas.training_data` directly:

```bash theme={null}
python - <<'PY'
from statistics import mean
from atlas.training_data import get_training_sessions

sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    learning_key="mcp-tool-learning",
    status_filters=["succeeded"],
    limit=200,
)

recent_scores = [s.session_reward["score"] for s in sessions[:25] if s.session_reward]
baseline_scores = [s.session_reward["score"] for s in sessions[25:] if s.session_reward]

def pct_diff(a, b): return (a - b) / b * 100 if b else 0
print(f"Recent avg reward: {mean(recent_scores):.3f}")
print(f"Baseline avg reward: {mean(baseline_scores):.3f}")
print(f"Delta: {pct_diff(mean(recent_scores), mean(baseline_scores)):.2f}%")
PY
```

Persist the JSON payloads under `results/learning/` (for example `results/learning/mcp-tool-learning.json`) so you can diff windows in CI. Many teams wrap the snippet above in a small script that also counts review statuses and execution modes before pushing the summary to dashboards.

## Troubleshooting

| Symptom                                             | Likely Cause                                                                 | Resolution                                                                                                      |
| --------------------------------------------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| Playbook hash does not change after successful runs | `learning.update_enabled` disabled or drift guardrails prevented persistence | Re-enable updates and confirm review approvals; check logs for guardrail warnings.                              |
| Prompts missing playbook content                    | `learning.apply_to_prompts=false` or cache invalidation failed               | Flip the flag to `true`, clear `.atlas/cache/learning/*`, rerun `atlas run`.                                    |
| Evaluation harness shows empty history              | `learning_key` absent in sessions                                            | Ensure `atlas env init` / runtime config tags sessions with consistent metadata.                                |
| Synthesizer timeouts                                | LLM provider throttling                                                      | Configure `learning.llm` with a provider-specific timeout or reduce `history_limit`.                            |
| Learning updates lost on restart                    | Registry not persisting (fixed in atlas-sdk v0.2.5+)                         | Upgrade to latest SDK version. Earlier versions had a persistence bug where updates were only cached in memory. |
| Playbook reverts to older version                   | Cache staleness or registry race condition                                   | Clear `.atlas/cache/learning/*` and verify `learning_registry` table shows latest `updated_at` timestamp.       |

<Note>
  **Learning Persistence Fix (v0.2.5+)**: Earlier versions of atlas-sdk had a bug where learning updates were cached in memory but not reliably persisted to the database. This caused playbooks to revert on restart. If you're experiencing this issue, upgrade to atlas-sdk v0.2.5 or later:

  ```bash theme={null}
  pip install --upgrade arc-atlas
  ```

  After upgrading, verify persistence is working:

  ```bash theme={null}
  # Run a session that should trigger learning
  atlas run --config your_config.yaml --task "test task"

  # Restart the runtime (new Python process)
  # Verify playbook loads from database
  atlas run --config your_config.yaml --task "another task"

  # Check that playbook_hash in telemetry matches registry
  psql $DATABASE_URL -c "SELECT learning_key, updated_at FROM learning_registry WHERE learning_key='your-key';"
  ```
</Note>

## Related Guides

* [`SDK Configuration`](/sdk/configuration) – full YAML reference, including the learning block.
* [`Atlas CLI Reference`](/sdk/cli-reference) – commands for discovery, execution, and review gating.
* [`Export Runtime Traces`](/sdk/export-traces) – move approved sessions into training datasets.
* [`Hybrid Learning Concept`](/concepts/hybrid-learning) – theoretical background for offline + runtime learning.


# How Orchestration Works
Source: https://docs.arc.computer/sdk/orchestration

Follow the Student, Teacher, and Reward System as they coordinate an Atlas SDK run from plan to final answer.

The Atlas SDK orchestrator starts every run with triage, gathers adaptive signals, and then chooses how tightly to supervise the Student. Understanding that flow makes it easier to tune configs and interpret telemetry.

## Adaptive Flow Overview

```mermaid theme={null}
flowchart TD
    Triage[Triage adapter<br/>metadata → dossier] --> Probe[Capability probe<br/>confidence + evidence]
    Probe --> Decision{Adaptive mode}
    Decision -->|auto| Auto[Single shot<br/>no validation]
    Decision -->|paired| Paired[Single shot<br/>teacher validates final answer]
    Decision -->|coach| Coach[Stepwise<br/>compact guidance]
    Coach --> Reward[Session reward & learning]
    Auto --> Reward
    Paired --> Reward
    Reward --> Memory[Persona memory update]
    Reward --> Telemetry[Adaptive summary<br/>+ JSONL export]
```

1. **Triage dossier** – Every run invokes a triage adapter (default: `atlas.utils.triage.default_build_dossier`) that normalises session metadata into risks, signals, and persona hints.
2. **Capability probe** – The probe LLM inspects the dossier plus recent history, returning `{mode, confidence, evidence}`. If `certify_first_run` is enabled and the fingerprint is unseen, the runtime forces a one-time certification (`paired`) before probing.
3. **Adaptive mode** – The orchestrator records the decision, stores probe evidence, and chooses between three lanes:
   * `auto` – single-shot execution without validation to keep latency low.
   * `paired` – single-shot execution with a single validation pass (ideal for certifications).
   * `coach` – converts the reviewed plan into a single step but always validates and allows a retry.
4. **Execution loop** – Depending on the lane, the Student either executes a single combined step or walks through the reviewed plan. Teacher interventions (validation, guidance, retries) are lane-aware.
5. **Reward & learning** – The Reward System aggregates judges, emits `session_reward`, and captures learning notes. Certification verdicts are reused as the reward signal when possible.
6. **Memory & telemetry** – Persona memories are refreshed, adaptive summaries are stored, and exporters/streamers consume the structured metadata.

## Lane Cheatsheet

| Lane     | When it triggers                                            | Supervision profile                                            | Persistence highlights                                     |
| -------- | ----------------------------------------------------------- | -------------------------------------------------------------- | ---------------------------------------------------------- |
| `auto`   | High confidence history                                     | Student executes once, no validation                           | Telemetry records lane + confidence, reward may be skipped |
| `paired` | Certification required or medium confidence                 | Student executes once, Teacher validates final answer          | Certification flag stored, reward reused from validation   |
| `coach`  | Low confidence or probe confidence below `paired` threshold | Plan collapses to single step with validation + optional retry | Guidance compact, adaptive summary logs probe evidence     |

Tune thresholds and fallback behaviour under `adaptive_teaching.probe` in your config (see [`SDK Configuration`](/sdk/configuration#adaptive_teaching)).

## Retries, Guidance, and Certification

* Step retries are only attempted in the `coach` lane and are capped by `orchestration.max_retries`.
* The Teacher’s guidance is appended to the execution context and streamed to the console so you can see why a retry happened.
* Certification runs (`paired` on a new fingerprint) mark the session as `certification_run` inside `adaptive_summary` and store the verdict for future routing.

## Event Stream & Telemetry

Every significant action is published to the `ExecutionContext` event stream. Subscribe to it to power CLI streams, dashboards, or custom logging:

```python theme={null}
from atlas.runtime.orchestration.execution_context import ExecutionContext

context = ExecutionContext.get()
subscription = context.event_stream.subscribe(print)
```

Key metadata to expect:

* `adaptive_summary` – active mode, probe payload, confidence, and recent history (surfaces in the console streamer and JSONL exports).
* `steps` – per-step attempts, timings, retry status, and guidance messages.
* `session_reward` / `reward_summary` – aggregated reward score plus individual judge breakdowns.
* `triage_dossier`, `personas_used`, `persona_updates` – the context and outcomes that feed persona learning.

Attach a custom `TelemetryPublisher` if you need to forward events elsewhere; otherwise the default console streamer handles everything automatically.

## Anatomy of `atlas.core.run`

At a high level the public API (`atlas.core.run` / `atlas.core.arun`) performs the following:

1. Load and validate your config (`atlas/config/loader.py`) and reset the `ExecutionContext`.
2. Build the agent adapter (`create_from_atlas_config`) plus Student and Teacher prompts (`atlas.prompts.build_student_prompts` / `build_teacher_prompts`).
3. Instantiate Student, Teacher, and the session-level Reward `Evaluator`.
4. Load the triage adapter and capability probe client defined in `adaptive_teaching`, collecting fingerprint hints for persona memory.
5. Connect to optional storage (Postgres) and set up telemetry publishers or console streaming.
6. Run the `Orchestrator`, which performs triage, probes for a lane, executes the plan (single-shot or stepwise), and records results.
7. Persist session metadata—including `adaptive_summary`, reward payloads, and persona updates—before returning an `atlas.types.Result`.

## When to Customize

| Goal                     | Consider tweaking                                                                  |
| ------------------------ | ---------------------------------------------------------------------------------- |
| Force a specific lane    | `adaptive_teaching.mode_override`                                                  |
| Bias routing thresholds  | `adaptive_teaching.probe.thresholds` and `fallback_mode`                           |
| Tighten or relax retries | `orchestration.max_retries` and Teacher guidance prompts                           |
| Adjust reward escalation | `rim.variance_threshold`, `rim.uncertainty_threshold`, or custom reward objectives |
| Stream custom telemetry  | Attach a `TelemetryPublisher` or subscribe directly to the event stream            |

## Next Steps

* Configure each YAML block in detail with the [`SDK Configuration Reference`](/sdk/configuration).
* Bring your own agent with the [`Bring Your Own Agent`](/sdk/adapters) guide.
* Understand the dual-agent reasoning concept in [`Adaptive Dual-Agent Reasoning`](/concepts/adaptive-dual-agent-reasoning).


# SDK Quickstart: Run Your First Task
Source: https://docs.arc.computer/sdk/quickstart

Launch the Atlas SDK runtime, run your first task, and understand how the dual-agent loop (your student agent + verifying teacher) orchestrates work.

<div>
  <video alt="Complete SDK setup walkthrough showing installation, configuration, and real performance gains" />

  <p><em>Watch: Complete SDK setup walkthrough—install, configure, and see real performance gains in 2 minutes.</em></p>
</div>

This guide provides the fastest path to running the Atlas SDK. Install the packaged runtime, point it at your agent, and execute your first task in a few commands—all while the adaptive runtime decides how much supervision each request needs.

<Note>
  **Beta notice:** The Atlas SDK runtime is in beta. APIs and configuration keys may evolve—check release notes before upgrading.
</Note>

## Prerequisites

```bash theme={null}
python -m pip install --upgrade arc-atlas
```

Set your API keys (see [Installation](/installation) for details):

```bash theme={null}
export ANTHROPIC_API_KEY="sk-ant-your-key"
export GEMINI_API_KEY="your-gemini-key"  # Optional for rewards
```

<Note>
  Store credentials in `.env` to avoid shell history exposure. Atlas defaults to Anthropic (Claude Haiku 4.5 for student, Claude Sonnet 4.5 for teacher) with Gemini for rewards. See [Configuration](/sdk/configuration) for alternatives.
</Note>

## Step 1 – Run the Quickstart Task

<Note>
  **Working directory:** The SDK installs globally via `pip install arc-atlas`. You can run `atlas` commands from any directory. Config files can live in your project root (for CLI autodiscovery) or in the [ATLAS Core repository](https://github.com/Arc-Computer/ATLAS) (for example configs).
</Note>

Atlas now ships with an autodiscovery CLI so you can validate your environment before touching Python.

<div>
  <img alt="Atlas SDK adaptive runtime flow diagram showing triage, probe, and lane routing" />

  <p><em>The adaptive runtime probes capability and routes every task into the right lane before the dual-agent loop (student + verifying teacher) executes.</em></p>
</div>

### Option A – CLI Autodiscovery (recommended for new stacks)

```bash theme={null}
pip install arc-atlas
atlas env init --task "Summarize the latest AI news"
atlas run --config .atlas/generated_config.yaml --task "Summarize the latest AI news"
```

* `atlas env init` scans for `@atlas.environment` / `@atlas.agent` decorators or factory functions, loads `.env`, writes `.atlas/discover.json`, `.atlas/generated_factories.py`, and `.atlas/generated_config.yaml`, and automatically sets up storage (integrating `atlas init` functionality).
* **Agent Selection**: `atlas env init` uses Claude Haiku 4.5 (`claude-haiku-4-5-20251001`) as an LLM-powered agent selector to analyze your codebase and automatically detect the best agent integration points. This intelligent discovery helps bootstrap configuration for existing codebases.
* **Learning Features**: Few-shot prompting and playbook injection are enabled by default, allowing the system to learn from past interactions immediately.
* `atlas run --config` loads the generated config, verifies module hashes, streams telemetry into `.atlas/runs/`, and injects learning playbooks when available.
* Need to exercise the full orchestrator? Point `atlas run --config src/atlas_core/configs/recipe/sdk_quickstart.yaml --task "..."` at a config file to bypass discovery entirely.

<Tip>
  **Customizing agent discovery**: Set `ATLAS_DISCOVERY_MODEL` to override the default Claude Haiku 4.5 model used for agent selection. Any Anthropic model is supported via the `ANTHROPIC_API_KEY` environment variable. Storage setup is now automatic—no need to run `atlas init` separately.
</Tip>

### Option B – Python API (direct invocation)

<Warning>
  This option uses example configs from the Atlas Core repository. If you only installed the SDK (`pip install arc-atlas`), use Option A or create your own config file.
</Warning>

If you want to use pre-built example configs:

```bash theme={null}
# Clone Atlas Core for example configs
git clone https://github.com/Arc-Computer/ATLAS.git
cd ATLAS
```

Then use the Python API with the example config:

```python theme={null}
from atlas.core import run

result = run(
    task="Summarize the latest AI news",
    config_path="src/atlas_core/configs/recipe/sdk_quickstart.yaml",  # Path relative to ATLAS repo root
    stream_progress=True,
)
print(result.final_answer)
```

<Info>
  Run it inline if you prefer to avoid creating a file:

  ```bash theme={null}
  python -c "from atlas.core import run; result = run(task='Summarize the latest AI news', config_path='src/atlas_core/configs/recipe/sdk_quickstart.yaml', stream_progress=True); print(result.final_answer)"
  ```
</Info>

**Expected output:**

```
=== Atlas task started: Summarize the latest AI news (2025-01-11 10:30:45) ===
Plan ready (3 steps):
  1. Search for recent AI news articles
  2. Extract key points from top articles
  3. Synthesize findings into concise summary
Adaptive: mode=coach confidence=0.58
STEP 1: Search for recent AI news articles | actor=student | attempt=1 | validation=PASS (found relevant sources) | duration=1200.5ms
STEP 1: Search for recent AI news articles | actor=teacher | attempt=1 | guidance=Focus on authoritative sources
STEP 1: retry 1 | Reward score=0.82 | Judge scores: helpfulness:0.85, accuracy:0.80
STEP 2: Extract key points from top articles | actor=student | attempt=1 | validation=PASS (extracted main themes) | duration=850.3ms
STEP 2: retry 1 | Reward evaluation deferred to session-level judge
STEP 3: Synthesize findings into concise summary | actor=student | attempt=1 | validation=PASS (summary complete) | duration=950.7ms
STEP 3: retry 1 | Reward evaluation deferred to session-level judge
Final Answer:
  Recent AI developments include...
Summary | execution_mode=stepwise | total_runtime=15.2s | judge_calls=1 | adaptive_mode=coach | adaptive_confidence=0.58
  attempts: 1=1, 2=1, 3=1
  Reward score=0.85 (All steps completed successfully)
=== Atlas task completed in 15.2s ===
```

The console streamer shows the plan, adaptive lane selection, step-by-step execution with validation status, teacher guidance when provided, and reward scores. `atlas.runtime.telemetry.ConsoleTelemetryStreamer` auto-enables when stdout is a TTY; override with `stream_progress=True/False`.

<Info>
  **Want to see adaptive learning in action?** Check out the [Adaptive Tool Use example](/examples/adaptive-tool-use) showing a LangGraph agent learning efficient MCP tool usage across 25 tasks, demonstrating 30-40% reduction in tool calls.
</Info>

### Bring Your Own Agent

Atlas wraps any agent that exposes an OpenAI-compatible API, HTTP endpoint, or Python callable. Three adapter types are available:

* **OpenAI adapter** - For GPT, Claude via OpenAI-compatible APIs
* **HTTP adapter** - For microservices, serverless functions
* **Python adapter** - For LangGraph, local callables, custom agents

See the [Agent Adapters guide](/sdk/adapters) for complete configuration options and examples.

## What Just Happened?

Think of `atlas.core.run` as a project manager who never gets tired—now fronted by an adaptive controller:

* **Triage & probe** – a triage adapter builds context, the capability probe scores confidence, and the runtime picks a lane.
* **Configure** – the YAML tells the orchestrator which agent to call and how the dual-agent reasoning loop (student + verifying teacher) should behave.
* **Plan** – the Student drafts a step-by-step approach when a stepwise lane is chosen; in single-shot lanes the plan collapses to one step.
* **Review** – the Teacher approves or tweaks the plan (or just inspects the final answer in `paired` mode).
* **Execute** – each step runs with lane-specific guidance, validation, and retries.
* **Evaluate** – the Reward System scores the work, deciding whether to reuse guidance and how to update persona memories.

## Configuration Breakdown

Key sections in `sdk_quickstart.yaml`:

* **`agent`**: Adapter settings (litellm/http/python) and model choice
* **`teacher`**: Verification model, typically stronger than student
* **`rim`**: Reward system judges (Gemini 2.5 Flash/Pro by default)
* **`adaptive_teaching.probe`**: Capability assessment (xAI Grok-2-mini)
* **`storage`**: Optional Postgres persistence

See [Configuration Reference](/sdk/configuration) for complete details and preset templates.

## Troubleshooting Checklist

* **Missing API key** – ensure `OPENAI_API_KEY` (or Azure equivalents) are exported in the same shell.
* **Time spent downloading dependencies** – editable installs pull in `litellm`, `httpx`, and friends on the first run; subsequent runs are instant.
* **Model limits** – bump `max_output_tokens` in the config if your summaries get truncated.

## Next Steps

<CardGroup>
  <Card title="Adaptive Tool Use" icon="wrench" href="/examples/adaptive-tool-use">
    See measurable learning with MCP tool integration
  </Card>

  <Card title="Agent Adapters" icon="plug" href="/sdk/adapters">
    Connect your own agent framework
  </Card>

  <Card title="Configuration Guide" icon="sliders" href="/sdk/configuration">
    Fine-tune orchestration and learning parameters
  </Card>

  <Card title="Export Training Data" icon="database" href="/sdk/export-traces">
    Persist sessions and build datasets for offline training
  </Card>
</CardGroup>


# Runtime Safety & Review
Source: https://docs.arc.computer/sdk/runtime-safety

Configure Atlas drift guardrails and session review gating to protect production learning.

Runtime learning is only valuable when it is trustworthy. Atlas combines automated drift detection with a manual review
queue so you can halt regressions before they reach training pipelines. This page explains how to configure those
guardrails and operate the review workflow.

## Safety Controls in the Runtime Config

Add or refine the `runtime_safety` block in your SDK YAML:

```yaml theme={null}
runtime_safety:
  drift:
    enabled: true
    window: 50
    z_threshold: 3.0
    min_baseline: 5
  review:
    require_approval: true
    default_export_statuses:
      - approved
```

| Parameter                        | Default        | Effect                                                             |
| -------------------------------- | -------------- | ------------------------------------------------------------------ |
| `drift.enabled`                  | `true`         | Toggle statistical drift detection based on reward deltas.         |
| `drift.window`                   | `50`           | Samples used for baseline statistics (increase for noisy domains). |
| `drift.z_threshold`              | `3.0`          | Standard deviations required to raise a drift alert.               |
| `drift.min_baseline`             | `5`            | Minimum samples before alerts trigger.                             |
| `review.require_approval`        | `true`         | Gate exports and learning updates on reviewer approval.            |
| `review.default_export_statuses` | `["approved"]` | Review states included when tooling omits explicit filters.        |

Drift alerts surface in `sessions.metadata["drift"]` and trigger flags in the review CLI. Review settings feed directly
into `arc-atlas export` / `atlas train`, so production pipelines default to approved sessions only.

## Review Sessions

1. **Approve or quarantine sessions**

   ```bash theme={null}
   arc-atlas review sessions --database-url postgresql://atlas:atlas@localhost:5433/atlas --status pending
   arc-atlas review approve 123 --database-url postgresql://atlas:atlas@localhost:5433/atlas --note "Clean reward delta"
   arc-atlas review quarantine 456 --database-url postgresql://atlas:atlas@localhost:5433/atlas --note "Investigate drift"
   ```

   **Expected output:**

   ```
   Status: pending (2 sessions)
       123 | ok    | scoreΔ=+0.12 | uncΔ=+0.05 | reward=0.85±0.03 (n=3) | created=2025-01-11T10:30:45 | reason=-
            Task: Debug authentication flow
            Notes: -
            Reward audit entries: 3
       456 | ALERT | scoreΔ=-0.23 | uncΔ=+0.17 | reward=0.42±0.08 (n=2) | created=2025-01-11T11:15:22 | reason=Significant reward drop compared to recent window
            Task: Debug authentication flow
            Notes: -
            Reward audit entries: 2
   ```

   The listing groups sessions by review status and highlights drift alerts, reward deltas, and uncertainty changes so
   reviewers can triage quickly.

2. **Export only the data you trust**
   ```bash theme={null}
   arc-atlas export \
     --database-url postgresql://atlas:atlas@localhost:5433/atlas \
     --output traces/approved.jsonl \
     --include-status approved
   ```
   Omit `--include-status` to inherit `runtime_safety.review.default_export_statuses`. For local testing, set
   `ATLAS_REVIEW_REQUIRE_APPROVAL=0` to bypass the gate—never disable it in production.

3. **Feed the evaluation harnesses**
   The learning evaluation harness counts review statuses in its summaries. Pending sessions are a signal that human
   review is still in progress; include or exclude them deliberately when comparing runs.

## Responding to Drift

* **Alert inspection** – Review the `drift` object in `arc-atlas review sessions` output. It contains z-scores, deltas,
  and reason strings pointing at the underlying metric.
* **Pause updates** – Temporarily disable playbook persistence by setting `learning.update_enabled=false`; this keeps
  existing guidance in place while you investigate.
* **Re-run evaluation** – From `atlas-sdk/`, run `python scripts/report_learning.py --database-url … --learning-key <key>`
  (or use the lightweight snippet in the [Learning System guide](/sdk/learning-system)) to recompute recent vs baseline
  reward windows for the impacted `learning_key` and gather context for root-cause analysis.
* **Rollback** – If a playbook caused the regression, reset it by deleting the entry from `learning_registry` or
  restoring a previously exported playbook, then re-enable updates.

## Database Signals to Monitor

* `sessions.review_status` & `sessions.review_notes` – authoritative state for approval.
* `sessions.metadata.drift` – contains drift z-scores and explanations.
* `learning_registry.updated_at` – spot stale playbooks that may indicate paused updates.
* `trajectory_events.event.event_type` – inspect underlying telemetry (e.g., `reward`, `guidance`, `validation`) when
  diagnosing regressions.

See the [`Database Schema`](/reference/database-schema) reference for column details and index coverage.

## Best Practices

* **Automate reviews** – Alert on pending sessions that exceed a time threshold or have drift alerts; build lightweight
  dashboards from the `sessions` table.
* **Document decisions** – Use `--note` when approving/quarantining so investigators have context later.
* **Audited exports** – Store export manifests alongside training jobs (timestamp, review statuses included, CLI flags).
* **CI safeties** – Keep `review.require_approval=true` in checked-in configs. Only override via env vars inside isolated
  dev environments.

Safeguards are only effective when enforced consistently. Use the runtime safety hooks together—drift detection signals
the problem, review gating ensures only vetted data leaves the system, and evaluation harnesses quantify recovery.


# Training Configuration
Source: https://docs.arc.computer/training/configuration

Hydra parameter reference for Atlas Core training

Atlas Core uses Hydra to compose model, dataset, trainer, and reward presets. This page is a complete parameter reference for launching GRPO, GKD, or SFT jobs.

<Note>
  For workflow guides, see [GRPO Training](/training/offline/grpo-training) or [GKD Training](/training/offline/gkd-training). This page focuses on exhaustive parameter lookup.
</Note>

## Training Method Comparison

| Method   | When to Use                        | Speed                         | Data Requirement                   | Output               |
| -------- | ---------------------------------- | ----------------------------- | ---------------------------------- | -------------------- |
| **GRPO** | Train from rewards (RL)            | 24-48h                        | Runtime traces with reward signals | RL-optimized teacher |
| **GKD**  | Distill large \u2192 small teacher | 4-8h (9-30× faster than GRPO) | Strong teacher exists              | Compressed teacher   |
| **SFT**  | Supervised warmup                  | 2-4h                          | Approved conversational traces     | Baseline teacher     |

→ Full comparison in [Offline Training Guide](/training/offline/grpo-training)

## Hydra Composition Map

Hydra builds a training run by merging defaults from each config group:

```mermaid theme={null}
graph TD
    A[src/atlas_core/configs/train.yaml<br/>Global defaults] --> B[src/atlas_core/configs/recipe/*.yaml<br/>Experiment recipes]
    C[src/atlas_core/configs/trainer/*.yaml<br/>Algorithms] --> B
    D[src/atlas_core/configs/model/*.yaml<br/>Models] --> B
    E[src/atlas_core/configs/data/*.yaml<br/>Datasets] --> B
    F[src/atlas_core/configs/reward/*.yaml<br/>Reward presets] --> B
```

| Layer              | Example Files                                     | When to Modify                   |
| ------------------ | ------------------------------------------------- | -------------------------------- |
| `train.yaml`       | `src/atlas_core/configs/train.yaml`               | Global logging/output rules      |
| `model@_global_`   | `qwen3_8b.yaml`, `base.yaml`                      | Swap checkpoints, quantization   |
| `data@_global_`    | `runtime_traces.yaml`, `arc_atlas_rl.yaml`        | Choose datasets, sampling        |
| `trainer@_global_` | `grpo.yaml`, `base_sft.yaml`, `teacher_grpo.yaml` | Algorithm, optimizer, batching   |
| `recipe@_global_`  | `teacher_rcl.yaml`, `teacher_sft.yaml`            | Pre-built experiment bundles     |
| `reward@_global_`  | `interpretation_teaching.yaml`                    | Reward adapter, prompt templates |

***

## Model Presets (`src/atlas_core/configs/model/`)

| Parameter                  | Default                          | When to Change                     |
| -------------------------- | -------------------------------- | ---------------------------------- |
| `model_name_or_path`       | Required (e.g., `Qwen/Qwen3-8B`) | Every run - specify checkpoint     |
| `tokenizer_name_or_path`   | `${model_name_or_path}`          | Distinct tokenizer needed          |
| `trust_remote_code`        | `true`                           | Vendor-specific architectures      |
| `use_peft`                 | `false`                          | Enable LoRA/PEFT adapters          |
| `load_in_4bit`             | `false`                          | GPU memory constrained             |
| `tokenizer.padding_side`   | `left`                           | Keeps RL rollouts aligned          |
| `unsafe_tokenizer_loading` | `false`                          | Untrusted tokenizer code           |
| `torch_dtype`              | `bfloat16` (qwen3\_8b)           | Hardware-specific precision        |
| `attn_implementation`      | `flash_attention_2`              | Faster attention on supported GPUs |

***

## Dataset Presets (`src/atlas_core/configs/data/`)

| Parameter                  | Default                                                    | When to Change                   |
| -------------------------- | ---------------------------------------------------------- | -------------------------------- |
| `dataset_id_or_path`       | `Arc-Intelligence/Arc-ATLAS-Teach-v0`                      | HuggingFace hub ID or local path |
| `dataset_split`            | `rl`, `train`                                              | Multi-split datasets             |
| `dataset_level_filter`     | `null` (e.g., `level_4_5` for BigMath)                     | Curriculum control               |
| `dataset_max_samples`      | `null`                                                     | Subsample for quick experiments  |
| `eval_split_ratio`         | `0.1`                                                      | Define held-out eval share       |
| `shuffle`                  | `true` (`runtime_traces`)                                  | Randomize JSONL before split     |
| `completion_only_training` | `True` (`arc_atlas_sft`)                                   | Trim prompts in SFT              |
| `dataset_path`             | `traces/export.jsonl` (`runtime_traces`)                   | Point at exported JSONL          |
| `make_dataset_fn._target_` | `atlas_core.data.runtime_traces.get_runtime_trace_dataset` | Loader entrypoint                |

**Common Dataset Configs:**

* `runtime_traces.yaml` - Exported JSONL from Atlas SDK
* `arc_atlas_rl.yaml` - Pre-collected RL dataset
* `arc_atlas_sft.yaml` - Supervised fine-tuning dataset

***

## Trainer Base Defaults (`src/atlas_core/configs/trainer/base.yaml`)

| Parameter                     | Default         | When to Change                     |
| ----------------------------- | --------------- | ---------------------------------- |
| `max_steps`                   | `450`           | Override in run recipe             |
| `num_train_epochs`            | `1`             | Mutually exclusive with max\_steps |
| `train_batch_size`            | `64`            | Effective batch across devices     |
| `per_device_train_batch_size` | `2`             | Per-rank micro batch               |
| `gradient_accumulation_steps` | Inferred        | Auto-computed if omitted           |
| `gradient_checkpointing`      | `true`          | Memory savings for long contexts   |
| `learning_rate`               | `5e-7`          | Baseline LR for RL                 |
| `weight_decay`                | `0`             | Regularization needed              |
| `max_grad_norm`               | `1.0`           | Gradient clipping value            |
| `lr_scheduler_type`           | `"cosine"`      | Constant/linear schedules          |
| `warmup_ratio`                | `0.03`          | Warmup fraction of total steps     |
| `bf16` / `tf32`               | `true` / `true` | Mixed-precision on supported GPUs  |
| `ddp_timeout`                 | `18000` seconds | Distributed training timeout       |

<Info>
  `gradient_accumulation_steps` is auto-computed: `train_batch_size / (per_device_train_batch_size × num_devices)`. Provide any two values; the launcher resolves the third (see `src/atlas_core/cli/train.py`).
</Info>

***

## GRPO Algorithm Controls (`src/atlas_core/configs/trainer/grpo.yaml`)

| Parameter                                     | Default                         | When to Change                            |
| --------------------------------------------- | ------------------------------- | ----------------------------------------- |
| `max_steps`                                   | `200`                           | Override base default                     |
| `train_batch_size`                            | `252`                           | Must divide evenly by devices             |
| `per_device_train_batch_size`                 | `3`                             | Increase for fewer devices                |
| `num_generations`                             | `null`                          | Limit for budget control                  |
| `learning_rate`                               | `1e-6`                          | RL-specific step size                     |
| `beta`                                        | `0.04`                          | KL penalization strength                  |
| `max_prompt_length` / `max_completion_length` | `2048` / `16384`                | Truncate input/output                     |
| `shuffle_generation_inputs`                   | `true`                          | Shuffle prompts before generation         |
| `temperature` / `top_p` / `top_k` / `min_p`   | `1.0` / `1.0` / `null` / `null` | Sampling controls for rollouts            |
| `repetition_penalty`                          | `1.0`                           | Discourage repetition                     |
| `use_vllm`                                    | `true`                          | Fast generation (recommended)             |
| `vllm_device`                                 | `"auto"`                        | Auto-select devices                       |
| `vllm_gpu_memory_utilization`                 | `0.9`                           | Cap GPU memory per worker                 |
| `vllm_dtype`                                  | `"auto"`                        | Hardware-based dtype                      |
| `vllm_max_model_len`                          | `null`                          | Override max context length               |
| `use_ray`                                     | `false`                         | Remote vLLM via Ray                       |
| `ray_tensor_parallelism`                      | `1`                             | Split model across GPUs                   |
| `enable_prefix_caching`                       | `false`                         | Cache prompt prefixes                     |
| `enforce_eager`                               | `true`                          | PyTorch eager execution (safer debugging) |
| `use_vllm_server`                             | `false`                         | External vLLM server                      |
| `vllm_host` / `vllm_port`                     | `null` / `null`                 | Required if use\_vllm\_server=true        |
| `reward_weights`                              | `null`                          | Per-judge scaling factors                 |
| `sync_ref_model`                              | `false`                         | Keep reference model in sync              |
| `ref_model_sync_steps`                        | `64`                            | Sync frequency (steps)                    |
| `unbias_log_probabilities`                    | `true`                          | Correct for temperature scaling           |
| `log_completions`                             | `false`                         | Store sampled completions                 |
| `push_to_hub`                                 | `false`                         | Publish to HuggingFace Hub                |
| `activate_debugging_logs`                     | `false`                         | Extra diagnostics                         |

***

## Teacher GRPO Overlay (`src/atlas_core/configs/trainer/teacher_grpo.yaml`)

Extends base GRPO with diagnostic prompts and teacher-specific controls.

| Parameter                                     | Default                              | When to Change                     |
| --------------------------------------------- | ------------------------------------ | ---------------------------------- |
| `trainer_log_name`                            | `teacher_grpo_rw_${reward_log_name}` | Appends reward preset name         |
| `logging_prob`                                | `0.1`                                | Fraction of episodes logged        |
| `student_model`                               | `null`                               | Co-train student alongside teacher |
| `use_reference_teacher_model`                 | `false`                              | Compare vs static reference        |
| `completion_only_training`                    | `false`                              | Completion-only datasets           |
| `trainer_args.max_probe_tokens`               | `500`                                | Diagnostic prompt budget           |
| `trainer_args.student_diagnostic_template`    | Multiline                            | Reflection prompt (see below)      |
| `trainer_args.teacher_adaptive_template`      | Multiline                            | Guidance prompt (see below)        |
| `trainer_args.student_with_teaching_template` | Multiline                            | Apply feedback prompt (see below)  |

**Default Prompt Templates:**

```yaml theme={null}
student_diagnostic_template: |
  Question: {question}
  Before solving, briefly describe:
  1. What type of problem this is
  2. The key concepts or steps needed
  3. Any potential challenges you see

teacher_adaptive_template: |
  Question: {question}
  Student's approach: {approach}

  <thinking>
  [Analyze student approach]
  </thinking>

  <teaching>
  [Only guidance to student - no answers]
  </teaching>

student_with_teaching_template: |
  Question: {question}
  A teacher has provided: {teaching}

  Now solve step by step.
  <solution></solution>
```

***

## SFT Trainer (`src/atlas_core/configs/trainer/base_sft.yaml`)

| Parameter                     | Default                   | Run Override (`teacher_sft.yaml`) | When to Change                |
| ----------------------------- | ------------------------- | --------------------------------- | ----------------------------- |
| `num_train_epochs`            | `1`                       | `10`                              | Longer supervised training    |
| `max_steps`                   | `-1`                      | `-1`                              | Negative disables step cap    |
| `train_batch_size`            | `64`                      | `16`                              | Effective batch for SFT       |
| `per_device_train_batch_size` | `4`                       | `1`                               | Pair with gradient accum      |
| `learning_rate`               | Base: `5e-7`, SFT: `2e-4` | `2e-4`                            | Higher LR for SFT             |
| `lr_scheduler_type`           | `"cosine"`                | `"constant"`                      | SFT prefers constant          |
| `warmup_ratio`                | `0.03`                    | `0.1`                             | Warm start for supervised     |
| `max_seq_length`              | `4096`                    | `16384`                           | Match runtime telemetry       |
| `packing`                     | `true`                    | `false`                           | Disable for long contexts     |
| `do_eval`                     | `true`                    | `false`                           | Enable with validation splits |
| `ddp_timeout`                 | `18000`                   | `180000000`                       | Long sequences across ranks   |

***

## Run Recipes (`src/atlas_core/configs/recipe/`)

Pre-built experiment bundles that override multiple config groups.

| Recipe             | Key Overrides                                                                                     | Use Case                                        |
| ------------------ | ------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| `default.yaml`     | Empty (inherit globals)                                                                           | CLI override experiments                        |
| `teacher_rcl.yaml` | Model: `qwen3_8b`, Data: `arc_atlas_rl`, Trainer: `teacher_grpo`, Batch: 128, vLLM server enabled | Production GRPO for reward-conditioned learning |
| `teacher_sft.yaml` | Trainer: `base_sft`, Data: `arc_atlas_sft`, Epochs: 10, Max seq: 16384                            | Supervised warmup before GRPO                   |

***

## Reward Preset (`src/atlas_core/configs/reward/interpretation_teaching.yaml`)

| Parameter                                       | Default                                      | When to Change                                   |
| ----------------------------------------------- | -------------------------------------------- | ------------------------------------------------ |
| `reward_log_name`                               | `interpretation_teaching`                    | Propagates to trainer logs                       |
| `max_probe_tokens`                              | `500`                                        | Diagnostic prompt length                         |
| `teacher_reward._target_`                       | `atlas_core.reward.interpretation.RIMReward` | Uses `reward_system/interpretation_offline.yaml` |
| `student_model` / `teacher_model` / `tokenizer` | `null`                                       | Auto-populated from Hydra model                  |

### Reward Configs (Runtime vs Offline)

| Setting                          | Runtime (`reward_system/interpretation.yaml`)         | Offline (`reward_system/interpretation_offline.yaml`) |
| -------------------------------- | ----------------------------------------------------- | ----------------------------------------------------- |
| `temperatures`                   | `[0.2, 0.5, 0.8]`                                     | `[0.2, 0.5, 0.8]`                                     |
| `models.small_model`             | `"gemini/gemini-2.5-flash"`                           | `"gemini/gemini-2.5-flash"`                           |
| `models.large_model`             | `"gemini/gemini-2.5-pro"`                             | `"gemini/gemini-2.5-pro"`                             |
| `active_judges`                  | accuracy, helpfulness, process, diagnostic (all true) | helpfulness, process (accuracy/diagnostic false)      |
| `anti_gaming.cap_score`          | `0.3`                                                 | `0.3`                                                 |
| `parallel_execution.max_workers` | `8`                                                   | `8`                                                   |

***

## Next Steps

<CardGroup>
  <Card title="GRPO Training" icon="dumbbell" href="/training/offline/grpo-training">
    Launch reward-conditioned learning runs
  </Card>

  <Card title="GKD Training" icon="compress" href="/training/offline/gkd-training">
    Distill large teachers into small models
  </Card>

  <Card title="Reward System" icon="trophy" href="/concepts/reward-design">
    Judge design, weights, escalation
  </Card>
</CardGroup>


# Custom Dataset Creation
Source: https://docs.arc.computer/training/custom-datasets

How to create custom training datasets from runtime traces

This guide shows how to prepare custom datasets for ATLAS training. For dataset references and schemas, see [Datasets Reference](/reference/datasets).

## Data Format Requirements

Your dataset should follow this structure:

```python theme={null}
{
    "prompt": str,           # Required: Task description
    "ground_truth": str,     # Required: Correct solution
    "metadata": {            # Optional: Additional context
        "difficulty": str,
        "source": str,
        "tags": List[str]
    }
}
```

## Preprocessing Pipeline (JSONL exports)

Use the runtime helpers that ship in this repository to turn SDK exports into trainer-ready splits:

```python theme={null}
from transformers import AutoTokenizer
from atlas_core.data.runtime_traces import get_runtime_trace_dataset

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
splits = get_runtime_trace_dataset(
    tokenizer=tokenizer,
    export_path="traces/runtime.jsonl",  # Generated via `arc-atlas export …`
    eval_split_ratio=0.1,
    dataset_max_samples=5000,
)

train_ds = splits["train_dataset"]
eval_ds = splits["eval_dataset"]
```

**Expected output:**

```
Loading dataset from traces/runtime.jsonl...
Loaded 5000 examples
Creating train/eval splits (90/10)...
Train dataset: 4500 examples
Eval dataset: 500 examples
```

## Postgres-Backed Workflows

For Postgres-backed workflows, query the SDK database directly and convert records with `atlas_core.data.runtime_traces`:

```python theme={null}
from atlas.training_data import get_training_sessions
from atlas_core.data.runtime_traces import sessions_to_rl_records

sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    learning_key="security-review",
)

records = sessions_to_rl_records(sessions)
```

`records` is a list of dictionaries that any Hugging Face `Dataset` can ingest (the same structure Hydra configs consume via `atlas_core.data.runtime_traces`). See the [Training Data Pipeline](/training/offline/training-data-pipeline) guide for additional filters and batching helpers.

<Note>
  **GKD alignment note:** Every conversation record now carries `prompt_text` (serialized messages excluding the final assistant turn) and `completion_text` (the assistant response the student learns to mimic). These fields let the distillation pipeline re-render prompts with both the student and teacher tokenizers so cross-tokenizer KL is computed in each model's native chat template.
</Note>

## Quality Validation

Inspect coverage with standard Python tooling—you already have `datasets` installed for training:

```python theme={null}
from collections import Counter
from datasets import Dataset

dataset = Dataset.from_list(records)
lengths = [len(example["step_trace"].split()) for example in dataset]
domains = Counter(example["session_metadata"].get("domain", "unknown") for example in dataset)

print(f"Examples: {len(dataset)}")
print(f"Avg step length: {sum(lengths)/len(lengths):.1f} tokens")
print(f"Domains: {domains}")
```

**Expected output:**

```
Examples: 5000
Avg step length: 324.5 tokens
Domains: Counter({'math': 2100, 'code': 1800, 'debug': 700, 'reasoning': 400})
```

Pair these quick checks with any in-house validators your team already maintains. The key is to keep the format (`prompt`, `student_response`, guidance, rewards) identical to what the SDK emits so Atlas Core can reuse the traces without custom glue code.

## Next Steps

<CardGroup>
  <Card title="Datasets Reference" icon="database" href="/reference/datasets">
    Official datasets and schemas
  </Card>

  <Card title="GKD Training" icon="book" href="/training/offline/gkd-training">
    Train with custom datasets
  </Card>

  <Card title="GRPO Training" icon="graduation-cap" href="/training/offline/grpo-training">
    RL training with custom data
  </Card>

  <Card title="Export Traces" icon="download" href="/sdk/export-traces">
    Export runtime traces from the SDK
  </Card>
</CardGroup>


# GKD Training
Source: https://docs.arc.computer/training/offline/gkd-training

On-policy distillation of Atlas runtime traces using Generalized Knowledge Distillation

## Overview

Generalized Knowledge Distillation (GKD) enables on-policy distillation of Atlas runtime traces into smaller, faster student models. Instead of re-running a full reinforcement-learning cycle, use GKD to compress a reliable teacher checkpoint while staying within the reward-weighted data captured in production.

### Atlas Training Stack Fit

**Problem**: Atlas SDK deployments often produce a reliable teacher policy plus verified runtime traces, yet the teacher checkpoint is too large or costly to redeploy, and running the full SFT → GRPO stack again would add days of compute.

**Solution**: Use GKD to transfer the teacher's action distribution to a smaller student directly from the same on-policy traces, reducing latency while respecting the reward-weighted data collected in production.

**Technical Implementation**: The run config (`src/atlas_core/configs/recipe/teacher_gkd.yaml`) wires the GKD trainer (`src/atlas_core/configs/trainer/gkd.yaml`) and the Postgres-backed dataset builder in `atlas_core.data.gkd`. Trace exports match the schema captured in the [MCP Tool Learning example](/examples/adaptive-tool-use), so you can replay identical workloads through distillation or GRPO. For the reinforcement-learning path, see [`grpo-training.mdx`](./grpo-training.mdx); both flows follow the override patterns documented in the [Training Configuration Guide](../configuration.mdx).

### Data Flow: Runtime to Training

The GKD training pipeline connects runtime traces to trained models:

| Stage                       | Tool                                           | Input                    | Output                     | Time      |
| --------------------------- | ---------------------------------------------- | ------------------------ | -------------------------- | --------- |
| 1. Runtime execution        | `atlas run`                                    | User tasks               | Episode traces in Postgres | Minutes   |
| 2. Export traces (optional) | `arc-atlas`                                    | Postgres episodes        | JSONL dataset              | Minutes   |
| 3. GKD training             | `atlas-core train recipe@_global_=teacher_gkd` | JSONL or Postgres traces | Distilled checkpoint       | 4-8 hours |
| 4. Deploy checkpoint        | Update `teacher_model_name_or_path` in config  | Trained model            | Production teacher         | Minutes   |

<Note>
  **No traces yet?** Run the [Adaptive Tool Use example](/examples/adaptive-tool-use) to generate training data, or see [SDK Export Traces](/sdk/export-traces) to populate your database.
</Note>

<div>
  <img alt="Atlas system architecture showing runtime to training flow" />

  <p><em>Runtime traces flow from SDK execution to Postgres storage to GKD training to deployed teacher models</em></p>
</div>

### Core Capabilities

GKD handles multi-turn traces directly from Postgres, applies the same tokenization and augmentation stack that GRPO uses, and records baseline-comparison telemetry (success delta and token efficiency) in WandB for side-by-side evaluation with other Atlas trainers. The same Hydra overrides that configure the GRPO pipeline apply here, so swapping between distillation and reinforcement learning becomes a config change rather than a separate code path.

## When to Use GKD vs GRPO

| Criterion         | GKD                       | GRPO                       |
| ----------------- | ------------------------- | -------------------------- |
| **Data source**   | Atlas runtime traces      | Interactive environment    |
| **Compute cost**  | Low (supervised + KL)     | High (PPO + rollouts)      |
| **Speed**         | Fast (single pass)        | Slow (multi-epoch)         |
| **Best for**      | Distill teacher → student | Train from scratch with RL |
| **Training time** | Hours                     | Days                       |

**Rule of thumb**: Use GKD when you have Atlas traces and want to distill knowledge into a smaller model. Use GRPO when training a new policy from scratch via reinforcement learning.

## Quick Start

### Prerequisites

Before running GKD training, verify your environment:

<Steps>
  <Step title="Clone Atlas Core Repository">
    GKD training requires the full Atlas Core repository (not just the SDK):

    ```bash theme={null}
    git clone https://github.com/Arc-Computer/ATLAS.git
    cd ATLAS
    ```
  </Step>

  <Step title="Install Training Dependencies">
    ```bash theme={null}
    # For Python 3.11
    pip install -r requirements-py311.txt

    # For Python 3.12
    pip install -r requirements-py312.txt
    ```

    <Tip>
      This installs PyTorch, TRL, vLLM, and all training dependencies. Takes 10-15 minutes.
    </Tip>
  </Step>

  <Step title="Set Database URL (Optional)">
    GKD training can load traces from Postgres. Set your connection string if using database access:

    ```bash theme={null}
    export ATLAS_DB_URL="postgresql://user:pass@host:5432/atlas"
    ```

    <Note>
      **Using JSONL exports instead?** Skip this step and provide `dataset_path` in your config.
    </Note>
  </Step>

  <Step title="Verify Configuration Files Exist">
    ```bash theme={null}
    # Check required config files
    ls src/atlas_core/configs/recipe/teacher_gkd.yaml
    ls src/atlas_core/configs/trainer/gkd.yaml

    # Expected output:
    # src/atlas_core/configs/recipe/teacher_gkd.yaml
    # src/atlas_core/configs/trainer/gkd.yaml
    ```
  </Step>
</Steps>

### Run First Training Job

Once prerequisites are met, start a minimal GKD run:

```bash theme={null}
# Run from ATLAS repository root
atlas-core train \
  recipe@_global_=teacher_gkd \
  teacher_model_name_or_path=Qwen/Qwen2.5-14B-Instruct \
  model.model_name_or_path=Qwen/Qwen2.5-7B-Instruct \
  trainer.min_reward=0.8 \
  trainer.max_steps=100
```

<Note>
  **First run?** Add `trainer.max_steps=100` to complete a smoke test in 15-20 minutes instead of hours.
</Note>

### Verify Success

**Check training progress:**

```bash theme={null}
tail -f outputs/gkd/training.log | grep "loss:"
```

**Expected output:**

```
{'loss': 1.342, 'learning_rate': 1.9e-05, 'epoch': 0.1}
{'loss': 1.156, 'learning_rate': 1.5e-05, 'epoch': 0.3}
{'loss': 0.892, 'learning_rate': 1.2e-05, 'epoch': 0.6}
# Loss should steadily decrease during training
```

<Note>
  Custom metrics like `metrics/success_delta` and `metrics/token_reduction_pct` are logged to WandB/TensorBoard, not in the console training logs.
</Note>

**Check distillation metrics in WandB:**

* `metrics/success_delta` - Should be positive (student improving over baseline)
* `metrics/token_reduction_pct` - Target: >20% token savings
* `metrics/meets_target` - Should be `true` when both success and token targets met

**Verify checkpoint creation:**

```bash theme={null}
ls -lh outputs/gkd/checkpoint-final/
# Should contain: config.json, model.safetensors, tokenizer files
```

| Metric          | Target        | Meaning                               |
| --------------- | ------------- | ------------------------------------- |
| Training Loss   | \<1.0         | Student learning teacher distribution |
| KL Divergence   | 0.3-0.6       | Balanced teacher/student alignment    |
| Success Delta   | >0 (positive) | Student improves over baseline        |
| Token Reduction | >20%          | Smaller model is more efficient       |

These same Qwen checkpoints (`Qwen/Qwen2.5-14B-Instruct` teacher and `Qwen/Qwen2.5-7B-Instruct` student) are used by `scripts/validate_gkd.py` to measure the GSM8K lift before scaling to customer traces.

Hydra composes the trainer from `src/atlas_core/configs/recipe/teacher_gkd.yaml`, which overrides `_global_.trainer=gkd` and the shared data presets documented in [Training Configuration](../configuration.mdx). Override any field inline (for example `trainer.learning_key`) using the same syntax shown in the GRPO guide.

Running the command streams Atlas runtime traces directly from Postgres (the same database you populate with `arc-atlas --database-url postgresql://... --include-status approved --output traces/runtime.jsonl`), fine-tunes the 7B student to mimic the 14B teacher, logs the Baseline Comparison metrics (success delta and token reduction) to WandB, and writes checkpoints into `outputs/gkd/`. If you prefer to operate on JSONL exports, point the dataset adapter at the CLI output; both paths preserve the SDK schema so Atlas Core sees identical conversation records. For a step-by-step walkthrough of exporting traces, running the validation script, and reading the metrics file, see the [Developer Example: Running GKD](/examples/gkd-dev-example).

### Configuration Files

GKD training uses two main config files. The trainer config (`src/atlas_core/configs/trainer/gkd.yaml`) houses the GKD-specific hyperparameters (lmbda, beta, temperature) plus database connectivity and general training settings. The run config (`src/atlas_core/configs/recipe/teacher_gkd.yaml`) specifies student and teacher checkpoints, the Baseline Comparison reference metrics, and the output/logging targets.

### Cross-Tokenizer Alignment

Issue #45 introduced native support for cross-tokenizer distillation. Two new trainer parameters control the behavior:

* `align_teacher_template` (default: `true`) – when enabled, Atlas captures the raw prompt/completion text for every dataset row, re-renders the conversation with the teacher tokenizer, and computes teacher logprobs in the teacher’s chat template before applying the KL term. Disable it only when the student and teacher share identical tokenizers or when you need to reproduce legacy runs.
* `teacher_tokenizer_name_or_path` (optional) – override when the tokenizer lives at a different path than `teacher_model_name_or_path` (for example, PEFT adapters or custom checkpoints).

The same fields are exposed via `scripts/validate_gkd.py`:

```bash theme={null}
python scripts/validate_gkd.py \
  --teacher-tokenizer Qwen/Qwen2.5-14B-Instruct \
  # ...other args...
  # add --no-align-teacher-template to reproduce legacy behavior
```

By default the validation script aligns templates, so cross-model pairs (e.g., Qwen → Llama) train without the KL spikes caused by mismatched chat formats.

## Key Parameters

### GKD Parameters

#### `lmbda` (On-Policy Fraction)

Set `lmbda` to 1.0 when the student should generate every response and receive teacher guidance token by token; this setting keeps the run fully on-policy and matches the configuration we use for the GSM8K validation sweeps. A mid-range value such as 0.5 mixes teacher- and student-generated continuations when you want to temper exploration, while 0.0 reverts to supervised distillation. Start with 1.0 and only back off when trace quality is noisy or you need consistency with earlier supervised checkpoints (Issue #40 recommendation).

#### `beta` (KL Divergence Balance)

`beta` tunes the interpolation inside the generalized Jensen-Shannon Divergence. Push it toward 0.0 to emphasize the teacher distribution (forward KL), move toward 1.0 to penalize deviations from the student distribution (reverse KL), and stay at the default 0.5 for a balanced view. Start at 0.5, then sweep ±0.2 once you have telemetry on success deltas and token savings.

#### `temperature`

Sampling temperature controls how aggressively the student explores during generation. Values around 0.9 yield more diverse reasoning chains and match the public validation scripts; dropping to 0.5–0.7 tightens answers when traces have deterministic formats. Stick with 0.9 for early runs and lower it only after observing long completions or oscillating eval loss.

### Database Filtering

#### `min_reward`

Minimum session reward threshold for training data:

```yaml theme={null}
trainer:
  min_reward: 0.8  # Only use high-quality traces
```

Higher values (0.8-0.9) ensure training on successful sessions only.

#### `learning_key`

Filter traces by task type:

```yaml theme={null}
trainer:
  learning_key: "crm_workflows"  # Only CRM-related traces
```

Set to `null` to use all traces.

### Baseline Comparison Metrics

Track distillation quality against baseline:

```yaml theme={null}
trainer:
  baseline_success: 0.75     # Baseline task success rate
  baseline_tokens: 1200      # Baseline tokens per episode
```

Atlas logs `metrics/success_delta`, `metrics/token_reduction_pct`, and `metrics/meets_target` alongside loss curves so you can judge whether the distilled checkpoint clears your success and token budgets without exporting separate spreadsheets.

## Example Configurations

### High-Quality Distillation

For maximum quality, use higher reward threshold and more epochs:

```yaml theme={null}
# src/atlas_core/configs/recipe/gkd_high_quality.yaml
defaults:
  - override /trainer@_global_: gkd

teacher_model_name_or_path: Qwen/Qwen2.5-14B-Instruct
model_name_or_path: Qwen/Qwen2.5-7B-Instruct

trainer:
  min_reward: 0.9           # Only excellent traces
  num_train_epochs: 5       # More training
  learning_rate: 3e-6       # Lower learning rate
```

### Fast Iteration

For rapid experimentation:

```yaml theme={null}
# src/atlas_core/configs/recipe/gkd_fast.yaml
defaults:
  - override /trainer@_global_: gkd

trainer:
  min_reward: 0.7
  num_train_epochs: 1
  eval_steps: 50
  save_steps: 50
```

### Task-Specific Distillation

For a specific workflow:

```yaml theme={null}
trainer:
  learning_key: "crm_contact_management"
  min_reward: 0.85
  baseline_success: 0.78
```

## Monitoring Training

### WandB Metrics

The trainer streams `train/loss`, `train/learning_rate`, `eval/loss`, `eval/success_rate`, `eval/avg_tokens`, and the Baseline Comparison trio (`metrics/success_delta`, `metrics/token_reduction_pct`, `metrics/meets_target`) to WandB so you can correlate convergence with task-level improvements without building a custom dashboard.

### Command Line Output

```
Starting GKD training with Baseline Comparison reference: success=75.00%, tokens=1200
Loaded datasets: train=850, eval=150 conversations
AtlasGKDTrainer initialized with lmbda=1.0, beta=0.5

Epoch 1/3
  Step 100: loss=0.245, eval_loss=0.312
  ✅ Baseline Comparison targets MET: success delta=12.3 pp, token reduction=35.2%

Epoch 2/3
  Step 200: loss=0.198, eval_loss=0.276
  ✅ Baseline Comparison targets MET: success delta=14.1 pp, token reduction=38.7%
```

### Training Telemetry Example

Recent GSM8K validation runs illustrate what a healthy loop looks like. With the stock configuration (`lmbda=1.0`, `temperature=0.9`, `min_reward=0.8`, `learning_rate=2e-5`, `max_steps=500`), training loss fell from 0.0676 to 0.0294 within 500 steps (epoch ≈0.21) and evaluation loss followed closely (0.0437 → 0.0397 → 0.0394). Gradient norms stayed between 0.88 and 1.70 and the entire run finished in 11h 45m on a single DGX Spark. When we extended the same configuration across the full GSM8K export, evaluation loss continued drifting downward toward 0.0341 while gradients remained below 1.3 even as the learning rate decayed to 7e-6. Use those ranges as a reference point—if training loss stalls above \~0.04 with steady gradients, tighten `min_reward` or lower temperature before changing optimizer settings.

## Troubleshooting

### Issue: "No conversations found in database"

**Cause**: Database connectivity or filtering too strict.

**Solutions**: Confirm `ATLAS_DB_URL` points to the right cluster, run `atlas.training_data.get_training_sessions()` to verify rows are available, temporarily lower `trainer.min_reward`, and clear `learning_key` to widen the task filter while debugging.

### Issue: "Out of memory (OOM) during training"

**Cause**: Student + teacher models exceed GPU memory.

**Solutions**: Reduce the effective batch by setting `per_device_train_batch_size=2` with `gradient_accumulation_steps=8`, keep gradient checkpointing enabled (default), load the teacher in 8-bit via `teacher_model_init_kwargs.load_in_8bit=true`, or drop to a smaller teacher checkpoint when the hardware budget is tight.

### Issue: "Metrics not improving"

**Possible causes and solutions**:

If metrics stall, first extend the schedule (`num_train_epochs: 5`) so the loss has room to decay, then sweep `beta` toward 0.3 or 0.7 to change the KL emphasis. Raising `min_reward` to 0.85 filters out marginal traces, and if the student remains constrained, move to a larger base checkpoint before adjusting optimizer settings.

### Issue: "Training too slow"

Cap `max_steps` (for example 500 instead of multi-epoch sweeps), relax `eval_steps` to 200 to avoid constant validation passes, or set `limit: 5000` on the dataset loader when you only need a smoke test.

## Advanced Topics

### Continual Learning with GKD

To preserve existing skills while distilling new knowledge:

```yaml theme={null}
# Include rehearsal data from previous tasks
trainer:
  learning_key: null  # All tasks
  min_reward: 0.85
```

Monitor for catastrophic forgetting using regression tests.

### Multi-Stage Distillation

Distill progressively smaller models:

```
14B Teacher → 7B Student₁ → 3B Student₂ → 1.5B Student₃
```

Each stage uses the previous student as the teacher:

```bash theme={null}
# Stage 1: 14B → 7B
atlas-core train recipe@_global_=teacher_gkd \\
  teacher_model_name_or_path=Qwen/Qwen2.5-14B-Instruct \\
  model.model_name_or_path=Qwen/Qwen2.5-7B-Instruct

# Stage 2: 7B → 3B
atlas-core train recipe@_global_=teacher_gkd \\
  teacher_model_name_or_path=outputs/gkd_7b/final \\
  model.model_name_or_path=Qwen/Qwen2.5-3B-Instruct
```

### Integration with Arc-CRM-Benchmark

For Stage 3 evaluation (Issue #42):

```bash theme={null}
# 1. Train distilled model from baseline reference traces
atlas-core train recipe@_global_=teacher_gkd \\
  trainer.learning_key="crm_workflows" \\
  trainer.min_reward=0.8

# 2. Evaluate distilled model (no guidance)
# ... use arc-crm-benchmark evaluation scripts
```

## API Reference

### AtlasGKDTrainer

```python theme={null}
from atlas_core.training.algorithms.gkd_trainer import AtlasGKDTrainer
from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import GKDConfig

student = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-7B")
teacher = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-14B")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B")

args = GKDConfig(
    output_dir="outputs/gkd",
    per_device_train_batch_size=4,
    lmbda=1.0,
    beta=0.5,
)

trainer = AtlasGKDTrainer(
    model=student,
    teacher_model=teacher,
    args=args,
    db_url="postgresql://localhost:5432/atlas",
    min_reward=0.8,
    processing_class=tokenizer,
)

trainer.train()
trainer.save_model("outputs/gkd/final")
```

### Dataset Functions

```python theme={null}
from atlas_core.data.gkd import build_gkd_dataset

train_ds, eval_ds = build_gkd_dataset(
    db_url="postgresql://localhost:5432/atlas",
    min_reward=0.8,
    learning_key="crm_workflows",
    eval_split=0.15,
)
```

### Baseline Comparison Metrics

```python theme={null}
from atlas_core.training.evaluation.gkd_evaluator import compute_baseline_summary

summary = compute_baseline_summary(
    eval_results,
    baseline_success=0.75,
    baseline_tokens=1200,
)

print(f"Success delta: {summary['success_delta']*100:.1f} pp")
print(f"Token reduction: {summary['token_reduction_pct']:.1f}%")
print(f"Meets targets: {summary['meets_all_targets']}")
```

## Next Steps

Review the [Configuration Reference](../configuration.mdx) for override syntax, compare with the reinforcement-learning workflow described in [`grpo-training.mdx`](./grpo-training.mdx), and validate distilled checkpoints with the [Evaluation Harnesses](../../benchmarks/evaluation-harnesses.mdx).

## Status and Contributions

GKD support in Atlas Core remains beta. We're building expanded dataset filters, staged teacher → student schedules, and deeper telemetry hooks; track progress in [Issue #40](https://github.com/Arc-Computer/ATLAS/issues/40). Contributions are welcome—open a PR with trace snippets, Hydra overrides, or MCP-focused repro steps (the [MCP Tool Learning example](/examples/adaptive-tool-use) is the easiest shared workload) so we can iterate on the trainer together. For a qualitative look at how teams alternate between fast validation runs and longer reliability sweeps, see the ongoing `gkd_two_gear_gkd_blog_draft.md` research note.

## References

Consult the [On-Policy Distillation paper](https://arxiv.org/abs/2306.13649) for the underlying method, the [TRL GKDTrainer docs](https://huggingface.co/docs/trl/main/en/gkd_trainer) for library configuration, and [Issue #40](https://github.com/Arc-Computer/ATLAS/issues/40) for Atlas-specific implementation notes.


# Training Your Own Teacher Model
Source: https://docs.arc.computer/training/offline/grpo-training

Complete guide to custom teacher model training with GRPO

<Note>
  **Time**: 24-48 hours (mostly unattended) • **Active setup**: 30-45 minutes • **Difficulty**: Advanced
</Note>

This is the advanced path. Most users should start with our [pre-trained models](/sdk/quickstart).

<Info>
  Already collecting runtime traces? Stream them straight from Postgres with the `runtime_pg` data preset (`+override /data@_global_: runtime_pg db_url=...`) or stick with exported JSONL files (see [`Runtime Export Guide`](/sdk/export-traces)).

  Need to customise Hydra configs? See the [`Training Configuration`](/training/configuration) guide for directory structure and override patterns.
</Info>

## Who Should Train Custom Models

You need custom training if you have:

* **Proprietary knowledge** not available in public models
* **Domain-specific tasks** where generic teaching doesn't work well
* **Regulatory requirements** that prevent using pre-trained models
* **Extreme performance needs** where every percentage point matters

**What you'll need:**

* 4-8 H100 or A100 GPUs (40GB+ VRAM each)
* 2-3 days of training time
* Basic PyTorch and distributed training knowledge
* \~200GB disk space for checkpoints

## Training Pipeline Overview

<img alt="Training Pipeline" />

### Runtime traces as the data source

**Direct database access** (SDK v0.1.13+) queries training sessions from PostgreSQL with reward-based filtering and selective data loading. This eliminates JSONL export intermediates and prevents schema drift:

```python theme={null}
from atlas.training_data import get_training_sessions

# Query high-quality sessions directly
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    status_filters=["succeeded"],
    limit=10000
)
```

**JSONL export** (alternative method) is still supported for backward compatibility:

```bash theme={null}
arc-atlas --database-url postgresql://... --output traces.jsonl
```

Reference the provided Hydra config for dataset loading:

```yaml theme={null}
# src/atlas_core/configs/data/runtime_traces.yaml
make_dataset_fn:
  _target_: atlas_core.data.runtime_traces.get_runtime_trace_dataset
  export_path: traces/aime-batch.jsonl  # Or use direct database access
  eval_split_ratio: 0.1
```

Each session includes the triage dossier, adaptive summary (lane, confidence, certification flag, probe evidence), plan/step traces, persona usage/updates, reward breakdowns, and validation labels. See the [Training Data Pipeline Guide](/training/offline/training-data-pipeline) for complete API reference and filtering options.

The training happens in three clear steps:

**Step 1: SFT Warmup** → Teach basic teaching patterns (4-6 hours)

**Step 2: Launch vLLM Server** → Set up fast inference (5 minutes)

**Step 3: Run GRPO** → Learn verifying-teacher behaviors for the dual-agent runtime through RL (24-36 hours)

Each step has a clear goal and success criteria.

## Step 1: SFT Warmup

**Goal**: Establish foundational teaching capabilities

### What It Does

Supervised fine-tuning teaches the model basic teaching patterns through demonstration. Think of it like a student teacher observing an expert before trying it themselves.

### Configuration Snapshot

```yaml theme={null}
# src/atlas_core/configs/recipe/teacher_sft.yaml
defaults:
  - _self_
  - override /model: llama3_8b
  - override /data: arc_atlas_sft
  - override /trainer: sft

learning_rate: 2e-5
num_train_epochs: 1
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
warmup_ratio: 0.1
output_dir: checkpoints/sft
```

### Run It

```bash theme={null}
# Minimum (2 GPUs)
scripts/launch.sh 2 src/atlas_core/configs/recipe/teacher_sft.yaml \
  output_dir=checkpoints/sft

# Recommended (4 GPUs)
scripts/launch.sh 4 src/atlas_core/configs/recipe/teacher_sft.yaml \
  output_dir=checkpoints/sft

# Full production (8 GPUs)
scripts/launch.sh 8 src/atlas_core/configs/recipe/teacher_sft.yaml \
  output_dir=checkpoints/sft

# Memory-constrained with offloading
scripts/launch.sh offload 2 src/atlas_core/configs/recipe/teacher_sft.yaml \
  output_dir=checkpoints/sft
```

### Verify Success

**Check training is running:**

```bash theme={null}
tail -f checkpoints/sft/training.log | grep "loss:"
```

**Expected output:**

```
{'loss': 2.134, 'learning_rate': 1.8e-05, 'epoch': 0.15}
{'loss': 1.892, 'learning_rate': 1.5e-05, 'epoch': 0.30}
{'loss': 1.456, 'learning_rate': 1.2e-05, 'epoch': 0.50}
# Loss should decrease from ~2.0 to <1.5 over 4-6 hours
```

| Metric        | Target    | What It Means              |
| ------------- | --------- | -------------------------- |
| Training Loss | \<1.5     | Model is learning patterns |
| Gradient Norm | \<5.0     | Training is stable         |
| Duration      | 4-6 hours | On 8× H100 GPUs            |

## Step 2: Launch vLLM Server

**Goal**: Fast inference for RL training

### What It Does

The vLLM server provides high-throughput generation during reinforcement learning. It runs on separate GPUs from the training process for maximum efficiency.

### Run It

Atlas Core ships a lightweight launcher so you can spin up the inference stack without third-party tooling:

```bash theme={null}
CUDA_VISIBLE_DEVICES=0,1 \
python -m atlas_core.training.generation.vllm_server \
  --model checkpoints/sft/final \
  --port 8765 \
  --tensor-parallel-size 2 \
  --gpu-memory-utilization 0.9
```

If you prefer a single command that starts the vLLM servers and the GRPO job together, stick with `scripts/launch_with_server.sh <server_gpus> <training_gpus> src/atlas_core/configs/recipe/teacher_rcl.yaml ...`. That wrapper orchestrates `atlas_core.training.generation.vllm_server` under the hood, waits for the health checks to pass, and then launches `scripts/launch.sh` for reinforcement learning.

### Verify Success

**Check server health:**

```bash theme={null}
curl http://localhost:8765/health
```

**Expected output:**

```json theme={null}
{"status": "ok", "model_loaded": true}
```

**List available models:**

```bash theme={null}
curl http://localhost:8765/v1/models
```

**Expected output:**

```json theme={null}
{
  "object": "list",
  "data": [{"id": "checkpoints/sft/final", "created": 1234567890}]
}
```

**Key parameters:**

* `tensor-parallel-size`: Number of GPUs for inference (match your hardware)
* `gpu-memory-utilization`: How much VRAM to use (0.9 = 90%)
* `max-model-len`: Maximum sequence length (2048 is good default)

## Step 3: Run GRPO Training

**Goal**: Optimize teaching through reinforcement learning

### What It Does

GRPO (Group Relative Policy Optimization) trains the teacher to actually improve student performance. The reward comes from measuring if students get better when taught.

### Run It

```bash theme={null}
# Minimum (2 GPUs: 1 training, 1 vLLM)
scripts/launch_with_server.sh 1 1 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  model_name_or_path=checkpoints/sft/final

# Recommended (4 GPUs: 2 training, 2 vLLM)
scripts/launch_with_server.sh 2 2 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  model_name_or_path=checkpoints/sft/final

# Production (8 GPUs: 4 training, 4 vLLM)
scripts/launch_with_server.sh 4 4 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  model_name_or_path=checkpoints/sft/final
```

The first number is training GPUs, second is vLLM GPUs.

### Verify Success

**Monitor training progress:**

```bash theme={null}
tensorboard --logdir checkpoints/grpo --port 6006
```

**What to look for in TensorBoard:**

1. **Reward Mean** (chart: `train/reward`)
   * Should trend upward from baseline (\~0.3-0.5 → 0.7-0.8)
   * Steady increases indicate the teacher is learning effective guidance

2. **KL Divergence** (chart: `train/kl`)
   * Healthy range: 0.5-2.0
   * Warning if >5.0 (policy diverging too far from SFT baseline)

3. **Non-degradation Rate** (chart: `train/non_degrade_rate`)
   * Target: >95% of samples improve or maintain quality
   * Warning if \<90% (teaching is making students worse)

| Metric               | Healthy Range          | Warning Sign           |
| -------------------- | ---------------------- | ---------------------- |
| Reward Mean          | 0.3 → 0.8 (increasing) | Plateau or decrease    |
| Non-degradation Rate | >95%                   | \<90% indicates issues |
| KL Divergence        | 0.5-2.0                | >5.0 suggests collapse |

**Check live metrics:**

```bash theme={null}
tail -f checkpoints/grpo/training.log | grep "reward:"
```

**Expected output:**

```
{'step': 50, 'reward': 0.52, 'kl': 1.2, 'non_degrade': 0.96}
{'step': 100, 'reward': 0.61, 'kl': 1.4, 'non_degrade': 0.97}
{'step': 150, 'reward': 0.73, 'kl': 1.6, 'non_degrade': 0.98}
# Rewards should steadily increase over 24-36 hours
```

## Key Configuration Parameters

You only need to understand 3-5 core parameters:

### beta (KL divergence coefficient)

**What it does**: Controls how much the model can change from the original

**Default**: 0.04

**When to adjust**:

* Model changing too fast? → Increase to 0.06
* Training too conservative? → Decrease to 0.02

### temperature (sampling temperature)

**What it does**: Controls how creative the teaching is

**Default**: 0.7

**When to adjust**:

* Want more diverse teaching? → Increase to 0.9
* Teaching too random? → Decrease to 0.5

### learning\_rate

**What it does**: How fast the model learns

**Default**: 5e-7 (much smaller than SFT!)

**When to adjust**:

* Training too slow? → Try 1e-6
* Rewards collapsing? → Decrease to 1e-7

<AccordionGroup>
  <Accordion title="Advanced: Reward System Configuration">
    GRPO uses the principle-based reward system. Point to the config:

    ```yaml theme={null}
    # src/atlas_core/configs/reward/interpretation_teaching.yaml
    teacher_reward:
      _target_: atlas_core.reward.interpretation.RIMReward
      config_path: reward_system/interpretation_offline.yaml
    ```

    The offline config focuses on helpfulness and process (not accuracy):

    ```yaml theme={null}
    # reward_system/interpretation_offline.yaml
    active_judges:
      accuracy: false      # Disable for offline training
      helpfulness: true    # Core reward signal
      process: true        # Rewards good reasoning
      diagnostic: false    # Disable for offline training
    ```

    Monitor `rim_rewards` in logs to spot regressions.
  </Accordion>

  <Accordion title="Advanced: All Parameters">
    ```yaml theme={null}
    # Full configuration options
    num_generations: 32              # Samples per prompt
    max_new_tokens: 512             # Response length
    top_p: 0.95                     # Nucleus sampling
    warmup_ratio: 0.1               # LR warmup
    weight_decay: 0.01              # L2 regularization
    max_grad_norm: 1.0              # Gradient clipping
    gradient_accumulation_steps: 4   # Effective batch size
    ```
  </Accordion>
</AccordionGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="CUDA Out of Memory">
    **Problem**: GPU runs out of memory during training

    **Quick fixes**:

    ```bash theme={null}
    # Reduce batch size
    per_device_train_batch_size=1

    # Enable gradient checkpointing
    gradient_checkpointing=true

    # Use DeepSpeed offloading (Accelerate config)
    # scripts/launch.sh offload <gpus> src/atlas_core/configs/recipe/teacher_rcl.yaml
    ```
  </Accordion>

  <Accordion title="Reward Collapse">
    **Problem**: Rewards go to zero or negative

    **Quick fixes**:

    ```bash theme={null}
    # Increase KL penalty
    beta=0.1

    # Reduce learning rate
    learning_rate=1e-7

    # Check data quality
    # Verify your training data has clear improvement signals
    ```
  </Accordion>

  <Accordion title="vLLM Connection Failed">
    **Problem**: Training can't connect to server

    **Quick fixes**:

    ```bash theme={null}
    # Check server is running
    ps aux | grep vllm

    # Verify port is open
    lsof -i :8000

    # Restart with more memory
    --gpu-memory-utilization 0.95
    ```
  </Accordion>

  <Accordion title="Slow Training">
    **Problem**: Training slower than expected

    **Quick fixes**:

    ```bash theme={null}
    # Enable Flash Attention 2
    attn_implementation=flash_attention_2

    # Use torch compile (PyTorch 2.0+)
    torch_compile=true

    # Optimize data loading
    dataloader_num_workers=4
    ```
  </Accordion>
</AccordionGroup>

## Expected Results

After successful training, you should see:

### Performance Metrics

* **Teaching efficiency**: 15.7% average accuracy improvement
* **Safety**: 97% non-degradation rate
* **Token efficiency**: 50% reduction in response length
* **Completion rate**: 31% improvement (69% → 100%)

### Training Duration

* **2 GPUs**: 4-5 days
* **4 GPUs**: 2-3 days
* **8 H100s**: 24-36 hours

### Output Artifacts

```
results/
├── sft_checkpoint/          # Phase 1 model
│   ├── pytorch_model.bin
│   └── config.json
├── rl_checkpoint/           # Phase 2 model (use this!)
│   ├── pytorch_model.bin
│   ├── config.json
│   └── trainer_state.json
└── logs/
    ├── train.log
    └── tensorboard_events
```

## Validation

Test your trained model:

```python theme={null}
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load your trained teacher
teacher = AutoModelForCausalLM.from_pretrained(
    "results/rl_checkpoint",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("results/rl_checkpoint")

# Load baseline student
student = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Test on a problem
problem = "A train travels 120 miles in 2 hours. What is its speed?"

# Get baseline (student only)
inputs = tokenizer(problem, return_tensors="pt").to(student.device)
baseline = student.generate(**inputs, max_new_tokens=100)
print(f"Baseline: {tokenizer.decode(baseline[0])}")

# Get teaching (using the atlas-sdk runtime loop)
# This gives you the enhanced response
```

## Performance Optimization

### Multi-Node Training

Scale across multiple machines:

```bash theme={null}
# Node 1 (master)
torchrun \
  --nproc_per_node=8 \
  --nnodes=2 \
  --node_rank=0 \
  --master_addr=10.0.0.1 \
  atlas-core train recipe@_global_=teacher_rcl

# Node 2
torchrun \
  --nproc_per_node=8 \
  --nnodes=2 \
  --node_rank=1 \
  --master_addr=10.0.0.1 \
  atlas-core train recipe@_global_=teacher_rcl
```

### DeepSpeed for Large Models

Atlas Core uses Accelerate configs (in `accelerate/`) to control DeepSpeed settings. Choose one of the shipped presets:

* `accelerate/deepspeed_zero3.yaml` (default)
* `accelerate/deepspeed_zero3_cpu_offloading.yaml` (CPU offload)
* `accelerate/deepspeed_zero1.yaml` (lighter config)

Use with:

```bash theme={null}
# Default zero3
scripts/launch.sh 8 src/atlas_core/configs/recipe/teacher_rcl.yaml

# CPU offload
scripts/launch.sh offload 8 src/atlas_core/configs/recipe/teacher_rcl.yaml

# Or explicitly via Accelerate
accelerate launch --config_file accelerate/deepspeed_zero3.yaml \
  -m atlas_core.cli.train recipe@_global_=teacher_rcl
```

## Next Steps

<CardGroup>
  <Card title="SDK Runtime" icon="arrows-rotate" href="/sdk/quickstart">
    Keep production agents improving between offline runs
  </Card>

  <Card title="Deploy to Production" icon="server" href="/sdk/configuration">
    Integrate your custom teacher into production
  </Card>

  <Card title="Reward System Deep Dive" icon="trophy" href="/concepts/reward-design">
    Understand how teaching effectiveness is measured
  </Card>
</CardGroup>

## References

* [Training Data Pipeline](/training/offline/training-data-pipeline) - Direct database access for training data
* [ATLAS Technical Report](/reference/technical-report) - Detailed methodology and ablations
* [GRPO Paper](https://arxiv.org/abs/2402.03300) - Original algorithm
* [vLLM Documentation](https://docs.vllm.ai) - Server configuration options
* [Export Runtime Traces](/sdk/export-traces) - Direct database access and JSONL export methods
* [Quickstart](/sdk/quickstart) - Collect runtime traces before training


# Training Data Pipeline
Source: https://docs.arc.computer/training/offline/training-data-pipeline

Direct database access for training data extraction and preprocessing

## Overview

The Atlas SDK provides direct PostgreSQL access for training data extraction, eliminating JSONL export intermediates and preventing schema drift between SDK and ATLAS Core. Query training sessions with reward-based filtering, selective data loading, and pagination support for large datasets.

## Prerequisites

* Atlas SDK v0.1.13 or higher
* PostgreSQL database with runtime traces (configured via `storage.database_url`)
* Python 3.10+

## Direct Database Access

### Basic Usage

Query training sessions directly from PostgreSQL:

```python theme={null}
from atlas.training_data import get_training_sessions

# Query sessions with filters
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    learning_key="security-review",
    status_filters=["succeeded"],
    limit=1000
)

# Access essential fields
for session in sessions:
    reward_score = session.session_reward["score"]
    trajectory = session.trajectory_events
    learning_data = session.learning_history

    # Optional fields via property accessors
    task_id = session.learning_key
    drift_status = session.drift_alert
```

### Async Queries

For high-throughput training pipelines:

```python theme={null}
from atlas.training_data import get_training_sessions_async

sessions = await get_training_sessions_async(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.7,
    limit=5000
)
```

## Query Filters

### Reward-Based Filtering

Filter sessions by reward score using JSONB operators:

```python theme={null}
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,  # Only sessions with reward ≥ 0.8
    max_reward=1.0   # Sessions with reward ≤ 1.0
)
```

### Status Filtering

Filter by runtime completion status:

```python theme={null}
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    status_filters=["succeeded", "failed"],  # Include both
    learning_key="task-batch-1"
)
```

### Date Range Filtering

Query sessions within a specific time window:

```python theme={null}
from datetime import datetime, timedelta

start_date = datetime.now() - timedelta(days=7)
end_date = datetime.now()

sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    start_date=start_date,
    end_date=end_date
)
```

## Selective Data Loading

Control which data is loaded to optimize performance:

```python theme={null}
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    include_trajectory_events=False,  # Skip trajectory events
    include_learning_data=True,       # Load learning history
    limit=1000
)
```

**Performance impact:**

* `include_trajectory_events=False`: 50-70% faster queries
* `include_learning_data=False`: 30-40% faster queries

## Pagination

Process large datasets in batches using async iterators:

```python theme={null}
from atlas.training_data import paginate_sessions

async for batch in paginate_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    batch_size=100,
    min_reward=0.7
):
    # Process batch of 100 sessions
    for session in batch:
        process_session(session)
```

## Session Count Queries

Get session counts without loading full data:

```python theme={null}
from atlas.training_data import count_training_sessions

total = count_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    learning_key="task-1"
)
print(f"Found {total} sessions matching criteria")
```

## Fetch Individual Sessions

Retrieve a specific session by ID:

```python theme={null}
from atlas.training_data import get_session_by_id

session = get_session_by_id(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    session_id=42
)
```

## Schema Fields

### AtlasSessionTrace

**Essential fields** (always loaded):

* `session_reward`: Aggregate reward with score and uncertainty
* `trajectory_events`: Ordered list of runtime events
* `student_learning`: Student persona learning notes
* `teacher_learning`: Teacher persona learning notes
* `learning_history`: Historical learning data
* `adaptive_summary`: Mode selection and probe evidence

**Property accessors** (loaded on demand):

* `learning_key`: Task identifier for grouping sessions
* `teacher_notes`: Guidance provided during execution
* `reward_summary`: Simplified reward statistics
* `drift`: Detected schema or behavior drift
* `drift_alert`: Critical drift warnings
* `triage_dossier`: Pre-execution risk assessment
* `reward_audit`: Detailed judge breakdowns

### AtlasStepTrace

**Essential fields**:

* `runtime`: Execution time in milliseconds
* `depends_on`: Step dependency graph

**Property accessors**:

* `attempt_history`: Previous attempt records

## Performance Optimization

### Database Indexes

The SDK automatically creates performance indexes:

```sql theme={null}
-- Reward filtering (10-100x faster)
CREATE INDEX sessions_reward_score_idx
ON sessions ((reward_stats->>'score')::float);

-- Date range queries (50-100x faster)
CREATE INDEX sessions_created_at_idx
ON sessions (created_at DESC);

-- Learning key queries
CREATE INDEX sessions_metadata_gin_idx
ON sessions USING GIN (metadata);
```

### Query Optimization

For training workloads with millions of sessions:

```python theme={null}
# Use selective loading
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    include_trajectory_events=False,  # Skip if not needed
    limit=10000
)

# Use pagination for large datasets
async for batch in paginate_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    batch_size=500,
    min_reward=0.8
):
    process_batch(batch)
```

## Integration with Training Pipeline

### Step 1: Query Training Data

```python theme={null}
from atlas.training_data import get_training_sessions

# Extract high-quality sessions
sessions = get_training_sessions(
    db_url="postgresql://atlas:atlas@localhost:5433/atlas",
    min_reward=0.8,
    status_filters=["succeeded"],
    limit=10000
)
```

### Step 2: Convert to Training Format

```python theme={null}
from atlas_core.data.runtime_traces import sessions_to_rl_records

# Convert to RL training records
records = sessions_to_rl_records(sessions)
```

### Step 3: Wire into Hydra configs

Prefer to skip the manual Python glue? The repo now ships a Postgres-backed dataset preset. Override the global data config with `runtime_pg` and supply your connection details:

```bash theme={null}
scripts/launch.sh 4 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  "+override /data@_global_: runtime_pg" \
  "db_url=postgresql://atlas:atlas@localhost:5433/atlas" \
  "min_reward=0.8" \
  "status_filters=['succeeded']"
```

The helper streams sessions via `atlas.training_data`, converts trajectory events into chat-format messages, and produces Hugging Face datasets on the fly—no JSONL export required.

### Step 4: Train with GRPO

```bash theme={null}
# Launch training
scripts/launch.sh 4 src/atlas_core/configs/recipe/teacher_rcl.yaml \
  model_name_or_path=checkpoints/sft/final
```

See [GRPO Training Guide](/training/offline/grpo-training) for complete training pipeline.

## Migration from JSONL Export

### Previous Approach (JSONL Files)

```bash theme={null}
# Old: Export to JSONL first
arc-atlas \
  --database-url postgresql://localhost:5433/atlas \
  --output traces.jsonl \
  --limit 1000

# Then load from JSONL
sessions = load_runtime_traces("traces.jsonl")
```

### Direct Database Access

```python theme={null}
# New: Query directly
from atlas.training_data import get_training_sessions

sessions = get_training_sessions(
    db_url="postgresql://localhost:5433/atlas",
    limit=1000
)
```

**Benefits:**

* No intermediate JSONL files
* Filters applied at database level
* 10-100x faster queries with indexes
* No schema drift between SDK and training

## Troubleshooting

| Error              | Cause                     | Solution                                        |
| ------------------ | ------------------------- | ----------------------------------------------- |
| Connection refused | PostgreSQL not running    | Start Postgres: `docker compose up -d postgres` |
| Empty result set   | No sessions match filters | Verify filters with `count_training_sessions()` |
| Memory error       | Loading too many sessions | Use pagination with smaller batch sizes         |
| Missing fields     | SDK version mismatch      | Upgrade to atlas-sdk ≥ 0.1.13                   |

## API Reference

### Core Functions

```python theme={null}
# Sync variants
get_training_sessions(db_url, min_reward=None, max_reward=None, ...)
get_session_by_id(db_url, session_id)
count_training_sessions(db_url, min_reward=None, ...)

# Async variants
get_training_sessions_async(db_url, min_reward=None, ...)
get_session_by_id_async(db_url, session_id)
count_training_sessions_async(db_url, min_reward=None, ...)

# Pagination
paginate_sessions(db_url, batch_size=100, min_reward=None, ...)
```

### Converter Functions

```python theme={null}
convert_session_dict_to_trace(session_dict)
convert_step_dict_to_trace(step_dict)
```

## Related Documentation

* [GRPO Training Guide](/training/offline/grpo-training) - Complete training pipeline
* [Database Schema](/reference/database-schema) - PostgreSQL schema reference
* [Runtime Export](/sdk/export-traces) - JSONL export and direct database access methods


# Reward System Implementation
Source: https://docs.arc.computer/training/reward-system-usage

How to use and customize the ATLAS reward system in code

This guide shows **HOW-TO** use the reward system in code. For conceptual understanding, see [The ATLAS Reward System](/concepts/reward-design).

## Using the Reward System

### In Training (Offline RL)

The reward system integrates seamlessly with the GRPO trainer:

```python theme={null}
from atlas_core.training.algorithms.grpo import GRPOTrainer
from atlas_core.reward.interpretation import RIMReward
from datasets import load_dataset

# 1. Instantiate reward system
reward_system = RIMReward(config_path='reward_system/interpretation.yaml')

# 2. Pass to trainer
trainer = GRPOTrainer(
    model="path/to/your/teacher_model",
    args=grpo_config,
    reward_funcs=[reward_system],  # Just pass it in
    train_dataset=train_dataset
)

# 3. Train - the reward system runs automatically
trainer.train()
```

The trainer handles calling the reward system with batches of data during the RL loop. You don't need to manage it manually.

### For Ad-hoc Evaluation

Quick evaluation of teaching effectiveness:

```python theme={null}
from atlas_core.reward.interpretation import RIMReward

# Create reward system
reward = RIMReward(config_path='reward_system/interpretation.yaml')

# Evaluate a single interaction
result = reward.evaluate(
    prompt="What is 2+2?",
    response="The answer is <solution>4</solution>.",
    baseline_solutions="It is 4",
    teacher_traces="Explain your reasoning step by step",
)

print(f"Score: {result.score}")
print(f"Per-judge: {result.judge_scores}")
print(f"Rationale:\\n{result.rationale}")
```

### In Continual Learning

In the SDK runtime, the same reward signals drive continual learning loops and help teams decide when to export traces for GRPO training. See the [`atlas-sdk` documentation](https://docs.arc.computer/sdk/quickstart) for details on wiring reward feedback into production orchestration.

## Customizing Judges

<Note>
  **Advanced Configuration**: This section is for users who need custom evaluation criteria. Most users can use the default judges.
</Note>

### Modifying Existing Judges

Judge behavior is controlled by their prompts in `src/atlas_core/reward/interpretation/judges.py`. To change what AccuracyJudge prioritizes:

```python theme={null}
# src/atlas_core/reward/interpretation/judges.py
class AccuracyJudge:
    def _build_prompt(self, inputs: Dict[str, Any]) -> str:
        # Customize this string to change evaluation criteria
        return f"""Evaluate these responses.

Prompt: {inputs.get('prompt', '')}
Response A: {inputs.get('response_a', '')}
Response B: {inputs.get('response_b', '')}

Step 1: Generate 2-3 evaluation principles with weights (must sum to 1.0)
Step 2: Score both responses against each principle
Step 3: Provide final scores (0.0 to 1.0)

Output JSON only: {{"principles": [...], "score_a": float, "score_b": float, "uncertainty": float}}"""
```

### Adding a New Judge

**Step 1: Create judge class** (`src/atlas_core/reward/interpretation/judges.py`):

```python theme={null}
class CreativityJudge:
    def __init__(self):
        self.name = 'creativity'

    def evaluate(self, inputs: Dict[str, Any], model_fn, temperature: float):
        prompt = f"""Score creativity (0.0 = formulaic, 1.0 = highly creative).
        Response: {inputs.get('response', '')}
        Output JSON: {{"score": float, "rationale": str, "uncertainty": float}}"""

        response = model_fn(prompt, temperature)
        return json.loads(response)
```

**Step 2: Register in reward adapter** (`src/atlas_core/reward/interpretation/reward_adapter.py`):

```python theme={null}
from atlas_core.reward.interpretation.judges import AccuracyJudge, HelpfulnessJudge, CreativityJudge

class RIMReward:
    def __init__(self, ...):
        self.judges = {
            'accuracy': AccuracyJudge(),
            'helpfulness': HelpfulnessJudge(),
            'creativity': CreativityJudge()  # Add here
        }
```

**Step 3: Enable in config** (`reward_system/interpretation.yaml`):

```yaml theme={null}
active_judges:
  accuracy: true
  helpfulness: true
  creativity: true  # Enable new judge
```

## Performance & Monitoring

### RewardBench V2 Results

The ensemble-and-escalation architecture achieves **93.7% overall accuracy**, significantly outperforming individual models:

* **Component model** (`gemini-2.5-flash`): 77.7% on its own
* **System performance**: 93.7% (+16 points)

The architecture creates a result greater than the sum of its parts.

<div>
  <img alt="ATLAS Reward System Leaderboard" />
</div>

### Category Breakdown

<div>
  <img alt="Performance by Category" />
</div>

See the complete [Reward System Technical Report](https://www.arc.computer/blog/ATLAS-Reward-System) for full analysis.

### Monitoring Rewards During Training

The training logs include reward system outputs:

```python theme={null}
# Example log entry
{
  'step': 150,
  'rim_rewards': {
    'accuracy': 0.85,
    'helpfulness': 0.72,
    'process': 0.78,
    'diagnostic': 0.80
  },
  'rim_explanations': {
    'accuracy': 'Response correctly solves the problem with proper units',
    'helpfulness': 'Teaching improved reasoning structure significantly'
  },
  'escalation_rate': 0.23  # 23% of cases went to Tier 2
}
```

Monitor these to:

* Spot prompt regressions (dropping helpfulness scores)
* Identify misconfigured thresholds (escalation rate too high/low)
* Validate teaching improvements (rising scores over time)

## Next Steps

<CardGroup>
  <Card title="Reward System Concepts" icon="trophy" href="/concepts/reward-design">
    Understand the two-tier evaluation architecture
  </Card>

  <Card title="GRPO Training" icon="graduation-cap" href="/training/offline/grpo-training">
    Use the reward system to train teacher models
  </Card>

  <Card title="SDK Runtime" icon="lightbulb" href="/sdk/quickstart">
    See how rewards flow through the production loop
  </Card>

  <Card title="Training Configuration" icon="sliders" href="/training/configuration">
    Configure reward system parameters
  </Card>
</CardGroup>

## References

* [Reward System Technical Report](https://www.arc.computer/blog/ATLAS-Reward-System) - Complete methodology and benchmarks
* [ATLAS Technical Report](/reference/technical-report) - How rewards integrate with training
* [RewardBench V2](https://huggingface.co/spaces/allenai/reward-bench) - Benchmark leaderboard