Configuration Guide

Quick Reference

ATLAS uses configuration files to customize behavior without code changes. Here’s what each directory contains:

configs/
├── wrappers/          🎯 START HERE - Wrap your existing agent
├── optimize/          📈 Optimization settings (API or vLLM)
├── examples/          ⚡ Ready-to-run configs
├── data/              💾 Dataset configurations
├── demo/              🔬 Full demo scenarios
├── rim_config.yaml    🏆 Reward system configuration
├── model/             🤖 Model architectures (advanced training)
├── run/               🔬 Training experiments (advanced training)
└── trainer/           ⚙️ RL algorithms (advanced training)

For Getting Started: Start with wrappers/ and optimize/. The model/, run/, and trainer/ directories are for advanced RL training (see Architecture docs).Root-level configs:

rim_config.yaml - Reward system settings (judges, models, thresholds). See Reward Design.
train.yaml - Global training defaults (advanced users only)

Common Configurations

Wrapping Your Existing Agent

A common starting point is integrating ATLAS with your existing agent. All integration configs live in configs/wrappers/.

HTTP API
Python Function
CLI Command
Real Working Example

Wrap any REST API endpoint with ATLAS teaching:

# configs/wrappers/my_api_agent.yaml
user_agent:
  type: custom
  config:
    integration_type: http_api
    endpoint: "http://localhost:8000/chat"
    prompt_field: "message"
    response_field: "response"
    headers:
      Authorization: "Bearer YOUR_API_KEY"
    timeout: 300

# ATLAS teacher model
teacher_model: Arc-Intelligence/ATLAS-8B-Thinking

# Dataset to test on
trainset: arc-atlas-rl
max_examples: 10

# Enable compatibility mode (recommended for custom agents)
compatibility_mode: true

generation_config:
  max_tokens: 2048
  temperature: 0.7
  diagnostic_max_tokens: 500

How to run:

./scripts/openai_agent_atlas.sh configs/wrappers/my_api_agent.yaml

Online Optimization Settings

Once you have your agent wrapped, configure GEPA optimization in configs/optimize/.

API Models (No GPU)
vLLM Server (Local GPU)
Default Config

Use API models for optimization (recommended for getting started):

# configs/optimize/my_api_optimization.yaml
defaults:
  - default

student_model: gpt-4o
teacher_model: Arc-Intelligence/ATLAS-8B-Thinking
reflection_lm: gpt-5

max_metric_calls: 50

Cost: ~$10 for 50 iterations with 10 examplesTime: ~2 hoursRun it:

python optimize_teaching.py --config-name my_api_optimization

Dataset Configuration

Configure which dataset to use for optimization in configs/data/.

Built-in Datasets
Custom Dataset
Use in Wrapper

Use ATLAS curated datasets from HuggingFace:

# configs/data/my_dataset.yaml
dataset_id_or_path: Arc-Intelligence/Arc-ATLAS-Teach-v0
dataset_split: rl
dataset_max_samples: null
eval_split_ratio: 0.1

data_log_name: my_dataset

make_dataset_fn:
  _target_: custom_data.arc_atlas_rl_data.get_arc_atlas_rl_dataset
  dataset_id_or_path: ${dataset_id_or_path}
  dataset_split: ${dataset_split}
  dataset_max_samples: ${dataset_max_samples}
  eval_split_ratio: ${eval_split_ratio}

Available datasets:

arc-atlas-rl: RL training data (default)
arc-atlas-sft: SFT training data
See configs/data/ for more options

Configuration Patterns

Pattern 1: Quick Start with Defaults

Use the quickstart config which inherits sensible defaults:

# configs/examples/quickstart.yaml
defaults:
  - ../wrappers/openai_existing_agent

# Override only what you need
# teacher_model: gpt-4o
# student_model: gpt-4o-mini
# max_metric_calls: 40

Run it:

./scripts/openai_agent_atlas.sh configs/examples/quickstart.yaml

Pattern 2: Environment Variables

Use environment variables for secrets and dynamic values:

# configs/wrappers/secure_agent.yaml
user_agent:
  type: custom
  config:
    integration_type: http_api
    endpoint: "${API_ENDPOINT:http://localhost:8000}"  # Fallback to localhost
    headers:
      Authorization: "Bearer ${API_KEY}"  # Must be set in environment

vllm_host: ${VLLM_HOST:localhost}
vllm_port: ${VLLM_PORT:8765}

Set them before running:

export API_ENDPOINT="https://api.example.com"
export API_KEY="your-secret-key"
export VLLM_HOST="10.0.0.5"
export VLLM_PORT="8000"

./scripts/openai_agent_atlas.sh configs/wrappers/secure_agent.yaml

Pattern 3: Compose Configs with Hydra

Create task-specific configs that inherit from base configs:

# configs/my_custom_experiment.yaml
defaults:
  - wrappers/openai_existing_agent
  - optimize/api_models
  - data/arc_atlas_rl

# Override specific fields
max_examples: 50
max_metric_calls: 100

generation_config:
  max_tokens: 4096
  temperature: 0.9

wandb:
  enabled: true
  project: my-atlas-project
  tags:
    - experiment-1

Key Configuration Fields

Model Selection

student_model: The model being improved

API models: gpt-4o, gpt-4o-mini, claude-3-opus
Local models: Qwen/Qwen3-4B, meta-llama/Llama-3-8B

teacher_model: The model providing guidance

Recommended: Arc-Intelligence/ATLAS-8B-Thinking (reasoning tasks)
Alternative: Arc-Intelligence/ATLAS-8B-Instruct (coding tasks)
API: gpt-5, gpt-4o (requires API access)

reflection_lm: Model for GEPA prompt optimization

Recommended: gpt-5 or gpt-4o (highest quality)
Budget: gpt-4o-mini (faster, cheaper)

Generation Settings

generation_config.max_tokens: Maximum output length (default: 2048)generation_config.temperature: Sampling randomness (0.0-1.0)

Lower (0.3-0.5): Focused, deterministic responses
Higher (0.7-0.9): Creative, diverse responses

generation_config.diagnostic_max_tokens: Max tokens for diagnostic probe (default: 500)generation_config.timeout: Request timeout in seconds (default: 300)

GEPA Parameters

max_metric_calls: Optimization budget (default: 50)

Development: 20-50 iterations (~$5-10)
Production: 100-200 iterations (~$20-40)

gepa_config.candidate_selection_strategy: How to select candidates

pareto: Multi-objective optimization (recommended)
greedy: Single best candidate

gepa_config.module_selector: Which prompts to optimize

all: Optimize all teaching prompts (default)
single: Optimize only teacher_adaptive_template (compatibility mode)

gepa_config.reflection_minibatch_size: Examples for reflection (default: 5)

Integration Settings

compatibility_mode: Enable for custom agents (recommended: true)use_vllm_client: Use local vLLM server (default: false)

true: Connect to vLLM at vllm_host:vllm_port
false: Use API models

max_litellm_workers: Parallel API workers (default: 100)

Higher: Faster but more API rate limit risk
Lower: Safer for rate limits

trace_storage: Where to save interaction traces (default: traces/optimize_traces.jsonl)output: Where to save optimized prompts (default: optimized_prompts.json)

Monitoring Settings

wandb.enabled: Enable Weights & Biases logging (default: false)wandb.project: W&B project namewandb.tags: Tags for this run (list)wandb.notes: Run descriptionExample:

wandb:
  enabled: true
  project: atlas-development
  tags:
    - experiment-1
    - baseline
  notes: "Testing ATLAS with our custom agent"

Troubleshooting

Error: Could not find wrappers.my_agent

Problem: Config file not found in expected locationSolution: Ensure config is in correct directory

# Correct structure
configs/wrappers/my_agent.yaml  ✅

# Wrong - won't be found
my_agent.yaml                   ❌
configs/my_agent.yaml          ❌

Error: KeyError: 'response_field'

Problem: Your API returns different field name than configuredDebug: Check actual API response

curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "test"}'

# If response is {"data": {"text": "..."}}, update config:
response_field: "data.text"  # Use dot notation for nested fields

Error: Timeout after 60s

Problem: Default timeout too short for slow modelsSolution: Increase timeout in generation_config

generation_config:
  timeout: 600  # 10 minutes
  request_timeout: 600

Error: API rate limit exceeded

Problem: Too many parallel workers hitting rate limitsSolution: Reduce worker count

max_litellm_workers: 10  # Lower from default 100

Teaching not improving performance

Problem: Teaching prompts not effective for your taskSolutions:

Verify teacher model is stronger than student
Customize seed_prompts for your domain
Increase max_metric_calls for more optimization
Check that examples in trainset are relevant

Out of memory with vLLM

Problem: Model too large for GPUSolutions:

# Use smaller model
student_model: Qwen/Qwen3-4B  # Instead of 8B or 70B

# Or use quantization (add to vLLM server launch)
# --quantization awq

Quick Decision Guide

Which config directory do I need?

┌─ I want to integrate my existing agent
│  └─ configs/wrappers/
│
┌─ I want to optimize teaching prompts
│  └─ configs/optimize/
│     ├─ Use API models → api_models.yaml
│     └─ Have local GPU → vllm_server.yaml
│
┌─ I want to use a different dataset
│  └─ configs/data/
│     ├─ Use built-in → arc_atlas_rl.yaml
│     └─ Use custom → create new yaml
│
┌─ I want to train teacher from scratch (advanced)
│  └─ configs/run/ + configs/trainer/
│     └─ See Architecture docs for details
│
└─ I just want to test quickly
   └─ configs/examples/quickstart.yaml

Next Steps

Wrap Your Agent

See complete example of wrapping an existing agent

Run Optimization

Use GEPA to improve teaching prompts

Advanced Config System

Deep dive into Hydra composition for training

Production Deployment

Deploy optimized configs at scale

Getting Started

Core Concepts

Guides

Examples & Case Studies

Integration

Training

Benchmarks

Architecture

API Reference

Reference

Configuration Guide

Quick Reference

Common Configurations

Wrapping Your Existing Agent

Online Optimization Settings

Dataset Configuration

Configuration Patterns

Pattern 1: Quick Start with Defaults

Pattern 2: Environment Variables

Pattern 3: Compose Configs with Hydra

Key Configuration Fields

Troubleshooting

Quick Decision Guide

Next Steps

Wrap Your Agent

Run Optimization

Advanced Config System

Production Deployment

Getting Started

Core Concepts

Guides

Examples & Case Studies

Integration

Training

Benchmarks

Architecture

API Reference

Reference

​Quick Reference

​Common Configurations

​Wrapping Your Existing Agent

​Online Optimization Settings

​Dataset Configuration

​Configuration Patterns

​Pattern 1: Quick Start with Defaults

​Pattern 2: Environment Variables

​Pattern 3: Compose Configs with Hydra

​Key Configuration Fields

​Troubleshooting

​Quick Decision Guide

​Next Steps

Wrap Your Agent

Run Optimization

Advanced Config System

Production Deployment

Quick Reference

Common Configurations

Wrapping Your Existing Agent

Online Optimization Settings

Dataset Configuration

Configuration Patterns

Pattern 1: Quick Start with Defaults

Pattern 2: Environment Variables

Pattern 3: Compose Configs with Hydra

Key Configuration Fields

Troubleshooting

Quick Decision Guide

Next Steps