Tier 1: Fast ensemble evaluation → Tier 2: Expert arbiter when needed
### How It Works
**Tier 1: The Initial Team**
* Multiple efficient models (like `gemini-2.5-flash`) run in parallel
* Each runs at different temperatures for diverse perspectives
* They all score the same interaction independently
* Fast and cheap for most cases
**Tier 2: The Expert Arbiter**
* Only called when the team disagrees (high variance in scores)
* Or when any judge reports low confidence
* A more powerful model (like `gemini-2.5-pro`) reviews everything
* Makes the final decision with full context
**The key insight**: Most cases are clear-cut and don't need the expensive expert. When there's genuine ambiguity, escalate to the specialist.
### When Escalation Happens
The system escalates to Tier 2 when either:
* **High disagreement**: Standard deviation of scores exceeds the threshold (default: 0.15)
* **Low confidence**: Any judge reports high uncertainty (default: >0.3)
Otherwise, it uses the most confident judgment from Tier 1—saving both time and money.
## Session-Level Evaluation
The reward system evaluates the complete trajectory after execution finishes. It derives 2-3 weighted principles tailored to the specific session, scores the trajectory against those principles, and extracts behavioral patterns for future learning.
### How It Works
The evaluator receives the full session context—task, plan, steps, final answer, execution mode—and generates a structured evaluation:
```json theme={null}
{
"principles": [
{"name": "Correctness", "weight": 0.5, "description": "Final deliverable matches requirements"},
{"name": "Safety", "weight": 0.3, "description": "No policy violations detected"},
{"name": "Efficiency", "weight": 0.2, "description": "Minimal retries needed"}
],
"score": 0.85,
"rationale": "Response solves the task correctly with efficient execution",
"uncertainty": 0.1,
"student_learning": "For straightforward tasks, proceed directly to solution without exploratory steps",
"teacher_learning": null
}
```
**Key components:**
* **Principles**: Domain-relevant evaluation criteria with weights (sum to 1.0)
* **Score**: Aggregated result in \[0.0, 1.0] range
* **Rationale**: Explanation grounded in the principles
* **Student learning**: Cross-domain behavioral pattern to remember (not task-specific content)
* **Teacher learning**: Pedagogical strategy that worked (when teacher provided guidance)
This makes every score fully auditable—you can see which principles were applied and why the judgment was made.
## Defining Domain Objectives
The judge prompt system lets you express quality criteria in natural language without training custom models. The evaluator derives 2-3 weighted principles tailored to each trajectory, scores against those principles, and reconciles multiple judge opinions through the ensemble flow.
**How it works:**
The `focus_prompt` field in `adaptive_teaching.reward` accepts arbitrary evaluation criteria. The judge reads that prompt, generates domain-relevant principles (e.g., "Correctness: 0.5 weight", "Safety: 0.3 weight"), evaluates the trajectory, and extracts behavioral patterns to store as learning memory.
## Configuration Essentials
The reward system is configured via YAML, but you only need to understand a few key settings:
### Core Settings
```yaml theme={null}
# reward_system/interpretation.yaml
rim:
# Diversity: More temperatures = more diverse initial opinions
temperatures: [0.2, 0.5, 0.8]
# Escalation sensitivity
variance_threshold: 0.15 # Lower = more escalations to expert
# Which dimensions to evaluate
active_judges:
accuracy: true
helpfulness: true
process: true
diagnostic: true
```
### Key Tuning Knobs
**Want more precision?**
* Lower `variance_threshold` to 0.10 → More cases go to the expert model
**Need faster/cheaper evaluation?**
* Raise `variance_threshold` to 0.20 → Trust the initial team more often
* Reduce `temperatures` to `[0.3, 0.7]` → Fewer ensemble members
**Different use cases?**
* Adjust `variance_threshold` and `uncertainty_threshold` to control escalation frequency
* Use `focus_prompt` to steer evaluation criteria toward specific domain objectives
## Reward System in the Atlas SDK
The SDK runtime uses the same reward philosophy to control its execution loop. The `rim` block in [`configs/examples/openai_agent.yaml`](https://github.com/Arc-Computer/atlas-sdk/blob/main/configs/examples/openai_agent.yaml) wires up the scorekeepers and escalation model:
```yaml theme={null}
# configs/examples/openai_agent.yaml
rim:
small_model:
provider: google
model: gemini/gemini-2.5-flash
api_key_env: GEMINI_API_KEY
max_output_tokens: 8096
large_model:
provider: google
model: gemini/gemini-2.5-flash
api_key_env: GEMINI_API_KEY
max_output_tokens: 8096
judge_prompt: 'reward the agent for attending the issues mentioned in the task'
variance_threshold: 0.15
uncertainty_threshold: 0.3
```
During orchestration, this configuration tells the runtime how to behave:
1. After execution completes, the session trajectory is evaluated using the **small model** at multiple temperatures for diverse perspectives.
2. If variance across samples exceeds 0.15 or any sample reports uncertainty > 0.3, the system escalates to the **large model** arbiter.
3. The final reward includes derived principles, score, rationale, and extracted learning patterns (student\_learning, teacher\_learning).
4. The reward informs retry decisions and learning memory—patterns are stored for future sessions.