Apple RISED: Rubric-guided data selection and self-distillation for multi-environment agents

Apple researchers introduced RISED, a training method for LLM agents that operate across multiple interactive environments. The approach uses rubric-based textual feedback alongside scalar rewards to guide data selection, policy supervision, and rollout generation. In tests across model backbones, RISED achieved the highest mean pass rate across environments and ranked first or second in every individual environment, according to the paper published in October 2026.

Rubrics as shared vocabulary

The core idea is a predefined rubric vocabulary shared across environments. An LLM judge tags each rollout using this vocabulary, producing a profile that describes the rollout's behaviours. Positive rubrics describe desired behaviours, while negative rubrics describe undesired ones. These profiles serve three purposes: they guide selection of data that aligns with the overall behavioural composition of the mixed-environment batch, they provide context for an on-policy self-distillation teacher that supplies token-level supervision, and they steer subsequent rollout generation away from recurring failure modes.

The motivation comes from two limitations of scalar rewards in multi-environment RL. First, scalar rewards provide limited information about cross-environment relationships, making it hard to compare rollouts across different environments. Second, when environments are learned at different rates, a batch can contain all-failure and all-success rollout groups, leaving those data without group-relative reward signals because the rewards are identical within each group. Rubrics add within-group contrast and cross-environment context, supplementing the scalar signal.

Selection and supervision mechanics

During training, the LLM judge tags each rollout with the rubric vocabulary. The selection step uses these profiles to pick data that matches the behavioural composition of the mixed-environment batch while limiting overlap with already-selected data. This addresses the problem of curriculum and data-selection strategies that allocate training at the environment level or prioritize local reward-based signals without considering relationships between current rollouts across environments for prompt-group selection.

The self-distillation component uses available positive rubrics as privileged context for an on-policy teacher. This teacher supplies additional token-level supervision beyond the scalar reward, helping the policy learn from successful behaviours. Negative rubrics guide rollout generation away from failure modes, effectively using the judge's feedback to shape future exploration.

Implications for agent training

For teams training agents across multiple domains, RISED suggests that textual rubrics can supplement scalar rewards in RL pipelines. The shared vocabulary across environments means the judge can tag rollouts consistently, enabling cross-environment comparisons that scalar rewards cannot provide. The self-distillation teacher adds a supervision signal that does not require additional human annotation, since it uses the same rubric profiles already generated for selection.

The paper reports results across model backbones. The claim of highest mean pass rate and top-two ranking in every environment is attributed to the authors' experiments. For practitioners, the key takeaway is that rubric-based profiles can serve as a multi-purpose signal for data selection, policy supervision, and rollout shaping in multi-environment RL, potentially reducing the need for hand-crafted reward functions.