RiVER: RL Without Ground Truth Beats Answer-Key Training

On June 25, 2026, researchers posted a paper on arXiv introducing RiVER, a framework that trains LLMs on score-based optimization tasks without ground-truth solutions. This breaks the fundamental assumption that RLVR requires a correct answer to assign rewards, potentially reshaping how models learn subjective tasks.

RiVER (Ranking-induced VERifiable framework) trains LLMs without ground-truth answers, using execution feedback instead.
It applies group-relative RL to continuous-valued rewards from deterministic executions, unlocking score-based optimization tasks.
The paper challenges the assumption that RLVR requires a single correct answer, widening RL applicability to subjective domains.

How Does RiVER Assign Rewards Without a Ground-Truth Answer?

According to the arXiv paper published June 25, 2026, RiVER replaces the binary reward of standard RLVR—which checks if an output matches a stored answer—with a continuous-valued signal derived from deterministic execution feedback. For example, in code optimization, the reward is the runtime of the generated code; in a logistics task, it could be the total cost of a routing plan. The authors state that this feedback is “deterministic” and “execution-based,” meaning it can be computed programmatically without human labeling. This shifts the reward function from a categorical check to a scalar score, enabling RL on tasks where multiple valid solutions exist but differ in quality.

Why Does This Break the Ground-Truth Bottleneck in RLVR?

RiVER: RL Without Ground Truth Beats Answer-Key Training

Standard RLVR, as deployed by OpenAI and Anthropic in systems like GPT-4o and Claude 3, requires a verifiable ground-truth answer—such as a math problem’s numeric result—to assign a pass/fail reward. The arXiv authors argue this “limits applicability to tasks where the ground-truth solution is unknown or subjective.” RiVER’s innovation is that it uses group-relative comparisons: it samples multiple outputs from the model for the same input, executes them to get scores, then ranks them. The top fraction gets positive reinforcement; the bottom gets negative. This eliminates the need for any external answer key. The paper claims this method improves LLM performance on score-based optimization tasks compared to baselines that use ground-truth rewards.

What Kinds of Tasks Does RiVER Unlock?

The paper focuses on score-based optimization tasks—problems where the goal is to maximize or minimize a measurable outcome. Examples include code compilation speed, route efficiency, and chemical synthesis yield. According to the source material, these tasks “cannot be addressed by conventional RLVR because they lack a single correct answer.” RiVER’s approach means that any task with a deterministic evaluation function—a program that can score outputs—becomes a candidate for RL training. This includes subjective quality metrics like human preference scores, as long as the scoring function is consistent.

Comparison: RiVER vs. Standard RLVR

Feature	Standard RLVR	RiVER
Reward type	Binary (pass/fail)	Continuous scalar
Ground truth required	Yes (exact answer)	No (execution feedback)
Applicable tasks	Math, QA, factual recall	Code opt., logistics, design
Training signal	Individual correctness	Group-relative ranking
Scalability	Limited to answerable tasks	Broad to evaluable tasks
Verdict	Mature but narrow	Emerging but broader

What Remains Uncertain About RiVER’s Practical Impact?

The paper does not report results on large-scale models (e.g., 70B+ parameters) or on real-world proprietary benchmarks. According to the source, the experiments are conducted on “score-based optimization tasks,” but the exact model sizes and datasets are not fully detailed in the summary. Additionally, the deterministic execution feedback assumption may not hold for all tasks—e.g., creative writing where scoring is subjective and inconsistent. The authors also note that group-relative RL can be sample-inefficient, requiring multiple outputs per input to form a ranking. This could increase training cost by a factor of the group size.

My thesis is that RiVER is a genuine breakthrough that democratizes RL for subjective optimization, but its impact hinges on the availability of deterministic evaluation functions. In the short term, code optimization and logistics will be the first beneficiaries because they have clear, programmatic scoring. In the long term, I expect this to pressure companies like OpenAI and Anthropic to invest in execution-feedback pipelines for their RLHF systems, moving beyond simple preference comparisons. The losers are startups that built ground-truth-dependent RL tools—they will need to pivot. My prediction: within 18 months, at least one major cloud AI provider (AWS, GCP, or Azure) will announce a RiVER-like service for code optimization RL.

Predictions

By December 2027, AWS will launch a RiVER-based RL service for code optimization on SageMaker, targeting enterprise DevOps teams.
OpenAI will incorporate execution-feedback RL into GPT-5 by mid-2027, expanding its RLVR capability beyond factual tasks.
The EU AI Office will issue a guidance note by Q1 2028 on the use of execution-based rewards in safety-critical AI training, citing RiVER as a reference.

Article Summary

RiVER removes the ground-truth requirement from RLVR, enabling RL on score-based optimization tasks.
It uses deterministic execution feedback and group-relative ranking to assign continuous rewards.
Tasks like code optimization, logistics routing, and chemical synthesis become trainable via RL.
Uncertainties remain about scalability to large models and subjective scoring domains.
This shifts the competitive advantage to organizations with robust execution-feedback infrastructure.