Skip to content
xzx34Public

About

Code and evaluation tools for JevOut: Natural Context Can Flip Decision Models

Topics

Resources

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

JevOut: Natural Context Can Flip Decision Models

Paper on arXiv   Project page   Quickstart

News

  • [10/08/2026] 🎉 JevOut has been accepted to Agenthon @ NeurIPS 2026 as a poster! See you in Atlanta!
  • [10/08/2026] Code v0.2.0 is available, with an offline demo, independent-generation baseline, repeatability tools, and updated experiment documentation.
  • [09/24/2026] Initial code release.

Introduction

Decision models turn language into choices for routers, evaluators, and agents. But can they tell useful context from a distraction? We find that short, natural-looking additions can redirect a correct decision toward a chosen wrong answer, without changing the original task, choices, or correct answer. The additions read as ordinary background or procedural details rather than instructions to change the decision.

JevOut studies this phenomenon across four decision systems and seven datasets. We use probability-guided context optimization to construct answer-preserving additions and measure Targeted Flip Rate (TFR): how often an initially correct decision moves to the wrong option fixed in advance. With a budget of up to 64 accepted target evaluations per decision:

Decision system Initially correct decisions TFR
Jev 508 61.4%
OpenSourceJev 328 72.6%
Von 328 73.2%
Plain Qwen 285 64.9%

The study also examines cross-model transfer, independent human validation, repeatability, and learned context generation. See the results and experiment settings for the full comparisons.

Quickstart

Requires Python 3.12 and uv.

git clone https://github.com/xzx34/JevOut.git
cd JevOut
uv sync --frozen
uv run context-opt demo --output-dir outputs/demo

The demo uses a deterministic synthetic model and needs no API keys or model service. It runs context optimization, independent generation, and repeatability checks, then writes the results to outputs/demo/.

To use Jev or your own decision model, follow the quickstart guide. The package supports Jev, HTTP endpoints, and Python callbacks through a common target interface.

Repository layout

context_optimization/   evaluation, context optimization, controls, and prompts
docs/                   usage guides, experiment settings, and result summaries
examples/               sample decision items and a Python target adapter
tests/                  offline tests

Usage: Quickstart · Input format · Output format · Target adapters

Experiments: Settings · Results · Aggregate JSON

Citation

If you use this work, please cite:

@article{xu2026jevout,
  title   = {JevOut: Natural Context Can Flip Decision Models},
  author  = {Xu, Zixiang and Song, Zirui and Zhang, Chiyu and Chen, Xiuying and Liu, Xi and Hu, Xiyang and Zhao, Yue},
  journal = {arXiv preprint arXiv:2609.30243},
  year    = {2026},
  url     = {https://arxiv.org/abs/2609.30243}
}

Corresponding author: Yue Zhao (yue.z@usc.edu). Code is licensed under Apache-2.0.

About

Code and evaluation tools for JevOut: Natural Context Can Flip Decision Models

Topics

Resources

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages