L-MARS

L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit

Accepted as a poster at the ICML 2026 AI4Law Workshop.

📄 Paper: L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit

🤗 Dataset: LegalSearchQA

L-MARS is a multi-agent legal question answering system designed for grounded answers over current legal information. It combines structured query decomposition, agentic web search, evidence filtering, and cited answer synthesis, and it is audited claim by claim on whether each cited source is reachable and actually supports the answer. The project also includes optional local retrieval over user-provided documents and CourtListener integration for case-law search.

L-MARS supports two operating modes:

• Simple Mode: a single-pass retrieval pipeline that decomposes the question, searches for evidence, and synthesizes a grounded answer.
• Multi-Turn Mode: an iterative search-and-verify loop that refines queries until the evidence is sufficient or a maximum number of iterations is reached.

The system can use the following evidence sources:

• Web search via Serper
• Local RAG over user-provided documents using BM25
• CourtListener for case-law retrieval

1. Query Agent parses the question into structured search intents (intent, jurisdiction, date scope).
2. Search Agent retrieves evidence from the enabled sources.
3. Judge Agent checks whether the evidence is sufficient (factual support, jurisdiction match, temporal specificity, contradiction) and flags missing information for the next turn.
4. Summary Agent writes the final answer with inline citations and rationale.

Beyond picking the right answer, L-MARS is audited on whether the source each claim cites is reachable and entails the claim. A VerifyAgent decomposes an answer into atomic claims and labels each claim against its cited source with a six-class taxonomy: supported, partially_supported, unsupported, citation_unreachable, no_citation, and verifier_error. Claims are scored with strict-ALCE precision and recall under cross-provider judging, where the answerer and the verifier come from different model families so a model does not grade its own output.

Faith-Search is an optional post-draft step that re-verifies each claim, then repairs an unreachable citation with a newly retrieved source or drops it rather than keep a broken one.

Here "faithfulness" means citation reachability plus snippet-level entailment. This is narrower than doctrinal validity, which would also ask whether the cited authority is still good law.

The paper evaluates L-MARS on two settings:

• Bar Exam QA (stratified 100-question audit): a reasoning-focused benchmark where retrieval barely moves accuracy, used to audit citation faithfulness claim by claim.
• LegalSearchQA (50-question case study): time-sensitive questions that require 2024-2026 legal knowledge, used for external validity.

The headline result is about citation faithfulness rather than accuracy. On Bar Exam, answer accuracy stays flat across retrieval settings, while the multi-turn judge loop lifts strict citation F1 from 0.13 (naive RAG) to 0.25 and cuts the no-citation rate from 34% to 13%. Faith-Search drops the unreachable-citation rate below 1%. All comparisons are reported as relative deltas under one fixed verifier; a blinded human pass is used as a stress test rather than as ground truth.

pip install -r requirements.txt

Quick legal research with online search only:

python main.py "Your legal question"

Enable offline RAG for local documents:

python main.py --offline-rag "Your legal question"

Enable all sources (offline RAG + CourtListener + web search):

python main.py --all-sources "Your legal question"

Verbose output:

python main.py -v "Your legal question"

Run iterative research with refinement:

python main.py --multi "Complex contract dispute..."

Set a custom number of iterations:

python main.py --multi --max-iterations 5 "Your question"

If you are reproducing the paper's evaluation pipeline:

python run/single_turn_pipeline.py \
  --dataset legalsearchqa \
  --model openai:gpt-4o-mini \
  --use-cache true \
  --output results/lmars_preds.jsonl

python eval/run_eval.py \
  --preds results/lmars_preds.jsonl \
  --judge-sample 20 \
  --llm_model openai:gpt-4o-mini

If you use L-MARS in your research, please cite:

@misc{yuan2026lmarslegalmultiagentagentic,
      title={L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit}, 
      author={Boqin Yuan and Ziqi Wang},
      year={2026},
      eprint={2509.00761},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2509.00761}, 
}