# Past Decisions Can Skew AI Refund Approvals

> A study found that changing past cases’ decisions and changing their facts can push language models’ refund approvals in opposite directions.

Oossa · 2026-11-03 · https://oossa.com/en/past-decisions-can-skew-ai-refund-approvals

A study posted on arXiv tested how three language models handle close-call refund and claim requests when shown three similar past cases. For DeepSeek V4.1 Flash, GPT-5.6 Luna and Claude Haiku 5.5, changing the cases’ recorded decisions from DENY to APPROVE raised approval rates by about 30 to 55 percentage points on study-written policies.

But when DeepSeek and Luna saw only the past cases’ facts, cases with facts supporting approval made them approve less often than cases supporting denial. The authors say this pattern also appeared on nine public policies paraphrased from real ones.

## The facts

- The study tested DeepSeek V4.1 Flash, GPT-5.6 Luna and Claude Haiku 5.5.
- On a separate DeepSeek run, changing decisions on cases resembling a different request shifted approvals by 9 percentage points, versus 43 when cases resembled the request.

## Why it matters

Organizations using AI to assess refunds or claims should test past-case facts separately from recorded outcomes, because either can shift decisions and the direction of the effect may differ.

## Sources & references

1. [Precedent Labels and Facts Shift LLM Approval Decisions in Opposite Directions](https://arxiv.org/abs/2611.00672) – arXiv, 2026-11-03

Last updated: 2026-11-03