Study finds asking vision-language models about a director’s view changes their actions

Researchers from arXiv report that vision‑language models (VLMs) behave differently when first asked what a director can see. In a “Director Task” where a large block is hidden from the director, asking the model which blocks the director sees raised the chance of picking the hidden block by 12 to 88 percentage points for six GPT models and by 17 points for Qwen‑3.8‑27B. The models still reported the director’s view correctly, but the prior question shifted their subsequent action toward their own perspective.