Researchers introduced Periscope, a way to make a frozen language model search very long texts without re‑training it. The technique splits a document into chunks, arranges them on a square grid, and asks the model to evaluate two kinds of spans: local groups of consecutive chunks and strided groups that sample the whole text. By combining the scores from these spans, Periscope builds an “evidence map” that points to the chunk most likely to contain the answer. The whole process needs only about √(s·c) tokens per probe, where s is the document length and c the chunk size, instead of reading the entire text in one pass.
How well does it work?
On the LongBench v2 benchmark, Periscope’s map ranked highest when reading only 9 000 tokens, matching the best window‑read performance of the same model across windows from 32 000 to 1 000 000 tokens. On InfiniteBench, where the median document is about 150 000 tokens, the method beat the best window read by five points. It also achieved the top NDCG@10 score among six methods on the BRIGHT long‑document corpora.
Why it matters for users
Periscope lets a 27 billion‑parameter model handle 4.5 million‑token contexts on a single 80 GB GPU. Previously, a single forward pass over that much text would need around 296 GB of GPU memory just to cache the tokens. This means users can run very long‑document queries on cheaper hardware, as the GPU only needs to store the model, not the entire text.
Why it matters
For developers and researchers, Periscope reduces the hardware cost of working with very long documents, enabling larger context analysis on standard GPUs. For end users, this could lead to faster, cheaper services that need to search or summarize massive texts, such as legal archives or scientific literature. Independent validation beyond the paper’s benchmarks is still pending.
