Back to journal

AI · · 1 min read

A long document still needs an evidence trail

How to make an AI-assisted document review easier to check, one passage at a time.

A long document still needs an evidence trail

Expanded from my Telegram note. The English edition is a translation and editorial adaptation.

Uploading a long report is only the beginning of a document review. A model can accept a large input without using every part of it equally well.

The research discussed in my Telegram note concerns the effect of information position on model performance. The earlier Lost in the Middle paper also examines how location inside a context can affect retrieval. Results depend on the model and task, so a dramatic result from one experiment should not become a universal rule.

Start with a question

Replace a broad request to analyse the file with a decision you need to make. For example: which customer complaints concern delivery time, and what evidence supports that grouping?

Separate collection from interpretation

Ask for relevant passages across the document before asking for the conclusion. For each passage, request the file name, page when available, and a short quotation. Then open the original and check that it exists and supports the interpretation.

Keep uncertainty visible

If the evidence is incomplete or contradictory, that should appear in the answer. For a large collection, review sections separately and combine the checked extracts afterwards.

This workflow adds review work. It also gives you something more inspectable than a polished paragraph with unexplained confidence. The prompt is a starting point for verification, and the source document remains the reference.

Ali Rashedi