There's been a lot of new releases about documents. Here's the lightning version for the 5 of them (for more detail and videos go to my news feed).

Also - Joe Barrow and I will be livestreaming a dive into post processing for OCR and the recent & related paper, Miner U Popo, tomorrow. Sign up here if you want a calendar invite with a link so you don't miss it.

PP-DocLayoutV3: Reading Order and Warped Text

Layout parsers often work sequentially (figure out block 1, then block 2, then block 3, etc.). In this model they predict which block is first, for each pair of blocks. Then they sort based on how many likely blocks are before it to get reading order. And they do block calculation and reading order predictions in 1 model (OmniDocBench saw ordering errors go from 0.189 to 0.041 by combining)

They also do segmentation which means the bounding boxes can be curved, angled, or warped. Very useful if photos aren't straight on (phone pic of an ID, security cam, robotics) or if the actual surface isn't straight (crooked scans, OCR on shampoo bottle, etc.)

Index documents with a large model, query with a small one

Perplexity released a 9B and 0.6B embedding models. Normally you must use the same model that you embed your documents with to embed your queries when users search. But in this case the models are trained to be compatible so you can use the big one for documents in advance, but the fast one for snappy user queries or queries on device.

OmniParseBench: look at the pages, not just the scores

Datalab's new benchmark gives separate scores for tables, handwriting, and handwritten tables, and shows how many tests each score is based on. They publish the data, tests, and scoring code. I like this benchmark a lot, but you still need to look at the pages and see whether they look like your documents. A bit of a grain of salt because it's parsing vendor making a parsing benchmark, but it looks like they are trying really hard to be fair.

MinerU-Popo: headings need the whole document

Most approaches, especially modern VLMs, read a PDF one page at a time. But PDF pages aren't really independent like that, so it causes errors like incorrect heading levels, or broken tables that span pages. MinerU-Popo works out which extracted text, tables, and images belong together across pages. For retrieval, that gives you a section with all the supporting material. Reminder, join our livestream for more details on this and what cases specifically it helps.

Xiaomi: a council of OCR models

Xiaomi used a council of eight OCR models to create training data for their OCR model. Basically, disputed examples go through extra Qwen checks for repair, then back through the agreement check, with human review if agreement is still too low. I like that they're not assuming their repair path works, but agreement doesn't really mean they're agreeing on the right answer.

You can find the full takes and sources here: https://isaacflath.com/news​

- Isaac