A confidence score is a number an AI attaches to a result to say how sure it is: that this field is the invoice total, that this knot is dead, that this word is spruce. Microsoft's documentation for its document extraction service puts it this way: "A confidence score indicates probability by measuring the degree of statistical certainty that the extracted result is detected correctly." It suggests using it "to determine whether to automatically accept the prediction or flag it for human review".
How it is used
The usual pattern is a threshold. Results above it are accepted automatically, and results below it go to a person. Set the threshold high and people check more. Set it low and more errors pass. The right level depends on what an error costs.
Calibration
A score works only if it is calibrated: a field marked 90% confident should be right about nine times in ten. That isn't guaranteed. A 2017 study by Guo, Pleiss, Sun and Weinberger found that "modern neural networks, unlike those from a decade ago, are poorly calibrated", in a study of image classification. Extractors trained on one business's standard forms can do better. General models reading unfamiliar pages may do worse. A 2026 benchmark found that "63-91% of model errors are prior-driven" when models read handwriting, meaning the output reflected expectation rather than the page. A score alone doesn't say which errors those are.
In a timber business
Scores help on documents such as scale tickets, delivery notes and invoices, where uncertain fields can go to a person with the page beside them. For volumes, weights and amounts, add a second check that doesn't rely on the model. Does the ticket weight match the scale record? Do the invoice lines add to the total?
What to check
Test calibration on a sample checked by people: group results by confidence and see how often each group was actually right.
Quarri for forest management is built around how a forest operation runs, from the cruise to the settled account.
Sources
- Microsoft Learn, "Interpret and improve accuracy and confidence scores", last updated 8 April 2026: learn.microsoft.com
- Guo, Pleiss, Sun and Weinberger, "On Calibration of Modern Neural Networks", arXiv 1706.04599, 2017: arxiv.org
- Zhang and others, "WildHandBench", arXiv 2608.22959, 24 August 2026: arxiv.org
Quarri is an AI-native data platform for the timber supply chain. It connects buying, production, sales and inventory for forest management, sawmill, wood products and pulp, paper and packaging operators.