When Arithmetic Can Be Known, Don't Ask AI

From the WALDHORN.AI Build Archive
Build date: May 29, 2026

Yesterday I added royalty-statement auditing to WALDHORN.AI. The basic problem is easy to describe: a contract says how somebody should be paid, a statement says what happened, and somewhere between those two documents may be an error, an unexplained deduction, the wrong royalty basis, a missing payment, or nothing wrong at all.

At first, this feels like an obvious AI problem. The documents can be messy, the terminology changes, a royalty provision may depend on definitions several pages away, and a spreadsheet may use different labels from the agreement. A deduction can make sense only after reading another section. That kind of ambiguity is exactly where a language model can be useful.

Then I reached the total and realized I was asking AI to do something I did not need AI to do.

Interpretation and arithmetic are different problems

Suppose the system has examined a royalty statement and identified several financial discrepancies. Eventually the analysis needs to express a number: the amount that may have been underpaid.

A language model can produce that number. It can read the surrounding text, reason through the line items, perform arithmetic, and return a plausible total. The question is not whether it can. The question is why I would make it the final authority if the application already has the numerical values necessary to calculate the total itself.

There are questions where the answer requires judgment, and there are questions where the answer can be known. Those should not automatically use the same mechanism.

Royalty statements are not clean mathematical exercises. If every agreement exposed the same variables in the same places and every statement followed one standard format, much of the interesting part of the audit would not need AI either. That is not the world I am building for.

A contract may describe compensation in prose. Definitions may modify later provisions. A royalty may apply to one base but not another. The statement itself may arrive as a PDF, an image, a CSV file, or a spreadsheet with labels chosen by whoever generated it.

Before anything can be calculated, somebody has to understand what the numbers represent. Which contractual term governs this line item? Is the reported amount being compared against the correct obligation? Does a deduction appear to belong here? Is there enough information to make the comparison at all?

Those are questions about meaning, not arithmetic.

Trying to replace all of that with fixed rules would create a different kind of fragility. Legal language is variable, financial documents are inconsistent, and the same economic idea can appear in many forms. I am not trying to remove AI from the audit. I am trying to stop using it after its comparative advantage has ended.

Precision can create false confidence

Numbers have a strange effect inside generated prose. A vague sentence still looks vague. A precise number looks authoritative.

If an analysis says that a provision may materially reduce compensation, the reader understands that some interpretation is involved. If the next line gives a dollar amount down to the cent, the psychology changes. Two decimal places create the appearance of measurement.

Formatting does not create certainty.

The amount may depend on a mistaken interpretation upstream. The underlying values may also have been interpreted correctly while the arithmetic itself is wrong. That second category bothers me because it is avoidable.

Once the software has the relevant numerical values in a structured form, addition and subtraction are no longer interpretive acts. They are computation. I do not need a model's opinion about the result. I need arithmetic.

Today I changed the audit so the financial findings do not exist only inside narrative analysis. The system can represent the relevant financial items separately and independently recompute the resulting total rather than accepting a number simply because the model produced it.

I am deliberately less interested in whether the model can add correctly almost all the time. If software can eliminate that particular failure mode, there is little reason to preserve it.

This is not distrust of AI. It is division of labor.

Deterministic does not mean true

There is an important trap in this idea. A deterministic calculation can be perfectly reproducible and still be wrong.

If the inputs are wrong, deterministic software gives the wrong answer with exceptional consistency. That means separating arithmetic from interpretation does not solve the difficult part of every financial question.

Imagine a contract where the royalty base itself is ambiguous. One definition may appear to include a category of revenue while another provision seems to narrow it. A deduction may be permitted, but the document may leave uncertainty about how it applies in the circumstances shown on the statement.

No calculator can resolve that.

If the system establishes that the expected amount is 100 and the reported amount is 80, calculating the difference is trivial. The difficult question is whether 100 was actually the correct expectation.

That still belongs to interpretation.

This is why deterministic should not become another decorative word for trustworthy. Determinism gives a narrower guarantee: given these inputs and these rules, the system will produce the same calculation. It does not prove that the inputs were interpreted correctly.

The distinction matters because different parts of an audit have different sources of confidence. A number taken directly from a statement has one kind of provenance. A contractual obligation inferred from several clauses has another. A difference calculated between two established values has another. A conclusion about whether that amount is actually owed has another still.

Putting all of those inside one seamless AI answer can make them sound equally certain. They are not.

The model should not certify everything it produces

There is also a structural problem with letting one model own the entire chain. If the same system identifies the discrepancies, calculates them, totals them, and then explains the result, every stage depends on the same source.

An error can travel through the whole answer without encountering anything independent enough to challenge it. The prose reinforces the calculation that came from the same reasoning process, and the calculation reinforces the conclusion that produced the prose. The result may be internally consistent while still being wrong.

Internal consistency is not independent verification.

I would rather introduce a boundary wherever the system has enough information to do so. Once an analysis produces values that can be checked mechanically, ordinary software should take over that part of the task.

This does not solve the entire reliability problem. It solves one part that does not need to remain probabilistic.

That principle is becoming increasingly important to me because AI products have a strange architectural temptation. Once a model is already inside the workflow, it becomes easy to give it everything.

Extract the values, classify them, interpret the provisions, calculate the result, format the answer, explain the answer. A single intelligent operation can feel elegant.

It also makes every part of the system inherit the uncertainty of the most probabilistic component.

There is no prize for using AI everywhere.

Two plus two does not become more sophisticated because a language model returns four. It simply becomes less constrained.

Boring software can be stronger software

Ordinary calculation is not impressive. It does not reason, stream, or produce a thoughtful explanation. It accepts defined inputs and performs a defined operation.

That limitation is exactly why it is useful here.

A calculator cannot reinterpret a clause halfway through the calculation. It cannot decide that one line item is more interesting and pay less attention to another. It cannot produce a different answer because the prose surrounding the numbers changed.

It has almost no intelligence.

For this part of the system, that is a feature.

The broader audit still requires judgment. A royalty tool that cannot understand the agreement behind the statement would be little more than a spreadsheet calculator. But once judgment has done its work, I want deterministic software to take over wherever it legitimately can.

The goal is not maximum AI. The goal is the smallest amount of uncertainty necessary to solve the problem.

That also changes how I think about the final result. If WALDHORN.AI says somebody may be underpaid by a particular amount, I do not want that figure to exist only as the last sentence of a long analysis. The result should be capable of being broken apart.

Which amounts were reported? Which amounts were expected? Which discrepancies contributed to the result? Which parts came from interpretation? Where is uncertainty still present?

A number that can be decomposed is easier to inspect than a number that arrives as a conclusion.

That does not mean exposing the internal machinery of the system. There is a difference between making an outcome accountable and publishing the recipe that produced it. I care about the first.

Certainty should stop where the evidence stops

The deeper I get into royalty auditing, the less comfortable I am with the assumption that every audit should end with one absolute number.

Sometimes it can. If the contractual terms are clear, the statement provides the necessary figures, and the discrepancy follows directly, the calculation may be straightforward.

Other times the correct result may be conditional. If one interpretation of a provision is correct, the financial difference is one amount. If that premise cannot be established confidently, the number should not pretend otherwise.

There is nothing weak about preserving that uncertainty.

In fact, I trust a conditional result more than a system that quietly turns uncertainty in the premise into precision in the output.

Software often feels pressure to collapse uncertainty because people want answers. I understand the pressure. I still think a conditional truth is more useful than an unconditional number built on a hidden assumption.

I started this work trying to make a royalty audit more reliable. I ended up with a rule that feels much broader than royalty statements:

Interpret what requires judgment. Compute what can be known.

The boundary between those two will not always be obvious. Sometimes something that looks deterministic depends on an interpretive premise. Sometimes a task commonly handed to AI can actually be established directly by ordinary software.

The architecture should care about the difference.

AI is useful to me precisely because the world refuses to arrive in clean rows and known variables. Contracts do not. Human language does not. Financial relationships often do not.

But once part of a problem crosses from ambiguity into something the system can establish exactly, continuing to ask AI is not sophistication.

It is giving uncertainty another place to live.