Document processing

Detect duplicate invoices with business keys and review

Reading time
6 minutes
Published
Written by
modernagents editorial teammodernagents is an AI automation agency for SMEs.
Illustration: a magnifying glass over two matching invoices, a raised hand stops the letter

Reliable duplicate-invoice detection combines several checks: identical files, matching business identifiers and similar documents with conflicting details. AI can assist with ambiguous cases, but blocking an invoice, connecting it to an existing case or allowing it through requires understandable rules and a controlled review process.

Key takeaways

  • A file hash detects identical bytes and is a useful first filter for exact repeats, but it does not reliably identify a newly scanned copy.
  • Invoice number alone is insufficient; matching needs business context such as company, supplier, amount, currency and date.
  • Matching identifiers with a changed amount create a hold for business review; the earlier version is not overwritten silently.
  • Every warning should explain what matches and what differs, showing both originals and the existing processing state.
  • When a response is missing after submission, first establish whether the transaction already exists before creating a new record.

01 / 08

Why are filenames and hashes insufficient?

File hash. The same invoice can arrive as a PDF, scan and photograph. The files differ while representing one transaction. A file hash detects identical bytes, making it a useful first filter for exact repeats. It does not reliably identify a newly scanned copy of the same invoice.

Invoice number. Invoice number alone is also insufficient. Different suppliers can use identical numbers, numbering schemes can change and records belonging to separate companies must not be merged. Matching needs business context.

ERP practice. ERP systems reflect this distinction. SAP's duplicate-invoice documentation identifies fields such as supplier, company code, reference, date and amount as relevant checks or configurable criteria. SAP Duplicate Invoice Check, checked on 26 September 2026. Actual behaviour depends on the product and configuration and needs testing in your environment.

02 / 08

Which layers of checking does duplicate detection need?

Duplicate-invoice detection works in five layers, each with its own comparison and intended outcome:

Layer Comparison Intended outcome
File Same hash in the same intake context Connect an exact repeated submission
Business key Company, supplier and invoice number Inspect an existing transaction
Supporting values Amount, currency and date Confirm a match or flag a conflict
Similarity Different spelling, number or scan Create a reviewable suspicion
Destination state Captured, approved or paid already? Choose an action appropriate to that state

Original and normalised values. Retain original values alongside normalised comparison values. Whitespace and case can be standardised for search, but do not remove leading zeroes or meaningful number components indiscriminately. Aggressive normalisation can make genuinely different invoices appear identical.

Explain every warning. Every warning should explain what matches and what differs. “AI says duplicate” is not enough for accounting to decide. Show both originals, the compared fields and existing processing state together.

Search scope. Consider how far back the comparison should search. Restricting it to today's inbox can miss a repeated invoice already processed last month. Conversely, an unrestricted number-only match across years and suppliers will create unnecessary warnings. Agree the search scope with accounting.

03 / 08

How should repeated, corrected and recurring invoices and credit notes be handled?

Repeated, corrected and recurring invoices and credit notes each need different handling:

  • Repeated submission: someone forwards the same invoice because they did not know it had already arrived. Connect the new intake event to the existing case, preserve provenance and avoid preparing another payment.
  • Corrected invoice: the reference is unchanged but the amount or lines differ. This needs business review. Do not overwrite the earlier version silently. Record which version applies and how any processing already performed must be corrected.
  • Recurring service: monthly invoices from one supplier can have the same amount. Without service period or an appropriate reference, blocking solely on supplier and amount is too broad. Similarity should initially create a review task.
  • Credit note: Credit notes also need separate handling. Referring to an earlier invoice number does not make a credit note a duplicate of the original invoice. The sign of an amount does not fully establish the business meaning either.

04 / 08

Synthetic example with four incoming documents

A company receives supplier A's invoice R-2026-081 for EUR 1,200. Two days later the same invoice arrives as a scan. Supplier B then submits its own R-2026-081 for EUR 900, followed by A sending a corrected version for EUR 1,150. This invented example is intended for rule testing.

The four incoming documents of this invented example lead to four different actions:

Incoming document Matching evidence Resulting action
Supplier A, R-2026-081, EUR 1,200 First invoice Creates the case
Same invoice as a scan, two days later Different hash; business key matches; amount and currency support the match Connected to the existing case
Supplier B, R-2026-081, EUR 900 Supplier identity differs Separate transaction
Supplier A, corrected version, EUR 1,150 Matching identifiers with a changed amount Hold; the owner checks whether the original was posted or paid and determines the accounting correction required

The first invoice creates the case. The scan has a different hash, but its business key matches. Amount and currency support the match, so the additional submission can be connected to the existing case. Supplier B's document is a separate transaction because its supplier identity differs.

The corrected version from A is neither discarded nor accepted automatically. Matching identifiers with a changed amount create a hold. The owner checks whether the original was posted or paid and determines the accounting correction required. The document history remains available.

Include all four cases in acceptance. Testing only two identical PDFs proves the easiest part while leaving the important business decisions unresolved.

Got a workflow like this?

Tell me about it. Honest take and a rough price, usually within a day.

AI document processing →

05 / 08

How do you prevent duplicates caused by technical retries?

Why retries create duplicates. A workflow can submit an invoice twice even when it arrived only once. A timeout after destination creation is a typical cause. Document checking therefore needs a stable case identifier and deliberate retry behaviour alongside it.

Submission state and destination identifier. Store submission state and the acknowledged destination identifier. When a response is missing, first establish whether the transaction already exists. Repeating a request must not automatically mean creating a new record. Also test concurrent intake: two simultaneous runs must not both decide that no record exists using stale information.

Depends on the interface. The implementation depends on the actual interface. Ask the provider to explain uniqueness, repeated delivery and destination reconciliation. OCR capability alone does not solve this operational problem.

Test an interruption. Include a deliberate interruption in testing. The operator should be able to identify the uncertain case, inspect the destination and resume without losing the original or creating another invoice. A process that requires deleting records blindly is not ready for unattended operation.

06 / 08

How do you measure protection and review burden?

Measures. Track true duplicates found, duplicates missed and legitimate invoices incorrectly held separately. A high number of warnings may simply indicate overly broad rules. Add review time and the age of unresolved holds.

Review screen. The review screen needs an owner, clear decisions and a recorded reason: connected to existing case, independent transaction, corrected version or unresolved. A decision to release a suspected duplicate should remain explainable. Do not delete documents automatically because a rule found similarity.

Where to start. Start with the most common intake channels and include records awaiting approval as well as already posted invoices. The goal is an understandable defence against repeated processing that still allows legitimate invoices to proceed and preserves the history of corrections.

07 / 08

Take the worksheet

Worksheet

Download the editable CSV table

LayerDetectsRemaining gap
File hashIdentical bytesSame invoice in another format
Business keySupplier and invoice number within tenantMissing or misread number
Similarity reviewSuspicious dates, amounts and linesWhether similar documents are actually duplicates

Columns to fill in, in the file: Your value / observation, Owner, Next action.

Download the editable CSV table (CSV). Add your own values, owners and open actions. This worksheet contains no measured client results.

08 / 08

These guides and service pages cover the decisions next to duplicate-invoice detection:

For a workflow review, start with the task, systems involved and typical exceptions. Discuss it with modernagents.

ShareEmail

Related reading

AI document processing

Got a workflow like this?

Tell me about it. Honest take and a rough price, usually within a day.

Robert Van Ysendyck · Founder & Managing Director