RAG costs: budget for setup, models and knowledge maintenance

Retrieval-augmented generation supplies a language model with material retrieved from a knowledge source. For budgeting, the important point is that retrieval needs usable, current information with appropriate access controls. An existing folder of files is not yet a maintained knowledge base.

The following table lists the cost categories of a RAG project with their billing basis and a question for the proposal.

Split every line into one-off work, recurring baseline effort and usage-based consumption. Otherwise, a proposal with a low subscription can appear cheaper while all source maintenance and subject-matter testing remain with your team. Internal hours belong in the calculation even when they do not produce an additional supplier invoice.

This separation also makes responsibility visible. A hosting provider may keep the application available without checking whether an old policy document has been replaced. A subject expert may maintain the policy without having access to diagnose a failed connector. Both activities need an owner and an allowance in the operating budget.

Source inventory. Start with a source inventory. Record page counts, file types, average length, change frequency, access groups and responsible people. A thousand short, current instructions can need less preparation than a hundred poorly scanned manuals containing conflicting versions.

For an initial RAG cost estimate, capture five quantities:

• pages in the first import
• pages changed each month
• stored text chunks
• monthly user questions
• average model calls per question

Add peak demand and concurrent users. The total number of accounts describes a different workload from the number of people sending requests at the same time.

Representative sample. Check the assumptions against a representative sample. If the sample contains only clean PDFs, it may exclude difficult tables, scanned attachments and duplicated contract versions. Record which content is deliberately outside the scope. This gives you a testable project boundary and prevents every newly discovered folder from silently becoming part of the original commitment.

First import versus updates. Distinguish the first import from ongoing updates. Processing only changed documents can involve much less material than repeating a full import every month. Whether incremental processing is actually implemented should be stated in the technical design. Savings that depend on change detection should not be treated as confirmed before that mechanism works.

Document preparation

Plain-text extraction, OCR and specialist document processing are different services. Their billing units depend on the selected processor. Google's Document AI pricing, for example, distinguishes processing products and volume units. A price for one extraction method is therefore not a universal price for processing every PDF. Source: Document AI pricing.

Put the exact service, billing unit and expected quantity into the cost sheet. Add manual correction of damaged tables or missing metadata as a separate line. If a person must repair the source, that effort remains a project cost even when the automated extraction itself is inexpensive.

Retrieval

Search services also use different pricing models. Microsoft's Foundry IQ/Azure AI Search pricing lists hourly dedicated capacity and a serverless preview billed for compute and storage. Serverless can scale to zero while idle, and additional features may incur separate charges. Specify the region and plan before comparing totals. Source: Microsoft search pricing.

Compare a normal month, a month with a large initial import and a quiet month. A single price per user may describe these situations poorly. Evaluate reserved capacity and usage-based operation against the same requirements for response time, data handling and operational support.

Do not assume that a preview deployment is interchangeable with every established production configuration. The relevant comparison is the configuration that meets your requirements and is available to your organisation, including its limits and support terms. The cheapest visible price row is only useful when it describes something you can actually use.

On 26 September 2026, Google's official pricing lists Gemini 3.1 Flash-Lite, standard service in the Global region, at US$0.25 per million input tokens and US$1.50 per million text output tokens, including billed reasoning tokens. Billing in other currencies uses the corresponding local SKU prices. Source: Google model pricing.

For our own calculation, assume 10,000 questions a month. Each question uses exactly one model call with 6,000 total input tokens and 500 billed output tokens:

The resulting US$22.50 covers only the model consumption in this example. It excludes retrieval, embeddings, document preparation, hosting, support and expert review. Selecting this price row is not a recommendation for the model or evidence that a Global configuration meets your processing-location requirements.

Input includes the context sent to the model and, where applicable, conversation history. If the system rewrites a question, plans several retrieval steps or sends an answer through another model for checking, add those calls. Once a pilot is running, replace the assumed consumption per visible answer with measured usage.

This is also why a short answer is not necessarily a cheap request. A concise paragraph might have required a large context and several intermediate operations. Record the complete request path before deciding where optimisation would make a meaningful difference.

The following euro example is independent of the dollar calculation above. Every amount is an invented planning allowance for a bounded internal knowledge project. It illustrates the structure of a budget; taxes, actual supplier quotations and individual contract terms need separate treatment.

Year one is €7,200 + 12 × €900 = €18,000. A second year with unchanged scope would be €10,800. Expansion, new connectors, migrations and major redesigns are not automatically included.

Consumption allowance. The monthly €100 consumption allowance covers several technical services. It cannot be compared directly with the US$22.50 generation calculation. Currency, workload, included services and actual usage would first need to match.

Sensitivity. Next, test how sensitive the budget is to its assumptions. Increasing knowledge maintenance from €300 to €600 a month adds €3,600 a year. Doubling infrastructure alone from €200 to €400 adds €2,400. This shows which uncertain quantities deserve closer investigation before a purchase decision.

Internal effort. Keep these allowances visible in the purchase proposal. If the project is approved using only external invoices, the team may discover later that nobody has time for source review. A budget that includes internal effort makes that resource decision explicit even when the cash outlay is lower than the total economic cost.

Calculate the cost per task completed correctly. An answer can arrive quickly and still create more work when its source, validity or access context is unclear. Measure search time, verification, rework and questions that still need fully manual handling.

Consider a deliberately conservative invented example. Without the assistant, 600 questions take six minutes each, or 3,600 minutes. With the assistant, the outcomes are:

The difference is 1,080 minutes, or 18 hours. At an assumed internal hourly value of €40, that represents €720 of released capacity. It is below the assumed €900 monthly operating cost, before recovering any setup expense. Claiming rapid payback from these numbers would be incorrect.

Released capacity is not automatically a payroll saving. It becomes economically useful when, for example, the team completes additional work, avoids overtime or reduces a bottleneck. Decide before testing how that use will be recorded and what minimum quality must be maintained.

Keep unsuccessful questions in the measurement. Removing them from the denominator makes the assistant look better while leaving the business with the same unresolved work. It is reasonable for a system to decline questions outside its scope, but the remaining manual effort still belongs in the overall business case.

Source changes. A source change can create work without any increase in user numbers. Examples include permission changes, new file formats, a revised folder structure and replacement of an integrated system. Agree which changes are covered by ongoing operation and which require a separate quotation.

Retesting. Budget for retesting after material changes. A different model or document-chunking approach may reduce consumption while making certain answers worse. Maintain a fixed set of typical and difficult questions, including deliberately unanswerable questions and access boundaries.

Two reserves. Two separate reserves help: one for variable technical consumption and one for additional work. A cloud spending alert does not limit unexpected internal maintenance effort. Conversely, having an available subject expert does not prevent a faulty import from repeatedly processing the same files.

Exit scenario. Include an exit scenario as well. If the assistant is discontinued, the organisation still needs its source material, ownership records and any reusable evaluation set. Clarify the handover effort in advance, especially when a supplier has built custom mappings or transformed documents into a proprietary format.

Request a defined scope rather than a single monthly number. A supplier proposal for a RAG project should:

• identify sources and user groups
• explain update handling
• show consumption units
• describe acceptance

Ask how export and handover work and who investigates an incorrect answer.

A useful comparison includes at least a normal operating month and a higher-load scenario. Ask suppliers to use the same quantities and test questions. A less expensive offer may be a good fit because it deliberately includes less; the boundary needs to be visible and compatible with your team's available capacity.

If Microsoft 365 licences or an existing search platform are already available, first assess the incremental effort of using them. A custom RAG assistant does not become the better option simply because model tokens are inexpensive. The decision depends on whether the necessary sources, access rules and workflows can be supported at a reasonable total cost.

These modernagents guides cover related decisions about AI knowledge assistants:

• AI knowledge assistants: sources, access and answers
• Microsoft 365 Copilot or an additional RAG assistant?
• RAG architecture from source document to checked answer
• AI automation costs: a first-year budget for SMEs

For a useful estimate, bring a source inventory, typical questions and the relevant user groups. Discuss a RAG project with modernagents.