Hardware for local AI: what does a small company need?

Write down the task before choosing the machine. Specify the model artefact, input material, configured context, expected output length and simultaneous active requests. Add the period during which employees need the service and the fallback if it stops.

Keep active generation separate from registered users. A user list does not say how many requests arrive together or how long each occupies the runtime. Use representative prompts and the expected burst when testing. Retain difficult cases that require longer inputs, rather than sizing only from a short demonstration.

Define a successful request in business terms. The output must satisfy the task, source and access requirements, not merely finish without a technical error. Use the knowledge-assistant acceptance guide alongside the performance test so the purchase decision considers both usefulness and capacity.

This article concerns buying or renting capacity. Installation, authentication and shared service deployment should be a separate operating workstream. Keep its owner in the procurement decision even when another person performs the hardware test.

With a discrete GPU, record its VRAM separately from system RAM. Do not add those capacities and label the sum GPU memory. Ollama's FAQ distinguishes system-memory and GPU-memory use; inspect actual loading and offload behaviour for the selected runtime.

The Mac mini specifications and Mac Studio specifications list unified memory for the Apple candidates below. That is a different configuration from a separate graphics card and system memory, but the full pool should not be treated as freely available model capacity. Leave room for the operating system and runtime, then measure the loaded application.

Ask the supplier or trial operator to report the memory categories explicitly. A quotation that lists only a large system RAM figure does not resolve whether the chosen model and workload fit the GPU path. Likewise, a machine with more unified memory is a candidate to test, not evidence of acceptable throughput.

Hardware and model values as of October 2026: Qwen3-8B's model card reports 8.2 billion parameters and 32,768 native context tokens. The name is rounded, and the context specification is not a machine-capacity guarantee.

The vendor lists the Q4_K_M file at 5.03 GB and Q8_0 at 8.71 GB. These are artefact sizes, not full runtime-memory requirements. Inspect the exact file you intend to load.

For a transparent calculation, assume uniform precision across the documented parameter count. At an assumed 4 bits per parameter, 8.2 billion × 4 / 8 = 4.1 GB of ideal weight-only storage. At an assumed 16 bits, the result is 16.4 GB, using decimal GB. These are illustrative figures, not the encoding specification of Q4_K_M.

The actual quantised file above is larger than that ideal lower-precision calculation. Runtime allocations must also accommodate the chosen context and cache. Use the calculation to understand why a parameter-only estimate is incomplete, then measure the real artefact.

Ollama documents that parallel requests affect context-memory allocation and provides settings for parallelism and KV-cache quantisation. Weight quantisation and cache quantisation are separate choices. Record both rather than assuming the downloaded weight format settles all memory use.

The documentation also needs care. The FAQ gives a blanket default context, while the dedicated context guide describes VRAM-dependent defaults. Do not size a purchase from an assumed default. Set context explicitly during the test and verify the behaviour of the pinned runtime release.

Check the memory reserve and supported workload for your chosen configuration. Ask for peak memory, offload or swap behaviour and completed-request timing under your intended concurrency. If a configuration fails, reduce the agreed workload or test another candidate; do not turn an isolated success into a shared-use promise.

As of October 2026: the following complete-machine candidates use Apple's German announcement prices dated 25 August 2026, including German VAT. They are starting prices, not current checkout quotations. Hardware compatibility and application performance still need testing. Use the tiers to organise your evaluation of purchase candidates.

These rows exclude peripherals, a backup target, UPS, deployment labour, service cover and redundancy. Current checkout prices and delivery dates remain to be confirmed. The largest candidate is not established as a requirement for routine office drafting, and the smaller candidates carry no concurrent-user guarantee.

A Linux/CUDA tower is a separate candidate. NVIDIA specifies 32 GB GDDR7 for the RTX 5090, 575 W total graphics power and 1,000 W required system power. Those are specifications as of October 2026, not a complete server quotation or measured average wall consumption.

Request a current EU quotation for a complete tower with compatible CPU, motherboard, RAM, disks, enclosure, cooling, power supply and service cover. A graphics-card price alone should not stand in for the purchase cost of the working system.

We recommend the following acceptance test. Pin hardware, operating system, drivers, runtime, model digest, weight quantisation, cache format, context and prompt set. Use the same task cases if comparing cloud execution.

As illustrative test inputs, trial 1, 2, 4 and 8 concurrent requests, explicitly configured contexts of 4,096 and 16,384 tokens, and a 512-token output ceiling. These values are test-design assumptions, not defaults or universal recommendations. Stop at the failure boundary instead of continuing to overload the service.

Record cold and warm starts, peak memory, swap or offload, time to first token, queue delay, completed-request latency and errors. Include the expected burst and sustained working period in repeated runs. Measure wall power and noise during those runs rather than copying a specification into the results.

For decode rate, Ollama exposes eval_count and eval_duration, with generation duration in nanoseconds. The calculation is eval_count / (eval_duration / 1,000,000,000) tokens per second. It excludes queueing and prompt processing, so also measure the complete request.

Agree task-specific latency and throughput thresholds beforehand. We recommend reporting the 95th percentile and the distribution, not only the mean. Reject candidates that lose requests, exhaust memory or fail quality and access tests. Retain the failing case so a configuration change can be checked against it.

Test reboot, interrupted requests, model reload and restoration of application data and configuration. Hardware specifications alone do not establish uptime or recovery performance. Ask the eventual operator to perform the restoration using the written instructions before accepting the system.

Apple specifies 155 W maximum continuous power, an operating range of 10–35 °C, and 5 dBA at the user position while inactive for the Mac mini, as of October 2026. The power value is not a measured inference average; inactive noise is not a promise about AI-load noise. Source conditions apply.

Likewise, the NVIDIA graphics-power and system-power figures above are not additive consumption readings. Measure the whole system at the wall during idle and loaded periods. Check ventilation and actual noise at the intended location before treating an office desk as a suitable permanent site.

For a hypothetical energy worksheet, 0.100 kW × 160 hours × EUR 0.30/kWh = EUR 4.80 monthly, using 16 kWh. All inputs are assumptions as of October 2026, not hardware measurements or a national electricity tariff. Replace them with measured power, actual powered-on hours and your site tariff.

Plan controlled physical access, backup scope, restoration and retention. The EDPB security guide includes premises and backup planning. Define which application state and configuration must be recoverable. A backup plan should have a tested restoration procedure and an owner.

Rental is a candidate when you want to test a discrete-GPU stack before purchase or expect temporary demand. Compare billed hours, storage, networking, support, operating labour and recovery. Do not assume the visible compute rate covers the complete deployment.

As of October 2026, Scaleway's public comparison page lists an L4 with 24 GB VRAM, starting at EUR 0.79 per GPU-hour, Paris availability and per-minute billing. Exact SKU, capacity, tax treatment, extras and stopped-state billing still need confirmation.

Using that starting rate with illustrative monthly hours, 160 × EUR 0.79 = EUR 126.40, while 730 × EUR 0.79 = EUR 576.70. These are compute-only sensitivity calculations, not quoted monthly bundles. Check which hours the selected service actually bills.

Hetzner's current matrix lists GEX45 with 24 GB GPU memory and GEX131 with 96 GB, with published German and Finnish location options. Confirm current setup and monthly rates, the specific configuration and location before comparison.

An EU machine location does not by itself approve the surrounding application's data handling. Inventory copies, access and deletion using the document-processing privacy guide. Assign workflow and recovery ownership with the self-hosting guide.

Keep the selected configuration, dated quotation, test inputs, failed and passed cases, operating owner and recovery evidence together. Record unresolved compatibility or performance questions as conditions to resolve, not promised capabilities. Compare successful work and the full operating budget before choosing a tier.

The RAG cost guide helps include application work and knowledge maintenance beyond the machine. Training belongs within the project: the operator needs recovery instructions, and employees need the permitted scope and fallback. Purchase capacity for the workflow that passed, then test again before expanding it. See our guide to local AI for further planning.