Three Ways to Run a Private AI for Your Business
Your staff are already using AI. Not the one you chose. It is the one in a browser tab you can’t see, fed whatever they happened to paste into it: a customer’s name, a contract clause, an invoice, a password someone was resetting. In an earlier post we argued the AI chat terminal is becoming the single interface for the whole business. If that is where work is heading, the question stops being whether your company runs on AI and becomes whose infrastructure it runs on, and how much of your data has to leave the building to get there.
You can own that terminal. There are three practical ways to do it today, on a spectrum from cheap-with-caveats to fully sovereign. They differ on one axis that matters more than any feature: where your sensitive data is allowed to stop. Decide that boundary first; the setup follows from it.
flowchart LR
A["1 · Redaction gateway<br/><i>self-host chat, rent the model</i>"]:::o1
B["2 · Rented EU GPU<br/><i>open models, EU host</i>"]:::o2
C["3 · Owned bare metal<br/><i>in-house, VPN-locked</i>"]:::o3
A -->|more control, more cost| B -->|more control, more cost| C
classDef o1 fill:#494fdf,stroke:#376cd5,color:#ffffff,stroke-width:2px
classDef o2 fill:#2f7d6e,stroke:#1f5a4f,color:#ffffff,stroke-width:2px
classDef o3 fill:#1e293b,stroke:#64748b,color:#f1f5f9,stroke-width:2px
The only decision that matters: where your data stops
To your staff, all three setups look identical: the same chat box, the same login, the same assistant that knows your systems. What changes underneath is where the model actually runs, and therefore what crosses your company boundary. That single choice sets your cost, your compliance posture, and how carefully you have to police what people type.
| 1 · Redaction gateway | 2 · Rented EU GPU | 3 · Owned bare metal | |
|---|---|---|---|
| Where the model runs | External provider, behind a gateway | Your rented GPU, EU datacentre | Your machine, on your premises |
| What leaves your boundary | Redacted prompts | Nothing (stays at the EU host) | Nothing (stays in the building) |
| Cost shape | Lowest; pay per token | ~€200–1,500 / month | ~€15k–30k up front, then power |
| Model quality | Frontier | Strong open-weight | Strong open-weight |
| Main risk | Residual leakage past redaction | Trusting one third-party host | Capex, and you run the ops |
Option 1: Self-host the chat, rent the model
The cheapest private setup keeps everything under your control except the model itself. You host the chat platform (Open WebUI or LibreChat) with your own single sign-on, user sessions, conversation history, and integrations to internal systems via MCP and n8n. Only the model lives outside, reached through an external API. In front of it sits a privacy gateway, Microsoft Presidio, that detects and masks the sensitive parts of a prompt (personal data, credentials, account numbers) before the request ever leaves your boundary.
The honest caveat: redaction is a filter, not a guarantee. An unusual identifier, a sensitive fact that isn’t obviously personal data, an edge case the policy didn’t anticipate: any of these can still slip through. So this option only works when two things are true. First, some residual leakage is an acceptable risk for the kind of data you handle. Second, you actually run the discipline it demands: anonymisation designed in from the start, a tight redaction policy you can inspect, strict logging and auditing of what left, and MCP tools scoped to least privilege. Done properly it is a genuinely good, low-cost solution. Done casually it is “private ChatGPT” theatre.
Best for: most SMBs whose data is confidential but not tightly regulated, and who want frontier-model quality without a hardware bill.
Option 2: Rent a GPU in the EU and run the model yourself
The next step removes the external provider entirely. You rent a dedicated GPU server inside the EU (Hetzner, for example, rents GPU machines in German datacentres) and run open-weight models on it yourself. No model vendor sees a prompt, nothing is logged on someone else’s terms, and your data stays with a single EU host you hold a contract and a Data Processing Agreement with.
This is where the bill moves. A usable GPU box runs from roughly €200 a month for a modest card to around €1,500 for one that serves a whole team comfortably. These are directional figures that depend on the GPU and the size of the model you want to run. In exchange, you largely stop policing what employees type: inference happens on hardware only you use, in a jurisdiction GDPR already covers. The trade-off is that you run more of the stack yourself (model serving, updates, scaling), and open-weight models sit a step behind the very newest frontier releases. For most business work that gap is now small.
Best for: teams that can’t send data to a model vendor even redacted, but don’t need the machine physically on site.
Option 3: Own the metal, in-house
The most private setup owns the hardware and never lets data leave the premises. The machine sits in your building, locked behind a VPN, running open-weight models. Nothing, redacted or not, crosses the company boundary, because there is no outside for it to reach. For confidentiality-critical work this is the setup where you stop asking what document someone just fed the assistant.
That protection is capex. A professional accelerator (an NVIDIA RTX 6000-class card) plus the workstation around it puts a complete, production-ready build in the low tens of thousands: call it €15,000–€30,000 all-in, directional, plus power and someone to keep it running.
Here is the part that ages well. Open-weight models are getting smaller and smarter at the same time. The model you can fit on a given card this year is meaningfully better than last year’s, and next year’s will be better again, on the same hardware. You upgrade the model, not the machine. Today’s field alone (Qwen, DeepSeek, Mistral, Google’s Gemma, OpenAI’s gpt-oss) already covers most business use, and every new open release flows to your in-house box for free. It is the shift from metered to owned we’ve written about before: intelligence stops being a utility you rent by the token and becomes capital equipment you own outright.
Best for: regulated or confidentiality-critical operations, or anyone who wants their AI to be a machine they own, not a subscription.
Which one is right for you
Start from the data, not the budget. Ask one question: what is the worst single thing a member of staff could paste, and what happens if it reaches a third party?
- “Uncomfortable but survivable, and we’ll audit it” → Option 1. The gateway plus real discipline is enough.
- “Must never reach a model vendor, but an EU host under contract is fine” → Option 2.
- “Must never leave the building” → Option 3.
Most SMBs start at Option 1, and they are right to. You can begin with the gateway and move workloads to a self-hosted model later; the chat interface, the integrations, and your users never change, only the boundary moves inward. None of the three is a bigger project than a normal internal-tools rollout, and each one is reversible into the next.
The takeaway
Owning your company’s AI is no longer exotic. It is a procurement decision with three known answers, priced from “almost free” to “a serious but finite capital purchase.” The one thing not to do is leave it unmanaged, with staff quietly pasting the business into tools you can’t see. Decide where your data is allowed to stop, then build inward from there.
We design, host, and operate exactly this. Our Private AI Platform service spans all three setups, from the redaction gateway to fully sovereign in-house hardware, wired into your own systems; the assistant on this very site runs on the same stack. If you want help working out which boundary your business actually needs, tell us what you’re protecting.
