EU-hosted, private or on-premise LLM: a decision guide for regulated organisations
Published · 7 min read
A regulated organisation that wants a language model in a production workflow has three ways to run it. Each is reasonable. Each has a price in money, effort and flexibility. This guide defines the options, lists the criteria that decide between them and explains why the choice should be reversible.
The three options
EU-hosted API. You call a model through an API offered by a provider that runs it in an EU data centre. You do not manage hardware or model serving. The provider's terms determine what happens with your prompts and outputs. This is the fastest option and the one with the least control.
Private LLM on European partner infrastructure. An open-weight model runs in a dedicated environment on a European cloud or GPU provider, under your tenancy. The model weights, the serving stack and the logs are in an environment you control, with keys in your own KMS or HSM. The provider supplies capacity and platform; it does not see your data as part of its service.
On-premise LLM. The same open-weight model runs on hardware in your own data centre or a facility you contract directly. Everything stays inside your perimeter. You, or a partner you appoint, run the GPUs, the serving layer and the updates.
The difference between the second and the third option is smaller than it looks. In both cases the model is open-weight and the stack is yours. What changes is who owns the hardware and who is on call for it.
Decision criteria
Data classification
Start with what goes into the prompt. Policy terms, claim documents, patient letters and citizen records are personal data and often special-category data. If the workflow retrieves documents from a core system with the user's access rights, assume the highest classification in that system applies. A public-facing FAQ bot and a claims intake workflow do not deserve the same answer.
Jurisdiction
Where the data is processed, and under whose law the provider operates. An EU data centre run by a provider subject to non-EU law is a different situation from an EU data centre run by an EU provider. Your legal team, not your engineers, should write this criterion down. Buyer surveys from 2026 suggest that control over location and keys matters more to buyers than any label; take that as indicative.
Latency and volume
A workflow that processes a few hundred documents a day has different needs from one that handles tens of thousands. Shared APIs scale easily but with less predictable latency and rate limits. A dedicated environment gives predictable throughput and lets you size for your peak. On-premise is sized once and then fixed until you buy more.
Cost per case
The useful unit is cost per completed case, not cost per token. Cost per token is only one input. Add the cost of the environment, the people who run it and the evaluations after every model swap. A dedicated environment has a floor cost whether cases arrive or not; an API has none but can be expensive at volume.
Team capacity
Who patches the serving stack at two in the morning? Who re-runs the evaluation set when a new model version appears? If the answer is "nobody yet", on-premise is a project, not a decision. Partner infrastructure with a run partner gives you the control of a private environment without building that team first.
Exit
Can you leave? For an API, exit means switching to another API and re-testing. For a private or on-premise model, exit means taking the weights, the images and the configuration with you. Under DORA, financial entities need a documented and tested exit strategy for ICT services supporting critical or important functions. Under the EU Data Act, providers of data processing services may not charge switching fees for a one-off switch from 12 January 2027.
Decision table
| Criterion | EU-hosted API | Private LLM on EU partner infra | On-premise LLM |
|---|---|---|---|
| Data classification | Low to medium | Medium to high | High, or policy requires it |
| Jurisdiction | Depends on provider's terms and law | EU provider, your tenancy, your keys | Fully inside your perimeter |
| Latency and volume | Elastic; shared limits | Predictable; sized for your peak | Fixed capacity |
| Cost per case | Lowest at low volume; grows with volume | Floor cost; stable at volume | Highest floor; cheapest at sustained high volume |
| Team capacity needed | Low | Low with a run partner | High, or a run partner |
| Exit | Switch API and re-test | Weights, images, config move with you | Already yours |
| Time to first production workflow | Weeks | Weeks (indicative: about eight) | Months, if hardware is not in place |
Treat the rows as directional. Your numbers depend on your systems and your volumes.
Why the model should be a role behind a gateway
The most important design decision is not which option you pick. It is making sure you can pick again later.
In a Clicksov workflow, the application does not call a model by brand. It asks an AI gateway for a role: "extractor", "classifier", "summariser". The gateway exposes an OpenAI-compatible API and routes the role to whichever model is configured in the model manifest. Behind that gateway, the model can be an EU-hosted API today, a private open-weight model on partner infrastructure next quarter and an on-premise model later. The workflow does not change.
What keeps this honest is the evaluation set: golden examples with thresholds. Every model swap starts with it. Every release is deployed to at least two cloud partners and one on-premise reference and must pass the same evaluation set. If a model behind a role changes, you know before your users do whether the output still meets the bar.
This is why Clicksov separates the stack into layers. Infrastructure, platform and models (layers 1 to 3) are the partner side. Data and knowledge, applications and agents, users and interfaces (layers 4 to 6), together with security, monitoring and governance, are the Clicksov side. The boundary between them is an open interface, so the partner side can be swapped.
Cost realism
Some honest statements, all indicative:
- A private environment has a monthly floor. It is worth it when the volume is steady and the data classification demands it, not before.
- Operations cost more than building. Vendor surveys from 2026 report that after go-live, running costs exceed build costs for many organisations. Budget day-2 from the start.
- The cheapest token is not the cheapest case. A model that needs more human corrections costs more per case than a dearer model that gets it right.
- Partner infrastructure should be passed through at cost and itemised: capacity, sovereignty, platform. If a vendor cannot show you the split, you cannot compare options.
Clicksov's own pricing follows this: a one-off onboarding (indicative from €25,000), a monthly result fee per completed case (indicative €4 to €8, capped) and partner infrastructure at cost, itemised.
Where this leaves you
Pick the option that fits your data classification, your jurisdiction requirement and the team you have today. Then put the model behind a gateway, build the evaluation set and keep the exit package current, so that the choice can change when your situation does. Data and models in the EU, keys in your own KMS or HSM, no training on your data. That is control first, and the word sovereign second.
Want to work through the table for your workflow? Book a conversation.
Sources
- Travers Smith, "The Data Act's changes to cloud services contracts", on switching rules and the 12 January 2027 fee ban: https://www.traverssmith.com/knowledge/knowledge-container/the-data-acts-changes-to-cloud-services-contracts-tipping-the-scales-on-switching/
- DORA, Regulation (EU) 2022/2554, Article 28 (text): https://www.springlex.eu/en/packages/dora/dora-regulation/article-28/
- CNCF, Certified Kubernetes AI Conformance programme, 11 November 2025: https://www.cncf.io/announcements/2025/11/11/cncf-launches-certified-kubernetes-ai-conformance-program-to-standardize-ai-workloads-on-kubernetes/