A private LLM is the AI a company officially provides to its people: a chat interface or model access with a company account, a contract and rules, instead of personal free accounts. Some call it a corporate LLM. There are three ways to get one: a vendor's business plan such as Claude or ChatGPT, a US model running in EU regions through a cloud like AWS or Azure, or your own server with an open model, on your premises or rented in Germany. Which path fits depends first on the class of data, then on cost, operations and quality. For most small businesses the first path with a data processing agreement is enough; I run especially sensitive data mostly through the third myself.
I use all three paths side by side and decide per data class, not per tool. I am a developer, not a lawyer, and this is not legal advice, just the operator's perspective. More on AI in daily work is on my AI page.
What is a private LLM?
LLM stands for large language model. "Private" or "corporate" does not mean the company trains its own model. It means company-owned access to a model: one interface with the same account for everyone, defined permissions, a data processing agreement and a rule about which data may go in. Other names are private AI or, when the model runs in your own building, on-premise AI.
A private LLM always has two layers: the model that computes, and the interface your people work with. A business plan gives you both from one vendor; with your own server you combine them yourself, for example an open model under an inference server and a chat interface such as Open WebUI, "a self-hosted AI platform".
Three paths to a private LLM
Path 1: a vendor's business plan
A team or enterprise plan is the fastest path with the best quality: user management, a data processing agreement, the vendor's newest model. The open question is location. For the Claude API, Anthropic lists only "global" or "us" as the inference location and only "us" as the storage location, as of 31 October 2026; there is no EU option. For the business plans, check the location in the contract and in the plan's privacy documentation, not on the marketing page.
Platforms like Langdock bundle several models under one interface with EU hosting; see Langdock vs Claude: What I Use and Why.
Path 2: a US model in an EU cloud
The same model, run by a cloud provider in EU regions. Amazon Bedrock has an EU profile for Claude Sonnet 5.5; called from Frankfurt, it lands in one of six regions according to the model card: Frankfurt, Stockholm, Milan, Spain, Ireland or Paris. On Google Vertex, Claude has a multi-region endpoint "eu" (10 percent above global from Claude Sonnet 4.5 on); regional endpoints such as europe-west1 serve only Claude Sonnet 4.6 and earlier, according to Anthropic. On Microsoft Foundry the counterpart is "Data Zone Standard"; Microsoft's EU Data Zone follows the EU Data Boundary and can include EFTA countries such as Norway and Switzerland. This does not apply to Claude: Anthropic lists only "Global Standard" and "US Data Zone Standard" for Claude in Foundry, and only "Global Standard" for Claude Sonnet 5.5. If you need Claude processed in the EU, Foundry does not currently get you there.
You get top models processed in the EU, at the price of a cloud account, permissions, per-token billing and no ready-made interface for everyone. The details for Claude are in Claude on AWS Bedrock: EU, GDPR and Cost.
Path 3: your own server with an open model
An open model such as Qwen, Mistral or gpt-oss runs on hardware you control, on your premises or rented in a German data center. No vendor sees your inputs, and no model change arrives unasked. In return, updates, outages and model changes are your job, and quality is below the best cloud models. Hardware per model size is in Your Own AI Server: Hardware and Cost, the open models themselves in Open Source AI Models 2026 Compared.
Four criteria for the decision
Data class comes first because it limits the rest: if a client requires that their data never leaves your network, path 1 is out. Cost you compare per use: seats for daily work, tokens for occasional use, a server for constant load. Operations is what almost everyone underestimates. Quality you test on your own tasks, not on benchmarks.
LLM hosting in Germany: who offers what
A middle path: open models that a German provider runs for you behind an API. Three examples, based on the providers' pages as of 31 October 2026, not on a test of my own; neither a recommendation nor a ranking.
- IONOS AI Model Hub. Runs only in data centers in Germany according to its documentation and offers Qwen3.8-27B, gpt-oss-120b, Mistral Small 24B and Llama 3.3 70B, among others, billed per million tokens.
- STACKIT AI Model Serving. Runs in region EU01 in Germany according to its documentation and offers Qwen3.8 27B, gpt-oss-120b, Gemma 4 31B and Llama 3.3 70B, among others, also per million tokens.
- Hetzner GPU servers. You rent the hardware, not the model. The GEX131 with an Nvidia RTX PRO 6000 Blackwell Max-Q and 96 GB of graphics memory costs, according to Hetzner, 1,499 euros a month excluding VAT, plus a one-time setup fee of the same amount. It is the only GPU model also offered in Falkenstein; the others are Helsinki only. Running the model is up to you.
Two of them offer Qwen3.8-27B, the model I run myself; the difference is who maintains the server and processes the data. All three process or host your data, so a DPA usually belongs with it. And models change: IONOS lists many retired models, including Llama 3.1 405B and Mistral 7B, so plan for your starting model to be replaced. Data handling, contracts and further providers in Germany and the EU are covered in European AI Providers Beyond ChatGPT.
My setup: three paths along a traffic light
My traffic light for client data has three levels, each with its path. Green is copy and anything without personal data. Yellow is personal data in a normal business context. For business data I use Claude Team because Anthropic offers a data processing agreement for it. I use Bedrock in production myself in some cases and have set it up for clients, wherever processing in the EU is required. Red is health data, third-party bank data and other special categories, which in my setup mostly run directly through local models, no cloud.
An Nvidia DGX Spark has been in my own network since December 2025; it runs Qwen3.8-27B under SGLang. The API speaks both the OpenAI and the Anthropic format, so tools that normally talk to Claude run against the local server without code changes. To be honest: on complex tasks with many files and many tools, Claude is ahead in my setup. The goal of the local model is a cloud-free option for cases where not a single byte may leave the building, not a replacement for my workforce on Claude. The full setup is in Qwen 3.8 Locally: DGX Spark Experience.
Common mistakes in the decision
- Starting with the technology. Data class first, then the path.
- Reading "Frankfurt" as "only Frankfurt". An EU profile keeps processing within a group of regions, not in a single city. If a contract requires one specific region, check the model card.
- Confusing local with GDPR-compliant. Your own server saves the DPA with a model vendor, but legal basis, records of processing and internal rules remain, and a hosting provider usually needs a DPA after all. More in Local AI and Privacy: Cloud or Server?.
- Not planning for operations. Models get replaced, inference servers change, the interface needs updates. Without someone who enjoys this, a self-run private LLM goes stale.
- No rule for the team. The best private LLM does not help if personal accounts keep running on the side. How to write that down briefly is in AI Usage Policy for Small Teams.
Frequently asked questions
What is a private LLM?
Company-owned access to a language model, with a company account, a contract and fixed rules about which data may go in. The company almost never trains the model itself; it comes from a vendor or is an open model.
Is a private LLM GDPR-compliant?
No tool is compliant on its own, only a setup is. A data processing agreement, the right processing location and a rule for the data belong to it, whichever path you take. This is not legal advice; for your specific case, ask your data protection officer.
What does a private LLM cost?
A business plan costs a fixed price per seat, an EU cloud bills per token, and a rented GPU server like the Hetzner GEX131 costs 1,499 euros a month excluding VAT according to the provider, as of 31 October 2026. With your own server, add working time for operations.
Which LLM hosting providers are there in Germany?
Examples are the IONOS AI Model Hub and STACKIT AI Model Serving, which provide open models through an API, and rented GPU servers, for instance from Hetzner. Check in each provider's documentation where inference runs, whether inputs are stored and which models are currently offered.
Do I need my own hardware for a private LLM?
Only for the third path on your own premises. Business plans and EU clouds need none, and open models are also available hosted in Germany. Own hardware pays off when data must not leave your network or a task runs constantly.
Where to go from here
Write down the data class of your three most common AI tasks; usually that already shows which path fits. If it points toward your own server, AI Server for Small Businesses is the next step, and setup from the Mac to the server is in Setting Up Local AI: Ollama to SGLang. How other businesses set up their AI and write their rules, we discuss in my community AI Employee Framework.
Your first AI employee up and running within an hour
Join the community$5/month founding price, first 50 members
