How Do You Run a Client-Tuned Open LLM on EU Infrastructure
- Jul 30
- 6 min read
You can build a client-tuned translation model on hardware you control in six steps, and the whole thing stands or falls on decisions you make before a single word gets translated. Pick the wrong license and legal has to unwind a production system. Send one controlled technical file to a shared public API and you may have made an unlicensed transfer under EU law. Get the setup right and you hold something most translation buyers can't: an AI translation pipeline where client data never leaves a tenant you can point to on a map.
Here is the workflow we run at AD VERBUM as our standard AI+HUMAN hybrid, laid out so an in-house localization engineering or compliance team can follow it.
The six-step pipeline
The sequence matters. Each step feeds the audit evidence the next one depends on.
Choose an open-weight model with a clean license. DeepSeek V4 ships under MIT, Qwen 3.6 and Mistral Large 3 under Apache 2.0. All three are self-hostable and fine-tunable, so nothing forces your content through a vendor API.
Stand up an isolated EU-hosted tenant. Deploy on your own hardware or a single-tenant EU enterprise-cloud instance, with no route for client data to leave that boundary.
Fine-tune on the client's Translation Memory and Term Base inside that tenant. The training data stays where the model runs, and the tuned weights become a client-specific asset rather than a shared engine.
Constrain generation at inference with the same TM and Term Base. The model proposes, your terminology decides, so approved terms and prior segments override a fluent-but-wrong guess.
Add certified human post-editing. A qualified subject-matter linguist reviews output under ISO 17100 and ISO 18587, and takes responsibility for the final text.
Log every step. Capture model version, training data, prompts, and reviewer actions as evidence for ISO 42001 and ISO 27001.
Steps 1, 2, and 6 are where most in-house builds quietly go wrong, so they get the detail below.
Step 1, read the license and the jurisdiction
A permissive license and a comfortable jurisdiction are two separate checks, and you need both.
MIT and Apache 2.0 let you deploy, modify, and fine-tune for commercial use without asking anyone. That's why DeepSeek V4, Qwen 3.6, and Mistral Large 3 are the clean starting points. Watch for "community" or "modified" licenses that add acceptable-use clauses or user-count thresholds, because those need legal review before scale deployment.
Jurisdiction is the second check. Self-hosting the weights on EU hardware neutralizes the data-residency question, since inference never touches the model publisher's servers. For defense and life sciences buyers, a European-built model such as Mistral Large 3 removes the argument entirely. We wrote about the trade-offs across the current field in our rundown of the best open LLM models for regulated translation.
Step 2, isolate the tenant
The point of running your own model is that content never reaches a shared public API. That promise only holds if the deployment is genuinely single-tenant.
For us that means EU-hosted infrastructure under ISO 27001 and ISO 42001, with no reliance on public cloud tooling for core processing. A client's TM, term base, and source files sit inside one boundary, and the tuned model sits there with them. The difference between this and a SaaS translation tool that happens to host in Frankfurt is control: you decide the retention, the access log, and the deletion. We compared providers on exactly this point in our look at which translation companies run client-tuned LLMs on EU-hosted infrastructure.
Steps 3 to 5, tune, constrain, then let a human sign off
Fine-tuning on a client's own TM and term base is what turns a general model into a translation asset that speaks the client's product names, regulatory phrasing, and house style. The tuned weights carry that knowledge, and the term base enforces it at inference so a fluent guess can't overwrite an approved term.
The human step is not optional for regulated content. A general LLM produces confident, readable output that can still be wrong in ways a non-specialist won't catch, which is why classic segment-level neural MT and raw LLM output both need a certified reviewer for audit-ready work. We set out that argument in full in our piece on NMT versus LLM translation. Under ISO 18587 the post-editor is a qualified professional who owns the final translation, not a spell-checker.
Step 6, log for the audit you will eventually face
An AI translation pipeline with no record of what it did is a liability during a data protection authority review or a client audit. ISO 42001 expects an AI management system with risk assessment, impact assessment, and traceable decisions, and ISO 27001 expects an access and change record. Capture the model version and its license, the training data and its source, the prompts and terminology constraints, and every reviewer action. That log is also what proves your dual-use posture, discussed next.
Where it goes wrong
Most failures trace back to a shortcut taken to save a week of setup. Watch for these.
Skipping the license check. A "modified" or community license can restrict the very deployment you built, and finding out in production means unwinding a live system.
Routing content through a shared public API. The moment a client's controlled file hits a consumer AI endpoint, your data-isolation promise is gone and so is the audit trail.
Ignoring dual-use exposure. Under Regulation 2021/821 Article 2, transmitting working knowledge tied to Annex I or Annex IV controlled items counts as technical assistance, so fine-tuning a model on controlled technical data or handing it to an unvetted linguist can be an unlicensed transfer. Vetted linguists and an EU-hosted tenant answer this, an NDA does not.
Treating logging as an afterthought. If the evidence isn't captured as the work happens, you can't reconstruct it for BAFA, DGA, or a notified body later.
Fix these four and the pipeline holds up to the review it was built for. AD VERBUM runs this as our standard workflow rather than a special project, which is the difference between a demo and a compliant production line. For a wider view of what governance the EU AI Act now demands of AI translation, see what ISO 42001 requires from an AI-governed translation company.
Our AI translation services
Our translation services for regulated sectors run on ISO 27001 and ISO 42001 certified, EU-hosted infrastructure, with no reliance on public cloud tooling for core processing. Every project runs through our AI+HUMAN hybrid workflow: we ingest client Translation Memories and Term Bases first, our proprietary LLM-based LangOps System generates output constrained by client terminology on client-tuned open-weight models, and our certified subject-matter experts review for technical accuracy and regulatory compliance. Our QA is aligned to ISO 17100 and ISO 18587, with sector-specific requirements such as the EU AI Act (Regulation 2024/1689) and Dual-Use Regulation 2021/821 controlled-data handling applied where relevant. We serve Life Sciences, Legal, Finance, Defense, and Manufacturing clients across 150+ languages with 3,500+ subject-matter linguists. For teams managing audit-sensitive content, contact us to discuss your security and compliance requirements directly.
FAQ
Which open LLMs have a clean enough license to self-host commercially?
DeepSeek V4 (MIT), Qwen 3.6, and Mistral Large 3 (both Apache 2.0) all permit commercial use, modification, and fine-tuning without a vendor agreement. MIT and Apache 2.0 are the two permissive licenses that carry no user-count or acceptable-use clauses. Community or "modified" licenses need legal review before scale deployment.
Does self-hosting a Chinese-origin model create a data-residency problem?
Self-hosting the open weights on EU hardware means inference never reaches the publisher's servers, so the data-residency question is answered by where you run the model, not where it was built. The license terms still bind you regardless of jurisdiction. For buyers who want to remove the argument entirely, a European-built model such as Mistral Large 3 is the straightforward choice.
How does fine-tuning on a Translation Memory differ from prompting?
Fine-tuning adjusts the model's weights on the client's own TM and term base, so the model internalizes house terminology and phrasing. Constraining generation at inference with the same term base then enforces approved terms segment by segment. Together they produce output aligned to the client's regulatory language rather than a generic register.
Is human review still required if the model is tuned on client data?
Yes. ISO 18587 requires a qualified human post-editor who takes responsibility for the final translation, and for regulated content that review catches errors a fluent model still makes. A tuned model reduces the editing effort but does not remove the accountability step.
What does ISO 42001 expect from an AI translation pipeline?
ISO 42001 is the AI management system standard, and it expects risk assessment, an AI system impact assessment, and traceable records of how the system is built and run. For a translation pipeline that means logging model version, training data, prompts, terminology constraints, and reviewer actions. That evidence is what stands up during a data protection authority or client audit.
When does using a translator trigger EU dual-use rules?
Regulation 2021/821 Article 2 defines technical assistance to include transmitting working knowledge, and Annex I and Annex IV list controlled items. Sharing controlled technical data with an unvetted linguist, or fine-tuning a model on it without controls, can be an unlicensed transfer enforced by authorities such as BAFA, DGA, and UAMA. Vetted linguists and an EU-hosted tenant give you a defensible position that an NDA alone does not.
