On-premise generative AI: data sovereignty in the public sector

The sovereignty issue

Using external generative AI services often means transferring documents and data outside the organisation’s perimeter, raising questions about confidentiality, jurisdiction and retention. For a public administration, these questions are constraints, not details.

The alternative: open-weight models within the perimeter

Open-weight models can run on-premise or in qualified cloud. Combined with search over the organisation’s documents (RAG) and security guardrails, they deliver useful assistants without data leaving the controlled infrastructure.

Five design requirements

1. Models running within the organisation’s perimeter.
2. Access control integrated with existing identities.
3. Source traceability in every answer.
4. Interaction logging and security monitoring.
5. GDPR and AI Act compliance by design.