{
  "video_id": "x2bH0RKPgdc",
  "channel_slug": "aidotengineer",
  "channel_handle": "AI Engineer",
  "title": "What Breaks When You Build AI Under Sovereignty Constraints - Bilge Yücel, deepset GmbH",
  "duration_seconds": 1149,
  "url": "https://www.youtube.com/watch?v=x2bH0RKPgdc",
  "upload_date": "20260519",
  "transcript": "Hello everyone. Can you all hear me\nwell?\nAll right, amazing. Uh thank you for\njoining this session. In this one we're\ngoing to talk about sovereignty.\nUh my name is Viguen. I work as a senior\ndeveloper relations engineer at Deepset.\nBut I want to start with a question. So\nhow many of you are familiar with the\nterm of sovereignty?\nWow, okay, amazing. Maybe I don't need\nto do a lot of introduction for this\none. And um just a little bit of context\nhere about Deepset.\nUh Deepset is the company behind the\nopen source orchestration framework\ncalled Haystack. We also have our\nenterprise platform. And with those\nproducts we are solving customer AI\nchallenges for big organizations like\nAirbus, Bosch, Siemens, but also public\nsector uh organizations like European\nCommission, Federal Ministry of Research\nand Technology uh and space in Germany\nand other ministries as well. So you can\nimagine sovereignty is a very big and\nimportant topic for us.\nSo here is a like a policy definition\nyou can say what sovereign sovereign AI\nis. Sovereign AI is the ability of an\norganization to design, deploy, and\noperate AI systems on its own terms. But\nwe are all engineers here. We are not\npolicy makers, we are not lawyers. So if\nyou if you\nturn that into a technical\nuh definition, it's basically having\nexplicit control over data flow, model\nchoice, infrastructure, observability,\nand operations.\nAnd um for us to understand it better, I\nwant to split that into four pillars. So\nthe first one is data sovereignty. So,\nit's about where you store your data,\nwhere does it get processed? The second\none is model sovereignty, who controls\nthe running models, the origin of the\ntraining data. The third one is\ninfrastructure sovereignty. So, where\ndoes compute happen? And the last one is\noperational sovereignty. So, is your\napplication, is your system traceable?\nWho can update it? Who owns the incident\nresponse?\nSo, let's go into the depths of this for\na minute. Um data sovereignty. Data is\nthe most important assets that we have\nin in an enterprise. And for us to have\ndata sovereignty, data should be\nprocessed and stored within trusted\njurisdictions to meet compliance\nrequirements. What does it mean? So,\nbasically, GDPR says that your the\nEuropean citizen data should stay within\nEurope. But, if you send that data to an\nembedding model, to an embedding API\nhosted in Virginia in the US, then you\nare already losing the control of your\ndata. So, that's against data\nsovereignty. And the other aspect is\naccess permissions. So, this not exactly\nabout how the data is stored and\nprocessed, but if there are users in\nyour organization, within your\norganization, that are that has access\nto data that are not that they are not\nsupposed to see, there's also a breach\nof data sovereignty.\nThe other one is infrastructure\nsovereignty. So, where does compute\nhappen? It means like we we have the AI\napplication layer. So, we have our rag\npipeline ingestion, we have agents, uh\nthese the tools for the agents, and they\nall run somewhere. And where this\napplication layer runs defines the\ninfrastructure sovereignty of your\nsystem.\nUm and this and there's this whole\nspectrum from max control to max\nconvenience. Maybe you run everything in\nan air-gapped environment, it gives you\nthe EU AI Act safe. Maybe you run on a\nuh private VPC, this gives you GDPR\nsafety. There is also sovereign cloud,\nthis kind of depends on the provider\nthat you're going with. And there's also\nat the very end there's SaaS. So with\nSaaS, you kind of face the cloud act\nrisk, meaning that if you are for\nexample using US headquartered company,\nalthough you store, you run all your\ndate all your application within Europe,\nuh they have ability to get access to\nyour running data. So that's against the\nsovereignty that you have over your\nsystem.\nThe other pillar is modern sovereignty.\nSo who controls the model and the origin\nof the training data?\nUm so the\nyou should have the freedom to choose\nand switch models. But what it means\nthat if your system can only work with\none specific model, you are tightly\ncoupled with that model provider.\nIf they are if the API is down, then you\nlose access. If they increases the\nprice, then you also have a cost issue.\nUm this is kind completely against the\nmodel sovereignty idea. And the other\none is swappability without\narchitectural changes. So maybe you're\nnot\ntightly coupled with that model\nprovider, maybe you can use other models\nas well. But if your system, if your\ncode doesn't allow that immediately,\nthen you are although technically you\nare not tied to that provider, but\nyou cannot just change the whole code\nbase within one day.\nAnd the last one is training data\norigin. So this is a bit controversial\nbecause we don't have a way to know like\nwhere the model was trained, which data\nwas used to train this model. Uh but\nuh if a model provider is a European\ncompany, then they have a better\nadvantage than other companies based in\nthe US.\nThe The last pillar is operational\nsovereignty. So, like building the whole\nAI system is one aspect, but monitoring\nSo, basic basically maintaining this\nwhole system by monitoring, evaluating,\nmanaging is also another. Uh operational\nsovereignty is about monitoring how\nthese systems behave in production,\nincluding model inputs and outputs. And\nin high-stake environments like HR or\nfinance, it requires human in the loop.\nAnd managing versioning, updates to\nmodels and the application layer uh in a\ncontrolled auditable way also goes under\nthis operational sovereignty.\nBut the good news is sovereignty is a\nspectrum. So, not everyone needs like\nneeds to be sovereign in all of these\npillars. Of course, like uh if you are\nin finance, in healthcare, in high-stake\nenvironment in high-stake domains, you\nmight need an fully air-gapped solution.\nBut if you are an enterprise or a\nstartup working in a different domain,\nthen maybe you don't need everything at\nall. You're You don't need to comply\nwith every sovereignty pillar that I\nshowed you. The important thing here is\nyou need to know the level of control,\nso the the level of vendor lock-in you\nhave with your system.\nSo, you are here in the conference this\nweek, but next week on Monday you're\ngoing to go back to work and maybe your\nmanager or CIO comes and says like,\n\"Okay, now we need to make like we have\nthis working system, but then now we\nneed to make it sovereign.\"\nUh and in this slide I want to show you\nwhat you do first usually and what you\nbreak in this existing system. So,\nprobably intuitively the first thing\nthat you do is the model and you replace\nthe frontier API that you have with a\nself-hosted model. And what happens?\nThen you need to translate this whole\nAPI logic to this new model\narchitecture. Maybe you need to update\nthe prompts. Basically, you need to\nevaluate the performance of the system\nfrom scratch. And you need to write a\nlot of code.\nThe other one is you move private data\ninto the required jurisdiction because\nyou notice that okay, we have some\ninformation hosted in in the US. Now we\nneed to take them somewhere in Europe\nand you do that, but you find yourself\nmanaging multiple databases and\ninstances. Then search becomes a\nproblem. So how do you handle search? Do\nyou do classification like query\nclassification first or do you send the\nrequest to both of those databases and\nget information from those? That's just\none challenge that comes with this\nchange. And the other one is you replace\nmanage infra with on prem and you\nimmediately notice how much you had\nvendor lock-in in this area. You because\nnow you also start thinking about okay,\nKubernetes cluster management, how do I\nlike deal with all these\nall these things that all these model\npro all these cloud providers were\nhandling for you? And there are also\nhardware limitations because now you\nneed to think about okay, I have this\napplication layer running in GP CPU, but\nnow I have my model run now running GPU.\nSo I need to connect them in a way and\nthis connection management, network\nmanagement also becomes a problem. And\nthe last one is you incorporate\nobservability and tracing. I mean, if\nyou if you didn't have observability\nuntil this point in your system, that's\nan important issue already. But if you\nnow you when you need to incorporate it,\nyou notice that you have this fully\nblack box thing. So you don't know\nactually what's happening in this AI\napplication layer. So you need to\nunderstand now because you need to log\nit somewhere so that your system is\nauditable. And there's is version\ncontrol aspect. How do you do version\ncontrol for your application layer, for\nthis whole system that you are managing?\nAnd I want to hear shamelessly plug\nHaystack because a good orchestration\nframework I think solves some of these\nproblems. I mean, good orchestration\ncannot solve your GPU limitation, but\ncan help you with so many different\nstuff. Uh the the first thing is uh\nHaystack is has this consistent\ninterface. So, basically, it when you\nwant to transition your system from\ncloud to a self-hosted application, you\ncan easily do so just changing couple\nlines of code and just focus on the\nhardware that you have and this hardware\nconnection. The second one is explicit\ndata flow. So, every input and output in\na Haystack application is typed and\ndeclared. So, you can read the pipeline\ndefinition all over the this whole\napplication\nuh in a in a in a regular way and know\nexactly what data was where and even in\nless deterministic architectures like\nagents and\nuh like agents the whole data this tools\ntool calls and tool outputs are\ntraceable.\nThe third one is YAML. So, the Haystack\napplications are serializable to YAML\nmaking them very easy to version. So,\nyou can just\nafter creating your application with\nHaystack, you can turn it into YAML, put\nit into your version control, and then\nwhen you need to go back to the history,\nyou just need to go back to the the\ncommit and you you can see the hash.\nAnd the last part is it's truly open\nsource. So, there's no black box, no\nhidden assumptions. And when you need to\ncustomize some code or extend one of the\ncomponents, you can easily do so because\nyou actually understand what's happening\nunder the hood.\nAnd here is one Here's a sovereign\narchitecture that I want to show you\num today just as like a high-level\noverview. Uh imagine you're building\nthis agent, but it needs to be some sort\nof sovereign. So, first thing that you\ndo probably you add some guardrails\nbefore, and then because this guardrail\nneeds to check if there's like a prompt\ninjection coming from the user input\ncoming through the user input, and it\nneeds to also check some specific\nregulatory checks maybe because I this\nis like a very specific agent that needs\nto be used in a certain way, and you\nwant to check if user has this\nintention. And if it's unsafe, it just\nleaves the application layer\nimmediately. Uh and if it's a safe uh\nrequest safe input, then it goes to the\nagent. And agent here is basically an\nLLM with a system prompt and lots of\ndifferent tools. And these tools can be\nAPI calls, maybe they're connected to\nyour knowledge base that you're you're\ndoing some sort of search. Maybe you are\nusing other agents if you're dealing\nwith a\num multi-agentic systems, and there are\nalso MCV servers, of course, that are\nconnected to your agent. And agent\ncreates an input, so does the uh does\nthe work for you, and there is the last\nguardrail doing a compliance checks\nbecause you also don't want to leak\nsensitive information to your user, and\nthen there becomes an output.\nAnd uh but of course, like how you\ndesign a system as a sovereign system is\na challenge. And the the duty of\nHaystack here is to make sure that\neverything is swappable, traceable, and\nwithout vendor lock-in.\nUm so, you know, you can work with the\nright level of sovereignty uh that you\nneed. And here are some of the uh the\ntools that I picked from\nthat I picked that you can work with\nHaystack. Um so, maybe you uh you decide\nyou need like different models based on\nthe task. Maybe you decide that for\npublic data, you can steal some\nproprietary models from those providers,\nso you don't worry about those. But for\nsome of the task, for guardrails, maybe\nfor your knowledge base, for uh for your\nLLM, you think that you need local\nmodels running on on like self-hosted\nlocal models from other different\nproviders like\nuh Mistral, Google, Nvidia, and Gina.\nAnd then, of course, uh there's\ntraceability aspect. And as I said, in\nHaystack, everything like every input\ncoming to a component and every output,\nbut as well as the traces in the agent\nis easily visible, so you can just\nconnect those spans to your LLM\nobservability tool. And you can since\nthere is OpenTelemetry integration, you\ncan also implement your own\nobservability.\nAnd as the last part, there's the\nstorage. I wanted to pick some of the\nproviders who can give you cloud and\nalso the open-source version of them, so\nyou can easily host these ones uh\nlocally\nuh on your prem.\nAnd here is the um here is a guardrail\num\ncode. So, uh basically, how you connect\na guardrail or how you define a\nguardrail is quite simple with Haystack.\nYou just start with a model provider.\nFor example, in this one, it's the\nNvidia chat generator. You give the name\nof the model that you want to use. And\nby connecting it to your LLM message\nrouter, it helps you do the\nclassification. So, it checks the input,\nif it's safe, it goes to the safe route,\nand if it's unsafe, it goes to the\nunsafe route.\nAnd then, you add you start adding\ntools. So, you connect your MCP server\nthat you host locally with uh with MCP\ntool set, and you give the names of the\ntools that you want to pick from that\nMCP server. Because probably you have\nlots of tools running on that on that\nMCP server. You don't want to get all of\nthem.\nYou don't want to have access to all of\nthem with just one agent. And you pick,\nfor example, knowledge base search and\ngenerating PDF report one. And but of\ncourse like you can add the different\ntools. You can add just like you can\nstart defining tools from a Python\nfunction or you can connect other\ncomponents in Haystack like agent\ncomponent into a tool or maybe there's\nsome functionality that you define like\ndata ingestion or rack pipeline and you\ncan also convert those workflows into\ntools as well. And by after this\ndefining all of these tools, you put\nthem in a searchable tool set. So this\ngives you a dynamic tool search with\nBM25 because you probably now have more\nthan hundreds of tools to for your\nagent, but you don't want to fill in the\nwhole context with just tool\ndefinitions.\nThen there's this agent component. So\nbasically you define a system prompt.\nYou say like you're a sovereign agent\nwith access to multiple tools and you\ndefine the intention of this agent and\nthen you put a brain to this agent.\nBasically a chat generator. If you are\nrunning this model locally on prem,\nmaybe you create your own custom\ncomponent saying now this is my own prem\nchat generator connected to this\ninternal company URL and getting the\nmodel inference from that. But of course\nif you have an open AI API compatible\nendpoint, you can already use an\nexisting component in Haystack and then\nyou give the tools and then you define\nconfirmation strategies. So basically\nyou incorporate human in the loop saying\nthat so if the user wants to submit a\nrequest, the agent should always ask for\nhuman approval, but if the agent wants\nto use the list payment request tool,\nmaybe ask for a permission first from\nthe human and then you can use this tool\nas much as possible,\nas much as you want in this whole cycle.\nThen you bring it all together. So, you\nstart defining your pipeline. You first\nadd the component as a tracer that\nconnects to your\nLLM observability. Then you put the\ninput guardrail. You connect your agent,\nthe output guardrail, and you you can\njust run this agent saying that pull the\noutstanding payment request for Q3 and\ngenerate a PDF for me. And by using the\ntools that this agent has, it creates\nthe PDF for you and saves it under this\ndirectory under this name.\nSo, with that,\nI want to give you just like when we are\ncoming to the end of the presentation, I\nwant to show you a checklist. So, if you\nwant to check if your\nsystem is sovereign, you can you can\ntake a look at this question. So, you\ncan think about can you swap models\nwithout changing the application logic?\nDo you have reproducible run logs stored\nin a complying way? And can your team\nrespond to an incident without calling a\nvendor like one of those hyperscalers?\nThank you for joining the session.\nAnd you can get the presentations and\nask the questions to me by filling in\nthe the form. And you can also find me\non social media if you want to connect\nthere. I'm happy to talk about you about\nagents,\nHaystack, and especially in a sovereign\nsetting. Thank you.",
  "transcript_chars": 16858,
  "ingested_at": "2026-05-21T19:18:53.684780+00:00",
  "source": "retry-no-transcript",
  "yt_meta": {
    "view_count": 2885,
    "like_count": 75,
    "channel_id": "UCLKPca3kwwd-B59HNr-_lvA"
  }
}