{
  "video_id": "JHDsM8ER7tk",
  "channel_slug": "ibmtechnology",
  "channel_handle": "ibmtechnology",
  "title": "Can you social engineer an AI? Plus: AI worms and the nonhuman identity problem",
  "duration_seconds": 1969.0,
  "url": "https://www.youtube.com/watch?v=JHDsM8ER7tk",
  "upload_date": "",
  "transcript": "Folks. Who do you think is more likely to fall for a social engineering scheme, a human being, or an\nAI agent? I feel like an AI. Right now, the answer is AI. AI is going to fall for it. I do think we're\ngoing to have to make AI a lot smarter, not just in the book sense, but in the street sense as well.\nHello and welcome to Security Intelligence, folks, IBM's weekly cybersecurity podcast, where our\nexpert panelists turn the biggest industry news stories into practical takeaways you can use. I'm\nyour host, Matt Kozinski, broadcasting live from some kind of dungeon. And joining me this week,\nwe've got a real classic Security Intelligence lineup. It's Claire Nuñez, creative director, IBM\nX-Force Cyber Range, Jeff Crume, distinguished engineer, master inventor, data and AI security. And\nNick Bradley, manager, X-Force Threat Intelligence. And on the docket today, we've got an AI-powered\nworm designed by University of Toronto researchers and the Sophos State of Identity\nSecurity 2026 report. But first, we're going to keep talking about AI falling for social\nengineering schemes, because some hackers tricked customer support agents into handing over\nInstagram account passwords. Now, 404\nMedia broke this story last week. It's about a spate of attacks where hackers just posed as the\nlegitimate Instagram account owners, messaged Meta's AI customer support agent and asked it to\nupdate their accounts with new emails, emails they controlled. And the agent just did it. It just\nhanded the accounts right over to these people, put their emails in there, and they were able to\nuse that to recover the passwords and take over the accounts. Now, the vulnerability has since been\nresolved, but I think there's something to learn here. And, Claire, I want to start with you, and I\nwant to ask, are you surprised by just how easy it seems like it was to pull off this attack? Does it\ngive you any concerns about AI in customer-facing roles? How are you feeling? I feel like it's\nsurprising on one aspect because like the whole premise of Instagram is like your\naccount and you being able to access your account. So you would think Meta would have that, you know,\nunder lock. But it's not surprising at the same time that something so seemingly simple has been\noverlooked. I mean, as Jeff mentioned, agents don't understand nuance, and they are very\nnaive. So, if you they're not going to ask why are you changing this email? So it's not that\nsurprising to me, but you would think that a company that kind of like, revolves around your\naccount and your ability to access that account for, like, everything, would\nhave looked at that a little more closely. Absolutely. And, you know, I know I think it reminds\nme of, like, it's very easy to trick a person. Right. Maybe I shouldn't say very easy, but it\ncan be quite easy to trick a person. And one of the promises of AI is supposed to be that it's\nit's, you know, it's got certain contexts that we don't have that maybe it won't fall for some of\nthese things, but we're seeing the opposite happen here. And, Jeff, I wanted to ask you, you know,\nsimilar question to Claire. Like looking at an AI agent fall for something like this. Does it give\nyou concerns about putting these things in customer-facing roles right now? How do you feel?\nAI is like this, this person that's got a thousand PhDs but has never spent one\nday above ground, you know, out of their parents' basement. So a lot of intelligence in one level,\nbut not a lot of common sense, not a lot of the kind of lived experience that allows us to\nrealize when somebody is trying to con us or not, when somebody is trying to ask for something\nthat's unreasonable or not. I mean, if you put in the system prompt, you're a helpful, customer\nsupport assistant. Well, then it's going to try to be helpful, and it's going to be very eager and\nwork as hard as it can to satisfy the request. And just like a person, if you haven't taught\nthem: Yeah, but every person that asks for your wallet, you don't hand it to them. You know, you\nwant to be polite, but you know, there's certain limitations to that kind of politeness and what\npeople ask you to do and reasonableness checks. This one was way too easy though. I mean, the the\nattackers just basically put their VPN so that it looked like they were in the area of where the\nthe person would have been and then said, you know, hey, send me a reset code for my account to\nthis new email address. And we know you shouldn't be able to do that without some other form of\nauthentication, but unless you explicitly say, no, don't do this, it's going to probably try to\ndo it. And that's that's the thing we all had to learn as people. You know, we were growing up\nand we got fooled a lot of times. Well, AI is really young, too. It might be really smart, but\nthat doesn't mean it's not very young and naive. And this is a classic example of that. Got to\nbe a lot, lot more specific in terms of what we tell them, what they can do. I think about my\ngrandson, he's four years old, and we're having to constantly tell him the kinds of things, you know,\nyou don't play in the street. And here's why. Okay. That sort of stuff that we take for granted. But\neverybody had to be taught that. And these AIs have to be taught this as well. That's a very good\npoint. And I like this comparison you draw actually with age. You know, young folks, I think\nmy son. Right. He is just about three years old. And it's a similar thing. I got to tell him, no, you\ncan't hang out with the cars in the streets. It's actually not. I know you like the cars, but you\ncan't go out there. It's not safe. And it goes back to this idea that you you brought up, which is\nthat, like, we have to be very specific with directions here. And so in ways that I was like\nslightly surprised by this attack, I think maybe I shouldn't have been surprised because like you\nsaid, Jeff, when you tell an agent you're here to help customers access, do things they need to\ndo unless you specifically tell them not to do that. They want to help customers. They they they\nthink they're doing the right thing and they're just going to try to do it. So I think that's a\nvery good point. Nick, I wanted to ask you, do you think there's any kind of meaningful difference\nbetween tricking a person with a social engineering attack, or tricking an AI agent with a\nsocial engineering attack? Like this is basically the same kind of thing? Are they slightly\ndifferent? How are you feeling about it? So there is a key word that I was looking for that I was\nhoping one of the three of you were going to drop and you didn't. So I'm kind of glad because then I\nwouldn't have had anything to say. And the word I'm, the word I'm looking for is is wisdom, right?\nBecause AI is AI. It's not AW. It's artificial intelligence, not artificial\nwisdom. And that's the difference between those of us sitting on this call and an agent. Right? We all\nhave intelligence, at least allegedly, and we all have wisdom. Some of us show it more\nthan others and the color of our hair at this point. But that's the point, is, it's the wisdom\nthat develops over time and learning behavior on do I trust someone or do I not trust someone?\nIt's that gut feeling or just that experience that comes with time that AI just doesn't\nhave that? I mean, every one of us has probably been in a situation where we went, this seems\nreally sus, right? AI is not going to do that. And it's not because of a failing of AI. It's just because\nit's not in its wheelhouse. It doesn't have that capability. And so this is a case of\nof, it's kind of a case of us being our own worst enemy because we're going to shove AI into\neverything, because AI will solve world hunger, world peace, and everything else it possibly can save.\nWe're just not asking ourselves how it's going to come about, how it's going to do it, how\nis it going to come about solving those problems, because it's going to do things that you would\nnever do as a human being, with the wisdom to know better. And so the question for me then becomes,\nyou know, we're circling around this issue of guardrails and needing to explain to AI what it\ndoes but how do you teach something wisdom? Like, can you? I don't know. I mean, Claire, I'm going to\nask you first any thoughts on like, what we do to maybe help AI get a little bit better against\nsocial engineering attacks? And any thoughts there? Well, I think the first thing you have to do is\nrecognize that AI doesn't have, like, a gut instinct. It doesn't have a gut to follow. So you\nneed to remember that AI doesn't. Doesn't have context like you and I do of\nlooking and feeling. It just has the context that we give it. So you have to remember that. You have\nto. You have to tell AI things that you don't necessarily think about. So even if it you kind of\nhave to think a little bit more about everything. Like you wouldn't think as an adult, like to tell\nanother adult to go, not go in the street when the cross sign is is off or on. But you need to tell\nthat to a child, right? Like, you need to think about almost as if you're explaining it to\nsomebody that is like completely new in the industry or completely new on Earth. Like, you have\nto just kind of think about giving it those contexts, which is a little more work. And I guess,\nlike you could in theory say, yeah, I'm going to ask an AI to do this for me. But I feel like you\nalso need to like reason as a human of what would I be doing and provide those guardrails. Yeah, I\nthink that's really good advice for AI use in general, right. No matter what you're using it for.\nLike, sure, you can give it the one sentence prompt, but if you really want it to do something like\nwrite good code for you, or maybe not give away account passwords, you do need to think about that\nextra context and give that to them. So I think you can apply that in general. Jeff, anything to\nadd there to Claire's thoughts on on kind of strengthening our AI against this? Sure, sure. If\nyou think about what AI is, the simplest definition I can think of is it's basically\ntrying to match or exceed human intelligence in a computer. Okay, so we're using ourselves as the\nmodel, and we're saying we want this system to be as smart or smarter than we are. And so we train\nit in the ways that we think. Well, look, your initial question about will humans or AI be more\nlikely to fall for this? It has implied in it that people, in fact, still fall for these things. And\nwe've had thousands and thousands of years of of developing common sense and developing a\ntradition where we teach each other, you know, don't fall for this, don't fall for that. And yet\nstill, phishing attacks work. So it's not like this is a solved problem for people. So\nit's definitely not a solved problem for AI. I do think we're going to need to spend more time than\nwe have on trying to teach AI what's right and what's wrong, because right now we've focused, I\nthink, most of our efforts on telling it, having it understand what's true and what isn't true. You\nknow, basic information and facts and things like that. Like you know, Nick mentioned wisdom and\nyou can think of this as kind of a pyramid. You know, you've got data, you've got information,\nyou've got knowledge. Wisdom is on top of all of that pyramid where we process each one of those\nthings to a greater degree. I didn't mention wisdom before because it's not a word I'm\nfamiliar with. I don't I don't deal in this in this area. As for those who know, but but the. I do\nthink we're going to have to make AI a lot smarter, not just in the book sense, but in the\nstreet sense as well. And we haven't spent a whole lot of time on that, because honestly, even though\nwe call it common sense, it's actually pretty hard to define. And, and what's common for one\nperson might not be common for the next, but this one clearly was a common sense case. And there\nshould have been, and I'm sure there are at these companies rules, procedures that say, if you're\ngoing to reset somebody's account, here are the things that must be met, the conditions that must\nbe met. And you have to make sure the AI follows those conditions just to the letter, you know, just\nlike anyone else would. It can be courteous, but it can say courteously, no, we're not going to, you\nknow, reset your account to this random email address. Yeah. And I like that you point out and I\nthink it's important to keep in mind, you know, we haven't solved this problem for people either.\nRight. So like the takeaway here isn't like, oh, the AI is not good. Like, you know, it's not like a\ngotcha kind of thing, right? It's like we are also not great at this. And so like as we're learning\nand trying to learn, we also need to be helping the AI try to learn. And like you said, Jeff,\ndevelop some of those street smarts we haven't focused on yet. Nick, to close this out, I want to\nask you a slightly different question, which is that so much of the conversation recently around\nAI in cybersecurity has revolved around these big, powerful models. You know, your Mythoses, your\nGPT-5.5s. And here's a story of a really, really simple, basic attack working. Is there a lesson\nhere for us? Yeah, we sometimes just miss the forest for the trees, right? We're trying to solve\nthe big problems when some of the small problems are still lingering about. Right. And then the\nquestion that I have in general is, can this be learned? And I don't have an answer. Can can wisdom\nor at least the the facsimile of wisdom can that be learned, right? Can it\nuse certain clues and logic and whatnot to determine if, okay, this person is asking\nto do said thing, but here are the things that should be the red flags on why I shouldn't do it\nright. So we have what we're calling that that gut feeling. Right. But AI is not going to be able to\ndo that. It's going to have to use logic. And is it possible? I don't know. Somebody smarter than me is\ngoing to have to figure that out. And we can look at this particular case and say, this was really\nstupid, you know, why did this happen? And therefore AI and we jumped to the conclusion, AI is not\nready for prime time. This stuff is junk, blah blah blah. Look, if if a single failure case was was\nenough to disqualify a technology, then humans would have been disqualified a long time ago.\nI guarantee you there are helpdesk agents who have made similar errors, and yet we don't\nsummarily dismiss all of humankind because of it. So. And there are and I do have to move this along,\nbut I have to mention it now that you said it, Jeff, is that I've had many conversations with folks in\nX-Force who do like, you know, social engineering testing. And they've always told me that, like, the\nmost successful attack we do is we just call the help desk, pretend we're the person, and need to\nreset a password. And it works almost every time. And so you're literally right. It can work against\npeople too. But I have to move this along to the next story. Before I do. I'm going to open it up.\nYou know, viewers on YouTube, if you have thoughts about whether we can teach an AI wisdom, let us\nknow. Maybe you have the answer, because I don't think any of us here do. But to move on. Our second\nstory for the week, University of Toronto researchers design a new AI worm.\nUsing an open source LLM, the researchers created what they call a self-replicating agent. It can\nspread from device to device, like any worm, using device resources to run a local model that can\nsupposedly reason its way through attacks, choosing different vulnerabilities and exploits\nfor each device it encounters. Now, I've been following the kind of AI malware story for a few\nyears now, and virtually every time I talk to people about it, they say it's not really a thing.\nLike, yeah, attackers use AI to generate malware code, but it looks a lot like regular malware code.\nAnd I'm wondering, have we finally reached a point now where AI malware is actually here? Like, is\nthis something different or more of the same? And Jeff, I'm going to ask you first, what do you think?\nIs this a genuine advancement or just more of the same? Well, my first reaction was, what took you so\nlong? I actually, I actually figured this was coming. To me, it was obvious. In fact, I've covered\nthis in some of the YouTube videos I've done on the IBM channel in the past as predictions as to\nwhere malware would go. Malware. I mean, I remember when we first started seeing the first samples of\npolymorphic viruses, these viruses that would change themselves as they, you know,\npropagated. And everyone thought that was going to be the end of the world. Well, you know, the world\nis is still spinning around on its axis and we're all still here. And so people that look at this\nprobably will jump to the same conclusion. And it's definitely a bigger risk. It's not\nsurprising to me that this happened. And as large language models become small language models,\nbecome even smaller language models, these things will be more containable and more portable and\nmore able to be sent around. So I fully expect to see more of the same,\nwhich means the good guys just have to use a similar but better technology to do the\ndetection and repelling and prevention and all that kind of stuff. But it's this is the arms\nrace. And like I said, I'm just surprised that it took so long for this to happen. I think that's a\nreally fair point. And I didn't even think about the bigger trend. Like you said, the models are\ngetting smaller and the smaller they get, the more it becomes possible to do an attack like this\nthat literally spreads a model from computer to computer. Claire, any thoughts on your end?\nLooking at the story of the AI worm, what's it got you thinking about? Anything come to mind in terms\nof risks? Things you're concerned about? My first thought was, I'm sure this isn't the first\nAI malware that we've seen out there that's been partially developed. I also don't think a hacker\ngroup is going to be like, I used AI to develop this malware that I'm sending to all these people.\nIt's kind of like you're not going to give away your, like, secret operating sauce. It's kind of\nlike quantum too, where it's like hackers are not going to, you know, most likely come out and say,\nI've used quantum computing like to decrypt all this data. I think it's probably similar where\nyou know, that they're they're using AI, but, you know, we'll start to see it more. I\nmean, at the end of the day, I've said this a million times on this podcast, but these are\ncybercrime businesses. They are trying to just make their products and services a little more\nprofitable for themselves. So they're just trying to improve ROI. And if AI-related malware does\nthat, then they're going to keep using it. It depends how successful it is, too, in terms of, you\nknow, how we see it propagate. Yeah, I mean, it ties back into what Jeff said about this being an arms\nrace, and it's an arms race in a pretty literal sense that, like, we're trying to develop these\ntechnologies, they're trying to develop these technologies. We're subject to almost the same\nkind of market pressures. And so I like that you brought it back to that, Claire. Reminded us that,\nlike, these are businesses. They're not just doing this for fun. They're trying to get some kind of\nreturn on it. Nick, how about you looking at this? Do you feel like this is a genuine leap\nforward? More of the same where you land in here? I think it is a logical expectation in the\nevolution of what we've been watching. Between the smaller models, you know, more compact, less of\na footprint on top of the of the things we're seeing with the frontier models. This was to,\nto use the same word again. This was inevitable. It was going to happen. And I agree with Jeff on why\ndid it take so long. But I'm going to I'm going to add on to that is I don't think it did\ntake so long. I think it's been there. And this is just the first time we've got good guys that have\ndone it and just said, here, look what we've done, and then you've got the bad guys going, \"Rats — shut up!\"\nYep. Yep. Kicking them under the table. Right. My my my thought on this\nthough is it's going to happen. So take advantage of it. We could use the same the same methodology\nto let this run and find those same vulnerabilities that it's going to reason its way\nthrough, but instead of taking advantage of them, give me a report, tell me how to fix it. Tell me\nwhat I need to go do to tighten up my environment so the same tool could be used in for for good\ninstead of evil. I'm glad you brought up that that take on it, Nick, because I saw people saying\nsomething similar, right. That like, hey, if we can use it to spread an AI to like look for\nvulnerabilities that it then exploits, why not use it to spread an AI that looks for vulnerabilities,\nthat it then patches and like that seems like a really. A lot of times when we talk about, you know,\nAI attacks, I end the segment feeling kind of down. I feel pretty okay about this one, you\nknow, because this is one where the development maybe can be readily used for, for good ends. But\nbut before we move on, I do want to talk a little bit more about, you know, the so what, the how do we\nprotect ourselves and and Jeff, I want to I want to ask you this question because you were on the\nepisode we did a little while ago about protecting open source AI infrastructure. And one\nof the interesting things about this worm is that they used an open source model. They didn't say\nwhich one, but they said they use an open source model. And part of the reason they did that was\nbecause that means there's no, you know, OpenAI or Anthropic watching what they do with the model\nand being able to kick them off the platform. Do you think this is a problem we're going to have\nto contend with, people using open source AI models for for bad purposes? And what do we do\nabout it? Any thoughts there, Jeff? Oh, no doubt it's an issue, but this is one where you know, the\nthere's, all the animals have escaped. There's no point trying to lock the barn door at this point.\nBy that I mean, we've got Hugging Face, which is an, an open AI model\nrepository. Think of it as GitHub but for AI models. There's more than 2 million AI models\non there already. Okay, so there's no way you're going to make that stop. There's no way to unring\nthat bell. Or toothpaste back in the tube, genie back in the bottle. If I can think of any more\nanalogies, I'll, I'll run them all down into the ground. But. So this is this is our new\nreality, and we have to accept it. I think I think there's a larger trend going on with regards to\nAI, because the change has occurred so fast that we've seen these kinds of of new attack\ntypes, or it seems new to people. It's really variations on a theme done at greater velocity\nand greater volume. So therefore it feels new. But that, that we, we've got\nthere's no point in saying, stop everybody, just stop it, you know, don't do this anymore. And people\nare saying that when it comes to using AI in the classroom, they're saying, don't use it with music,\ndon't use it, you know, in this area, that area. All these other different areas. Look, it's already out\nthere. So now we have to accept the reality that it's out there, or use an AI model that's too\ndangerous for everyone to have because you know it, it uncovers vulnerabilities. Well, actually, that\ncould be a good thing if it's in the hands of the right people. So all of these things have the\npotential equal potential to do harm as they do to do, you know, good things. And it all depends\non whose hands it's in and what their motivations are. I was just going to say I loved all the\nanalogies, but the one I want to stick with, well, it's not even an analogy, but it was something you\nsaid on the don't do that. You know what I want to do the second you tell me not to do something, I\nwant to do it. Right. And that's that's human nature. Don't push that button. Don't open that\ndoor, don't touch that. I'm going to do all the things, right? And and that's just. The best way\nto to get me to do something is tell me don't do it. 100%. And so we need to understand that\nPandora's box is open. Everything's out. Now we just have to deal with it. Nick, that's what I\nwanted to stress, too, that part of the of Jeff's response, because I think you're really right to\npoint out. Look, people can sit there and say, I'm not going to use AI for XYZ reason. Okay, fine. The\nattackers are going to use it, though, like they're going to stop using it. So sure, you can\nwillingly take yourself out of that race, but it just means you're going to get left behind. You\nknow what I mean? At a certain point, you kind of got to get in there. Claire, to round out this\nsegment, though, for us, any last thoughts on, you know, what this might mean for security? Any advice\nyou would give organizations? What's your take here? I think a lot of organizations know by now\nthat security is moving so much faster and evolving every day. I think something that you\nsaid, Matt, that or maybe Nick, you said it, that if you're not, like, in the race with it, you're\ngoing to get left behind. So, I mean, if you're not looking at AI security solutions, if you're\nnot following AI and security at all. You've kind of been left behind already. You can catch up and\nyou should catch up. You can stick your head in the sand if you're afraid of the rain, but all\nyou're going to do is drown. Yep. On that note, folks, I'm going to move this along to our final\nstory for the week. This is the State of Identity Security 2026 report.\nThis is Sophos' survey of 5000 IT and cybersecurity leaders. And it found that 71% of them\nsuffered at least one identity-related breach in the last year. I don't think this comes as a\nsurprise to any of us. You know, IBM, we do the X-Force Threat Intelligence Index every year, and\nidentity-based attacks are a number one attack vector. They were number two this year, but still\nlike 32% of attacks involved them. So like we know it's a big thing, right? What was especially\ninteresting to me about this report though, and the reason I included it here, was some of the\nstats around non-human identity specifically. Sophos kind of broke out which attacks involved\npeople stealing user identities and which involved them stealing non-human identities. You\nknow, AI agents. You know, APIs, service accounts, those kinds of things. And they found that\nnon-human identities were involved in 41% of successful identity breaches, and only about a\nthird of organizations regularly audit or rotate their NHIs or their credentials. Nick, I want to\nask you, because this is something that's come up over and over again on the show. What is it about\nnon-human identities, non-human credentials that makes them such a weak spot? Why are we\noverlooking them still? Any thoughts there? Because no one's watching. If I steal your password\nand log into your account, your machine, or whatever it is you do. It's only a matter of time\nbefore you figure out that somebody has gotten ahold of your credentials. But if I get a hold of\na service account, are you ever going to find out if you're not watching it or auditing the\nactivities of that account? So it's it's brilliant, to be honest, because I'm stealing something that\nno one was even watching in the first place. So it's going to continue to happen unless we put\nbetter mechanisms in place to monitor these things. Because as it was said in the article, some\nof these, these service accounts come into play and they're used forever. They're not rotated,\nthey're not monitored. They're just they're there until they fail. And then someone tries to figure\nout, wait, what's broken? Who set up this account? The person that set up the account is not even\nhere anymore. So I think that's an extremely good point. And it's a very simple one. And it's one\nthat again, it didn't cross my mind. But but you're right. Like somebody is looking at a user account\nevery day. Someone goes in there. Most of them are not looking at service accounts every day.\nNobody's like you said, nobody looks until they fail. Jeff, any thoughts on how we start to maybe\nget a little more visibility into this kind of thing, or start protecting these accounts better?\nWhat's your take there? Sure. Well, first of all, I'll make the the master of the obvious statement.\nIdentity is hard. It always has been. It's seemingly always going to be. I mean, I've\nbeen working in the identity and access management space for more than a quarter of a\ncentury, and we still haven't solved all the problems. We're still solving the same problems\nthat we've had for, you know, over and over and over again. And, and I look at it this way. When we\nfirst started off, you know, if I wanted to give you an account, well, you filled out a paper form\nand that got bucked around and somebody signed it, and then an administrator went and created your\naccount. And then we put that in a file cabinet. I mean, it was all a very manual process, very\nerror prone, very inconsistent. And then we started moving. And yet still some organizations have not\nfully done this, moving to more automated identity management systems, where we can do provisioning\nand deprovisioning automatically based upon your job role and things like that. So that sped\nthings up and give us more insight and accountability into the systems. There's a new\nevolution that has to occur as well, that even this report I don't think has anticipated, and it's\nrelated to non-human identities. But the fact that we're going to need way more than we think we do.\nAnd creating some of these non-human identities. I mean, they're not going to go fill out a form.\nThey're not going to go get hired by HR and then have a job role that we can map them to and so\nforth. They're going to need privileges for maybe a few seconds and then that's it. They pop up. They\ngo away. When we're talking about agents. A lot of these kinds of capabilities, these are ephemeral\nIDs. These are things I need to go do this right now. Okay. We'll provision you to do that. But\nprinciple of least privilege says we're not going to let you do anything more than just what is\nabsolutely necessary. Even though you're an agent, I don't trust you any further than I can throw\nyou. And so I'm only going to I'm going to bound you to this and only for this period of time. And\nthen your ID goes away. This is the kind of systems that we're needing to build now. And for\nthe most part, people don't have those. And things like OpenClaw that allow anybody to run an agentic\nframework on their laptop, I guarantee you they haven't thought about all of this. So they're\nrunning these things under the main user account in many cases. You know, the essentially the root\nuser on the system, you know, the sysadmin, the the the, the superuser account on that, you know,\nWindows or Mac system. And therefore if the agent makes a mistake, well, it's got the full\nprivileges of that user and it can make a real big mess in a hurry. A friend of mine\non, on LinkedIn just recently sent me an article where it referred to, what was it?\nIt, it basically, AI is a fast fool, and that's what it is. It can do these things really\nfast and miss a point that we would have been able to have, have gated and, and\npaced more if a human was doing it. So that's another aspect of identity management that is\nonly going to get more complicated as we move forward. Absolutely. And I'm glad you bring up this\nidea of like, especially spinning up ephemeral IDs and ephemeral permissions and how we manage that.\nAnd it reminds me a lot of this idea I see coming out of the kind of HashiCorp wing of IBM right\nnow around security lifecycle management and how you automate things like provisioning these\naccounts, making sure they have just the right privileges for just the right time, just in time\naccess. A lot of people are starting to think about this thing. But like you said, Jeff, you know,\nidentity is hard. And so we're working our way through it. Claire, I want to bring you in here.\nLooking at the report or things we've talked about so far, what's what's coming up for you?\nWhat's sticking out? What are you thinking about? I think a common misconception with non-human\nidentities is that there's, like, no human attached, like at any point. But at the end of the day, a human\ndid provision this at some point or it provisioned it at some point in the chain. So like\nwhatever mistakes the the person made at some point are then moving around and going\ndown the chain, it's kind of like if you think about when a fish eats plastic and then a bigger\nfish eats that, and like the plastic just builds, it's like something it's like the mistakes just\nkind of keep rolling. And we're seeing a lot of clients, like, consider,\nnon-human identities and their experiences, and sometimes their executive teams are a little bit\nlike, oh, wait, what does this even mean? So it is something too, that's like a little bit meta where\nit's like, what do you mean? We have people and we don't have people at the same time. Like we just\nhave all these identities. But it's interesting because at the end of the day, they do\nalways tie back to a human somewhere far back in the line. Yeah, that makes a lot of sense to me. You\nknow, and it is funny to point out, like non-human identities, this thing we talk about a lot now, but\nlike, it feels kind of far out, you know what I mean? Like, am I talking about aliens? Like, what do\nyou mean by a non-human identity? And it's no, these, these machines that are active. And I think\nfocusing on that person who's there at some point, like I think that's a that's a good place to\nstart looking, right. Like what are they giving. How are they setting these things up? It's like Jeff\nsaid before, people setting up OpenClaw without knowing what they're getting into. If they know\nwhat they're getting into, maybe we can head off more things at the pass, lest that plastic gets\nthrough. Like you said. Claire. Nick, I have to close this out. But before I do, last\nthoughts on this issue of non-human identity security? What we do to better going forward? Any\nlast words for us? I think we're going to have to figure out some type of dynamic behavior analysis\nthat we haven't conceived of yet. And sadly, it's probably going to take AI to solve it. So we've\njust come full circle. The AI solves the problems that the AI makes for us folks. That is the story\nof security today in many ways, but that does it for this episode. I want to thank our panelists,\nClaire and Nick. And Jeff. Thank you to the viewers and the listeners and our producers. Subscribe to\nSecurity Intelligence wherever podcasts are found, so that you never miss an episode. Stay safe out\nthere and remember, when it comes to their susceptibility to social engineering and\ncredential theft, non-humans are basically humans, too.\nFolks.\nWho do you think is more likely to fall\nfor a social engineering scheme,\na human being or an AI agent?\nFeel like an.\nAI right now?\nThe answer is AI is going to fall for it.\nI do think\nwe're going to have to make AI a lot\nsmarter, not just in the book sense,\nbut in the street sense\nas well.\nHello,\nand welcome to Security Intelligence,\nFolks, IBM's weekly cybersecurity podcast,\nwhere our expert panelists\nturn the biggest industry news stories\ninto practical takeaways you can use.\nI'm your host, Matt Kozinski, broadcasting\nlive from some kind of dungeon.\nAnd joining me this week.\nWe've got a real classic\nsecurity intelligence line up.\nIt's Claire Nuñez, creative\ndirector, IBM X-Force Cyber Range,\nJeff Crume, distinguished engineer, master\ninventor, data and AI security.\nAnd Nick Bradley, manager, X-Force\nThreat Intelligence.\nAnd on the docket today, we've got an\nAI powered worm designed by University\nof Toronto researchers and the Sophos\nState of Identity Security 2026 report.\nBut first, we're going to keep talking\nabout AI falling\nfor social engineering schemes,\nbecause some hackers tricked\ncustomer support agents into handing over\nInstagram account passwords.\nNow, 404 Media broke this story last week.\nIt's about a spate of attacks.\nWere hackers just posed as the legitimate\nInstagram account owners messaged Meta's\nAI customer support agent\nand asked it to update their accounts\nwith new emails, emails\nthey controlled and the agent just did it.\nIt just handed the accounts\nright over to these people,\nput their emails in there,\nand they were able to use that\nto recover the passwords\nand take over the accounts.\nNow, the vulnerability\nhas since been resolved,\nbut I think there's something to learn\nhere.\nAnd, Claire, I want to start with you,\nand I want to ask, are you surprised\nby just how easy it seems like\nit was to pull off this attack?\nDoes it give you any concerns\nabout AI in customer facing roles?\nHow are you feeling?\nI feel like it's surprising on one aspect\nbecause like\nthe whole premise of Instagram\nis like your account\nand you being able to access your account.\nSo you would think Meta would have\nthat, you know, under lock.\nBut it's not surprising at the same time\nthat something so seemingly simple\nhas been overlooked.\nI mean, as\nJeff mentioned, agents don't understand\nnuance, and they are very naive.\nSo, if you they're not going to ask\nwhy are you changing this email?\nSo it's not that surprising to me,\nbut you would think that a company\nthat kind of like, revolves\naround your account and your ability\nto access that account for, like,\neverything,\nwould have looked\nat that a little more closely.\nAbsolutely.\nAnd, you know,\nI know I think it reminds me of, like,\nit's very easy to trick a person, right?\nMaybe I shouldn't say very easy, but\nit can be quite easy to trick a person.\nAnd one of the promises of AI\nis supposed to be that it's it's,\nyou know, it's got certain context\nthat we don't have that maybe won't fall\nfor some of these things,\nbut we're seeing the opposite happen here.\nAnd, Jeff, I wanted to ask you,\nyou know, similar question to Claire,\nlike, looking at an AI agent fall\nfor something like this.\nDoes it give you concerns\nabout putting these things\nin customer facing roles right now?\nHow do you feel?\nAI is like this?\nThis person that's got, a thousand PhDs\nbut has never spent\none day above ground,\nyou know, out of their parents basement.\nSo a lot of intelligence in one level,\nbut not a lot of common sense,\nnot a lot of the kind of lived experience\nthat allows us to realize\nwhen somebody is trying to con us or not,\nwhen somebody's trying to ask\nfor something that's unreasonable or not.\nI mean, if you put in the system prompt,\nyou're a helpful, customer\nsupport assistant.\nWell,\nthen it's going to try to be helpful,\nand it's going to be very eager and work\nas hard as it can to satisfy the request.\nAnd just like a person,\nif you haven't taught them.\nYeah, but every person that asks for\nyour wallet, you don't hand it to them.\nYou know, you want to be polite,\nbut you know, there's certain limitations\nto that kind of politeness.\nAnd what people ask you to do.\nAnd reasonable is text.\nThis one was way too easy, though.\nI mean, the attackers\njust basically put their VPN\nso that it looked like\nthey were in the area of where the\nthe person would have been\nand then said, you know, hey,\nsend me a reset code for my account\nto this new email address.\nAnd we know you shouldn't\nbe able to do that without some other form\nof authentication,\nbut unless you explicitly say,\nno, don't do this, it's\ngoing to probably try to do it.\nAnd that's that's the thing\nwe all had to learn as people.\nYou know, we were growing up\nand we got fooled a lot of times.\nWell, AI is really young, too.\nIt might be really smart, but that doesn't\nmean it's not very young and naive.\nAnd this is a classic example of that,\ngot to be a lot,\nlot more specific in terms\nof what we tell them, what they can do.\nI think about my grandson,\nhe's four years old,\nand we're having to constantly\ntell him the kinds of things,\nyou know, you don't play in the street.\nAnd here's why.\nOkay, that sort of stuff\nthat we take for granted.\nBut everybody had to be taught that.\nAnd these AIs have to be taught\nthis as well.\nThat's a very good point.\nAnd I like this comparison.\nYou draw actually with, an age,\nyou know, young folks, I think of my son.\nRight. He is just about three years old.\nAnd it's a similar thing.\nI got to tell him, you know, you can't\nhang out with the cars in the streets.\nIt's actually not.\nI know you like the cars,\nbut you can't go out there. It's not safe.\nAnd it goes back to this idea that you\nyou brought\nup, which is that like we have to be\nvery specific with directions here.\nAnd so in, in ways that I was like\nslightly surprised by this attack.\nI think maybe I shouldn't have been\nsurprised because like you said, Jeff,\nwhen you tell an agent\nyou're here to help customers,\nyou know, do things they need to do,\nunless you specifically\ntell them not to do that.\nThey want to help customers.\nThey they think\nthey're doing the right thing\nand they're just going to try to do it.\nSo I think it's a very good point.\nNick, I wanted to ask you,\ndo you think there's any kind\nof meaningful difference\nbetween tricking a person\nwith a social engineering attack,\nor tricking an AI agent\nwith a social engineering attack like\nthis is basically the same kind of thing.\nAre they slightly different?\nHow are you feeling about it?\nSo there is a key word\nthat I was looking for that I was hoping\none of the three of you were going to drop\nand you didn't.\nSo I'm kind of glad because\nthen I wouldn't have had anything to say.\nAnd the word I'm the word I'm looking for\nis is wisdom, right?\nBecause AI is AI.\nIt's not AW, it's artificial\nintelligence, not artificial wisdom.\nAnd that's the difference between those of\nus sitting on this call and an agent.\nRight?\nWe all have intelligence,\nat least allegedly,\nand we all have wisdom.\nSome of us show it more than others\nand the color of our hair at this point.\nBut that's the point, is, it's the wisdom\nthat develops over time and learning\nbehavior on do I trust someone\nor do I not trust someone?\nIt's that gut feeling,\nor just that experience that comes\nwith time that AI just doesn't have that\nI mean, every one of us has probably been\nin a situation where we went,\nthis seems really sus, right?\nAI is not going to do that.\nAnd it's not because\nof a failing of AI, it's\njust because it's not in its wheelhouse.\nIt doesn't have that capability.\nAnd so this is a case of\nit's kind\nof a case of us being our own worst enemy\nbecause we're going to shove\nAI into everything because AI will solve\nworld hunger, world peace,\nand everything else that possibly can save.\nWe're just not asking ourselves\nhow it's going to come about,\nhow it's going to do it,\nhow is it going to come about\nsolving those problems, because\nit's going to do things\nthat you would never do as a human\nbeing with the wisdom to know better.\nAnd so the question for me then becomes,\nyou know, we're circling around\nthis issue of guardrails\nand needing to explain to AI what it does,\nbut how do you teach something\nwisdom like, can you?\nI don't know,\nI mean, Claire, I'm going to ask you first\nany thoughts on like,\nwhat we do to maybe help\nAI get a little bit better\nagainst social engineering attacks\nand any thoughts there?\nWell,\nI think the first thing you have to do\nis recognize that\nAI doesn't have, like, a gut instinct.\nIt doesn't have a gut to follow.\nSo you need to remember that AI doesn't\ndoesn't have context like you and I do\nof looking and feeling it.\nIt just has the context that we give it.\nSo you have to remember that.\nYou have to you have to tell AI things\nlike, you don't necessarily think about.\nSo even if it you kind of have to think\na little bit more about everything.\nLike you wouldn't\nthink as an\nadult, like to tell another adult to go,\nnot go in the street\nwhen the cross sign is is off or on.\nBut you need to tell that to a child,\nright?\nLike you need to think about\nalmost as if you're explaining it\nto somebody that is like\ncompletely new in the industry\nor completely new on earth.\nLike you have to just kind of think about\ngiving it those contexts,\nwhich is a little more work.\nAnd I guess, like you could in theory say,\nyeah, I'm going to ask an AI\nto do this for me, but I feel like\nyou also need to like reason as a human of\nwhat would I be doing\nand provide those guardrails.\nYeah, I think that's really good.\nAdvice for AI use in general, right?\nNo matter what you're using it for.\nLike, sure,\nyou can give it the one sentence prompt,\nbut if you really want it to do something\nlike write good code for you, or\nmaybe not give away account passwords,\nyou do need to think\nabout that extra context\nand give that to them.\nSo I think you can apply that in general.\nJeff, anything to add there?\nTo Claire's thoughts on on kind\nof strengthening our AI against this.\nSure, sure.\nIf you think about what AI is,\nthe simplest definition I can think of is\nit's basically trying to match or exceed\nhuman intelligence in a computer.\nOkay, so we're using ourselves\nas the model, and we're saying\nwe want this system\nto be as smart or smarter than we are.\nAnd so we train it\nin the ways that we think.\nWell, look,\nyour initial question about will humans\nor AI be more likely to fall for this?\nIt has implied in it that people, in fact,\nstill fall for these things.\nAnd we've had thousands\nand thousands of years\nof of developing common sense\nand developing a tradition\nwhere we teach each other, you know,\ndon't fall for this, don't fall for that.\nAnd yet still, phishing attacks work.\nSo it's not like\nthis is a solved problem for people.\nSo it's definitely not a solved problem\nfor AI.\nI do think we're going to need\nto spend more time\nthan we have on trying to teach\nAI what's right and what's wrong,\nbecause right now we focused,\nI think most of our efforts\non telling it, having it understand\nwhat's true and what isn't true.\nYou know, basic information and facts\nand things like that.\nLike you know, Nick mentioned wisdom\nand you can think of this\nkind of a pyramid.\nYou know, you've got data, you've got\ninformation, you've got knowledge.\nWisdom is on top of all of that\npyramid where we process\neach one of those things\nto a greater degree.\nI didn't mention wisdom before\nbecause it's not a word I'm familiar with.\nI don't I don't deal in this in this area.\nAs for those who know, but but the I do\nthink we're going to have to make AI a lot\nsmarter, not just in the book sense,\nbut in the street sense as well.\nAnd we haven't spent\na whole lot of time on that,\nbecause honestly,\neven though we call it common sense,\nit's actually pretty hard to define.\nAnd, and\nwhat's common for one person\nmight not be common for the next,\nbut this one\nclearly was a common sense case.\nAnd there should have been,\nand I'm sure there are at these companies\nrules, procedures that say if you're\ngoing to reset somebody's account,\nhere are the things that must be met,\nthe conditions that must be met,\nand you have to make sure the AI follows\nthose conditions\njust to the letter,\nyou know, just like anyone else would.\nIt can be courteous, but it can say\ncourteously, no, we're not going to,\nyou know, reset your account\nto this random email address.\nYeah. And I like that.\nYou point out\nand I think it's important\nto keep in mind,\nyou know, we haven't solved this problem\nfor people either. Right.\nSo like the takeaway here isn't like,\noh, the AI is not good.\nLike, you know, it's\nnot like a gotcha kind of thing.\nRight?\nIt's like we are also not great at this.\nAnd so like as we're learning\nand trying to learn,\nwe also need to be helping the AI\ntry to learn.\nAnd like you said, Jeff, developed\nsome of those street smarts\nwe haven't focused on yet.\nNick, to close this out, I want to ask you\na slightly different question,\nwhich is that so much of the conversation\nrecently around\nAI in cybersecurity has revolved\naround these big, powerful models.\nYou know, your Mythos, your GPT-5.5s.\nAnd here's a story of a really,\nreally simple, basic attack working.\nIs there a lesson in here for us? Yeah.\nWe sometimes just miss the forest\nfor the trees.\nRight?\nWe were trying to solve the big problems\nwhen some of the small problems\nare still lingering about. Right.\nAnd then the question that I have in\ngeneral is can this be learned?\nAnd I don't have an answer.\nCan can wisdom or at least,\nthe facsimile of wisdom,\ncan that be learned.\nRight.\nCan it use certain, clues and logic\nand whatnot to determine if,\nokay, this person is asking to do\nsaid thing, but here are\nthe things that should be the red flags\non why I shouldn't do it right.\nSo we have what\nwe're calling that that gut feeling.\nRight.\nBut AI is not going to be able to do\nthat.\nIt's going to have to use logic\nand is it possible?\nI don't know, somebody smarter than me\nis going to have to figure that out.\nAnd we can look at this particular\ncase and say,\nthis was really stupid, you know,\nwhy did this happen?\nAnd therefore AI\nand we jumped to the conclusion,\nAI is not ready for prime time.\nThis stuff is junk, blah blah blah.\nLook, if if a single failure case\nwas was enough to disqualify\na technology, then humans would have been\ndisqualified a long time ago.\nI guarantee you\nthere are helpdesk agents who have made\nsimilar errors, and yet we don't,\nsummarily dismiss all of humankind,\nbecause of it. So.\nAnd there are\nand I do have to move this along,\nbut I have to mention it\nnow that you said it,\nJeff, is that I've had many conversations\nwith folks in X-Force who do like,\nyou know, social engineering testing.\nAnd they've always told me that, like,\nthe most successful attack we do is\nwe just call the helpdesk, pretend we're\nthe person, and need to reset a password.\nAnd it works. Almost every time.\nAnd so you're literally right.\nIt can work against people, too.\nBut I have to move this\nalong to the next story.\nBefore I do.\nI'm going to open it up,\nyou know, viewers on YouTube,\nif you have thoughts about whether\nwe can teach an AI wisdom, let us know.\nMaybe you have the answer,\nbecause I don't think any of us here do.\nBut to move on, our second story\nfor the week,\nUniversity of Toronto researchers\ndesign a new AI worm.\nUsing an open source LLM,\nthe researchers created\nwhat they call a self-replicating agent.\nIt can spread from device to device.\nLike any worm, using device resources\nto run a local model\nthat can supposedly reason\nits way through attacks, choosing\ndifferent vulnerabilities and exploits\nfor each device it encounters.\nNow, I've been following the kind of\nAI malware story for a few years now,\nand virtually every time\nI talk to people about it,\nthey say it's not really a thing.\nLike, yeah, attackers use AI to generate\nmalware code, but it looks a lot\nlike regular malware code.\nAnd I'm wondering,",
  "transcript_chars": 50829,
  "ingested_at": "2026-06-17T04:32:29.761405+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": null,
    "like_count": null,
    "channel_id": null,
    "categories": null,
    "tags": null
  }
}