{
  "video_id": "7pOom_qlAH4",
  "channel_slug": "mitcsail",
  "channel_handle": "mitcsail",
  "title": "PART 2: MIT Professor on How AI & LLMs are Shaping Financial Advice, Analysis, & Risk Management",
  "duration_seconds": 989.0,
  "url": "https://www.youtube.com/watch?v=7pOom_qlAH4",
  "upload_date": "",
  "transcript": "(air whooshing)\n(text buzzing)\n- \"What are the current limitations\nof LLMs in financial contexts\nand how might future advancements in areas\nlike causal reasoning and\nknowledge representation\nunlock new possibilities?\"\nWell, that's a really\ninteresting question.\nThere are a number of limitations\nthat I think many people know about.\nLLMs are not that good at numbers per se.\nThey're very good with language,\nbut for whatever reason,\nwhen you start asking numerical questions,\nthey easily get tripped up.\nA friend of mine recently\njust gave me an example.\nTry asking LLM to name all of the capitals\nof European countries and\nrank them by population.\nThat seems like a pretty\nstraightforward question,\nbut you'd be surprised how difficult it is\nfor typical LLMs to get that correct.\nAnd there are all sorts of conjectures\nas to why that might be the\ncase, but we don't really know.\nAnd so at this point, we\nneed to start coming up\nwith better ways of informing LLMs\nwith respect to these kinds\nof numerical questions\nand to be able to integrate them\ninto their text-based abilities.\nThat's a key aspect of financial context\nbecause everything that we do\nin finance is both numbers and language.\nIn terms of causal reasoning,\nnow this is getting on the\nborder of the theoretical\nand beyond it the science fiction.\nThe question that I sometimes\nget asked by my students is,\ndo LMS really understand?\nAnd you know,\nI have to answer the\nquestion with the question,\nwhich is, well, what do\nyou mean by understand?\nAnd that's a very difficult\nconcept to deconstruct\nbecause we ourselves don't understand\nwhat we mean by understand.\nBut lemme give you an idea\nof how I'm beginning to\nunderstand understanding.\nThe way that you test\nwhether somebody really\nunderstands something is very much\nlike how we test our PhD candidates\nwhen we give 'em the oral exam.\nAnd I'll tell you how I give my oral exam.\nI'll start with an easy question,\nand if the student\nthat's taking the exam\nanswers it correctly,\nI'll move on, and I'll\ngive 'em a harder version.\nAnd if they answer that correctly,\nI'll give them yet another\nversion that's even harder.\nAnd I'll keep increasing\nthe level of difficulty\nand changing the context of the question\nto really probe the limits\nof that student's knowledge.\nBecause what I wanna understand\nis how much they understand,\nhow deep is their understanding.\nSo at some point, I'll get to the point\nwhere a student will get it wrong,\nand that tells me that they\ndon't really understand\nthe particular question that I'm asking.\nAnd I'll ask them one\nmore question after that,\nthat is deeper still,\njust to see whether or not it\nwas just a careless mistake\nor whether or not it's really true\nthat now we're in deeper waters\nthat they really haven't ever tread.\nAnd if they continue to\nfalter, I will have determined\nthat that's their level of understanding.\nSo let's apply that approach\nto a large language model.\nWe ask it a question,\nit provides an answer,\nand the answer seems correct.\nWe'll provide a harder version,\nand that seems correct.\nAt some point, we will reach a level\nwhere the LLM will not be\nable to answer correctly.\nAnd so you can think of it as going down\nbelow the level of knowledge to the point\nwhere there's no more\nanswers to be gotten.\nThat kind of depth of knowledge\nis a form of understanding.\nRight now, I would say that\nLLMs are quite shallow.\nThey can answer in complete sentences.\nMost of those sentences make sense.\nAnd if you ask them questions,\nthey can usually provide\nreasonably correct answers.\nIf you ask them to write a piece of code\nthat engages in certain calculations,\nthey can usually do that correctly,\nperhaps with some modification.\nBut that tells me that their level\nof understanding is very shallow.\nThe next generation of LLMs\nought to be able to answer questions\nat 2, 3, 4 levels of depth.\nAnd there will come a point in time\nwhere they will exceed our capacity\nto answer those questions.\nAnd at that point,\nI believe we will then\nsay they understand,\nand then causal reasoning\nwill be possible.\nSo it's coming, but in the world of LLMs,\nit's gonna take a while\nbefore they get to the point\nwhere we consider it to be\ntruly causal understanding.\nWe've got a question from one\nof our social media followers.\n\"Applications of machine learning\nto forecast in clinical trial outcomes\nand improving investment performance\nin the biopharma industry.\"\nThank you for that.\nThat's referring to some research\nthat I've done over the last few years\nwith a number of my collaborators\napplying machine learning techniques\nto predicting the outcome\nof clinical trials.\nI got interested in that\nbecause a number of friends\nand family were dealing\nwith various kinds of\ncancer a few years ago,\nand I realized that actually\nmoney plays a huge role\nin drug development.\nIt turns out that money can be scarce,\nparticularly at the very\nearliest stages of drug discovery\nbecause that's when the\nrisk is the highest.\nAnd so the so-called valley\nof death between, you know,\nearly stage research and the\nfirst in human clinical trials,\nthat's the area that I\nstarted thinking about\nhow do we get more money\nto come into that space\nto help get over this valley of death?\nAnd the answer of course\nis you need investors,\nbut how do you attract investors?\nThe way you attract\nthem is by demonstrating\nthat there are some\nattractive opportunities\nto make money with their capital.\nAnd in order to do that,\nyou need to have some kind\nof statistical framework\nfor thinking about risk and reward.\nOver the last few years, my collaborators\nand I have applied machine\nlearning techniques,\nand now we're using large language models\nto try to read all of\nthe body of information\nabout medical clinical trials\nand what kinds of features tend\nto make them more successful\nand other features that\nmake them less successful.\nAnd use those features to predict outcomes\nso that we can tell investors,\nunder what circumstances\nare you likely to do better\nor worse, we can actually\nprovide them with a framework\nfor thinking about how to manage risk.\nAnd if we do that,\nwe assume that they will\nthen be more willing\nto deploy their capital as\nthey have in other industries\nwhere these tools have\nbeen made available.\nSo stay tuned,\nand hopefully we'll be able\nto tell you more about that\nover the course of the next few years\nas we apply these models\nin practical circumstances.\nOkay, another social media post.\n\"What's the dark side of AI,\nand what's the bright side,\nand which is more likely to occur?\"\nWell, that's the age old\nquestion of good versus evil,\nand happy to talk about both.\nI think I already mentioned a\nbit about the dark side of AI.\nPowerful tools can be\nused for adverse purposes.\nThings like cheating on your tax returns,\ncoming up with fake news\nthat will fool a certain\npart of the population\nfor financial or political gain.\nThose are the kinds of things\nthat not only can AI be used for,\nbut I suspect that has\nalready been used for.\nAnd so I think that we really need\nto consider ways of guarding against that.\nWe need antidotes for fake\nnews and for manipulation.\nThat really is not what we\nintend with these amazing tools.\nOn the positive side, I think\nthere is tremendous potential\nfor AI to transform society.\nI'll give you one very,\nvery simple example\nthat's already being done.\nIf you take patient medical records\nand look at the various\ndifferent kinds of behaviors\nthat people engage in\nand the diseases that they suffer from,\nwe now are able to come up\nwith all sorts of hypotheses\nfor certain kinds of things\nthat we can do to improve our health.\nNow you already know some of\nthose things, like exercise,\nhave a proper balanced diet,\nsleep well, so on and so forth.\nBut there are lots of nuances\nto those broad-based pieces of advice\nthat have to be tailored just to you.\nAnd so this is where\nLLMs can really shine.\nThey can take your medical\nrecords, they take the sum total\nof all of the knowledge\nthat's in the medical system\nand ultimately tailor that advice\nto be suited to your\nparticular health goals.\nNow we're not there yet because obviously\nthe hallucination problem\ncan be very, very dangerous\nin the medical context,\nbut I don't think we're that far away,\nparticularly given the rapid pace\nat which we're making progress.\nOkay, another social media post.\n\"How can non-engineers\nget involved in AI?\"\nThat's a wonderful question.\nOne of the reasons that I\nwas so excited by the advent\nof broad-based large language models,\nand you've already played\nwith many of them, I'm sure,\nlarge language models have democratized AI\nso that you don't have to be\nan engineer to make use of AI.\nThat wasn't true a couple of years ago.\nIf you wanted to use machine learning,\nyou needed to have programming experience.\nThere were very few generic,\nnon-technical machine\nlearning engines, if any,\nthat would allow people\nwho didn't understand the intricacies\nof these various different\nrandom forest models and so on\nto be able to use them in any\nway that would be beneficial.\nWith large language models,\nwe can actually speak in plain English.\nAnd moreover, there are\na number of non-engineers\nthat I've met who have\nbecome extraordinarily good\nat formulating just the right\nprompt in order to be able\nto get their desired outcome\nfrom these different LLMs.\nSo I think the answer is absolutely\nnon-engineers can become\nextremely adept at AI.\nAnd the way to do it is to use it,\nstart playing with your favorite LLM\nand just talk to it, ask\nit a bunch of questions.\nIn a way,\nthat's kind of how we\ndevelop relationships\nright now with our human friends.\nWe begin by asking them questions\nand gauging their answers.\nAnd based upon what we hear,\nwe either decide to become\nbetter friends with them\nor decide stay away, we're not interested.\nWith LLMs, we need to talk with\nthem in order to understand\nhow they operate, understand\nwhere they're coming from,\nunderstand what they can do for us.\nAnd once we develop that\nkind of, dare I say rapport,\nwe will then be adept at\nbeing able to prompt them\nin the most efficient manner.\nYou do not have to have\nan engineering degree\nto be able to do that.\nAnybody watching this video can do this,\nand I would urge you all to try\ndoing it as soon as you can.\nAnother social media post,\n\"Are cryptocurrencies\na temporary phenomenon,\nor are they here to stay?\"\nYes, they are here to stay,\nbut no, the cryptocurrency\nthat your trading may not be here forever.\nThe idea of cryptocurrency\nis extremely powerful,\nand it provides enormous benefits\nfor financial transactions of\na variety of different sorts.\nAt the same time, it also\nhas a number of flaws\nthat can be exploited by bad actors\nand the improper usage of crypto.\nA lot's gonna change over the\ncourse of the next few years.\nIn fact, one of the biggest changes\nthat I think we all\nshould all watch out for\nis when a very large and\nstable sovereign entity decides\nto issue crypto of its own.\nFor example, imagine the\nUnited States decides\nto issue Fed Coin and that\nbecomes the cryptocurrency\nof preference for the entire world.\nIf that happens,\nthen the existing cryptocurrencies\nmay not do as well.\nWithout a doubt,\nsome of them will become\nworthless overnight.\nAnd if that's your currency\nthat you're holding your wealth in,\nthat's not good news for you.\nIn a way, this happened in the 1800s,\nbecause as the west was\nbeing developed in the US,\na number of banks arose to\nfinance that kind of development,\nand these banks would\nprint their own currency\nto be able to hand out to\nworkers that wanted to get paid.\nAnd of course, these are like bonds\nthat the banks were issuing.\nThe problem is that every once in a while\nthese pieces of paper\nwould become worthless\nbecause the bank would go outta business,\nthey would extend too far,\nand they couldn't basically make good\non all of these pieces of paper.\nNow, these pieces of paper\noften had pictures of wildcats on them\nbecause that's what,\nthat's the animal that dominated\nthe population of the west\nas it was being developed.\nAnd so this phenomenon\ncalled wildcat banking\nbecame quite prominent in the mid-1800s.\nAnd at some point, the number\nof failures became so bad\nthat the US government stepped in\nand basically shut down this\nwhole wildcat banking system\nby creating the Federal Reserve\nand providing a much broader base of laws\nand the US currency\nthat supplanted all of\nthese wildcat currencies.\nSo pretty much overnight,\nas the US began to issue its own currency,\nthat made these particular\nsmaller currencies worthless.\nIf a sovereign entity were\nto issue a cryptocurrency\nand that ended up taking the place\nof a number of existing\ncurrencies, you need to worry about\nwhether or not your currency is the one\nthat's gonna become worthless.\nSo that's one of the challenges\nof investing in crypto.\nBut without a doubt,\nthe technology that is used\nin crypto is here to stay.\nAnd I believe that\ncryptocurrencies themselves\nas an asset class are here to stay.\nBut you need to think about\nhow to manage that risk very carefully.\nCan't believe it, but\nthis is the last question\nfrom our social media friend.\n\"How can we deal with large\nscale market manipulation\nnow that LMS are so cheap\nto deploy at large scale?\"\nWow, what a question to end on.\nSo it turns out that one of the things\nthat large language models\ncould be very capable of\nis providing fraudsters\nand other bad actors\nwith the appropriate technology\nfor disrupting financial stability.\nNow, this may seem farfetched,\nbut I bet you if you ask\nthe large language model\nhow to build an atomic bomb,\nyou'd probably get a\npretty reasonable answer.\nAnd so it actually scares\nme that now the knowledge\nof the entire financial\nliterature is available\nto bad actors who can\nuse large language models\nto summarize the ways\nthat they can involve\ntheir various different nefarious plots\nwithin the financial system.\nThe regulators also have access\nto large language models,\nand I believe that they are\nalso using the tools of AI\nto figure out how to detect\nthese kinds of schemes\nwhile they're being hatched\nand before they occur.\nBut as I said earlier, it is an arms race,\nand this is one of the reasons\nwhy we at CSAIL are making these videos.\nWe want all of you to start thinking about\nhow to use these tools to\nmake society a better place.\nThere's no doubt that\nthere's tremendous value\nin these new technologies,\nbut there's also some potential dangers\nthat we have to worry about.\nAnd the only way that we're\ngonna get better at it\nis if more and better people are involved.\nWell, that's it for the questions.\nCan't believe that we're all done,\nbut thank you so much for\ntaking the time to submit them.\nIt was fascinating,\nand I look forward to seeing\nyou in the next video.\nBye-Bye.",
  "transcript_chars": 14734,
  "ingested_at": "2026-05-15T10:56:32.910474+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 13230,
    "like_count": 451,
    "channel_id": "UCBpxspUNl1Th33XbugiHJzw",
    "categories": [
      "Science & Technology"
    ],
    "tags": [
      "MIT",
      "Massachusetts Institute of Technology",
      "CSAIL"
    ]
  }
}