{
  "video_id": "K18Gmp2oXIM",
  "channel_slug": "machinelearningstreettalk",
  "channel_handle": "machinelearningstreettalk",
  "title": "Every Definition of Intelligence Is Wrong. Here's Why — Michael Bennett",
  "duration_seconds": 3945.0,
  "url": "https://www.youtube.com/watch?v=K18Gmp2oXIM",
  "upload_date": "",
  "transcript": "How close are we to AGI? I mean,\nit's interesting how much it has stuck\naround. We have just replaced the pineal\ngland with a touring machine.\nYou're a big fan of what I would call\nbiologically inspired intelligence.\nA biological systems with a a tiny\nfraction of the energy and learning data\ncould do so much more.\nIs is that fair?\nBecause whatever that software does has\nto pass through an interpreter and the\ninterpreter decides what it does.\nConsciousness is basically an illusion.\nOne of my supervisors accused me of of\nwriting libertarian biology because one\nof the results of one of my thesis is\ncalled the lore of the stack\nwhich I like dramatic names.\nMLST is proud to be sponsored by\nProlific. This is Enzo Blindout.\nYeah, that's that's kind of the the the\nthe goal that we're working towards. So\nwe're trying to make uh human data or\nhuman feedback uh or actually any kind\nof feedback at that uh um we treat it as\nan infrastructure problem right we try\nto make it accessible we make it cheaper\nyou see this pattern in almost any\ncompany and and even in academic\nresearch as well every academic\nresearchers cares about the quality of\ntheir data let's abstract it away let's\nput a nice API around it to make it just\nlike the same way you also do CI/CD or\nyou do model training pipelines we\neffectively democratize access to this\ndata.\nYeah. Okay. I'm trying to think of my\nbeliefs. Give me one second.\nWhat were they again?\nEntirely cognizant of it.\nUh should I look at the camera? Should I\nlook at you?\nLook at me.\nOkay.\nYeah.\nUm my name is Michael Timothy Bennett. I\nam a computer scientist who has to use\nhis middle name because there are too\nmany Michael Bennett in the world. I am\ninterested in understanding AI uh\nintelligence um life uh the universe and\nthe nature of existence uh and I spend\nall my time doing that and I have a a\nside hobby trying to build AI.\nI I got in touch with you actually it\nwas quite a few months ago. It was when\nyour paper called what the f is\nartificial intelligence. Did I get the\nname right? Yeah, I mean I just a couple\nof extra letters, but yeah.\nOkay. Yeah, that was doing the rounds\nand I flick through it at the time. I've\nnow just spent the last couple of hours\nlike reading it word for word and it's\nactually brilliant. I do recommend that\nfolks at home read that especially for\nfolks in the MLST audience because uh\nwe're a little bit eclectic in our\ntaste. We are ideas collectors and we\nlike you know different approaches to\nAGI and also hybrid approaches and a\nlittle bit of philosophy and\nconsciousness and whatnot. So um\ncertainly in that respect you might be\nthe perfect guest.\nThank you.\nSo this is all very good. This is all\nvery good. Um in that paper you were\ntalking about what intelligence is and\nvarious approaches to AGI and and also\napproaches to categorizing them.\nTell us about that.\nOkay. So uh intelligence is a hotly\ndebated topic. It's been for a long\ntime. Um I sort of started off with the\nleg cut hutter definition of the ability\nto satisfy goals in a wide range of\nenvironments. Uh but as I delved more\ninto biological intelligence and other\nthings, I sort of arrived at a\ndefinition as um the ability uh as the\nefficiency of adaptation. So how sample\nand energy efficient you are. And then\nlater I found uh Paywang's definition\nwhich preceded mine by several years was\nadaptation with limited resources which\nI think is really succinct and clear. Uh\nand of course there's myriad other\ndefinitions but my favorite is pay\nwings.\nMe too. I I could pay if you're watching\nthis I'm sorry we haven't published your\ninterview yet. Like basically the reason\nis I I loved his defining artificial\nintelligence paper so much that we've\nbeen meaning to make a special edition\non it. So I kind of held back his\nmaterial for quite a while and we still\nhaven't still haven't done it. But even\nthough it's my favorite definition as\nwell and of course it informs his Nars\nframework, his his non-acimatic\nreasoning framework you know some might\nsay it's almost a bit tortological you\nknow just to say well yeah intelligence\nis about adaptation with with\ninsufficient resources and and in his\npaper he went he was at pains to kind of\nsay well we need to have simplicity we\nneed to have fruitfulness in the\ndefinition. It's actually a really\ndifficult thing to get a handle on.\nYeah. And I think a lot of people who\nsort of look at what is intelligence,\nthey end up writing very long and\ncomplicated definitions and uh then you\nspend more time trying to figure out\nwhat the definition means than actually\nthinking about intelligence. Uh that's\nanother reason I like pace I guess.\nYes.\nClear.\nSo as you know I'm a big uh you know fan\nof France. Um for much of MLST history\nthere was a rule that I was only allowed\nto mention Francois's name once per\nshow. Um we've relaxed that a little bit\nrecently but um may maybe that's that's\na good place to start. So how does\nCharlay define intelligence and and you\nalso said that Shallay was kind of\ninspired a little bit by Lean Hutter\ncertainly in terms of the use of like\nyou know cormraph complexity. Yeah, I\nmean his his work is very much descended\nfrom that sort of uh way of uh thinking.\nHe defines it in terms of the ability to\nuh acquire skills um which is you know I\nsuppose um perhaps a more benchmark\nfocused version of pay's definition uh\nwhich makes sense because Chile was\nproposing a benchmark in that paper. Uh\nso he was looking at each perhaps he was\nlooking at each test question as a skill\nand the ability to acquire it as what it\nwas testing. Uh which makes a lot of\nsense. Um but his uh formalism which\nalmost seems uh an afterthought in that\npaper is uh is where you can see much\nmore of uh lean hutter's influences. It\nit's once again uses comograph\ncomplexity. it sort of frames things in\nthe same terms even though in that paper\nChalet goes on to say that he doesn't\nthink compression is sufficient for\nintelligence. He he was very clear about\nthat in the paper and then he uses it.\nUm so\noh I think that might be because um yeah\nhe describes like a a metaenerating\nprocess which is the intelligence and\nthat produces skill programs. So the\nprograms are a compression but the\nprocess which created them is doing\nsomething more maybe.\nYeah.\nI felt like that part was um my uh I\nchose to focus on the test part at that\npoint. I'm like oh this isn't the bit\nthat he was really focusing on. I don't\nknow. I'd have to ask him. Uh I guess uh\nyes but you did say something\ninteresting is so yeah may maybe a\ndistinction from is is that and and he\nthinks of LLMs as being a kind of\ninterpretive collection of skilled\nprograms. So he you know he thinks\nprograms are the output of an\nintelligence system not the intelligence\nitself. And with um you know let's say\num AXY and we should we should talk\nabout what that is um I don't think the\nconcept of a program was an explicit\noutput artifact. is more a definition of\nan agent which can succeed in in an\nenvironment. Is is that fair?\nYeah, I mean I suppose a lot of this is\nkind of semantics a little bit but the\nthe model is a general reinforcement\nlearning agent. So it just takes the\nstandard reinforcement learning thing\nand tries to make it as um sort of a a\nuh what you might call an upperbound or\na super intelligence based on that using\nSolomon off induction. uh and solomongh\ninduction is uh you can think of it as a\nformalization of Okam's razor uh it's\njust if I have two explanations I pick\nthe simpler one and so the idea is that\nI achieves this upper bound intelligence\nbecause if you if you accept that Okam's\nrazor is uh some sort of optimal\nhuristic that you can use and it does\nthis using complexity which is sort of\nthe optimally compressed version of a\nmodel so if I can compress something\nmore then it's simpler and so if I just\ntake the most compressible models then I\ncan get the simplest ones. Uh and this\nis you know useful for thinking about\nwhat a super intelligence might do. And\nbecause it's a general it's a\nreinforcement learning agent we can sort\nof model it out. We can we can build\napproximations of it. Um I disagree with\nsome of the theoretical foundations. Uh\nand I you know a lot of my publications\nare about like what we could do better\nbut I find the overall idea of very\ncompelling and it has informed a lot of\nmy work. Yes, because I I guess for you\nreading reading your work, you said that\nif you know if we wanted to create AGI,\nit would be it would be something that\nlooked like a scientist because you know\nif we if we frame it at the right level,\na scientist can generate hypotheses and\nyou know they're an agent, they can act\nin the world, you know, they're embedded\nin in environment. So you're framed at\nat a sort of sufficient level of\nembedding that you can actually capture\nthe dynamics of the system and and in\nthat respect is an agent and it has this\nprinciple of compression and actually\nmaybe you can contrast it to active\ninference because that's quite similar.\nIt's about this agent that you know\nbalances energy and entropy. So sort of\nlike you know predictive control and\nsimplicity in some you know so-called\nnatural way. How is that different from\nI mean they're very different\nformalisms. I would love to see someone\ntry to active inference. Um but it it's\njust I guess active inference has sort\nof it has a simplicity bias built into\nit. um you know there's a like a just a\nregular the there's but it's it's sort\nof a the focus is more on\nuh explaining something else like\nexplaining I mean and also I think\nactive inference is more okay so there's\na whole bunch of ideas there you got the\nfree energy principle you got active\ninference you got all this stuff about\nmarov blankets and maintaining the\nborder of an organism and having an\ninternal and external world and um\nso the targets kind of different\nbut uh\nI I think that I think that's fair. we\nwe have an agent and the agent is doing\nprediction in the environment and it can\nact and and so on and it they are both\nin some sense I see and active inference\ntrying to um produce a simple model\nright so there's this assumption or\nprinciple if you like that simplistic\nmodels if they predict well must be good\nyeah yeah um and uh this is a very\npopular almost orthodox assumption to\nmake cuz AAM's razor does kind of work\nbut uh Even as far back as 10 years ago,\num there were people pointing out that\nthis assumption is based on you know\nwhat I mean something like Solomon\ninduction there's it it performs uh\nreliably within within bounds uh based\non the original assumptions but once you\nput it in an interactive setting like\nwith iixy um now you've got the\nsubjective notion of complexity that the\nagent has which is used for its version\nof simplicity because it's it's it's\nsort of perceiving the through an\ninterpreter. Uh in the case of a\nuniversal touring machine, but um you\ncan think of it as just just think of it\nas an instruction like a language. When\nI when I say something in a language,\nhow long it takes me to say it depends\non the language I use. If I have some\nmimemetic single syllable word to\ndescribe a complicated concept, then the\nlength of that concept is one in my\nlanguage. And in in sort of my\nsubjective world, that's fine. But if I\nhave an external world that assesses\ncomplexity in the case of this is sort\nof like uh like looking at leg cut\nhutter intelligence which is sort of a\nmeasure of intell measuring the\nintelligence of an agent based on the\ncomplexity of the model it comes up\nwith. It's got a different concept, a\ndifferent sort of uh you know it it the\nyou can make these\nyou can make it perform arbitrarily well\nor arbitrarily poorly by sort of\nshifting the goalposts of\ninterpretation. You can make it so that\nuh if if I am to uh you know\nwell yeah you can essentially make\nsimplicity completely disconnected from\nperformance if you like. Um, not that\nthat actually happens in reality. That's\nlike it's not so cut and dried, but it's\nit's certainly not optimal. Uh, which is\nI think what a lot of people were hoping\nfor with the original\nYes.ization.\nAnd of course, it's so interesting that\nactually when you dig into this deeply,\nit it just becomes apparent how\ndifficult this problem actually is. You\nknow, like many people might just think,\noh yeah, defining intelligence, we we've\ngot that now ages ago. I I think there\nwas a distinction as well that certainly\nlean Hutter they were very focused on on\nthis simplification of of the model and\nOkam's razor and I think Shalet did\novercome one hurdle which is um task\ngenerality as well this developer aware\ngeneralization. Yeah. So so for's\ndefinition is not a general definition\nof intelligence. So it's it's very much\na specialized definition. So it's it's\nit's intelligence relative to a scope of\ntasks.\nOh yeah. And then the generalization\ndifficulty is is like the relative\nentropy from that scope of tasks for you\nknow from the wider scope of tasks and\nand if I understand correctly like I I\nthink that it's been years since I read\nthat 2007 paper by Lean Hutter but but\nit it was about an agent minimizing\ncommon complexity which can like do well\non a on a the expected performance on a\nwide range of environments.\nThe definition of task here is\nimportant. I um I agree with Chillette\nthat the tasks are what's important. I\njust disagree about what constitutes a\ntask. Um so like something like Leen\nHutter's definition with the\nenvironments and the goals and the sort\nof it's the reinforcement learning\nframework. You've got things like\nactions.\nThese are all like highle abstractions\nthat that we humans use to simplify the\nworld. And uh as the uh problem of\nrelative complexity in an interactive\nsetting kind of illustrates, if I use a\ndifferent set of abstractions to achieve\nthe same ends, I can make it I can make\nsomething very difficult or very easy.\nUm so if I'm trying to talk about tasks,\nthen I need to talk about embodiment as\nwell. I can't just rely on this idea of\na software mind because whatever the\nsoftware mind does depends on the the\ninterpreter or hardware. that you have\nto look at the system as a whole. And in\ncognitive science, they've got the\nthey've got this idea of inactive\ncognition uh which is not just embodied\nbut in the environment as well because\nif you change the environment then the\nsame goals change difficulty which is\nkind of an idea that you can see in lean\nHutter's definition but it is um it is\nsomething that needs to be formalized as\npart of the process of intelligence\nbecause if if you just sort of assume\nyou have a set of actions or assume a an\nenvironment you're kind of bypassing a\nlot of what um what intelligence needs\nto do to solve a task. And so\nit doesn't make sense to think of a\ncomputer program, you know, in a in an\nabsolute sense. Programs have purpose.\nThey have they are situated in a\ncontext, in a world, in an environment.\nAnd I guess more broadly, you're a big\nfan of what I would call biologically\ninspired intelligence, which is that we\nshould create intelligence which, you\nknow, has properties like\nself-organization and um delegation and\ncausal learning and and all of this kind\nof stuff, you know, because that's much\nmore like how it works in the real\nworld.\nYeah. So to solve the um so I you know\nhave uh you know had harder very briefly\nas a supervisor during my masters and\nthen sort of continued working on that\nsort of thing um uh as I progressed\nthrough my PhD and I wanted to address\nthis sort of subjective complexity\nsubjective performance thing and that\nturned out to be about defining this\nprocess of coming up with an abstraction\nlayer. If you think of the touring\nmachine on which uh with respect to\nwhich is computed as a I mean or like\ncutter intelligence um if you think of\nthat as an abstraction layer uh then\nyou've got a software mind on a hardware\nabstraction layer and then that's sort\nof interpreted by physics and in a\nconventional computer you've got like\nPython interpreted by a C program\ninterpreted by and it just goes all the\nway down to hardware but it doesn't\nreally stop at hardware because hardware\nis sort of a state of a physical\nworld and it's interpreted by whatever\nphysical laws according to which that\nworld runs. And you could then say well\nknowledge of physics is kind of\nincomplete. So where does the\nabstraction end? And if you just if you\nreally want to make an objective claim\nor a claim about objective behavior to\nbe more abs or exact um you need to\nformalize what must be true of all\nabstraction layers, not just a uh sort\nof a fixed subset assuming some basic\nlayer that you can identify cuz we we're\nsort of all interacting with the world\nthrough our own abstraction layers\nanyway. So um if we want to make claims\nthat generalize to other abstraction\nlayers, it sort of helps to have this\nframework. Um,\nwait, where were we at the start of\nthis?\nNo, no, that that that that's great.\nThat makes sense. Let's bring in the the\ncausality component, right? So, in your\nin your paper, you were describing\nalmost like single direction arrows of\ncausality. So, you know, we have the the\nhardware, we have, you know, like the\nthe C compiler, the interpreter, all of\nthis kind of stuff. And so part of what\nwe were saying is that you know to to\nbuild a living breathing lifelike system\nyou you need to sort of like respect the\ncausality. You can't just you know take\nsomething out on on its own. But I was\nalso more broadly interested in is it\nalways the case that the causality goes\nin in one direction or is it actually\nkind of like quite multi-cale\nbirectional? It is definitely multiscale\nbirectional because in the same way that\nuh so uh cells uh cells can network\nright each cell has its sort of own\ngoal- directed behavior this is in\nbiological systems and they can uh\nnetwork uh with other cells in their\nsort of perceptual field if you will and\nuh and then they are constrained by the\ncollective of cells of which they're\npart in the same way that a human uh\nwithin a legal system is constrained by\nthe behavior of the other humans around\nuh they're not going to suddenly run\ndown the street naked. Um so it's uh\nyeah definitely there is top down\ncausation and we can see it in our own\nmultiscale architecture that we're a\npart of as a species. One really cool\nthing you did in your paper was you drew\na plot and described what it is, but you\nhad abstraction on the y- axis and\ndelegated control on the x-axis and you\ngave an example of like um you know like\na centralized you know form of\ngovernance would be in the top left and\nlike a free market would be in the top\nright. Tell me about that.\nYeah, I actually have a much better\nversion of that graph uh that is coming\nout in the final version uh when that\nbecause it's uh provisionally accepted.\nSo it's that'll that'll be much clearer.\nBut so it's like it's uh the the new\nversion of the graph has got like um\nlike food stamps versus UBI to\nillustrate something that is uh\ndifferent levels of delegation of\ncontrol. So these things both distribute\nresources to uh members of an entire\npopulation. But the food stamps central\nuh doesn't delegate control. It only\ndelegates some of the resources. there\npeople are very restricted in in what\nthey can do with that. Um and so that\nillustrates the difference between sort\nof uh sort of uh decentralization and uh\ndelegation of control. And then you can\nthink of every system as a stack of\nabstraction layers not just so a\ncomputer is typically arranged into like\nthis hardware you know um machine code\nassembly se all this the stack but so\nare human organizations. We've got um in\na military organization we've got like\nsoldiers then you've got a squad platoon\nand uh so you've got these different\nlevels of abstraction at which you can\nlook at the system and each layer is\nsort of the behavior of the parts of the\nlayer below and in biological systems\nthe the in the same so their behavior\ncan be an organ and the behavior of\norgans can be an organism and the\nbehavior of a set of organisms like\nhumans can be language. So you just can\nmove and so you can use this framework\nof abstraction layers to understand uh\nsome of the relative advantages that\nbiological systems have. Uh should I\nkeep going or\nwell does that imply that the\nabstractions are real?\nRight. So you know like I see you as an\nagent and I see you as factorized quite\nneatly into organs and brains and eyes\nand and whatnot. But if we want to build\nan artificial intelligence, one approach\nis we we handcraft the abstraction\nhierarchy. Um another one is that we\nadaptively learn it.\nYeah. And and adaptively learning it is\ndefinitely the way to go because we we\nhave learned the abstractions we have in\norder to uh because these things are\nuseful to us. So a chair for example is\nuseful to me. It is something that that\nis a cause of veilance to me or is a\nstep on the way to causing some positive\nor negative veilance. It has utility\ntable same thing. I don't have the\nconcept of half a chair that I think\nabout like it's just not useful to me to\nthink. I have to combine this other\nconcept of half to even describe it. And\nthe world is in uh is in every aspect\ndivided into these simplifications,\nthese classifiers. And these are if if\nyou know we want to go back to talking\nabout an intelligence or something that\nbuilds programs. I'm building all these\nclassifier programs for television,\nlight, chair, u table, like this is all\nstuff that matters to me. I don't build\nclassifiers for things that don't\nmatter. And so you can even you tie this\nin with things like the fmy paradox of\nlike why we don't even notice something\nthat might be classified as intelligent.\nits behavior is just not relevant to\nanything that causes us veilance which\nwould tie in with Mike Leven's work on\nmind blindness and um\nvery good very good and you spoke about\nthe need for um actually learning causal\nrelationships between things as well\ntell me about that\nright I started with a bit of pearl\nstuff because I was reading this book of\nwhy and looking at optimal agents of\ncourse I was thinking well if it's going\nto be optimal it has to learn some sort\nof representation of its own\ninterventions in the world. Uh so that's\nto say that if I want to know whether if\nI want to be able to get food and\nnavigate my environment, I need to be\nable to tell the difference between if\nI'm a fly on my shoulder for example and\nthe world moves around the fly. There's\ntwo reasons that could have happened.\nEither my shoulder moved or the fly\nmoved. Fly needs to know which or it's\ngoing to get squished.\nSame with humans. Same with uh in and in\nuh and in insects all the way up. This\nis uh you know others have already\nsuggested this is sort of important to\nthe notion of subjective experience\nbecause you need a a subject to have\nexperience right you need to have an eye\nto know that I did something um and\nuh so I started looking at to how you\nwould arrive at how you would construct\nthat as a self-organizing system uh and\nthis came into this this this\nalternative to simplicity that I was\nworking on for like optimal learning I\ncall weak uh weak policy optimization or\nuh weak constraints or even uh after\nfeeling particularly cocky called it\nBennett's razor. That one hasn't caught\non, but I'm working on it. Um\nwell, you never know.\nMaybe if anyone uh\nYeah. Benn Bennett's razor everyone.\nUm\nso so\nwe can get this uh so the point was if\nwe can define an optimal agent that\nlearns optimally it must construct this\nsort of representation of itself sort of\na I call it um a causal identity for\nself. It's like I I have an identity\nthat I associate with the causal effects\nof my actions. But it's also like if I\ndon't start off with a world divided\ninto objects, then I would also do that\nfor other things like a a chair, a\ntelevision. And it wouldn't just be a\npassive classifier in the sense of like\na um that we think we tend to think of\nthings as value neutral because it's\nsimple. But these are things are not\nvalue neutral if you have a system that\nis sort of uh you know impelled uh uh by\nsort of attraction and repulsion from\nthe ground up, right? It's not like it's\nnot like it's going through an\ninterpreter and having veilance attached\nafter the fact. Uh it is uh you know in\nthe case of like a biological organism a\nnetwork of cells each of which are being\nattracted and repelled. Each collective\nof cells within that is sort of being\npushed and pulled and by attractive and\nrepulsive forces and the organism as a\nwhole is being attracted and repelled.\nSo you can think of this as sort of a\ntapestry of veilance. um and uh it's\ndeveloping representations and\nclassifiers of the world not just of\nitself but of other objects and all of\nthese objects would be sort of the\ncauses of that veilance. So um or in in\nsome way causally relevant to what\ncauses veilance.\nSo there was there was scale maxing\nwhich is the San Francisco thing. Yeah.\nAnd then there's a simp maxing which\nlet's make it simple. So like I see\nwould be an example of that. Yeah.\nUm, and then there was the W maxing and\nW I think means world.\nUh, well, it meant weakness, but I just\nthought it was funny cuz it was like\nwind maxing, but\nOh, interesting.\nWhat was what was the interest? Because\nyou you were talking about things like,\nyou know, an active cognition, you know,\nwhere we have like we consider the whole\nenvironment and everything. Is that\nroughly?\nYeah. So, it's like if if I is a sort of\ndualist appro. When I say dualist,\nright, I'm referring to like uh the for\nanyone who's unfamiliar, there's um\ncartisian dualism is the idea that you\nhave mental substance and physical\nsubstance. And he was trying to come up\nin the 16th century with an explanation\nof the mind that conform to church\ndoctrine. Uh and so he said that you had\nmental substance which are all our\nthoughts and things interacts with the\nphysical substance of the world through\nthe pineal gland and the animal spirits\naround the pineal gland. it sort of you\nknow bumps it and then it bumps us and\nthen we act. Uh now this even in the\n16th century came under some criticism\nbut it sort of stuck around and um it's\ninteresting how much it has stuck around\nbecause we've kind of done the same\nthing with AI. We have just replaced the\nthe pineal gland with a touring machine.\nUm and an active cognition is the idea\nthat uh your cognition is in the world,\nright? It's not a mental substance of\nsort of uh it's not just embodied in the\nsense of like I'm not just a body, but I\nam part of the world around me. I store\nmy memory extends into the world. I can\nwrite things on a piece of paper. Um I\nand I enact my cognition by interacting\nwith the people around me and the\nobjects around me.\nYes. I'm so I'm so so glad you brought\nthis up because the other day when we\nwere chatting you were talking about the\npineal gland\nand and I and I thought oh my god what\nthe what the [ __ ] is he talking about\nand um and and yes so you're saying in\nthe 1600s we had this cartisian dualism\nyou know that the mind and the body two\ndifferent ontological substances and and\npeople at the time thought the pineal\ngland was almost like the mediating\nthing in the brain between so obviously\nyou're not you're not saying that the\npineal gland is the mediating thing\nmaybe you are I don't know\nI'm definitely not saying that.\nOkay, good. Just get that right. But but\nbut the very interesting analogy is that\nyou're saying that there is this thing\ncalled computational dualism and that\nand and I want to press on this a little\nbit because I don't know are you are you\nsaying that any form of functionalism or\ncomputationalism is computational\ndualism or are you saying this this um\nuse of a touring machine in some\nformalisms is computational dualism? I'm\nsaying I'm I'm using I use computational\ndualism more to poke fun at the idea of\nuh defining just a software intelligence\nbecause uh if we just have software by\nitself and we just we don't say anything\nabout the hardware then it can't really\nbe intelligence if intelligent if\nintelligence is measured in terms of\nperformance in the environment\nbecause whatever that software does has\nto pass through an interpreter and the\ninterpreter decides what it does. So we\ncan just make it arbitrarily stupid if\nwe want.\nYeah.\nUm and so it is really uh it was I I\nused that term computational dualism as\nalmost like a rolled up newspaper to\nwhack people on the nose because I was\ngetting frustrated with repeating\nmyself.\nOh, I like it. I like it. How does this\nidea relate to I mean I spoke with a few\nfolks about um mortal computation,\nright? You know, so touring machines and\nprogram, you know, these are great\nbecause they allow us and and also, you\nknow, people talk about panc\ncomputationalism. It's a really neat\nidea to think about computation in the\nabstract and think about programs in the\nabstract. You know, these are things\nwhich can run on any computer\npotentially on different substrates and\nand whatnot. And the world isn't really\nlike that.\nNo. No. Um there are finitely many\ncopies of every piece of software we\nmake. Um\nI love the branding. mortal and immortal\ncomputation. But there's no such thing\nas immortal computations. There's just\nfinitely many of uh copies of the\nsoftware we make. And I get that people\nmight quibble about but yeah, you can\ncopy the thing. But in the context of\ntrying to define intelligence, it is a\nterrible concept. There's just there's\njust mortal computation. Let's not\ncomplicate the matter by adding in an\nextra concept that doesn't apply. Um\nYes. Yeah.\nYes. And then just to help folks\nunderstand, so mortal computation is\nthat the the stuff is the computation.\nThere's like no meaningful disconnection\nbetween like the program and the stuff\nwhich does it.\nI've seen a few people give different\ninterpretations of it. Uh the best is\nprobably that of Auroria and Friston\nwhere they talk about\nOh yes, I interviewed him by the way,\nAlex.\nOh, right. Cool.\nYes. Cool guy.\nYeah. So they've got like quite a\nrigorous definition that extends over\nseveral pages. Um the most common\ndefinition I've seen is the internet\nversion which is just people going it's\nimmortal. Um but uh and I think it was\nfirst the the term was used as sort of\nan afterthought by Hinton at the end of\na paper that was mostly about feed\nforward neural networks.\nOh yeah. Was it his new his new proposed\narchitect? Was it like the feed forward\nforward from 2022 in Europe?\nI I think so. Yeah. Yeah, I can't\nremember the name of the paper, but\nyeah,\nbut I think it it sort of and I remember\nI saw that and I'd already been writing\nabout like embodiment and how like you\nyou got to like take into account the\nabstraction layer if you want to like\nhave uh any claims that hold up about\nperformance.\nUm and so uh and my response to that was\nto come up with the term computational\ndualism and write an irritated paper.\nBut um\ngrumpy papers are my favorite papers\nwhich is amazing. Okay. Very good. Very\ngood. And so in in this kind of um you\nknow categorization of different\napproaches because we haven't we haven't\nspoken about you know the Silicon Valley\nscale maxing and so on. I mean maybe\nlike what what drives you I mean how\nclose are we to AGI? I mean the folks\nover over there think that we've already\ndone it right but just by scaling.\nWell, I mean, if you want something that\ncan do jobs uh and automate a lot of the\neconomy, then sure, we've got some form\nthat form of AGI. If we if you want\nsomething that's actually intelligent\nlike a human though, we do not have\nthat. Uh and a lot of that is because um\nlike even just interacting with and like\nI use things like you know, Grock and\nchat GPT. It's not like I'm not\ninteracting with this stuff, but if it\nwas if it was anywhere near as\nintelligent as a human, I wouldn't have\nto do all the work that I do. I would um\nbe able to offload a lot of it's not\nsample efficient. I know um there's\nclaims about the ARC test, but just\ninteracting with these agents, you can\nsee that it's not really sample\nefficient. I don't know what they did\nthere to get those results on ARK 1, and\nI don't know what uh what the claims are\nabout ARK 2. It's very hard to tell what\nis true when people don't release the\ncode and the results and everything that\nyou can sort of puzzle through and work\nout. So, uh I think we're probably a\ngood ways off something that really\nresembles human intelligence and we need\nto look at something that isn't just\nlike a software innovation but hardware\ninnovation. The reason I think that is\nbecause we're using these abstraction\nlayers in the form of like you know the\nthe hardware we have purpose-built for\nvery useful standardized software\napplications that we can roll out and\ncopy and run on many different\ncomputers. Um but if we want something\nthat's as adaptive as a biological\nsystem we need something that is sort of\nmodular and cellular and efficient like\na biological system. We can't be\nexpecting some I mean a biological\nsystems with a a tiny fraction of the\nenergy and uh and learning data can do\nso much more.\nYes.\nUm yes\nwhich you know it's great because I like\nhaving a job. So let's uh\nabsolutely by the way hot off the press.\nDid you hear that um Elon released Grock\n4? I don't I don't know who's released\nit but um Greg you know from Arch Prize\nhe posted this morning. Greg's a good\nguy and apparently it's scored about 16%\nwhich is sort even even because you my\nforens you know like um Muhammad and and\num Jack I think they're around 15 and a\nhalf% at the moment so yeah Grog Grog\nfour is now in the lead I mean how do\nyou interpret that you think it's just\nsort of like dent of memorization or\nI don't know I mean I'll have to\ninteract with it right maybe it'll be\nreally impressive maybe it'll uh but I I\nsuppose the proof's in the pudding uh\nlike if if this starts to do a bunch of\nreally useful jobs across the economy.\nUm, then we can say with certainty that\nwe're closer to it. But these benchmarks\neven like and the ARGI benchmark is a\ngreat benchmark, right? I've been\nlooking at that my whole PhD is it was a\nit's it's great, but it's not perfect.\nNone of uh Chillette would not claim it\nto be perfect.\nUm, and 16%'s a great result, but uh I\nwant to see if it can add long numbers.\nThat probably\nthat would be a good start.\nI don't know. That's like that's my\nusual thing. I sort of oh, it's a new\ntoy. Let's see if it can add long\nnumbers. And it almost always can't.\nIt's just\nWell, I mean, devil's advocate. Um,\nyeah, no one's going to argue with you\nif you say vanilla LLMs are, you know,\nbasically databases. No one's going to\nargue with you, right? But you can add\ntools to them. I mean, so, so you were\nstarting to talk in your paper about\nthere are some very interested\ninteresting hybrid approaches. or you\nknow obviously on the LLM side you add\ntools and there's an interesting\ndiscussion to be had there whether you\nknow you train them with stocastic\ngradient descent they use tools what can\nthey do but you're also talking about\nsome other interesting like you know\nthere's um uh the non axiomatic\nreasoning system from pay and there's\nthe hyperon system from Ben Girtz and my\nhonest like I don't know anything I\ndon't know much about those systems but\nhonestly when you were describing them\nit seemed a little bit like they were\neverything but the kitchen sink so\nthey're like you know they can do a bit\nof basian inference over here and they\ncan do some neural networks over there\nand I mean that that could work but what\nwhat's your assessment?\nYeah. So, oh should I go around should I\ngo over that like tool?\nOh yeah yeah yeah. Just just sort of\nweave weave a path.\nSo I sort of um said there like taking\nthe inspiration from Sutton's bitter\nlesson where he talked about sort of uh\nsearch and learning. Um I sort of\ndivided in we got sort of two basic\ntools right we got approximation which\nis what the LLMs are. I mean and by\ndefinition they're inexact and that's\nreally great for like you know um you\nknow trolling through large amounts of\ndata and coping with noisy data because\nit's an approximation. You can do a lot\nwith that and then with the\ncomputational resources we have\napproximation works beautifully. And\nthen you got search which is like like\niterating through a flow diagram or\nsomething like that and that is great\nfor precision or things like navigation\non your phone. Uh and when you combine\nthese things you get something like\nalpha go or or um uh yeah I mean so you\ncan use the approximation part for a\nsort of a huristic to guide the search.\nYou can you can combine these in many\ndifferent ways and these hybrids allow\nus to create much more more effective\nintelligent systems of some one form or\nanother. So in the case of um you know\nthere's well-known examples of this are\nthings like Alph Go or Alpha Star um but\nthere's also\nuh you know the sort of more\ncomprehensive architectures that are\nmeant to kind of emulate the versatility\nof a human mind. So something like uh\nlike NARS is uh a system that sort of\nyou can integrate many different\ncomponents and I've seen some of the\nexperiments involving NARS and LLM that\nwere at the 2023 AGI conference and like\nHyperon is like an inherently modular\nsystem that is just meant to allow you\nto sort of plug and play lots of uh well\nthat's uh lots of different modules and\nit's meant to be decentralized and\nadaptable so you can plug all these\nthings in as they develop.\nUm, and yeah, that's that is definitely,\nyou know, like it can include the\nkitchen sink if you want to plug that\nin. But I guess the uh\nI think it was Yeah, sorry. It was more\num hyper on I was thinking about the\nkitchen s one may maybe you'd do a\nbetter job than me at this but r roughly\nspeaking it's about like building up a\nwhole bunch of reasoning about something\nwhich I don't know much about and then\nadapting and modifying it over time\nuntil which I can make you know\ndeductions and inferences about things\nand I remember there was some stuff with\ntime constraints in there so if it got\nstuck on something it would move on and\nwould rank things by it would take into\naccount the resources it had So in\npractice it would actually be quite\nuseful. You can put it on a little\nrobot, have the little robot run around\nthe room and do stuff.\nYes.\nUm.\nYes.\nWhich is cool. Uh but and they they seem\nto every year come up with better and\nbetter benchmarks results, but it\ndoesn't seem to get much attention in\nthe sort of mainstream machine learning\nspace. Uh\nyeah. What do you think about benchmarks\nby the way? Because Grock um I actually\ninterviewed the guy who created\nhumanities last exam the other day, Dan\nHendris, and um I think it was at 26%.\nToday with Grock 4 it's about 46%.\nYou know what what is the point of\nbenchmarks if they're so easily\nsaturated? Great marketing. Um I mean\nit's it's like we love measuring sticks.\nI it's just it's a it's a it's nice to\nhave measuring sticks. It lets us know\nthat we're progressing in a direction,\nwhatever direction we point the stick\nin, I guess. Um but like humans are uh\nadaptive. If we set up a measuring stick\nand it's less than perfect, we will find\na way to exploit that.\nUm, I think this is\nthis isn't necessarily a bad thing. It's\njust that I think people should\ninterpret benchmarks as what they are,\nwhich is\nmeasuring sticks.\nYes. Talk to me about consciousness.\nOkay. Um\nwell uh that whole spiel about um and\nabstraction layers kind of led me down a\nvery long and winding rabbit hole uh\nwith the abstraction layers thing\nbecause um after doing that I mentioned\nbefore the idea of coming up with a\ncausal representation of the self and\nthe this idea of tapestries of veilance\nand if you keep scaling up the ability\nto sort of learn these causal uh causes\nof veilance then you don't just get like\na a sort of a a do operator for the self\nor a representation of the self uh which\nothers have already oh actually I should\nstart there that self thing um when I\nwas looking at that in the original\npaper I uh that I put that in I I sort\nof think well if you got this self and\nit is sort of has is inherently veanced\nuh and it's sort of made up of sensory\nmotor activity then wouldn't that be the\nsort of the basis for a subjective\nexperience and explain something of\nconsciousness. So I I put that in the\npaper and people liked it and I thought\nthis consciousness thing is great. I'm\ngoing to keep doing this. Um I like this\nand people aren't laughing at me. So um\nI kept going and then one of my\nsupervisors said hey that's like my\ntheory. I did a causal self thing uh and\npointed me at his paper and his paper\nwas about something called reaffirence\nin uh the insect central uh central\ncomplex which is also uh he was trying\nto show that flies have subjective\nexperience. And this tiled back to some\nwork from like 20 years ago where\nsomeone was saying that well this is\nthis this sort of representation of the\nself is like where human subjective\nexperience uh comes. we have uh\nsomething called reafference in the\nmamalian midbrain. Um and so many\nanimals have this. It's what enables us\nto tell when I am pressing down on the\nchair versus the chair pressing up on\nme. And this is very useful for causal\nrelations. And then I kind of started\nthinking about consciousness more\ngenerally and how well we could make up\nour subjective experience with these\ncauses of veilance. I was talking about\nyou know things like the television, the\nchair, whatever. Um and someone uh\nstarted um beating me over the head with\na copy of David Charalmer's work on the\nhard problem of consciousness.\nOh yes.\nUm and after a lengthy argument uh over\nthat I decided well now I have to write\nabout it. So I started writing a uh what\nand ended up as a 70page paper much to\nthe um my supervisors at the time would\nplease tell him please stop just\ngraduate just just just finish the\nthesis. Um but I kept writing this and\nthen it got cut down to a uh some much\nshorter paper. I ended up writing\nbringing on one of my supervisors as a\ncollaborator. Found another new\nsupervisor who was sort of expert in\nconsciousness to come on and help me\nfinish that and talked about the hard\nproblem of consciousness. Um, and I\npropose to solve it by showing that uh\nwhat's called a philosophical zombie is\nimpossible in every uh conceivable\nworld. And this is because if you go\ndown all the abstraction layers, you can\nsay well every conceivable world is um\nis sort of must uh whatever it is right\num it must include just change or\ndifference otherwise there's just a sort\nof universal oneness and as I put it in\nmy thesis becoming one with the universe\nis beyond the scope of my work. Um\nthe so you got a set of states and you\ncan build up a formalism from that\nand um and this formalism\nI argue describes all conceivable\nworlds. So if you can show us a\nphilosophical zombie that is and just to\nexplain what that is um it's something\nthat's like you or me but not conscious\nit's but in every way identical to you\nor me. So, it's as efficient\nenergetically, it's as it's as smart,\ndoes everything we do, just missing\nconsciousness. And I I got really into\nphilosophical zombies when I read the\nwork of of Peter Watts, uh, who is this\nscience fiction author that writes\ncosmic horror about philosophical\nzombies. Um, do you want to do you want\nto continue? I've got I've got a couple\nof questions, but\nNo, no, let's do the questions because I\ncan go on ranting for ages.\nWell, no, no. I mean, this this is\nbrilliant. Yes. So, philosophicals on\nme. I mean, um, I I love Charas. We've\nhad them on all of the function,\ndynamics, and behavior without the\nlittle bit extra about the the\nphenomenal component.\nAnd so, I I might be a phenomenal\nzombie.\nSo, I think what you're saying is\nbecause there's this kind of um\nlowercase S and capital S subjectivity.\nAnd if I understand correctly, you're\nsaying that the capital S subjectivity\ncan be discounted. There are no\nphenomenal zombies. I don't know whether\nyou're saying it's because there is no\nhard problem of consciousness like that\nthat consciousness is basically an\nillusion. So we should think about\nsubjectivity in terms of purposeful um\nperspectival\nrepresentations\nright but we shouldn't assume that\nthere's some magical extra realm on top\nwhich is called consciousness.\nI suppose uh there's a lot of lot to\ninterpret there. I mean if we're saying\nis there like a mental anything\nnon-physical that is if if we take\nphysical to mean something that\ninteracts directly with the physical\nworld and is like not sort of\nI suppose the advantage of doing that\nall conceivable worlds thing is I don't\nneed to talk about physical and\nnon-physical um and I like that because\nthen I can entertain sort of ideas of\nmagical worlds or whatever but beside\nthe point um\nI'm not saying that consciousness isn't\na thing or that it's an illusion.\nRight?\nI'm saying uh that if uh we can sort of\nenumerate a bunch explain a whole lot of\nthe features of consciousness as a\nnecessary consequence of uh you know\njust the state of the environment\nchanging from one to another in all\npossible worlds, all conceivable worlds.\nAnd if one accepts that my description\nof a conscious organism uh without\nhaving said that anything is just\nphysical or whatever is uh is compelling\num and I think it is then there is no\nconceivable world where you can have\nsomething that is as is as intelligent\nand acts the way that I do without or\nwithout without being conscious. The\nconscious is a necessary adaptation. uh\nthis ex and that the idea of information\nprocessing without consciousness is uh\nis implausible.\nYes. Very good. Very good. So\nyou are saying basically that it is it\nis just something that happens when you\nhave configurations along the lines that\nyou are describing.\nI I guess in a way I can't challenge you\nbecause you're already saying they need\nto be biological and and like you know\nphysical and real and so because I mean\nknow my my obvious retort to that would\nbe well um you know like a computer\nsimulation of those things obviously\nwouldn't be conscious but you you're not\nreally saying that are you?\nNo. And I uh there's one question that\nat the end of my thesis I get I sort of\narmanar about cuz it's like uh and I\nlike this question. I like that I\ncouldn't figure this out cuz I it's fun\nto think about. Um so that tapestry of\nveilance I mentioned with like all the\ncells getting pushed this way and that\nand you can sort of say that you know a\nconscious state is a tapestry of\nveilance. Now if I simulate a bunch of\ncells and give it a tapestry veance and\nall the uh necessary ingredients like\nthe first and second and third order\ncells that I only talked about the first\norder self just before but you can keep\nscaling the system up and getting\npredictions of predictions of\npredictions of uh stuff and if you you\nget all the necessary ingredients for\nconsciousness and you just simulate it\nin a computer um\nthere is two possibilities one either a\nconscious state has to be realized at a\npoint in time by one state of the\nenvironment\nor it doesn't. If it does have to be\nrealized by a state of the environment,\nthat is to say all of the parts of the\nconscious state are at the exact same\nmoment there. Um and in the in the\nformalism that has is a much clearer\nstatement but\nwell um\nthen it's just humans and organisms and\nhighly distri you could make a nanobot\nswarm that's conscious but you can't\nmake like a single thread CPU conscious\ncuz it's not really doing all this at\nonce right it's kind of like looking at\na thing program counter sort of loads\nsomething into a register does some\nstuff shuffles some stuff around it's\nspread smeared over time and so the idea\nis that smearing consciousness over time\nwould kind of kill it. The other\npossibility is that how the how would we\nknow? Am I allowed to swear on this?\nYeah, of course.\nYeah. How the [ __ ] know if we're we're\nbeing simulated? We don't we nowhere\nknow I think I'm existing at a point in\ntime. Um and uh I could be running this\nwhole thing could be running on a single\nthread CPU in some kid's basement. Now\nI'm not endorsing the simulation\nhypothesis. I'm just saying. I mean, I\nlike the idea that so everything is\nclearly there's like an infinite stack\nof abstraction layers. Maybe those\nabstraction layers end in some kid's\nbasement. I don't think they do. Doesn't\nmatter. That's I'm making fun of the\nsimulation thing. But um if it's smeared\nacross time, if I can have a single\nthread CPU load stuff into a register,\ndo all simulate all the stuff that is to\ndo with consciousness,\nthen I could make like populations of\nhumans or what are called liquid brains\nof ant. So a solid brain is like\nsomething like a a human brain with a\npersistent structure that supports like\nuh you know in the case of a human brain\na bioelectric abstraction layer that can\ndo information processing very useful\nbut it requires a certain stability.\nIt's it has to be maintain its form. A\nliquid brain is something like a\npopulation of humans doesn't have to\nmaintain its form. Its computation is\nnot in the form of sort of electrical\nsignals but people moving around doing\nactions and interacting with the world.\nNow maybe we'll network ourselves up. I\ndon't know but that's a liquid brain and\ncolony is another liquid brain according\nto my theory if if a consciousness has\nto be at a point in time then liquid\nbrain not conscious if you can smear it\nacross time then you can have a\nconscious liquid brain which is cool but\nuh\nhas some weird implications\nthat very cool by the way we we'll be\nclipping that part so you'll see you'll\nsee that part on on Twitter\num there is just one potential objection\nwhich is that you know What a lot of\ntheories of consciousness do is is they\nkind of they they brush it to one side\nand they treat it as something which is\nepifenomenal which means it it's not\nlike causally embedded in in the system.\nAnd we did ask Friston about this and he\nhad a bit of an interesting response. He\nsaid that you know phenomenal states are\njust um parts of the generative model.\nYou know it's it's all it's all in\nthere. It's all kind of like part of the\nthe causal nexus or whatever. What do\nyou think about that? So I I very\nexplicitly argue that a phenomenal state\nis a tapestry of veilance that that the\nsort of what the um that I am not just\ndoing a representation that's value\nneutral that these that my classifiers\nof the world like television so on is me\nbeing attracted or repelled from a\nphysical state um in at like at\ndifferent levels of abstraction at\ndifferent scales all happening at once.\nSo there's a lot of sensations going on\nthere which why it's not it's not just\nloading a file. I feel something as I\ntry to process this information because\nI'm being impelled by it. But isn't\nconsciousness something immaterial,\nsomething unobservable? So for me it\nmakes sense to say that it's a coralate\nof something physical in in your\nmodeling, but it has to be kind of\nsomewhere else outside of the system.\nYeah. And I guess uh so that's the idea\nof like a a firsterson ontology. There's\nsomething that I know by being conscious\nthat you can't know by watching me. Uh\nbut um\nbut if if\nI mean that's the point of the\nabstraction layer thing I guess is that\nif I if I do this formalism of all\nconceivable worlds and I have the luxury\nof saying that I I have this like god's\neye view into everyone's head. So in the\nformalism it's nice and neat like that\nfrom a sort of an explanation point of\nview here. Yes, I am stuck in my first\nperson ontology and I can only say what\nI'm interacting with through my\nabstraction layer. But it's also kind of\num kind of like saying that uh you know\nit's if we if we buy into the idea that\nwe can't say anything about things\noutside of our abstraction layer, we're\nbasically giving up on science\naltogether. And I think that why would\nwe draw a different line for\nconsciousness than for everything else\nwhen um we clearly we could draw the\nsame line for everything else. That's\nanother thing. I'm not saying that it's\nnot a line we could draw. We could say I\ndon't know the television's on. It's\njust it might be on. I don't know. Uh\nprove it to me. Uh according to my first\nperson ontology and I and then so you\nknow I can continue to hold whatever\nbelief I like really. Uh\nvery good. Very good. I've enjoyed this.\nI've enjoyed it, too.\nYeah. So, Dizzy, the Diverse\nIntelligences Summer Institute,\nuh was something I found out about about\num 3 days before the application\ndeadline. I thought, \"Oh, I should do\nthat.\" Um cold weather and talking about\nintelligence. Uh so I have come here and\nmet a lot of people who uh do everything\nfrom machine learning to biology to\nphilosophy and had very intense\ndiscussions about these things for 3\ndays.\nI uh I'm not sure my brain still\nfunctions but I know it's ticking away\nthere with with something. We're\nsupposed to come up with a project. Uh I\num I have a weird problem. There's too\nmany projects. There's too many good\nprojects.\nYes. Um, but it's it's interesting\nbecause it sort of ties in with all this\nuh there's a lot of people who are also\nbig fans of Mike Leven's work and\nI uh well I've been writing more\nrecently I've been writing a lot with\nthe tapestry of veillance stuff and\nthere talking about abstraction layers\neverything I'm doing has become about uh\nthat because it's just what I'm enjoying\nat the moment. I've been looking at say\nAI safety is for one thing as like well\nit's not about the AI and is isolation\nit's another um it's another swarm\narchitecture in which we are a liquid\nbrain into which the AI plugs in it's\njust a part of the organism and so\nrather than worrying about aligning a\npolicy or or whatever um it's really\nabout just designing the system as a\nwhole to accommodate these different\ncomponents and make use of them in the\nsame way that um that a biological\nsystem sort of uh you know makes use of\nresources available to it and then\nnetworks cells together to form stuff\nthat that's actually very interesting.\nYeah. So um you know like the way\nbiology works is that it is very\ndecentralized and there's this\ndelegation and canalization\nand weirdly it seems quite orchestrated\neven though it is so decentralized. So\nhumans um maturate in similar ways. We\nhave similar behaviors and so on. But\nbut we also have a degree of agency and\nfreedom on top of that.\nI mean if you were to to design like an\nartificial or or augmented AI system,\nhow could you kind of have your cake and\neat it and have something decentralized\nand steerable? So, one of the results I\num one of my supervisors accused me of\nof of writing libertarian biology\nbecause one of the results of of one of\nmy of my thesis is called the law of the\nstack, which I like dramatic names, but\num it's basically that if you've got a a\nhigh level of abstraction, say um you\nknow, some software running in a\ncomputer uh and it's learning, uh its\nability to learn and adapt hinges on the\nability to adapt at lower levels of\nabstraction.\nUm and uh so like the hardware level\nlearning and adapting there uh like in\ncomparison if you compare biology and\ncomputers and so there's a paper coming\nout um soon called uh are biological\nsystems uh more intelligent than\nartificial intelligence and it is\nbasically it says well um in addition to\nthe waxing thing that uh that biology\nseems to do better um it seems to\ndelegate adaptation down the stack.\nWhereas computers are like an inflexible\nbureaucracy that makes decisions only at\nthe top. And if we want to make\nsomething as adaptive and efficient as a\nhuman body or whatever, we need to\nemulate this sort of delegation. And if\nwe want to have and and then I go on to\nsay um because I I watched way too many\nMike Leven interviews and he was talking\nabout cancer and how cancer is something\nwhere uh can be seen as where you've got\na cell that becomes isolated from the\ninformationational structure of the\ncollective of which it's part and it\nreverts to like primitive uh\ntranscriptional behavior which basically\nmeans it just starts you know\nreproducing and eating itself eating and\nlike expanding and cancer. Um, and so\nyou could think of it as like well what\nwhat under what circumstances does part\nof a collective system become isolated\nfrom the informationational structure.\nSo I formalized that in this stack thing\nand said well each each cell is like a\nlittle task with a little policy and if\nthere are are not any correct policies\navailable for the overall collective\nthen the only way for the thing to\ncontinue is the the overall like higher\nlevel thing to continue is to break off\nsome of the parts until there are some\ncorrect policies that exist.\nThere's two ways this could happen. One\nyou impose lots of nasty stuff from the\noutside. Make the thing hard uh make it\ntoo hard to to have a correct policy.\nit's just too difficult, right? So, in\nthe case of biological organism, like\nlighting it on fire. Um, another way to\ndo this uh would be to just impose too\nmuch top- down control and sort of just\nsort of cut yourself off at the ankles\nand eliminate otherwise good policies by\nover constraining the members of your\ncollective. And if you think of this in\nterms of like humans, like uh this would\nbe like an overly um this is why my\nsupervisor was making jokes about\nlibertarian biology because I was saying\nthat it's just like if you over\nconstrain the members of the collective\nthen they're going to break off and and\ndo crazy [ __ ] Um so it's like having\ntoo much of a totalitarian state or\nsomething like that. Too many laws, too\nmuch restrictions. And you could I used\nto live in Italy and I could see this in\nthe way people used lines. Um they\ndidn't line up. They just kind of\nYeah. Yeah. Went towards the front of\nthe shop. There were rules for\neverything in Italy. So, nobody obeyed\nany of them.\nYeah.\nUm,\nother than that, it's a great place.\nYeah. I loved living there. Um, as soon\nas I got over the whole, you know,\nnobody standing in line thing, it was\nfine.\nYeah. And don't drive.\nYeah. Don't Well, I did a lot of that.\nThat was terrifying.\nUm, I had to stop at the side of the\nroad and my boss, uh, who was Italian\nand he's like yelling at me. He's like,\n\"Why aren't you driving?\" I'm like, \"I\nhad to stop. Everyone's crazy. They're\nall driving like maniacs.\" I know. I\nknow. It's insane. Yeah. Go on.\nThe AI if you over if you put too many\nconstraints on it, it's like you're more\nlikely to end up with that. You want to\njust constrain it in the areas you\nactually need it to be constrained if\nyou're trying to sort of avoid some sort\nof dangerous behavior. Don't sort of\nlike constrain something to a set of\nimpossible circumstances. It's just\ngoing to break.\nYes. Yes. Because I'm I'm interested in\nthis concept of what it means to be\nalive, right? Because you because we're\nkind of dancing around this a little\nbit. We want to create artificial\nsystems that that have the the the\nvibrancy, you know, and and we could\nhave perspectives on what that means,\nyou know, maybe it's sort of like\ncertain patterns of information\nprocessing, diffusion and whatnot. And\nthere are some existence proofs like you\nlook at Conway's game of life, you know,\nartificial life and linear, and you look\nat these things, you think, wow, that\nseems very lifelike. I can't really put\ninto words why it is, but it seems like\nit is. And then on on the other side of\nthings, I love building even with\ncomputers distributed agent-based\nsystems using the actor pattern. You\nknow, I I love I love sort of like\nseparating things out into autonomous\nlittle units of computation that can run\ndistributed, but but they're very very\nmuch not alive. I mean, they have some\ncool computational properties, but you\nknow, they're they're brittle. You know,\nmaybe I could make them do meta\nprogramming. So, you know, have a little\nLLM and it can kind of heal itself and\nupdate itself, but still still not\nreally like, you know, it's knocking on\nthe door of it, but it's not really\nwhere we want to go. There are some\ninteresting approaches like\nneuroscellular automter where you sort\nof do this emergentist sort of\noptimization where you know it's\nself-organizing and it can heal itself\nfrom the bottom up but that's quite\ndomain specific. So we're dancing around\nthis idea of creating systems which\nwhich are alive. Yeah. Okay. There is\nthere's two things I want to talk about\nthere.\nYeah.\nOne is you mentioned healing, right? So\nI was interested as simply\n[Music]\nextended universe stuff gets there's\nonly so much stuff you can cram into a a\nbounded system or finite space and so um\nif I want my system to say I've got you\nknow a biological system made of a just\ncoming up with abstraction layers to\nprocess information then as it delegates\ncontrol in order to make efficient use\nof space it's going to\nweaker constraints take simple forms.\nAnd so if you delegate as much as you\ncan, you're generally going to get uh\nsimplating with waxing\num as you sort of scale things up and do\nthis. So this means that something that\nmaintains homeostasis like a self-reping\norganism is going to need to delegate\ncontrol a lot in order to do this. And\nthat then so\nbeing alive almost requires this kind of\ncombination of simping and waxing. And I\nwas thinking about this because um I\nwanted to know uh why things are alive\nand I figured this would be a fun side\ngig with my thesis. Um so I uh I'm\nwaiting for examiner feedback on this.\nBut um I thought well\na rock uh is something that simpes and\njust kind of persists through simping as\nin by just simp maxing as the universe\nsort of transitions from one state to\nanother it's going to destroy some\nobjects and preserve others. And if\nsomething is simple why would it\npersist? Well, it would persist because\nsomething that's simple is more likely\nto sort of stumble into a kind of a suit\na weak constraint because there is this\nsort of basic correlation thing we've\ngot going on. Um, and so rocks and lots\nof objects like that in the universe\npersist by being simple.\nY\nbut then if you take something that\nself-repairs, it's doing the opposite.\nIt is becoming more complex than the\nsame thing if it didn't self-repair. It\nhas increased complexity and massively\nincreased its ability. It's its sort of\nability to embody weak constraints. And\nso I would argue life is that which uh\nwhich waxes at the expense of simp\nwhich are not alive just can simp.\nAnd we were just talking about healing\nand and like selforganization and so on.\nAnd it doesn't just come from within. It\nalso comes from the outside.\ninteractions with the surrounding\nenvironment.\nExactly. Because I think like a lot of\nwhat agency and intelligence is is about\nstoring a history of information. It's\nlike, you know, it's almost like you've\ngot like there's there's a there's a\nhard drive of human knowledge which is\nour culture and and this is all being\nstored and it survives and and it's kind\nof like ontoically um imbued into all of\nus during our lifetimes. and and and\nthis is why David Krakow has said that\nculture is evolution at light speed,\nright? Because we've we've like language\nand culture help us to transgress the\nphysical limitations of DNA and physical\nevolution. So it's there are just many\ndifferent ways of thinking about how\nthis like machine works. I sent an\nabstract of my thesis to David Krakow\nlast night and I don't I don't know what\nhe thinks of it yet. I I will wait and\nsee. He he might think it's terrible. Um\nbut\nI think he would be honest with you if\nif it was.\nYeah. But um but I think these ideas are\ncompatible. I mean I'm I'm sort of I\ndon't know. With the start of my PhD, I\nwas I sort of saw the free energy\nprinciple thought what is this? I don't\nknow about this. And then I read more\nand more and I'm like oh okay I can come\naround to it now. I like it. So um so\nmaybe there's maybe there's something\nI've missed but I I feel pretty\nconfident in this. And just to be clear,\nthe the the life is waxing without\nsimpaxing thing is not something that\nhas gone through peer review yet. I have\nsubmitted it for peer review and we'll\nwait and see what sort of hate mail I\nget in return. Um\nbut um it's it's definitely something I\nwould want to run past Fristant if I get\nthe opportunity in future and and hear\nwhat he thinks of it.\nUm\nyeah.\nYeah.\nVery cool. Very cool. Well, Michael,\nwhat I will say is you should join our\nDiscord server.\nAll right. I think I think the the folks\nwould love it if maybe one Friday\nafternoon we all got together and, you\nknow, shoot the [ __ ] for a little while.\nI'd love that. Yeah, that sounds you'd\nhave a lot of fans on there. Um,\nMichael, this has been amazing. Thank\nyou so much.\nThanks so much for having me on.\nAwesome.",
  "transcript_chars": 64661,
  "ingested_at": "2026-05-12T00:42:51.147972+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 20844,
    "like_count": 623,
    "channel_id": "UCMLtBahI5DMrt0NPvDSoIRQ",
    "categories": [
      "Science & Technology"
    ],
    "tags": [
      "artificial intelligence"
    ]
  }
}