{
  "video_id": "EInEmGaMRLc",
  "channel_slug": "machinelearningstreettalk",
  "channel_handle": "machinelearningstreettalk",
  "title": "When AI Discovers the Next Transformer — Robert Lange",
  "duration_seconds": 4687.0,
  "url": "https://www.youtube.com/watch?v=EInEmGaMRLc",
  "upload_date": "",
  "transcript": "I think a lot of sort of analogies from\nevolution transfer to scientific research,\nright, in the sense that we traverse a\ntree of different ideas or different\nexperiments and then in the paper we\nreport 1 path through that tree.\nWhen we run LLMs autonomously Yeah.\nThey they tend to just kind of like\nnothing interesting happens.\nBut oftentimes innovation for a specific\nproblem might require first inventing a\ndifferent problem, right? Sort of\nautomatically coming up with this\nreduction or like this let's say,\nrecursive nature of problem solving is\nsomething these systems right now not\nnecessarily have built in intrinsically.\nOftentimes it's easier to generate a lot\nof solutions than to actually like hard\nverify them. Right? The reason why I'm not\nthat worried yet about labor market\ndisruption is I still believe deeply that\nhumans are the source of deep\nunderstanding and creativity in the world.\nIf I didn't believe that, I would be very\nworried.\nSo I think it's gonna be an amplifier of\nsort of these these latent dimensions\nhumans are great at.\nRight? And I think 1 of the Rubicon\nmoments is when the the new transformers\narchitecture or something massive is\ndiscovered by AI, and we're all using\nit. NVIDIA GTC starts Monday in San Jose,\nand it's free\nto attend virtually online.\nThere's already been a leak this week of\nsomething called NemoClaw, which is an\nopen source agent platform.\nAnd if it's real, it could be 1 of the\nbigger announcements this year.\nSo it's definitely worth watching Jensen's\nkeynote for that alone.\nI'm giving away a DGX Spark.\nNVIDIA just hikes the price $700.\nYou probably heard about these memory\nshortages.\nRight? So, yeah, it's now $4,700, which is\nvery, very expensive.\nAnd Merve from Hugging Face, by the way,\nshe got 1 for her birthday and she said\nshe literally cried.\nSo it's a really cool bit of kit.\nIf you register through my link in the\ndescription and you attend at least 1\nsession, then you are in the draw.\nThis is a massive conference.\nPhysical AI and robotics are gonna be the\nbreakout theme.\nAnd Jensen does the keynote Monday at 11AM\nPacific.\nThe link is in the description.\nDon't miss it. Robert Lange, it's amazing\nto have you on MLST.\nThank you, Tim. It's a pleasure to be\nback.\nSo you're working for Sakana?\nTell us about that.\nSakana AI is a Japanese AI startup working\nmostly\non AI for Japan.\nAnd at the same time, sort of exploring\nexploring, let's say, novel or ambitious\nambitious ideas on the research side.\nIt's been around for over a year now.\nYou're on you're 1 of the founding\nresearchers.\nRight? Exactly. So Sakana has been around\nfor now, like, almost 2 years, like, 1 in\n3 quarters, I would say.\nAnd, yeah, it's pretty fascinating to to\nlook back and to look at the early days\nand how much the company sort of\norganizationally has changed.\nBut in spirit, like, we're we're trying to\nsort of embrace Ken Stanley's open\nendedness idea and sort of explore many\ndifferent ideas, which might not get the\nresources right now in the ML community\nmore general.\nAnd we've we've got a few interviews\ncoming out with Sakana that that we filmed\nhere in Japan. So I'm not I won't spoil\nthe surprise, but the the CEO is David And\nDavid, you know, like there are these\nepic, you know, giants out there like, you\nknow, Clune and Stanley.\nDavid is 1 of these people.\nDavid's work has had a lot of influence on\nmy personal PhD.\nRight? He he did a lot of fascinating work\non hyper networks and sort of modulation\nin in neural networks, but also on\nevolutionary computation and evolutionary\noptimization. And, yeah, that sort of also\npainted, yeah, my path during the\nPhD. You've you've released a paper called\nShinka Evolve and we and we were just\nsaying that that kind of means evolve\nevolve because in in Japanese, is evolve,\nbut that's quite common. It's common thing\nto do to have these like multilingual, you\nknow, double double namings in in\nJapanese.\nJust before we get there, so we\ninterviewed the AlphaEvolve team and I\nalso interviewed Jeremy Howard a few weeks\nago.\nAnd your paper is is very much like a more\nsophisticated version of those in the\nsense that it's using language models to\ngenerate programs and it's doing an\nevolutionary approach where we generate\nthe program, we refine the\ngenerated program and we have an an\nevaluator and we do this over several\nsteps. And and your your your approach\ndoes many things that that the other ones\ndon't do. Tell me about the paper.\nFirst off, of course, this was partially\ninspired by AlphaEvolve.\nI think it's great work. I know Alex and\nMate, and I think they're doing incredible\nscience. 1 thing that sort of is important\nabout sort of using all of these\nevolutionary LLM driven methods is sample\nefficiency.\nRight? So many of these systems sample,\nlike, let's say, 1000 programs for a given\ntask. And what we try to do with Shinka\nEvolve was try to essentially cut down\ncosts as well as sort of computation\nevaluation time by introducing a set of\nsort of technical innovations to this\nevolutionary search.\nAnd we showed that it's possible with very\nfew program evaluations to\nbasically improve upon, like, example, the\ncircle packing canonical result that they\nshowed in their paper. And, yeah, more\ngenerally speaking, I think we\nare right now at a point or like at an\ninflection point where these sort of,\nlet's say, evolutionary driven LLM systems\ncan really revolutionize scientific\ndiscovery. And, yeah, we hope to have made\na step forward to\nmaking this more democratically\naccessible.\nRight? So the code is open source\navailable and, yeah,\nby its sample efficient nature, we hope\nthat many people can interact with the\nsystem and can make their own scientific\ndiscoveries as well.\nYeah. That that's actually a really\nimportant point because I suppose we can\nuse these foundation models.\nAnd first of all, isn't it just\nfascinating to reflect that we have these\namazing models out there that we can\naccess.\nSo like GPT 5 and Grok 4.\nAnd they are so much better when you get\nthem to refine their\nsolution in in several steps.\nWhat why is that? I mean, I suppose a\nnaive question would be, why why aren't\nthey just good out of the box?\nPotentially, like with enough random\nsamples.\nRight? It's sort of this monkey typing on\nthe keyboard.\nThey would potentially be able to get\nthere.\nRight? But in principle, it's sort of\ncoming back to the principles\nof evolution. Right? In the sense that you\nneed to collect a bunch of stepping stones\nfirst and then build on top of them to to\nreally find innovations or to tune\ninnovations down the line.\nAnd I think language models with the right\nsort of evolutionary hardness\nare extremely powerful in terms of scaling\nup to to to make\ndiscoveries. And, yeah, I think Jeremy as\nwell as the AlphaEvolve paper\nas well as sort of work we've done on,\nlike, the Darwin Gödel Machine, for\nexample, shows that this sort of stepping\nstone accumulation plus\niterative verification and collecting sort\nof information and\nevidence from the real world real\nsynthetic evaluator is really important\nfor that. Very cool.\nAnd stepping stone collection.\nSo that this is it came from Kenneth\nStanley.\nIt's a wonderful paper, Why Greatness\nCannot Be Planned.\nAnd he said that it's it's better to have\nsystems that don't\nconverge. So in natural evolution, we are\njust trying all of these different things\nAnd greatness quite often follows a\ndiverse path, which means you have to do\nthings which initially seem quite stupid.\nAnd then later on, they turn out to be\nincredibly useful.\nYeah. We're trying to design algorithms\nthat can kind of allow for a\npopulation of slightly weird things.\nAnd and then we kind of lock in and and\nconverge a little bit.\nSo we we're still converging though.\nSo we're still building systems that don't\ndiverge forever.\nWhat are we losing? 1 1 thing I find\nextremely important after having done\nShinka Evolve is sort of this problem\nproblem.\nRight? So with all of these systems so\nfar, maybe except for the AI\nscientist, which we can also talk about,\nthe problem is given.\nRight? So you have an evaluator, you have\na correctness checker, and you sample\nprograms only on that single problem.\nRight? But oftentimes, innovation for a\nspecific problem might require first\ninventing a different problem.\nRight? So for example, I think in the\nmatrix multiplication result that\nthe AlphaEvolve people show, you can\nrecursively apply sort of the algorithm to\nlarger matrices, so it's actually an\nimportant result.\nRight? But sort of automatically coming up\nwith this reduction or in like this,\nlet's say, recursive nature of problem\nsolving is something these systems right\nnow not necessarily have built in\nintrinsically.\nRight? So I think going forward, it's\ngonna be really important to not only sort\nof do open ended, let's say, optimization\nof solutions, but sort of do the co\nevolution of problem and solution together\nin order to collect even more\ndiverse stepping stones and to really kick\noff this this open ended process.\nBecause also to me, like, of the the big\nlife goals or achievements\nI would wanna see is really having a\nprocess that can run not only for, let's\nsay, a week or many weeks, but, like, for\nyears even potentially.\nRight? Collecting even more diverse\ninteresting stepping stones.\nYeah. I spoke to Joel Lehman and he was\ntalking about the and uncertainty, which\nis that machine learning algorithms aren't\nvery good with unknown unknowns.\nAnd and in a sense, the unknown unknown is\ntalking about these these stepping\nstones that might be useful later.\nMhmm. And when we run these algorithms at\nthe moment, it's the same with LLMs and\nreasoning systems, is that they're very,\nvery good when we give them a specific\nthing. Mhmm. And what you're pointing to\nis we might need to invent a new\nunrelated problems and find the solutions\nwhich might then be related to what we're\ntrying to do. So that feels like a bit of\na catch 22 situation.\nMhmm. Right? So we're saying, you know,\ncircle packing.\nMhmm. Here's my evaluation function, and I\nwant you to sort of diversify and then,\nyou know, kind of and then converge\ntowards the solution.\nMhmm. It's just I I had the same thought\nwith Genie, by the way, that it it gives\nyou exactly what what you ask for.\nSo you put put a prompt in on, know, like\na Swiss lake with, you know, with boats on\nthe water and mountains on the side.\nAnd I was thinking, where are the birds?\nOh, I forgot to put birds in the prompt.\nMhmm. Right? So how can we meaningfully\nbuild systems that actually kind of bring\nin other unknown things that might be\nuseful?\nI think 1 inspiration or thing I would\npersonally wanna sort of research\nare systems like outlined in in PowerPlay\nor Poet by by by\nJeff Kline and others. Right?\nSo where there is essentially like a a set\nof tasks and a solution\ngenerator, and both of them sort of\ncoevolve in this almost like auto\ncurriculum play like style.\nRight? And I think sort of the in POET,\nthe natural first application was sort of\nreinforcement learning, but I think this\ncan now be broadened up to to, yeah,\nscience more generally.\nRight? At least when there is a simulator\navailable to for for running these\nevaluations. And by doing such a co\nevolution, you you\nalways try to to max out the capabilities\nof that generator\nwhile sort of increasing this this\nconvex hull or potentially even, yeah,\nmore diverse problems while\ndoing so. And I know that there's always\nthe leading thought that even with POET,\nwhich was this thing where you had like a\npopulate you you had like a load of\nenvironments and agents.\nMhmm. And the environments were in\ncomplexified.\nMhmm. So the agents would have a kind of\neffective curriculum to to learn things in\nincreasing complexity. But even then,\nisn't there a kind of design bias in the\nsystem where there's some code somewhere\nwhich complexifies the environment step by\nstep. Mhmm. And wouldn't that also just be\ndesigned by the humans?\nSo it would also just give you exactly\nwhat you ask for.\nUltimately, this comes down to like the\nhypothesis that language models can\npotentially do extrapolation or\ninterpolation.\nRight? In the sense that even though these\nthings might be in the end designed\nby humans, there are many unknown\nunknowns.\nRight? That we humans didn't think of\nwhile designing them.\nRight? So potentially, it is possible for\nan LLM to, yeah, find a novel\ndiscovery simply by us not having thought\nabout it before.\nRight? When we run LLMs autonomously Yeah.\nThey they tend to just kind of like\nnothing interesting happens.\nMhmm. So depending on the prompt you give\nthem Mhmm.\nThey'll kind of go a few steps in that\ndirection and then no new interesting\nnovelty emerges.\nAnd I think even if you wire them with\nenvironmental feedback Mhmm.\nThey they still seem quite parasitic on\ntheir starting conditions.\nWith an LLM, could we build a system which\nactually adapted to novelty that could\nactually discover new things?\nI think it really kind of also depends on\nwhat do you give the\nLLM as a starting point.\nRight? So for example, in Shinka Evolve,\nwe from time on time saw that if you\ngive an initial solution program, which is\nalready pretty optimized on the problem at\nhand, you still kind of get stuck in in\nlocal optima, right, where not a lot of\nnovelty is introduced. Right?\nWhile if you start off from like an\nimpoverished solution, there's much more\nroom for diversity.\nAnd I think this is sort of coming back to\nsort of what I did before in in my\nresearch namely meta learning, sort of\nthis classical trade off where you can\neither start out from something very,\nlet's say, unconstrained\nfrom, like, a very simple solution and\ngive much more room for the optimization.\nBut this might actually require open\nendedness and a long time to find a good\nsolution or you start out from something\nthat is already very constrained by\ninductive biases, let's say, and then you\nmight be much more efficient in terms of\nconvergence, let's say.\nBut you don't have the sort of open ended\nbig novelty sort of benefit from it.\nYes. I suppose where we wanna get to is\nbuilding systems which are not\ndesigned by humans. Mhmm.\nSo for example, if if I'm leveraging my\ndeep understanding, you know, LLMs are\nreally good if you if you understand\nsomething deeply.\nMhmm. And similarly, we could kick off a\nChinker Revolve, and we could could\nput a starting solution in there which\nleverages my understanding.\nMhmm. We want to have AI systems that that\nanyone could use.\nMhmm. So just a non expert could say, want\nto solve this problem and and it will\nsolve the problem. We should talk about\nthe the evolutionary approach.\nRight? So to maintain diversity Mhmm.\nYou had a population of programs and they\nwere separated into islands.\nMhmm. Tell me about that. The way how\nShinka Evolve similar to AlphaEvolve works\nis you keep an archive, like a database of\nprograms, and then you sample\nparent programs with a set of sort of\ninspiration programs.\nAnd then you ask an LLM to basically make\nan improvement to that program.\nRight? So to provide code edits or rewrite\nan entire program or to potentially even\ncross over different programs.\nAnd then you you basically you query the\nLLM, you get a program out, and\nyou evaluate it on the problem at hand.\nRight? So for example, increasing the sum\nof the radii of a bunch of circle in a\nsquare. You run this basically each time\ncollecting evidence from the evaluator,\nadding it to the database, and then sort\nof repeating this process.\nAnd you don't do this sort of\nsequentially, but you do this in parallel\nfor many different programs. And each time\nsort of a program is added, you\nessentially try to diffuse the knowledge\nthat was collected by that program across\nthe entire sort of database.\nRight? So 1 way to think about this is you\nhave a tree, a tree where each node in the\ntree represents a program, and then you\nyou sort of branch off of it based on the\nparent nodes. Right? And interestingly,\nlike, these approaches\ndo tend to scale, but\nideally, can make the scaling at a faster\nrate.\nRight? And this is something we tried in\nChinka Evolve by sort\nof doing a bunch of innovations, including\nsort of model ensembling.\nSo we're not using just Gemini, but we're\nusing basically all Frontier model\nproviders and figuring out a smart way how\nto use each\nmodel for a given parent.\nRight? So if you have a certain program in\nsome situations, it might be better to\nuse a GPT model, in other settings it\nmight be better to use a Gemini model,\nand we sort of introduce a sort of\nadaptive prioritization scheme that can\nadapt sort of the evolutionary algorithm\non the fly while running the the\nalgorithm. And this sort of also comes\nback to the naming.\nRight? So Shinka Evolve evolve evolve kind\nof means that this evolutionary algorithm\nthat we apply using LLMs sort of also co\nevolves at the same time\nwhile we optimize the programs.\nAnd on this while we're on this circle\npacking problem.\nSo you you had this plot showing how it\nconvergent and seemed to converge quite\nquickly. So and we'll show the plot on the\nscreen now.\nSo very quickly, the performance jumped up\nand then it slowly converged.\nAnd you said in the paper that it was\nusing 3, I think 3 core innovations.\nAnd my thinking was, if you ran this 50\ntimes,\nwould it be the same every single time?\nAnd how to what extent is it thinking\noutside the box?\nYou know, Sebastian Bubeck is always\nposting on Twitter talking about how\nGPT-five has just, you know, discovered\nnew things.\nAnd there's always the question of, well,\nis it just searching the Internet?\nIs it just finding things that have been\nfound before?\nAnd, yeah, combining things together in in\na new way.\nBut could it really think outside the box?\nMhmm. Yeah. I think this is almost like a\nsubjective question.\nRight? So first off, I don't know all\nproblems on the Internet that try doing\ncircle packing. Right? But what I can see\nin the tree that we also depict is,\nthere's for example, like a crossover\noperation between 2 programs happening\nwhere sort of\ndifferent concepts are combined.\nRight? So 1 important part is, for\nexample, the the initialization of the\ncircles. Another 1 is, like, the\noptimization.\nSo basically, like a constrained\noptimization program is executed.\nAnd then the final part is basically like\na reheating stage, right, when noise is\nadded and sort of more stride to be\nsqueezed out.\nAnd to me, like this sort of propagation\nof information through the\ntree is 1 that's really really\nfascinating.\nRight? Where in some sense, these stepping\nstones are actually used and\nso in a complementary fashion.\nRight? And with regards to rerunning the\nprogram multiple times, right, of course,\nthere's some stochasticity in that.\nRight? So we're using language models and\nsort of due to, like, the the queuing\ndevice scheduling on on their server side,\nbasically, we can't get rid of all the\nall the noise. We we've seen that at least\nfor the general quality of the\nsolution, so what is the right afterwards,\nit is possible to re obtain this.\nBut sometimes with a different program\nlike most of the times just by\nstochasticity. Right? So it's not like\nthere's for many problems, there's, like,\nnot 1 solution that achieves that score,\nbut there is, a spectrum or, like, a a\nregion, let's say, in the program space\nthat that resembles the same.\nRight? I think 1 thing that was very\ninteresting about the circle packing\nproblem, sort of also coming back to the\nproblem problem that I discussed initially\nwas that originally, we we used a\nformulation where\nthe correctness is checked with, like, a\nvery tiny amount of slack.\nRight? So the the circles could overlap a\ntiny little bit.\nAnd then afterwards, we we we sort of\nreduced the radii and the solution\nwas exact. Right? This didn't change the\nscore by too much, so it's still state of\nthe art, but it was essentially like a\nproxy problem.\nWe then reran the the Shinka Evolve on the\nexact setting, and we\nfound that it took a little bit longer to\nactually obtain the same quality of a\nsolution. So I think this already points a\nlittle bit in this direction of what I\ndiscussed in the beginning, like sometimes\nsort of surrogate problems might actually\nbe extremely valuable in in making such\ndiscoveries.\nAnd having an automated way for designing\nthese surrogate problems in an efficient\nway might be something really important\ngoing forward.\nYeah. That's absolutely fascinating.\nIt reminds me of support vector machines\nwhere we make the optimization tractable\nby introducing slack variables and you can\nthink of that as a kind of surrogate\nproblem. But then I'm thinking what would\nShinka Evolve or AlphaEvolve, would it\nknow to introduce a surrogate problem?\nBecause, you know, as designers who\nunderstand, you know, we can think outside\nthe box and and we can do stuff like that.\nBecause presumably, if the fitness\nfunction had the constraints that there\nwere no circle intersections, then it\nwouldn't it wouldn't occur to the\nalgorithm to come up with a surrogate\nproblem.\nExactly. Yeah. This is a big limitation\nright now.\nRight? So at this current point in time,\nwe take the problem to be fixed and we\noptimize for that problem.\nBut when you think about humans, we're\nreally really good at of inventing our own\nproblems, right, or reformulating the\nproblem so that we can actually sort of\nwork with it. Right? So I think a lot of\nsort of the innovations in,\nlet's say, mathematics come from taking a\nvery different perspective on a problem.\nRight? So taking sort of number theory and\napplying it to linear algebra or the other\nway around. And I think right now, these\nsystems are not yet at the point of of\nachieving such level of, let's say,\ntransfer.\nYes. And it reminded me, I spoke to Llion\nabout this.\nYou've got this Sudoku bench.\nAnd a lot of folks watch cracking the\ncryptic YouTube channel.\nAnd that's exactly what they do. They\ninvent new problems based on abstractions\nthat capture the essence or aspects of the\nproblem you're solving.\nAnd then they do something which is\nsimilar to Shinka Evolve.\nThey do this kind of evolution where they\ntake these different solutions and they\nkind of combine the best aspects of both\nof them.\nAnd they forge a divergent path to a new\nsolution.\nYeah. And that seems to be the essence of\nof what we need to do.\nYeah. For sure. I I mean, is some work\nalso by Jeff Clune, Shengran Hu, and Cong\nLu on automatic automated capability\ndiscovery.\nSo there, they look at language models\nthat generate tasks.\nRight? But it's in a, let's say,\nunstructured way in the sense that it's\nnot done in order to enable the solution\nto 1 target\nproblem. Right? And I think sort of doing\nthese connections is gonna be very\nfruitful down the line. Very cool.\nNow, other thing, we'll show the graph on\nthe screen, the evolutionary graph.\nSo for the circle back in problem.\nI was looking at that and first of all, it\nlooked incredibly parsimonious, which is\ngood. It it looked like it had found an\noptimal path to the solution very quickly.\nAnd I was thinking in my mind, well, maybe\nthere's some natural pattern that that\nthere's there's there's there's something\nabout that that we could use in the\nabstract to guide the evolution in the\nfuture.\nBut the other thing I'm thinking about is\nright now, the problem with machine\nlearning is that we don't really have\nsemantics baked in.\nSo what we're doing is we have a verifier,\nwe're looking at the rewards, and we're\nsort of like doing patterned exploration,\nand we're taking steps towards the, you\nknow, towards the target.\nAnd I love mechanistic forms of reasoning\nwhere we actually know something\nabout what the program components mean.\nAnd the reason this is important is when\nwe're merging together the best performing\nprograms from 2 different islands.\nThat's a kind of first order interaction.\nIt might not make sense to merge them\ntogether.\nIt's wonderful that LLMs, you can give\nthem any pairs of programs and it will\nfind a way to merge them together.\nBut wouldn't a more principled way be of\nthere's there's some kind\nof semantic primitives here and we know\nthey fit together.\nSo there's this Lego analogy that we're\nkind of building up based on principles\nrather than forging our path based on the\nperformance.\nYeah. That's a good point.\nSo 1 thing we do in Shinka Evolve as well\nis we keep essentially a scratch\npad. So each program is being summarized.\nAnd then from the program summaries, we\nkeep sort of a set of global insights,\nlet's say, then we're shared or, like,\nextracted from these programs.\nAnd then based off of the scratch pad, we\nconstruct sort of meta recommendations\nthat then become part of the system\nprompt.\nRight? So that way, you can try to sort of\nsemantically\ngrasp some of the discoveries.\nBut a general problem, which is again sort\nof task dependent is\nthereby you sort of diffuse that knowledge\nacross the tree.\nRight? But sometimes you want things to be\nmuch more isolated.\nRight? It's always like a trade off where\nyou somehow have to find for your problem\nthe right position on the spectrum of how\nmuch non diffusion do you wanna have and\nhow much sort of, let's say, hard islands\nof programs do you wanna have.\nRight? And, yeah, we're to make steps in\nthe direction of sort of automatically\nadjusting this in an optimal way, but\nagain, it's very program sensitive.\nAnd then sort of, I think, another point\nwhere you're already sort of going into is\nsort of Jeremy Jeremy's solution to Arc\nAGI.\nRight? And sort of doing solution\nevolution in the instruction space,\nright, instead of the program space.\nI do think that this is something\nimportant, and we're\nlike I said, with, like, the construction\nof this meta scratch pad trying to do sort\nof both at the same time.\nAgain, it's problem dependent.\nLike, I played around a little bit with\nARC AGI 1 and ARC AGI 2.\nAnd I think on ARC AGI 1, actually, the\nthe transform sort of program direction is\nactually quite effective. Right?\nIt's like Jeremy said, it's deterministic,\nand it's easier to sort\nof get clear signal to improve on during\nyour evolution process.\nWhile on others, like ARC AGI 2, like this\nwhole sort of semantic evolution seems to\nbe more efficient. So I think ideally we\nwe can get a system that can\nautomatically in some sense decide whether\nor not it wants to take like a\nprogrammatic approach in settings where\nit's actually feasible and easier to to\nbootstrap off, or it takes the semantic\napproach of evolving instructions or\nlike LLM driven input output mappings.\nYeah. It's it's so interesting because,\nyou know, like a a symbolic AI person\nwould say, oh, I don't like connectionism\nbecause it doesn't under you know, the\nonly semantics in connectionism is this\nnotion of similarity.\nIt doesn't really understand things.\nSo so they would say, well, just just\nstart with a a an entity relationship\ngraph and then just kind of build up\nusing, you know, composition and first\nprinciples. That that that doesn't work.\nRight? So we're using neural networks\nbecause they're incredibly flexible and\nthey understand a lot of things about the\nworld, but they don't have the kind of\nconstraints that we want. So what we do is\nwe use these tricks.\nSo Jeremy, we evolved program\ndescriptions.\nOn your program selection, you had a\nsemantic novelty detection, you know,\nusing like a Embedding based similarity.\nYou had like a kind of self similarity\nmetrics and you know, based on the\ncosines. And indeed, you've got this meta\nscratch pad.\nSo what we're seeing is this fascinating\nspectrum of possibilities where still\nusing neural networks, you can imbue\nsemantics in using all of these different\ntricks, but they all come with trade offs.\nYeah. For sure. Like, I think it's it's\nkind of interesting.\nWe we've had a long period of computer\nscience where algorithms were sort of\ndesigned by humans. Right?\nThen we had sort of this Android Kapathy\nsoftware 2 paradigm\nwhere, like, we trained neural networks\nthat then performed a certain function.\nAnd now we're sort of at this point where\nwe're using LLMs to design algorithms\nor solutions more generally.\nRight? And I think, actually, like, even\nthough, like, large\nfrontier language models are extreme,\nlike, let's say, black boxes or it's very\nhard to get a full mechanistic\nunderstanding of them, the outputs can be.\nRight? The programs, the instructions, and\nso on.\nRight? So I think it opens up a very sort\nof new paradigm of\ndoing research or basically doing\nanything.\nRight? If you if you think about it.\nBut I think we're we're just sort of at\nthe starting point of figuring out the the\nright user interface for that.\nSo the other innovation in the paper was\nusing UCB, which is upper confidence\nbound. It comes from the multi arm bandit\nliterature, which is this problem where\nyou can pull these these levers and at the\nbeginning, you don't know which levers to\npull. And and over time, you kind of\nreduce your uncertainty and you can kind\nof pull the ones that work. But there's\nthis exploration exploitation dilemma.\nAnd you've implemented that for figuring\nout which LLM.\nSo it could be Gemini. It could be like,\nyou know, Grokfur or something to figure\nout which 1 to use.\nWe're we're using like a model ensemble,\nright, to propose program mutations.\nAnd\nintuitively, 1 could say like the the best\nfrontier model on on SWE bench is always\nthe best mutation proposal model.\nBut that's actually in practice not always\nthe case.\nRight? And in general, it's extremely hard\nin this evolutionary setting to\nassign clear credit to a single model.\nRight? So you have, for example, like, 1\nimprovement is implemented by\nGPT 5, and then the next 1 is implemented\nby SONNET 4.5.\nAnd it's unclear basically if the\nperformance gain you get from the second\nmutation actually originated from GPT 5\nsort of collecting the first stepping\nstone or from SONNET 4.5.\nSo instead of sort of uniformly sampling\nmodels, what we do is we\nimplement this bandit based approach where\neach model is basically 1 arm of\na bandit. And then we look at how often\ndid this model\nimprove performance of a sort of parent\nnode by\ncreating a mutation. And we then adjust\nsort of this posterior probability\nto sort of first explore all arms once,\nright, and then essentially\nchange over the course of time in order to\nprefer models that sort of yielded\nimprovements before for similar nodes.\nThe great thing about using a UCB like\nalgorithm is is you can it it actually has\na theoretical regret Mhmm.\nWhich means it's not it's it's like only\nlog worse than the optimal\nswitching path if if that makes sense.\nBut if I understand correctly, UCB is\nbased on a\na sort of like a global rating, like a a\nmean score of every single\nLLM. And I think what we want is to have\nmore of a contextual\nswitching decision, which means we know\nfor this particular program,\nGemini is better.\nAnd do I understand correctly at the\nmoment that it might converge to a single\nfrontier model and then in a nuanced\nsituation, we might still get the wrong\nmodel. So in general, like there is some\namount of probability associated like\nallocated to all models.\nRight? So it's not like it can just peak\non 1 model and then you stop using the\nothers. Right? So there's still a chance\nfor open endedness and serendipity, if you\nwill. And we in general, like, for the\nproblems we consider, we we haven't seen\nthat, like, 1 model clearly dominates all\nthe others.\nRight? We've seen then it really depends\non the course of this evolutionary\nprocess, like which model is better and\nUCB or\nlike the the banded approach that we take\ndynamically adjust this in in an efficient\nway. And would it be possible in the\nfuture to use an LLM to make this\njudgment? Potentially. In some sense, in\nthat case, again, you think of the LLM as\na surrogate model. Right?\nIn some sense, you can think of, like, a\nGaussian process as a surrogate regression\nmodel, and there has been some work sort\nof showing that language models can act as\nsurrogate models.\nAnd the real question to me is, like, how\ndo you represent the information to\nthe LLM, right, in the sense that if you\nuse, like, the raw programs and their\nfitness evaluations, you you quickly run\nout of context.\nRight? So you need some amount of\ncompression in order to present the\ninformation the right way to the LLM in\norder to do this prioritization of the\nmodels. I hadn't appreciated how long the\ncontext is.\nI I I was thinking, you know, could we use\nlike an 8,000,000,000 LAMA model\nand we're doing active fine tuning.\nSo we're saying, just ran it on, you know,\nI just ran this program on Grok Yeah.\nAnd and it got this score.\nYeah. And and then over time that, you\nknow, this thing for the given run of this\nevolution, it will kind of know that Grok\nis good at these problems.\nYeah. Potentially. I'm not sure, like, how\nefficient this fine fine tuning is if if\nwe're only evaluating, like, a 150\nprograms.\nBut in principle, 1 could imagine.\nI think it's on the engineering side, not\nnecessarily like the prettiest to do.\nYeah. It could it could in fact happen.\nBut I think, like, for all of these\nthings, we started out sort of with the,\nlet's say, most intuitive algorithmic\ncomponent that we had, and UCB was 1 that\nreally did the job here.\nAnd, yeah, much credit to Edoardo Cetin\nwho introduced us to to Shinka.\nSo let's talk about the diffs and and the\nmutations.\nSo we've we generate programs and I I\nthink you folks\nwere inspired a bit by AlphaRevolve.\nSo they actually had this gating where\nwhere you kind of gate part of the code\nwhich is mutable. Tell me about all of\nthat.\nA program is just, let's say, a long\nstring.\nRight? And in order to to make sure that\ncertain parts which are sort of essential\nto the evaluation, for example, into the\nimports and so on, we're not sort of\ndeleted by the LLM mutations.\nThey are so called markers, which\nbasically state which parts of the code\nare mutable and evolvable.\nAnd it's easy to like programmatically\nsort of make them actually mutable when\nyou get a diff proposal and these will not\nbe changed.\nSo only the the rest of the\nthe code snippet will be changed.\nWe sort of implement a type of rejection\nsampling with reflection approach where if\nan LLM by chance, for example, tries to\nmutate this part, it's gonna be rejected\nand you resample a new proposal.\nAnd, yeah, thereby you you can somewhat\nmitigate certain security or\nsafety problems and, yeah, get a robust\nsort of\nmutation. 1 of the sort of I think the the\nbigger questions is how can you\nturn this from a single file mutation\nsetup to a multi file mutation setup.\nSo working on entire code bases.\nIn principle, you can represent many code\nbases in a single file.\nRight? But the hierarchical structure\nmight be actually useful.\nAnd there are some ideas from, let's say,\nAdder, this this\ncoding tool where you construct like a\nrepository map and\nsort of have some level of abstraction,\nbut they also come again with positive and\nnegative trade offs basically.\nAnd I love Ada by the way.\nMhmm. It it feels that in the future, the\nthe audio, know,\nlike the cogeneration systems will will\nactually resemble Synchrovolve.\nAnd if if you think about it, it'll be\nusing some kind of git repo.\nMaybe cursor already does this because in\ncursor, can restore previous checkpoints.\nBut it can be exploring different branches\nand and merging checkpoints together\nand and you know, obviously you just say\nin natural language what you wanna do.\nBut we didn't talk about mutation by way.\nSo so we just spoke about diffs and\nthere's also an option to do the full file\nrewrites. Exactly. But there's also this\nnotion of of of crossover.\nSo how how does that work? A small\ninnovation on top of AlphaEvolve and where\nI believe they only use sort of diff based\nmutations is that here we wanted\nto have more flexibility to entirely\nrewrite the program, right, to\ncome up with a completely different\nstepping stone if you will.\nSo, again, there you can make parts of the\ncode mutable, but instead of proposing,\nlet's say, a patch to change certain parts\nof it, we essentially rewrite the entire\nprogram. And this sometimes is helpful.\nRight? It's not always like a clear\nbenefit, but it it allows you to\nessentially get more diversity into the\nsearch.\nRight? So this is 1 type of mutation next\nto sort of this\ndiff patch based approach, and the other 1\nis a crossover mutation where we sample\nbasically not only a single parent\nprogram, but sort of 2 different ones, and\nwe ask the system to sort of make a\ncomplementary improvement.\nAnd again, on some problems, this is\nreally helpful and on others, it's not.\nBut in generally, we found that sort of\nhaving a diversity in terms of\noperators is also helpful in discovering\nnew things.\nAnd I wanted to to sort of follow-up on\nthe point you made before about this sort\nof being a new paradigm. Think so too.\nI'm really convinced. I think right now,\nwe're sort of at the beginning where we we\nstill think a lot about sort of this chat\nassistant interface as the way how we\ninteract with LLMs, but it's most of the\ntimes inherently single threaded.\nRight? So we're sitting in front of the\ncomputer.\nWe're interacting with the chat.\nWe're seeing sort of changes as they occur\nin the editor.\nWe accept them and so on.\nBut I think this is sort of also just a\nstepping stone towards sort of a more,\nlet's say, distributed way about thinking\nabout research, optimization, and so on.\nSo I like to sort of think of vibe coding,\nvibe\nchatting, and on the other hand, we have\nsort of vibe optimization and vibe\nresearching where sort of my ideal future\nscenario is 1 in which you as a\nresearcher sort of during the day co work\nwith, like, a system like Shinka or\nthe AI scientist. You sort of steer the\nship like a shepherd in some sense.\nAnd then during the night, you you you\npress play and you go to bed and in this\nin the background, you have multiple\nexperiments running and automatically new\nones being proposed by LLMs, evidence\nbeing accumulated.\nAnd then in the morning, you come back and\nsort of you have an multi threaded sort of\nsystem running in parallel.\nAnd you're more like the shepherd of the\nship than the the person actually\nexecuting\nexperiments and analyzing. Oh, yeah.\nYou're still analyzing, but you're not\nexecuting.\nThis is happening sort of by the system\nitself.\nYes. And increasingly, this might be semi\nsupervised or even proactive.\nI mean, know, there's that new product\nfrom OpenAI where it knows what you're\ninterested in and while you sleep, it's\ngoing off and, you know, find your pulse.\nThat's right. And you know, we're in the\nsituation now where we're reasonably\ntechnical people. So you know, MATLAB and\nMathematica, they're\nsupremely powerful.\nBut you need to know how to express\nproblems precisely.\nWhereas I can imagine a future where we,\nexpress problems\njust in natural language or maybe just\nbased on our interactions with language\nmodels. The platform knows what we're\ninterested in and it can just go and find\nthings on our behalf because this is about\ndemocratizing this technology to people\nwho perhaps don't know exactly what\nthey're looking for.\nI think 1 of the bigger problems there is\nsort of this verification aspect to it.\nRight? In the sense that oftentimes it's\neasier to generate a lot of solutions than\nto actually, like, hard verify them.\nRight? Language models are capable of\ndoing sort of soft verification, looking\nat code, and sort of latently running like\na like a stack trace of execution.\nRight? But it's not exact.\nRight? And I think sort of these notions\nof reward hacking and sort of\nnot doing real discoveries, but sort of\nshortcutting them is 1 where we need to\nput more time and effort into to figure\nout, yeah, how to make sure that this\nactually moves in the right direction.\nRight? And I would hope that language\nmodels at some point can do this\nefficiently themselves.\nRight? So either implementing in code or\nlatently doing it.\nBut this is also, like, part of the\nproblem problem.\nRight? It's not only coming up with the\nproblem, also with the automatic\nverification at the same point.\nYeah. Isn't it a tantalizing idea that\nthere are natural patterns in the\nworld and the building blocks to construct\nnovel\nsolutions are already there.\nRight? And and maybe they're there for a\nreason.\nMaybe they just reflect natural\nregularities in in the universe.\nBecause there's always this question of,\nyou know, intelligence is about adapting\nto novelty. So the world is always\nchanging.\nAnd the world tomorrow will have things\nthat we can't explain, you know, with our\nwith our knowledge today. But we do have,\nlike, abstract knowledge that could\nbe easily recombined to explain the\nfuture.\nAnd LLMs might already have those building\nblocks.\nYeah. For sure. I think, like, in some\nsense, the more you think about sort of\nOccam's razor applying to everything in\nour world, like, let it be language or let\nit be sort of science,\nis is pretty interesting because, like,\nthese artifacts now go into our language\nmodels of today, and potentially, is some\namount of this being captured.\nI think though it might also be an\ninactive bias that leads to a local\noptimum at some point, right, and you need\nmore complexity.\nBut I do think like with systems that sort\nof do this evolutionary mutation sort of\nstyle approach, you might still sort of\npush the system out of this local optima\neventually. Yes. And then there's also the\nnotion of the importance of adaptivity.\nSo this is what Charle says in\nintelligence is.\nAnd since we've had these models that\nactually do adaptivity\nat inference time, so things like test\ntime, active fine tuning, and the\nreasoning models and so on.\nThey started getting non trivial\nperformance on ARC.\nNow, it's very, very expensive to have\nadapting huge\nfoundation models. It's just a practical\nconcern why we haven't done that yet.\nBut what we can do is build systems like\nshrink or evolve that leverage the best\nof both worlds. So they leverage frozen\nfoundation models, but they give you\nadaptivity. And the purpose of adaptivity\nis to respond to novelties,\nto create new building blocks, synthesize\nnew building blocks in this principled\ntree like structure that allow us to adapt\nto novelty.\nYeah. So we are having our cake and eating\nit.\nI have to say I found it very interesting\nthat Jeremy basically in your podcast when\nyou asked him about Shinka was saying\nlike, he doesn't believe that there are a\nlot of sort of percentage points to be\ngained by using a system like Shinka, but\nyou can make it much more efficient.\nRight? That was sort of the gist of his\nanswer.\nTo me, it's like once you have made it\nmuch more efficient, you can scale it up\nagain. Right? So if you essentially have a\ncheaper system that can generate many more\nsort of instructions, I would expect that\nby the nature of open endedness,\nyou might get some amount of improvement\nout of it.\nRight now, I don't have any evidence for\nit.\nI would love to collect that evidence.\nIt's again like the magic of open\nendedness that comes into play that as\nlong as sort of these training examples of\nArcGi give you a good signal for a final\ntest submission, and you should be able to\nto progress.\nYes. And that and that is a great segue\nbecause certainly on on the circle packing\nproblem, it was so sample efficient that\nin less than 200, you know, interactions\nwith an with an LLM, you converge on the\nsolution.\nBut I was thinking that great, but it's\nstill quite\ndependent on the starting conditions.\nYou know, we talk about this design bias\nand and and so on.\nSo what we put in is very important.\nBut now what we could do is scale out.\nSo we could run this 1000 times and we\ncould have another process which\nprompts, generates, breeds the starting\nconditions because because every time we\nrun Chinker Revolve, what it's doing is\nit's it's searching parts of the epistemic\ntree. Sure. And what would happen if we\njust scaled that out massively?\nWe haven't tried, but you could even start\nwith, like, an empty program.\nRight? Which be it would be basically the\nsame.\nRight? And then you would branch off of\nthat empty program, I would expect.\nYeah. We haven't done this simply out of\nsort of cost and\ntime reasons.\nBut I do think, in many ways, sort of,\nthis is the question that will push\nus towards like this true open ended\nvision of running a system for like a\nmonth or so. Right? Really trying to\nsqueeze this out.\nYeah. I'm not sure if we're entirely there\nyet, but I will do my best that we will.\nAnd the reason this is interesting is we\nknow as a practical matter that we can't\nstart with nothing. Mhmm.\nIf we were just sort of like starting from\nthe most primitive building blocks, the\nsearch space would just be huge and\nthere'd be no learning signal.\nSo we know we need to start a little way\nup the stack, but we can massively\nparallelize that. So that you let's say we\nhave 1000 different instantiations of\nShinka Evolve. It doesn't have to be\nembarrassingly parallel.\nWe could still have some sharing.\nSo during their execution, we could still\nhave a little bit of like crossover and\nand maybe then we could we could run all\nthe Shinka Evolve instantiations in\na in a similar kind of meta evolution\nloop.\nAnd my suspicion is contrary Jeremy, I\nagree with you.\nWe know there are diverse stepping stones\nout there that could dramatically\ndramatically improve many of these\nsolutions.\nWe simply haven't scaled it up yet.\nYeah. I also believe that using a system\nlike Shinka Evolve\ncould be able to sort of automatically\ndetect whether or not like an instruction\nbased optimization approach for a given\nproblem or a transform based\napproach is actually the right thing to\ndo.\nAnd sometimes potentially, it's like even\nthe mixture.\nRight? There's some things you can\nprobably easier even articulate in Python\nthan you can articulate in in sort of\nlanguage.\nRight? So I would be really interested in\nsort of exploring that.\nYeah. I mean, you said earlier about\nJeff's clean what what was Jeff clean's\npaper? The the thing that generates\ncapability discovery.\nI did speak to him about this at Neuros,\nbut something like that could be\nfascinating as well. You know, where we're\nalso generating the problems and solutions\nand then kind of moving them back in.\nBut I I I think the way this will land\ncommercially is there'll be a new type of\nGPT where everyone is solving different\ntypes of problems and and the system,\nit'll be like a kind of Shinka Evolve but\na massively distributed version where\nmathematicians are using the platform over\nhere to solve this problem and it will see\ncommonalities and and it will kind of like\nlink them together.\nBecause you need to leverage like human\ncreativity in this process as well, I\nthink. Like a big challenge going forward\nis going to be like how do we change our\nincentive system for this to actually\nscale.\nRight? I think like for example, some\namount of economy will be needed\nor some amount of mechanism design in\norder to make sure that everyone is still\nhappy to engage in it.\nRight? So maybe we're gonna have many more\nleaderboards for whatever is numerically\nsort of scorable.\nAnd I think this this will be really,\nreally interesting to see how sort of\ncompute these automated agents, human\nshepherding and steering will\nultimately sort of change and\nrevolutionize science and, I guess,\nsociety more generally. And Rob, looking\nat the future, we've got a load of people\nin in San Francisco that that wanna scale\nlanguage models.\nAnd they are adding in implicit forms of\nadaptivity and composition.\nSo that they're building controllers and\nthey're doing reinforcement learning with\nverifiable feedback and so on.\nI think that you subscribe to the slightly\ndifferent idea that that we need to be far\nmore open ended and we need to be using\nevolutionary algorithms and so on.\nBut do you think that they are on a path\nto nowhere?\nDo you think they might change tack?\nDo you I mean, where where is this going?\nSo I I actually think that these things\ncan be complementary, right, in the sense\nlike, let's say you fine tune a model to\nbe like a circle packing\nexpert. Right? So I I do believe that\nmixing in sort of different sort\nof RL fine tuned models into sort of the\nensemble of models and then having a good\nway to adaptively select which 1 model to\nuse is is not a bad idea.\nRight? So to me, I just very fully\nsubscribe to this\nphilosophy of open endedness and reading\nKen's and Joel's book was\nreally like a fundamental moment in my\nlife.\nAnd I want to see how far we can push\nthis.\nAnd I think we're we're not yet at sort of\nconvergence where either in the\ncapabilities of the models has converged\nor the the way how we scaffold around them\nor the way how we humans\ninterface with them. So to me, they're\nreally like these 3 points, like model\ncapability, model scaffolding, and sort of\nthe user interface.\nAnd I think we have a lot still to push on\nall 3 angles.\nBeautiful. The only thing we didn't talk\nabout was we spoke about the circle\npacking problem, but you also applied it\nto a few other things.\nCan you tell us about that? So 1 thing we\ndid was we sort of\nused a framework called ADAS, automatic\ndesign of agentic system, where\nbasically instead of manually writing an\nagent scaffold, you use an LLM to write\nagent scaffolds for a specific task.\nRight? So what we did is we looked at\nmathematics tasks.\nSo AIME and we used Shinka to evolve\nbasically\nan agent. Right? So using an agent to\nevolve an agent.\nAnd we found that there we could\ndramatically improve sort of the\nperformance of very cheap models like GPT\n4.1 nano, but the agent scaffold was also\nable to either, like, generalize to other\nlanguage models or to\ndifferent years of of AIME.\nRight? That was 1 application.\n1 important other application that we did\nwas to ALE bench.\nALE bench is basically work done by other\nfolks at Sakana,\nincluding Yuki, who's also part of the\npaper, which is considering heuristic\nprogramming contests, sort of previously\ndone and executed by\nAtCoder, which is like this famous\nJapanese competitive programming\norganization. And we sort of showed that\nShinka can also work very well\nas a coscientist. So basically, we we took\ninitial solutions obtained by\nan ALE agent that was previously designed,\nand then we optimized on top of these\ninitial solutions with Shinka and showed\nthat on 1 of these\nsort of programming tasks, if the\ncombination of this agent and Shinka would\nhave competed in the challenge, it would\nhave ranked second place,\nbasically. So I think there's some\nevidence that Shinka can work as\na coscientist and not only for LLM agents,\nbut potentially\neven for humans like we discussed before.\nAnd then finally, the final application\nthat we looked at was designing sort of\nmixture of expert load balancing loss\nfunctions.\nSo at Sakana, we've done some previous\nwork called DiscoPop.\nI think we discussed this during the last\npodcast we did where we are using LLMs to\ndesign objective functions, and back then,\nwe did it for preference optimization and\npost training. And here, did it for load\nbalancing of mixtures of experts.\nAlso there, we found that within, I think,\nlike, even only 20 sort of\ngenerations, we were able to sort of\nexplore, let's say, not only a\nsingle objective function, but sort of,\nlet's say, a convex hull where there\nare different trade offs between sort of\nperformance and load balancing and so on.\nSo I think this is another application of\nShinka where it's not only basically about\nsort of finding the best solution, but\nessentially illuminating a program\nspace where there are always potential\ntrade offs between, like, let's say, for\nexample, runtime and the quality of the\ncircle packing.\nRight? And having a system that can\nexplore all of these is important as well.\nI'm very excited to see you apply this to\nthe ARC challenge.\nMhmm. Like, what what are what are your\nthoughts about that?\nI still need to collect results.\nSo I I don't wanna make any claims, like,\nhard claims before having done this.\nBut I would hope that there is some chance\nof, for sure, improving sort of the the\ncost of these systems and then potentially\neven performance.\nBut, yeah, to be seen. Oh, very so you've\ndone some experiments.\nExciting news is potentially coming.\nI've started looking into it.\nYeah. And I mean, what what are your\nthoughts in general about about ARC\nthough? I think it's great.\nI think it's it's really important, and I\nthink it\nfills an important gap.\nAnd I do really deeply respect François\nand sort of\nread the paper when it first came out, and\nno 1 thought of actually being able to to\nget numbers above 10%.\nRight? And it's also pretty fascinating on\na society level how far we've\ncome since then. And sometimes while\nyou're sort of deep in the, say, battle\nmode or work mode, you can forget where\nyou were 1 year ago and\nthen just looking back, it's pretty\namazing.\nAlso, how far we've come since o '1.\nIt's insane. I I think François doesn't\nget enough credit because it's such a good\nbenchmark. And not necessarily for reasons\npeople think because François is always\nsaying that we need to have a benchmark\nwhich is easier for humans and hard for\nAIs. And and in a sense, that's not quite\nthe case.\nI I said when Arc v 2 came out that it's\nactually very difficult for humans that,\nyou know, there was 1 task where Dagar was\nstumped for about 15 minutes.\nI agree. There was 3 of us looking at it\nand we we just it's and 1 of those things\nthat depending on your perspective, you\nmight get it straight away or or you might\nnot. So there's that criticism. And people\nhave said that ARC v 3 is even harder.\nYeah. Yeah. You know, but I I think that's\nrather missing the point.\nI I think he's saying that with with with\na lot of these competitive coding\nproblems, The the dataset is contaminated.\nThese are problems that have been solved\nbefore in in part or in whole, which means\nwhen you look at the epistemic tree, many\nof the building blocks for solving them\nare very high up in the tree.\nHe's he's looking at these these problems\nthat there is very little dataset\ncontamination. And they need to be solved\nfrom very abstract building blocks.\nSo you're starting much lower down the\ntree and you're synthesizing a model by\ncomposing together very abstract building\nblocks, is the essence of intelligence.\nYeah. And and I think for that reason, ARC\nis is really kind of pushing us to build\nadaptive systems which we could say are\nintelligent.\nYeah. I agree.\nI I mean, like, in many ways, I'm I'm\nreally looking forward to the next years\nand seeing how far we can push this and\nthen also how much generalization we can\nget afterwards. Because I I I believe,\nlike, when you look at sort of the more\nrecent models, they're getting much better\nat the transform style\ncode evolution or outputting for ARC than\nthey are on\nthe instruction based level.\nAnd I think this might already be like a\nsmall sign of some amount of\novertraining on ARC AGI 1 at least.\nRight? I do believe there are some aspects\nof work which will\nbe automated before it comes to sort of\nfully science automation\nand the type of work I'm doing.\nBut I could imagine that certain parts of\nthe dimensions that I deal with every day\nare for sure going to be hit by AI.\nAnd then the question is, are there gonna\nbe new dimensions opened up that we as\nhumans will fill in? Right?\nAnd I think what I said before about,\nlike, shepherding and so on, I\nreally hope that that's the way forward,\nright, in the sense that humans are the\nones steering the ship while just being\nmassively amplified in their productivity.\nRight now, I am not really seeing the kind\nof job market disruption that\nwas being predicted. I know from personal\nexperience that in in a\nsense, it's made it very difficult to hire\npeople.\nYou know, script writers use ChatGPT.\nI can spot it instantly.\nAnd\nwriters and copy editors are actually in\nmore demand than they were before fixing\nall of the crap that's been generated with\nChatGPT.\nAnd there's the cloud analogy as well.\nSo, you know, IT system administrators who\nwere earning, you know,\n£60,000 a year in The UK, they rebranded\nas as cloud or DevOps\nengineers, and they more than doubled\ntheir pay.\nAnd people are very adaptive.\nThey they see new trends, new bandwagons,\nand and they just adapt and and they add\nvalue on top. And that has been the trend\nfor, you know, for a very long\ntime. Do you think that AI is going to be\nso transformative\nthat it will transcend people's ability to\nadapt?\nI think it's just a question of speed.\nRight? So I was talking about sort of\ncultural evolution and technological\nevolution. And it seems like we humans, we\nneed more adaptation\nand more time to to get used to the\ntechnology to carve out these niches\nwhere we we can fill in and it's\ncomplementary.\nRight? So first off, I I think we're we're\nstill not at the ceiling of the sort of\ntechnological progression.\nRight? So maybe in a couple of years, we\nwill need less of sort of slop editing\nlike you said. But I do think we we need\nsome more time to adapt to the different\nmodalities of interacting with these\nsystems.\nRight? I think everyone can sort of\ninteract with chat assistant,\nbut I think this is the most sort of naive\nform of interacting with\nAI agents, for example.\nRight? So, yeah, I think we need to get\nthe pacing of all of this\nright and we need to do much more\nexploration in human machine\ninterfaces, UI UX design and how to make\nsure that\nhumans sort of fill or feel fulfilled\nduring this experience.\nThis is particularly relevant because, you\nknow, you were behind the AI scientist\npaper and there's now version 2 of that.\nAllow me to be a tiny bit skeptical.\nYou know, we were talking about when we\nevolve systems to do a to do\na particular thing. And at the moment, it\nfeels like as good as they are, they\nare still quite\nparasitic on the instructions and\nintentions of the human supervisor.\nSo it's very much an exchange between the\nhumans and the\nsystem. Because the implication is that in\nthe future, we might have systems that are\nso autonomous and so open endedness and\ncan figure out valuable\nthings to research that humans wouldn't be\nneeded anymore.\nAnd the reason why I'm not that worried\nyet about labor market disruption is I\nstill believe deeply that humans are the\nsource of deep understanding and\ncreativity in the world.\nIf I didn't believe that, I would be very\nworried.\nI agree. To me, like the AI scientists\nlike v 1 and now v 2 are sort of\nglimpses into a potential transformation.\nBut I fully agree in order to make really\nbig scientific\nbreakthroughs, like multiple of them,\nlike, every day or whatever, you still\nneed humans in the loop to sort of either\nseed or guide the direction\nin which to explore or to to verify,\ncheck, and actually,\nyeah, transfer these insights.\nRight? So I think\nit's not gonna be like all ML PhDs will\nwill be unemployed.\nIt's it's more gonna be a sort of core\nevolution of humans\nwith this technology and potentially,\nlike, in an ideal future for me, like, it\nwill allow humans to focus on what they're\nreally, really great at.\nRight? So I think it's gonna be an\namplifier of sort of these these latent\ndimensions humans are great at.\nRight? I think something that's critical\nis that we as humans try to interact with\nthese systems as early as possible in\norder to actually, like, have influence\nand ownership over, like, this development\nprocess.\nRight? It's ultimately collective\nintelligence that will shape all of these\nsystems together. And do you think these\nsystems can become\nincredibly sophisticated such that they\nare, you know, somewhat detached from\nhumans? Well, I mean, with the AI\nscientist v 2, we sort of released\nthat 1 paper that we submitted to an\niClear workshop was able\nto sort of pass the acceptance threshold\nbefore meta review.\nSo I do think at least for sort of\nworkshop level contributions,\nwe're we're getting there.\nWhile not every submission in AI scientist\npaper does is or\nis reaching that threshold, we're we're at\nthe point where we can even talk\nabout sort of noisy review processes and\nthis actually being, yeah, something\nthat as long as you have a large budget,\nyou might get something out of it.\nI think going forward for the bigger\ninnovations and so on, for now, you\nstill need humans, but we're sort of at\nthe GPT 1 moment of of making\nthis sort of a reality and potentially in\n10 years, this is gonna look very, very\ndifferent once the sort of also the\ninfrastructure for it has been built up.\nRight? So there are places like periodic\nlabs, right, which sort of now are\nbuilding like real physical labs with\nrobotic systems to\nautomating automatically sort of execute\nexperiments.\nThis will take some time, but it is sort\nof\nimaginable for sure that as we sort of do\nRL on these types of\nsystems, and we actually also account for\nnegative results and for actual, like,\nhypothesis testing. So getting these\nsystems to be a real good\nhypothesis testers with verifiers in the\nloop that we might be able to unlock many\nmore capabilities. Yeah.\nI mean, I suppose I I don't want to sound\nlike a Luddite.\nSo it's entirely possible that this is\njust, you know, I I don't have the\nimagination to think about the future.\nSo it is possible that in the future that\nthese systems might\nunderstand very deeply and be creative.\nKnow, I think right now the problem is\nthey only understand things a few levels\ndown in the epistemic tree.\nSo they can do some surface level\nrecombination and they can discover new\nthings in the basin of things that have\nalready discovered.\nBut but we understand things very deep\ndown in in the epistemic tree, means our,\nyou know, our cone of creative potential\nis is much wider.\nIt's possible that that gap might be\nclosed.\nWhat would happen then? The way how I\nkinda think about the scientific process\nis like a tree search ultimately.\nRight? So I think a lot of sort of\nanalogies from evolution transfer to\nscientific research, right, in the sense\nthat we traverse a tree of different ideas\nor different experiments, and then in the\npaper, we report 1 path through that tree.\nAnd I think what I kind of alluded to\nbefore, we need much more,\nlike, full tree datasets for training\nthese LLM systems to actually learn\nhow to do this exploration and this\nforaging, basically.\nAt the same time, I I feel like evolution\nwill also take place on a cultural level,\nlike, us. Right? We will get better at\nsort of steering the ship, and I can\nimagine that in in the future world, sort\nof the way how we do research will be\ncompletely different. And I'm pretty sure\nthat right now already 99% of machine\nlearning research is done with sort of AI\nassistance.\nRight? Think about JetGPT brainstorming,\ncursor coding, Claude code,\netcetera. In the long run, we're gonna\nmove on that spectrum from sort of with AI\ncloser to by AI and then sort of more high\nlevel sort\nof orchestration and overseeing by humans.\nThere's also the notion of how\nintrinsically coupled to\nhumans is the value function.\nSo 1 school of thought is that AI will\ndevelop a mind of its own and it will, you\nknow, basically transcend humanity and it\nwill just have agency which is not\nparasitic on on ours.\nI personally don't subscribe to that view.\nBut the other view is that it is like\nlet's say the AI scientist, know, like\nversion 10. It's going to be continually\nepistemic, you know, epistemic\nforaging. It's going to be finding new\nthings that are useful.\nAnd they kind of have to be useful to us\nbecause if it finds things that are not\nuseful to us then we just won't use them\nand then nothing will happen.\nSo so do do you think there'll always be a\nkind of coupled value function to humans?\nJeff Koon had this work on Omni, right,\nand using LLMs as sort of amortized\nnotions of interestingness for humans.\nRight? And I think ultimately the way how\nwe train these systems is coupled in in\nhuman data. Right? And going forward, it\nwill also be coupled with\nhuman data that is collected using\nverifiers.\nRight? So I have a hard time believing\nthat in the long run,\nwhen you run this open endedness sort of\nparadigm with AI scientist agents, it's\ngonna completely divert to to something\nthat's either fully\nnoninterpretable or unrelated to problems\nwe as humans care about.\nRight? And then again, like, humans can\nsteer to a certain degree where,\nlike, the search happens.\nRight? So you can tell a system, okay, try\nto do cancer research.\nRight? And sort of work on problems that\nwe care about.\nAnd, ultimately, like, we are the ones who\ncontrol how much flops are being pushed\ninto this. Yeah. Because as a thought\nexperiment, I can imagine, let's say, in\nthe world of mathematics, what if an AI\nscientist could come up with entirely new\nproblem formulations and then solve them?\nAnd these are things that humans had never\nconceived of before.\nAnd maybe they would be less interested in\nthe answer because humans hadn't spent\ntime thinking about it.\nAnd if you think about it, we could just\nexplore the phylogeny of mathematics just\nto the nth degree. And at some point,\nmaybe we just wouldn't care anymore.\nMaybe we can just carve out that space\njust forever and ever.\nYeah. But maybe down the road, there is a\nstepping stone that enables a new\ninnovation and a different field that we\nactually care about.\nRight? So it's very hard to say a priori\nwhether or not something is\ninteresting or not.\nRight? Yes. And there's also the notion of\nI love this idea of diverse\nintelligences and diverse minds.\nAnd maybe we we could just create\nartifacts in a space which is\ncompletely alien to us. And we might even\nascribe moral value to\nthem and we might not want to turn off,\nyou know, the the power because we we want\nthese alien artifacts to stay alive.\nMaybe. Like, I read a lot of science\nfiction but I\nwould sort of shy away from from\nspeculating about all of this.\nBut I do think 1 thing I'm extremely\ncertain of is that the way how we\nconduct research and science is going to\nfundamentally change in the next 5 years,\n10 years, and 20 years.\nAnd I hope that we're going to be able to\nsort of tackle some of the biggest\nproblems which are still sort of seemingly\nunreachable right now with\nand by AI.\nSo Terence Town has posted that he's been\nusing GBT 5 to and and it's\nit's been speeding him up.\nIt's it's taking away a lot of the, the\ndrudgery.\nBut the cynical take is that and Scott\nArison Arison posted something similar as\nwell. The cynical take is that maybe\nlaziness is is stepping in and\nin some pernicious way, using AI models is\nactually\nstopping us from thinking outside the box.\nSo it's it's encouraging us to kind of\nsearch in the neighborhood of things that\nare known and that is very useful.\nIt's very useful to have an artifact that\nknows all of the experiments, of the\nthings that were ever done by people 20\nyears ago.\nBut now we don't have people really kind\nof applying their\ntheir brilliance, their talent in\ncompletely new areas.\nSo first off, it's great that these\nexperts are already using the technology\nin their day to day work.\nRight? And I think it's also important\nthat really really top level scientists\ntry to push what's capable with these\nsystems or squeeze out where there might\nbe sort of black spots or stuff where you\nthese systems can't do.\nSecond off, I think it comes down sort of\nto discipline and how we raise sort of the\nnext generation. Right? So discipline on\nthe personal level, like how much do you\njust sort of tap accept everything that's\nbeing proposed by these systems\nand responsibility in terms of educating\nthe next generation in the sense that\nwe need to sort of teach our kids that\nultimately what comes out of these systems\nmight not always be be true, that facts\ncan be sort of\nsubjective if you will, and that there\nneeds to be more research about\nwhat's being given to you. And I think\nthis will be like I said, this cultural\nevolution that we have to step through and\ntry to make the best out of.\nYeah. The autopilot thing is very\ninteresting because there is a tendency\nusing cursor just to, you know, at at some\npoint the models are getting so quickly\nthat you can't even read the tokens coming\nat you and then you just press accept and\nyou press accept. It's the same thing in\ncars that as soon as you have too strong\nof an autopilot, you just completely\nswitch off.\nAnd and then you see a divergence because\nthere's something about thinking that it\nmust be grounded on your path.\nThere's this path dependence.\nAnd when you start kind of becoming\nparasitized by this other train of\nthought, then you stop thinking about your\npath and then you're not in the driver's\nseat anymore. This is now like a bit of a\nharsh statement, but sometimes I wonder if\nthese systems like these coding assistants\nare almost like drugs.\nRight? In the sense that you become\naddicted, you you use up all your sort of\nbudget and then you need to load up again.\nAnd once you you fully reached sort of the\nthe budget limit, you feel like, okay,\nwhat am I gonna do now? And I think once\nthat happens to you, you should really\nsort of rethink the way how you work.\nRight? And to me, right now, there are\ncertain parts where, like,\nsort of auto accepting is acceptable, and\nthere are certain parts where it's\ndefinitely not, and you really need to go\ndeep into it.\nAnd I think right now, we're sort of in\nthis weird non equilibrium state where\nthings are moving constantly.\nRight? So the systems or the models are\nchanging.\nThe features are changing.\nThe sort of points where the systems are\ngood is chain are\nchanging all the time, and we humans need\nto constantly adapt to to that.\nRight? And I think it's a big cognitive\nchallenge, and I think\nwe just all need to be aware that there\nare certain problems and certain\nchallenges that we have to adapt to.\nI think the best way to do so is just\ninteract with this technology as much as\nyou can and\nmaybe find new research ideas for out of\nthat experience.\nAnd how is AI scientist v 2 different to v\n1?\nIn v 1, we we use sort of a template based\napproach.\nSo we had, like, a base experiment.\nAnd then for that base experiment, we\nasked sort of an LLM to generate ideas\nsort of with semantic scholar calls and\nsort of literature search, and then it\nimplemented sort of these ideas based on\nthe template.\nRight? It did it basically code diffs.\nAnd then it linearly executed like an\nexperiment plan and wrote a paper in the\nend. And so what could happen was that\nthere was an idea and that idea\ndidn't work out.\nRight? But then in the end, the paper like\nthe experiments were still executed\nlinearly when you wrote a paper.\nAnd this was already impressive in the\nsense that it looked very much like like\nscience. But if you think about human sort\nof science and, like, the\nscientific method, it's much more like\ntree search, like I said before.\nRight? You sort of adapt what you're gonna\nexecute next, and you sort of refine based\non evidence that you accumulated.\nRight? So this is sort of the the notion\nof falsificationism from from Karl Popper.\nRight? In the sense that we collect\nevidence for hypotheses and reject we\nreject others, and we do so in in a loop\nbasically until we we want\nto publish or we find something.\nAnd we try to take this notion and\ndirectly build it into the agentic\nscaffolding for the AI scientist v 2.\nSo now it's basically like an paralyzable\nagentic tree search where\nthere's no longer a template experiment\nneeded, but this is drafted up by the LLM\nitself. And thereby, the AI scientist v 2\ncan be applied to many more sort of\nsettings if you will. So at the core is\nsort of this new agentic tree search\nparadigm, and then we use sort of a couple\nof minor\ntechnical chain changes like using a VLM\nreviewer for sort of\nfiguring out if captions of a paper are\naligned with the figures and we we scale\nthis up to many more sort of computational\nnotes and then write a paper\nin the end again. So I'm trying to say\nthis in the most polite way possible, but\na critic might say I I don't wanna use the\nword slot, but a critic might say,\nwe\nare producing papers which appear like\npapers.\nSo they they have figures and they have\nresults and they have things written in a\ncertain way, but they're not grounded deep\ndown the epistemic phylogeny which means\nthat they they they have you know near\nnear the top of the tree we're\nseeing some novelty and composition\nhappening but it but it doesn't reflect a\ndeep understanding. What would you say to\nthat charge?\nIt's for sure that not every paper that\ncomes out of the AI scientist v 2 is a\nnature worthy publication.\nRight? That that's for sure the case.\nSo definitely there is some amount of,\nlet's say, slop or\ncontent that is not like a scientific big\ndiscovery being written up by the AI\nscientist v 2. But ultimately, like, we we\nshowed that it was possible to obtain\na workshop level paper.\nI do think this is sort of the first time\nbasically where we can see that at least\nnow we're able to fully autonomously spend\ncompute, spend API calls to\nobtain some amount of scientific insights.\nAnd for me, at least right now, it's a\ngood way to sort of prototype ideas or to\ninvestigate a certain field, get like\ninitial starting point, initial results,\nand then to to work on top of it.\nBut for sure, more work needs to be done\nto make this entire process more\nrobust, more efficient, and essentially\nproduce many more sort\nof true positives as you will.\nYeah. And it might be 1 of these things,\nyou know, like when we moved from GPT 3 to\nGPT 4, there was just a massive increase\nin fidelity.\nBecause the thing is with with Slop, to\nme, it simply means lack\nof deep grounded understanding.\nAnd there's no reason in principle why\nthese things couldn't have a deep grounded\nunderstanding. They just don't have it\nyet.\nYeah. So it's something that could improve\nover time, but it's likely to improve\nquite slowly. And then at some point, we\nmight just think, oh my god, we've got an\nAI scientist. Yeah. I mean, like, to me,\nthis this kind of comes back to what\nwe were discussing about before.\nSo first off, there is a verifier in the\nloop.\nRight? Or in the sense that experiments\nare actually executed on a computer.\nRight? So the numerical results can be be\nfed back or are fed back\ninto the system to come up with the next\nthing to explore.\nBut like we haven't made like a let's say,\ndiscovery, like a residual connection,\nso it's something that have diffused into\neverything in machine learning.\nAnd I think what we really need is to make\nthese systems be much better at\nsort of integrating knowledge over\nmultiple experiments and sort of become\nbetter at sort of formulating the next\nhypothesis based on previous insights.\nAnd, yeah, this might require some amount\nof post training on sort of these\ntraces basically, but I'm pretty positive\nthat we might also get there\nwith just diversity and scaling these\nsystems up in in an efficient\nbut scaled up way.\nI'm just thinking that the the first\nbreakthrough discovery, would it resemble\nthe AI scientist paper or would it\nresemble Shinka Evolve?\nSo for example, we we could do, a\nmassively scaled up, Shinka Evolve.\nAnd we could say, want to discover a new\narchitectural design.\nYeah. And would that happen?\nAnd then we would get the AI scientist\npaper to kind of write it up and do\nablations and stuff. May maybe that would\nbe the the pattern of it.\nTo a certain degree, I've been thinking a\nlot about how you can potentially even\ncombine these 2 paradigms.\nRight? The AI scientist and and and Chinka\nor AlphaEvolve\nstyle optimization algorithms.\nAnd I do think there is some amount of\nwork to be done on sort of this auto\nverification sort of aspect to it, on the\nsort of problem formulation aspect to\nit. The paper writing part is actually\nthe least important about the AI\nscientist.\nRight? It's a form factor that we humans\nare sort of used to, and it helps anchor\nour mental model of, a scientific\ndiscovery.\nBut, ultimately, I'm not sure if the paper\nis going to be the the\nknowledge transmission medium in, let's\nsay, 20 years.\nRight? Something else I've been thinking\nof a lot is whether or not\nwe can make papers much easier identically\naccessible, right, in the sense that right\nnow it's it's it's a LaTeX\ndocument, but you could imagine sort of\nequipping every paper with sort of\nseveral model context protocols so that\nevery figure is reproducible, data is\naccessible, and essentially make it much\neasier for the LLM agents to\nessentially either replicate work or to\nwork off of them\nafterwards. Right? Doing sort of absolute\nimprovements, ablations yourself through\nthat interface to a paper.\nBut to be entirely honest, I'm not sure if\nit's gonna happen because there have been\nmany great ideas for improving sort of,\nlet's say, the the format of scientific\nartifacts out there and people still seem\nto to like the paper\nformat which has existed for, let's say,\nhundreds of years.\nRight? So I think it's a question of\nincentives again and really showing that\nif something like that would exist, it\nwould enable much faster progress\nof AI agents for scientific discovery.\nYeah. Paper is a great human interface.\nIt's a similar thing with automated\ndriving.\nRight? That we could revolutionize the\nroad network to have sensors, and we could\ndramatically improve the the monitoring\nand observability and optimization.\nBut I'm fascinated by the idea.\nSo so you're saying not just\nreproducibility of the experiments, but\nalso the way that the figures are designed\nand and the code and and so on.\nBecause then we could create this huge\nplayground where agents can\nrepurpose, recombine, restudy work that\nhas been published by other\nscientists. And it also made me think,\ndoes like having an automated\nscientist, does that make peer review more\nor less important?\nI do think it actually makes it more\nimportant at least for now.\nRight? In the sense that we now have a\nmechanism or could have a mechanism that\ngenerates many many papers.\nRight? And it first increases like the\nworkload on on on human\nreviewers and we need some effective way\nfor filtering and then essentially\nonly taking the cream of the crop for\nhuman verification afterwards.\nRight? So I think for now, like the\nultimate verification is still like the\nhuman and the diffusion of the result\nthrough the community and we need\nbetter tools for doing this automatic\nfiltering and verification.\nLike we have the AI reviewer that comes\nwith sort of the AI scientist,\nbut you actually probably need some form\nof experiment execution for actually\nverifying everything.\nYeah. But there is, for example, work by\nOpenAI on on paper bench and trying to go\ninto that direction using sort of LLM soft\nverification and these types of things.\nSo I I'm hopeful that we're going to\nfigure this out in the next years.\nYeah. And I think 1 of the Rubicon moments\nis when the the new transformers\narchitecture or something massive is\ndiscovered by AI and we're all using it.\nMy worry, I suppose, is that probably\nfolks like Google who have enough\ncompute power, they're gonna be running AI\nscientists and they're going to own many\nof these discoveries, which is why it's so\nimportant to have work which can\nefficiently discover new things in\nscience.\nAnd it's important to have work that's\nopenly available.\nRight? I think like with the AI scientists\nin Schinka, we're really trying to to make\nsure that we can sort of apply the\ncollective intelligence of all of us to to\nshape how this might look in the future.\nAmazing. Well, Rob, this has been so\nfantastic to have you on the show.\nSokana is hiring amazing engineers by the\nway.\nSo if if this sounds like an it it is an\namazing opportunity, get in touch with Rob\nand and the guys. And I trust you're\nworking on some exciting new things that\nare coming up. Yes.\nAnd I I hope to be able to talk to you in\nthe future again about some of this.\nAbsolutely. Rob, thank you so much for\ncoming.\nThank you so much, Tim.",
  "transcript_chars": 79837,
  "ingested_at": "2026-05-12T00:40:15.975027+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 26882,
    "like_count": 707,
    "channel_id": "UCMLtBahI5DMrt0NPvDSoIRQ",
    "categories": [
      "Science & Technology"
    ],
    "tags": []
  }
}