{
  "video_id": "53kYaFx7nHs",
  "channel_slug": "mitcsail",
  "channel_handle": "mitcsail",
  "title": "MIT Professor on Generative AI & Computer Vision: Part 1",
  "duration_seconds": 617.0,
  "url": "https://www.youtube.com/watch?v=53kYaFx7nHs",
  "upload_date": "",
  "transcript": "(swooshing tune)\n(electric tune)\n- Hi, I'm Phillip Isola\nand I'm a faculty member at MIT in EECS.\nAnd I study computer vision\nand machine learning in robotics,\nand I'm happy to answer some questions\nabout those topics today.\nSo let's go to the first question.\nOkay. What is embodied intelligence?\nEmbodied intelligence is\nthe study of AI systems\nthat have to interact in a physical world,\nin the real world.\nSo think robotics.\nBut the study of embodied\nintelligence is really,\nencompasses a lot of different fields.\nSo we need to be able to\nperceive the 3D world around us.\nThat's a vision problem.\nWe need to be able to take\nactions and make decisions.\nSo these are problems in\ndecision making and robotics.\nSo embodied intelligence is\na very kind of holistic take\non the AI problem.\nLet's go to the next question.\nCan you tell me about your contributions\nto the field of computer vision?\nSpecifically, could you detail\nsome of your key work in\napplying machine learning\nto image processing?\nMy work has been on a\nfew different topics,\nbut one of the central themes has been\non trying to understand\nhow should we represent\nthe world around us,\nspecifically how should we\nrepresent the visual world\naround us.\nThere's a lot of different\nanswers to that question.\nAnd you know, one answer might be\nyou could have 3D models of objects,\nand you could have the\ngeometry represented\nand you could have like\nthe semantics represented.\nFor example, I would know\nthere's a cat in the photo\nor a dog in the photo.\nMy work has been on a\nslightly different direction,\nwhich is, let's not kind of pre-specify\nor hand engineer the way of\nrepresenting the visual world,\nbut let's let it emerge\nfrom a learning algorithm.\nAnd I've specifically worked\non using deep neural networks\nto learn visual representations from data,\nwhere the representation that's learned\nis what's called a neural embedding.\nIt's what the neural net has come up with\nas a way of organizing\nthe data that it sees.\nAnd in particular, I've worked a lot,\nmy group has worked a lot on\nan approach which is called\nself supervised learning.\nSo in this approach, you\ndon't teach the neural network\nfrom kind of human specified labels,\nbut instead you train the network\nto try to predict properties\nof the raw data itself.\nSo for example, you could take a image\nand you could mask out half\nof the pixels in the image\nand try to predict the missing pixels\nfrom the pixels that are observed.\nSo this is something that you can train\non just raw data\nwithout having any labels\nor humans specifying\nwhat you know, the\nrepresentation should be.\nBut the neural net learns how\nto make these predictions.\nAnd by doing so, it comes\nup with a representation\nof the world, which knows\nsomething about objects\nand continuity.\nLike it knows that if I have, you know,\nthe left half of the table here,\nthere's probably a right half of the table\non the other side.\nThis idea that just by\npredicting missing data,\nmissing visual information,\nyou will get a representation\nwhich knows how the world works.\nAnd that's a representation\nthat can be used\nto make meaningful\npredictions about the future.\nFor example, what will\nhappen if I open this door?\nWhat will I see?\nWill I, what will happen\nif I drive my car forward?\nWill I go through a red light?\nIf I can make these predictions,\nI can make good decisions.\nAnd that's the type of\nvisual representation\nthat I've worked on.\nAny advice on how to tell the difference\nbetween AI generated\nimages and natural images?\nRight, this is a hard one\nbecause I can't tell anymore.\nWe're just at a point in history\nwhere if I see a photo online,\nI don't know if it's real or if it's fake.\nAnd that wasn't true a few years ago.\nThis is very much a new thing.\nThe main way that I would try to assess\nif an image or media\nonline is real or fake\nis not gonna be by looking at artifacts\nor the visual content.\nIt'll be, you know,\nlooking at the context.\nIf there is some kind of\ncrazy implausible photo\nof some celebrity on Mars,\nI'm gonna think it's probably fake.\nIf I don't trust this source,\nif it's an anonymous source,\nI might not trust it.\nThat's the state that we're in right now.\nLuckily, there are some\ntechnical tools that can help.\nSo there are AI systems\nthat have been trained\nto detect if a photo\nor some kind of content\nis made by another AI or if\nit's real authentic content.\nAnd those sort of work.\nWe're in an arms race where\nthe generative AI methods\nget better and better at making\nrealistic synthetic content.\nAnd the detectors get better and better\nat detecting any little artifact\nthat gives away that the image\nor the content is not real\nand it's gonna go back and forth.\nThat arms race is not gonna\nhave necessarily a clear winner,\nbut we'll see what happens\nover the next few years.\nUltimately, I would say I trust media\nwhen I trust its source.\nI can't verify just with my eyes anymore\nwhether or not an image is real or fake.\nOkay. Let's go to the next question.\nCould you explain to me what deep nets are\nand the kinds of\nrepresentations that they learn?\nSo deep nets are, you know, super cool.\nThey're the big thing in\nAI that everyone uses now.\nDeep nets are basically\njust a class of functions\nthat can be used to make decisions\nor learn representations of the world.\nAnd there are class of\nfunctions which are modeled\noff of how the brain works to some degree.\nSo in our brain we have a neural network,\na real neural network, and a deep net\nis kind of an approximation to that.\nIt turns out that deep nets\nare really a simple class of\nfunction, in a certain sense.\nThey're just a stack of layers\nand every layer takes the input data\nat the bottom of the stack\nand it kind of transforms it.\nAnd layer by layer the data gets processed\nand transformed until at\nthe top of the network\nyou get a better kind of\nrepresentation of the data\nthat can be used to make\npredictions or decisions.\nAnd these layers are all\ndoing very simple operations.\nThey're just doing what's\ncalled a linear operation.\nAnd then a simple thing\ncalled a point non-linearity.\nIt's just this linear,\npoint wise non-linear,\nlinear, point wise non-linear.\nSo these are very simple\nmathematical objects,\nbut they can do amazing things\nwhen they learn from data.\nAnd what kinds of representations\ndo deep nets learn?\nWell, they learn to\nsomehow organize the data\nin a way that makes it easy\nto solve the task of interest,\nwhich might be to make a\nprediction about the data,\nto classify the data\nor to make a decision on the basis of\nwhat the network is seeing.\nOkay. So let's go to the next question.\nHow did your research in\nimage to image translation\nshape your current work?\nOkay, this is taking\nme back a little ways.\nBut back during my postdoc\nabout seven or eight years ago,\nI worked on this problem called\nimage to image translation,\nwhich is to try to take\nan image that is presented\nin one kind of style or\nmodality, like a color image,\nand predict what it would look\nlike viewed a different way.\nFor example, translate\na photo into a sketch\nor translate a sketch into a photo,\nor maybe take a photo and try to say,\nwhat would it look like\nif some artist, Cison,\nhad painted that scene?\nSo you can kind of visualize\nthe world in different ways\nand we called this a translation problem,\ntranslating from one visual\nformat to another one.\nSo this project was really fun,\nit was popular, and it\ndefinitely affected my thinking\nand my ongoing thinking about AI.\nTo take you back, in the\nera before we did that work,\nthere were a lot of special purpose\nways of solving problems\nin computer vision.\nSo if you wanted to solve a\nproblem, like object detection,\ntry to label all the objects in a photo,\nyou would use a fairly different method\nthan if you wanted to solve a problem,\nlike estimating the depth of\nall the objects in the photo,\nlike predicting what's called a depth map.\nSo that didn't really satisfy me.\nI really wanted something\nthat would be a more uniform,\nunified solution to all those problems\nbecause what we kind of realized is that\nall of these different\ncomputer vision problems\ncould be thought of as taking\nsome set of pixels as input\nand predicting some other\nset of pixels as output.\nSo the output pixels could\nrepresent where the objects are,\nor they could visualize\nthe depth map of the scene.\nSo these were all pixel to pixel problems\nand that led to us calling\nthis project Pix to Pix.\nThat was the name we gave\nit, Pixels to Pixels.\nEssentially, we had a more\ngeneral purpose framework\nfor solving all of those problems,\nrather than having specialized\nsolutions for each one.\nAnd that made me realize essentially\nhow powerful a more general purpose\nand unified solution can be.\nAnd really the big trend\nover the last, you know,\n5 or 10 years of AI, has been toward these\ngeneral purpose solutions.\nThe biggest phase of that now is\nwith these large language models\nwhere one big model is trained to do\na lot of different tasks.\nSo when you interact with a chat bot\nor a language model, you can\nask it all kinds of questions.\nYou can even ask it to make an image\nor to solve a math problem.\nAnd it's one model that with\na kind of generic framework\nis able to solve all those problems.\nSo I think that's an interesting trend\nand that's what I'm always\nkind of on the outlook\nfor right now.\nSimple, unifying principles and methods\nthat can be applied to a\nwide variety of problems.\nAny recommendations for the\nbest AI image generation models?\nHmm, okay.\nOne I really like right now\nis called Stable Diffusion.\nI like this model\nbecause it's open source,\nso we get to use it and intervene on it\nand really study it as scientist.\nAnd it's one of the image\ngeneration models that works well.\nSo my lab mostly uses Stable Diffusion\nin our research as the current model.\nAnother one that I like a\nlot is called Control Net.\nControl Net is kind of the\ncurrent most popular iteration\nof image to image translation.\nWe did, you know, old work like X Depicts\nand now people have gone way beyond that\nand Control Net is the latest.\nSo what it does is it allows you\nto condition your\ngenerative model on an image\nand produce another image.\nFor example, you can take a sketch\nand you can turn that sketch into a photo.\nSo Control Net, Stable Diffusion,\nthese are some of the fun ones\nthat I'm playing with now.",
  "transcript_chars": 10293,
  "ingested_at": "2026-05-15T10:56:12.715871+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 6835,
    "like_count": 265,
    "channel_id": "UCBpxspUNl1Th33XbugiHJzw",
    "categories": [
      "Science & Technology"
    ],
    "tags": [
      "MIT",
      "Massachusetts Institute of Technology",
      "CSAIL"
    ]
  }
}