{
  "video_id": "36He5BAKKLI",
  "title": "Google's NEW Multimodal Model - Gemini Embedding 2",
  "url": "https://www.youtube.com/watch?v=36He5BAKKLI",
  "transcript": "Your current embedding model might handle text great, but we live in a multimodal world. More and more often, our AI systems expect it to handle all types of data. Text, images, audio, video. And honestly, most pipelines are still treating multimodal data as second-class citizens if they handle them at all. But Google just shipped one of the first truly multimodal embedding models able to handle text, audio, images, videos, and PDFs in a single vector space. So let's take a look into what that actually means and what you can build with it. Embedding models take data, like text, and convert it into a string of numbers called a vector. These numbers in the vectors aren't arbitrary. They encode meaning, so data with similar meaning gets similar vectors. That's what makes Vector Search work. Instead of matching keywords, you're measuring distance between vectors, which tells you exactly how similar or different they are. However, two different embedding models will encode the relationships and meaning of data differently, which means the vector spaces generated by two different embedding models are incompatible. A vector from model A means nothing to model B. This is what makes multimodal embedding so hard If you want to search across text images audio and video you can use separate models You need a single model that puts all of them into the same space with consistent meaning So a photo of a lion, the word lion, a recording of a roar, and a video clip all need to land close together in that same space. That's the problem that Google's new model is designed to solve. So Google has just released their Gemini Embedding 2 model, which supports text, image, audio, video, and PDFs out of the box. So it's truly a fully multimodal model. It maps all of these modalities into a single unified vector space so you can search with any modality and get any modality in return. Because it's built on the Gemini architecture, it already supports over 100 languages and ranks on the top 5 of the MTib multilingual leaderboard. The output dimensions are flexible between 128 and 3072 and Matrios Corepresentation Learning is available as well. It's available with the Gemini API in Vertex, and we've already added support for it in Weviate, so let's build a demo. For our example project in this video, we're going to build a PDF RAG pipeline. We'll embed each PDF page as an image using the Gemini Embeddings2 model and add it to our Weviate database. Then we query our database with text and get PDF page images as responses which we will then give to a Gemini Flash generative model to generate an answer to our query using the document context The vectorizer module in Weviate will automatically handle the vectorization orchestration and will also use the generative module in Weviate to handle the generation orchestration. The dataset we'll be using for this project is a set of PDFs that have these needles hidden within the document. So when we ask what's the secret flower, the pipeline should be able to pull this from the context and respond correctly. In order to do this, it needs to use its multimodal understanding of text and images to answer correctly. So let's first install our requirements. We have Weviate, PDF to image to change the PDFs to images, hugging face for the data set, and a couple of others. Then we'll initialize our Weviate client with all of our API keys and configure our documents collection with three properties, doc page as an image blob, document ID as text, and page number as an integer. In our vector config, we'll set the Gemini embeddings to preview as our model and set the property that we want Weave8 to embed using that model. Then we can download the dataset using the Huggy Face Hub and change the PDF pages to images and then append all of this to our list of data. We will batch add the data into Weave8 and then we can get started on our generative pipeline First we set the query text what is the secret flower and define our prompt Then we can use a generative module to search with our query text get three results and give that to our Gemini Flash model to AMP. We'll then print out our query, generated answer, and all the retrieved document content. So let's run it. And you can see here that the answer correctly identified that the secret flower was on the first page, where the text, the secret flower is A, and a picture of a rose is. We can also see the proper return context in our document. Pretty cool. So now we've made a fully multimodal rag pipeline using state of the art model. Text query in, PDF page image retrieved, answer generated, all running with WeVate's native Google model integration that's handling the vectorization and generation orchestration automatically. This isn't just about PDFs either. Because all the modalities share the same vector space with this embedding model, the same pattern works whether you're querying with text or an image or video or audio, or whether you have text or images or audio or videos in your embedding space. That means the possibilities here are honestly pretty endless. An e-commerce app where you can use image search to find similar products, a YouTube video search where you can ask questions about a creator's channel, or a podcast search, or honestly all of them combined. As always, all the resources, including the notebook, are linked in the description. I can't wait to see what you built.",
  "transcript_chars": 5437,
  "transcript_filled_at": "2026-06-06T14:39:52.754457+00:00",
  "transcript_filled_by": "tk-bulk-groq-retry-20260606"
}