{
  "video_id": "Fians2EMCeY",
  "channel_slug": "assemblyai",
  "channel_handle": "assemblyai",
  "title": "Building Earmark: Real-time voice AI, privacy by design, and founder lessons",
  "duration_seconds": 1514.0,
  "url": "https://www.youtube.com/watch?v=Fians2EMCeY",
  "upload_date": "",
  "transcript": "Hi everyone, it's Mark from Assembly.\nI'm joined here today by Mark and Sandon\nfrom Earach. We're going to be talking\nthrough how they built earmark and why\nthey chose Assembly and how they can\nrecommend other founders and PMs to be\nthinking about voice AI in their apps.\nHi Sandon, hi Mark. Thanks so much for\ncoming on today. Maybe just walk us\nthrough first what year does.\nYeah, so uh earmark is a productivity\nsuite where the work completes itself.\nUm, so what earmark does is it listens\nto your meetings in real time and turns\nwhat's said into finished work, things\nlike docs, tickets, updates, uh, and\nnext steps. And unlike generic AI\nmeeting tools, um, it actually produces\nlike real artifacts, right? Real work.\nUh, so product teams can move forward\nwithout all the manual follow-up.\nNice. Um, was that how earmark started?\nLike could you walk me kind of through\nthe origin story and what the first\nversion looked like? Yeah. So the the\norigin story in the first version was\nradically different than what we're\nworking on now. Uh we actually started\nat earark as a vision pro product you\nknow. So essentially what it was was a\nARVR uh rehearsal experience right where\num we wanted to help product engineering\nleaders be more preparatory like as\nthey're influencing. I mean a lot a lot\nof folks are certain leaders a lot of\npeople in product uh nobody reports to\nthem right. So the the concept was how\ndo we make people effectual like in\ntheir ability to influence. So we\nstarted with um this idea of real-time\nspeech coaching, right? So, you know,\nyou could go into an immersive\nenvironment, you know, in your vision\npro, you know, you'd have your Google\nslide deck, you know, uh uh presented\nand [snorts] then you would have uh\nreal-time feedback in terms of, you\nknow, whether or not you're breathing,\nyou know, should you enunciate a little\nbit better, should you speak up more, uh\nmaybe you're speaking too quickly,\nright? And we provide real-time feedback\naround those concepts. Uh and then we\ndecided to, you know, conduct a bunch of\nuser research, right, around that\nparticular solution. Uh and it turns out\nthat uh you know we learned and the key\ninsight was that nobody really really\nprepares for anything. [laughter]\nSo so we made a rehearsal product for\npeople who don't prepare for\npresentations. Uh so that was the key\nlearning. Uh and then what we did was we\ntook um essentially we we just pivoted\nand and we took the idea of a real time\nuh feedback experience right and we put\nit on on the web. Uh and the concept\nthere was could we essentially uh enable\nproduct folks to be uh more uh uh uh\ninformed in the moment. Right? So that\nwas like sort of the first thread we\nstarted pulling as a service. Uh and\nthat evolved to uh um automated creation\nof artifacts and deliverables of work\nwhile people were having conversations.\nUh and we're about five iterations in\nsince our vision pro solution. Um and\nyeah, just in market and having\nconversations with customers and\nprospects every day.\nThat's really cool. I like that. I guess\nit kind of developed and matured over\ntime and now you're building something\npeople want. So that's always a good\nplace to be in.\nUm do you dog food earmark internally at\nall? Like does your own team use it\nday-to-day?\nYeah, we we use it every day. Um so one\none concept would be I mean like for\nSandon and I and you know and and Dylan\nour our go to market uh leader uh what\nwe'll do is we'll have uh just\nunstructured conversations you know that\nare that you know could be like\nbrainstorming right an idea would be you\nknow could you like you know so we would\ntalk we'll talk through st sales status\nright things about customer sentiment\nthings about the product uh and then\nwe'll actually like go through ideation\nin real time uh the thing for earmark\nthat's really powerful is it'll take\nthese unstructured conversations and\nturn them into structured artifacts\nfirst and foremost, right? So, you know,\nrequirements or maybe uh support\ndocumentation for our customers as an\nexample or maybe go to market messaging\nin terms of uh you know, type of\nlanguage we're using maybe based off of\ncustomer conversations we've had. Right?\nSo, uh that's been really useful. But um\nsomething that's actually more useful um\nin our most recent iteration of the\nproduct is the ability to essentially\npush to cursor um you know push to vzero\npush to codeex um and actually uh have\nconversations lead to prototypical flows\num and so so for us the the concept and\nthe the the true problem we're trying to\nsolve is this idea of can we essentially\num author you know the deliverables that\nyou normally would create maybe a day\nafter sever several days after uh in\nthat 30-minut increment or hour\nincrement of the meeting that you're\nactually in, right? And the real unlock\nthere is, you know, if we're talking\nthrough concepts, right? And then\nactually looking at protocol protocol\nflows, um that's a great way to kind of\nclose that cycle time up. Um you know,\nbecause R D teams are just overt taxed,\nyou know, Mart, I'm sure you understand\nlike how hard uh just the business of\nR D is, you know, and for us, the\nbiggest thing is, you know, can we just,\nyou know, uh help cycle times for\neverybody? And then also you know for\nproduct managers and a lot of folks that\naren't in engineering you know like a\nlot of the like R D is moving so much\nfaster with AI um can we essentially\nenable um you know product folks to to\nkeep up and maybe even stay ahead of\ntheir engineers that are running five\ntimes 10 times faster.\nYeah. Yeah, I think my, you know, my\nfavorite use case is, you know, like\nimagine you're you're in a meeting as\nMark was mentioning, you're you're\nbrainstorming on features and you're 30\nminutes in, you're 40 minutes in and\nthen suddenly you share your screen and\nthen you'd be like, is this what you\nwere talking about? And like just having\nthat there like in that context is so\ncool because like everyone's minds is\nlike already thinking about that. Um,\nand then you can continue to riff and\nit's just like like the quality is so\nmuch better and just reducing like that\ncycle time and in in in not having to do\nthose follow-ups is kind of such a huge\nbenefit.\nI I I honestly the thought of working by\njust talking is a dream come true for me\nand it has been for me personally now\nthat I use Super Whisper on Mac like\nmost of the things I write now are\nhardly with the keyboard. It's all with\na voice and having kind of like an\ninteractive platform to do that on like\nmeetings obviously but um having\nartifacts come out coming out of those\nconversations that's like a game\nchanger.\nYes.\nUm so you guys use assembly um under the\nhood. Could you tell me more about why\nyou chose assembly? What was that\ndecision looking like? Did you do an\neval? Did you do a vibe eval instead? Um\ndid you have another solution prior to\nassembly? Why did you change? maybe kind\nof walk me through the story of how\nAssembly got into the picture.\nYeah, there were two there were two big\nreasons uh for why we chose assembly.\nSo, we we were using another provider uh\nfor transcription before um but we were\nrunning into two big issues. I would say\nthe first issue was kind of like the\nplumbing. It's almost like we had to\nbuild a ton of abstractions just to get\nit to work. like you can think of things\nlike microphone management, the\nwebsocket life cycle, a lot of\nreconnection logic and just just tons of\nwork to make it reliable. Um, but I\nthink the biggest piece was that we were\nkind of getting slammed on the\nconcurrency limits and it was really\nhard for us to predict scale or even\neven like a launch when we did like a\nproduct hunt launch or like hey like\nwe're not sure like we seem to be like\naround the edge. Um, but if we want to\nget over this we have to sign like this\nreally expensive like enterprise\ncontract. Um, so it was just kind of\nlike a lot of unknowns. Um, and actually\nfunny story, just kind of like right\nbefore launch, um, we kind of discovered\nAssembly AI. We did a quick, uh, you\nknow, some quick tests with it. We\ndiscovered, hey, like this\ntranscription, it's not only really\nfast, but it's also really accurate and\nmore so than what we were using. Um, and\nwe actually swapped it out in 4 days\nright before launch. And we we launched\nwith that, and it's it's been it's been\ngreat ever since.\nThat must have been just a mad rush.\nlike 4 days before launch, you're trying\nto like get like all the plumbing moved\nover from one provider to another.\nUm could you\nI want to like double tap on like that\nconcurrency limit conversation.\nUm do you open one session for everyone\non the call or does like each person get\none stream?\nEvery person will get one stream. Um so\nif there's one person on the call uh\nwho's actually using earmark they'll get\none stream but if four people are using\nit simultaneously that will be that will\nbe four streams. So if you think about\nthat, like even one meeting at like one\nworkplace could be using four streams at\nthe same time and then obviously\nmultiply that out within that one\nworkspace and that's a lot. Um and then\nof course over different companies and\nand whatnot. So that's like you know a\nton of concurrency streams. And what's\nreally cool about uh assembly is having\nthis unlimited concurrency stream where\nthere's almost kind of like a backoff\npolicy like we get you to a certain\nthreshold.\nUm and then it just continues to add on\nbased off of that. Um so that's that's\nbeen that's been fantastic for for us\nscaling and uh yeah I'm not sure how we\nwould have survived without that.\nNice. And what are you guys focused on\nbuilding now? Like where's the product\ndata?\nYeah. So, so Mark, the one thing that\nwe're we're trying to focus on this idea\nof a true, you know, chief of staff,\nuh, for product teams that is predictive\nin terms of what individuals need, what\nteams need.\nUm, the ability to not only task or\ndelegate work to your chief of staff,\nbut then also um have proactive tasking.\nyou know, being aware of um maybe if you\nthink about sort of a a multiplayer\nsetting. Um if let's say um there's a\ndelivery team, you know, that's like\noffshore, right, that had a blocker last\nTuesday. Um you know, could you be\nproactively notified, right, of what\nthose blockers are? Uh you know, sort of\nday in day out, like what should I truly\npay attention to? Um you know, as as a\nproduct or engineering leader. So I\nthink for us you know just the idea of\nlike sort of this this co-presence that\nis really helping you know like every\naspect of your work. Um we know that uh\nyou know product leaders and engineering\nleaders are so over tax right in terms\nof um not only meeting schedules but\nthen the types of deliverables that\nresult from them. Uh the the one thing\nthat uh that that we've learned is you\nknow every 30 minutes you know is\ndifferent you know in terms of the\naudiences they're speaking to. uh every\naudience requires a different type of\nartifact or deliverable, right? And\noftentimes they require different levels\nof fidelity of those artifacts based on\nseniority or whatever the immediate need\nis, right? Um and that's just a huge,\nyou know, sort of contextual lift, you\nknow, for folks that are working 60\nhours a week, right? Like, um so I think\na big part of that is, you know, can we\nhelp those folks like in their roles to\ncreate capacity to uh be more strategic,\nright? Like you know for a lot of\ncustomers we we speak to um oftentimes\nthey haven't talked to customers in\nmonths because they're just sort of\nbeholden to the needs of internal teams\nand deliverables around those teams and\nkeeping their teams mobilized and fed.\nUm so the idea for us is you know this\nchief of staff that basically creates\ncapacity for folks to basically you know\ndo the things that got them into product\nin the first place or engineering in the\nfirst place right um so that's the the\nconcept of the tool. The other piece is\nthis idea of like you know a second\nbrain you know which is kind of similar\nto this idea of achiev staff but could\nwe have a second brain for product teams\nyou know which is a a queryable you know\npool of context right organized by\nproject um and can you essentially um uh\nyou know have essentially a system of\naction right relative you know relative\nto maybe a system of record in more of a\ntraditional form\nlike a brain\nhonesty kind of sounds like a\nsorry about that um no sorry\nhonestly kind sounds like a game changer\nfor founders where like they're coming\nin and out of these meetings with\ndifferent people, with customers, with\nlike employees of the company. Um like\nnot every meeting looks the same and not\nevery meeting needs the same artifact to\ncome out of it. So like they could\nreally benefit from earmark by like\nbeing productive all day when they're on\nthose calls talking to people and like\ngetting things done even though like\ntheir calendar is full.\nYeah. the that the the the unlock for\nour customers which has been which has\nbeen a really cool thing to see is this\nidea of um unlimited task agents you\nknow that are running in real time like\nin the background you know like as\nconversations and the workday progresses\num for a lot of folks that we speak to\nit's like they they can't imagine not\nhaving unlimited task agents operating\nin the background as conversations\nprogress you know so it's a so that's a\ncool like sort of unlock for for for a\nlot of people\nit kind of reminds me of my coding\nworkflow where like Nowadays, I have\nlike four different tabs open on my\nterminal just with open code chipping\naway at different projects. But since\nit's a real-time like product earmarked,\nlike those terminals are constantly\nbeing open and we can have like way more\nthan just four. Like it could in theory\nbe unlimited.\nCorrect. Yeah. Yeah. We haven't actually\nuh uh tested the unlimited part, but uh\nbut we're [laughter]\nyou know it it's it's you know we're\nfairly certain that uh that you can put\nquite a few task agents towards towards\nwork. really nice. Uh maybe you guys\ncould show me around the product itself\nand we can give like the audience an\nidea of what it looks like and how you\ncan use it.\nYeah, let me pull that up for [sighs]\nyou a quick second here. So, welcome to\nwelcome to Earark. Um so, I'm kind of in\nmy main page. Uh I have a meeting that\nI've already pre-recorded here. Um kind\nof one of our our retros here. Um, but\nbasically what we wanted to to do is\nkind of like design almost like a\nutility type tool that was just super\nstraightforward to use, just really\nquick to get in. Um, where one a user\ncan kind of start capturing their\nmeeting. Um, and during a meeting or\nwhen they're done with their meeting,\num, they can create essentially any\nartifacts that they want, which is kind\nof completed outputs. Um, so kind of\nlike what we were talking a little bit\nearlier, one of my favorite use cases um\nis to come up with um engineering specs.\nSo essentially based off the meeting,\nbased off the transcript um is\nessentially pulling in kind of what it\nthinks are um actual work items that\nengineers can work on. Um so for\nexample, on this meeting um that I have\nhere, uh the users were talking about a\nmissing 404 page. And what's really neat\nabout this is I have just kind of like\nquick actions to essentially kind of\nlike build a cursor. Um, which would\nopen it up right into the external app\nand I could literally just kind of start\ngoing um and get that running um get\nthat running from here. Or likewise,\nokay, maybe if I didn't want to jump\ninto cursor um real quick, maybe I need\nto save this for later. Why not just add\nthis right into linear? Cool. This looks\ngood. I'll create the issue and now\nthat's on my that's on my tracker. Um,\nso those are kind of examples of just\nkind of like getting to action really\nquickly. Um, for more communication\nexamples, we have kind of a bunch of\ndifferent templates such as, hey, if I\nwant to follow up with my team on Slack,\num, I have this nice template that has\nkind of emojis. It's like really short\nand condensed. Um, has all of the action\nitems in here. Or maybe I, you know,\nneed a more kind of like traditional BR\nPRD that I just want to get started and\nuh get onto a first uh draft here. Um,\nall right. Um, so yeah, here's a\ntraditional PRD that uh someone might\nsee. And uh on the left, um I think I\nthink most people might be used to\nessentially kind of like chatting uh to\nchange. we're kind of introducing this\ntopic called uh uh vibe uh vibe docking\num which is kind of on the left is\nessentially the format of which you see\non the right and what's really cool\nabout this is I can just kind of kind of\ngo in here and maybe like you know add\num add emojis kind of like under this\nexecutive summary section and what\nthat's going to do is it's just like\nit's going to like regenerate\num live for me just kind of like based\noff of what I might tweak in here or\nlike maybe I like I add another section\num like um I don't know customer quotes\num if there were any quotes um you know\nin in this particular meeting and uh\nlet's see what it comes off okay cool so\nyeah indirect feedback of quotes um so\nit's it's neat because you can really\njust kind of like adjust and fine-tune\non the spot um and then send this out um\nto your platform of choice when you are\ndone. But uh yeah, that's that's your\nmark.\nThat's real neat. Thanks so much for for\nsharing this with us. Um I want to kind\nof take the conversation in another\ndirection and maybe start to think about\nhow like the future way that we\ninterface with computers and with AI is\nwith voice. Um you mentioned sort of\nlike this chief of staff uh that lives\nuh on your computer. Maybe you guys can\nshare a bit more of like a vision of how\nyou think we would use this tech 5 to 10\nyears from now when you know ASR becomes\nlike really accurate, really perfect um\nsuper fast and of course we we start to\nbuild up a lot of this context um from\nwork that we've done and artifacts that\nwe've already generated.\nYeah. one one of our one of our goals is\nhow to turn essentially a knowledge work\nfrom being really uh reactive to\nproactive. So, you know, imagine kind of\nlike a true chief of staff like as you\ncome into work um like in the morning as\nyou're walking down the halls. Yeah.\nImagine someone is being like, hey, like\novernight this vendor might have\nrenegotiated the deal or maybe there was\nan engineering team that's in a\ndifferent time zone as you and they ran\ninto a blocker and like here's that\nblocker and just kind of surfacing those\nthings for you. Um so you don't have to\nfind out kind of later in the day or\nmaybe where we have about hey like you\nknow what's the status on this? Those\nthings just come to you so you you can\nthen decide okay like where do I want to\ntake action today? And it's almost like\ncoming into work like every day and\nknowing what are like the top three\nimportant things to work on and those\nare all in real time. And the extra\nkicker on that is since we know the\ncontext now imagine if earmark could\nactually take action on that. So imagine\nlike okay now I actually want to\ndelegate some of this work off or maybe\nI want to take on some of this work that\nuh you know that um has but uh that's\nthat's kind of our our fiveyear like a\ngrand vision for\nyeah and the other piece too is um you\nknow we talk a lot about the the\nrelevance uh or maybe slightly ill\nirrelevance of systems of record you\nknow today where a A lot of work you\nknow for knowledge workers is um\nessentially entry you know to make sure\nthat um you records are kept right that\nyou can have a credible report to your\nkey stakeholders you know because\neverybody is you know uh entering you\nknow whatever is required like within\ntheir system of record. Um we think that\nthe future work is going to evolve to\nwhere systems of record might be a\nlittle less important maybe in a\ntraditional state because if you capture\nall conversational context right you\nthere's nobody that has to play scribe\num to enter these things you know to\nhave visibility in terms of what's\nhappening in R D as an example um so\nthat's a really powerful unlock and then\nto to Sandon's um point around um\nproactive agents right or it being smart\nenough to sort of self task itself self.\nUm, you know, this this sort of\nevolution to systems of action, right,\nis something that we're really um\noptimistic about as well, where, you\nknow, it's it's not entry, you know,\nsort of passive, right? You still have\nto go and, you know, execute whatever\nthe work is within the system of record.\nYou know, that the idea that um things\nwill basically selfseed themselves in\nterms of tasking is a really powerful\nconcept um and something that uh that we\nreally look forward to.\nYeah, it's really nice. Sorry, if I\ncould just add one more thing. Um, I\nthink one of the reason why we're really\nbullish on voice is because that's where\nlike 90%\nof like the conversations happen at at\nwork. And so like if you imagine like\nyou have all of your documents, you have\nall of your Slacks. Like that's great.\nThat's a lot of data, but there's still\nso much that happens in conversations in\nmeetings or side conversations. Um, and\njust imagine that also captured in kind\nof with your second brain and what you\ncould do with it.\nIt's really nice. I like the idea of\nlike a daily brief coming into work and\nlike just being caught up on everything\nand conversations that happened while I\nwas asleep and leveraging that to be\nmore productive in my day. That's like\nthat's a really nice vision for the\nfuture. Um, so do you guys have any like\nadvice for founders who are building in\ngeneral but also with voice AI? like um\nI know I guess your product has uh\nmatured and it's grown. Um I guess maybe\nyou can share some insights from from\nthere.\nYeah, there two big pieces of advice I\nthink. One is like privacy is is design.\nUh voice data is very sensitive by\ndefault. Um so you need to be like\nreally intentional about what you store,\nhow long you store it, do you encrypt\nit, um do you avoid storing it? Um and\nactually for us for your mark um you\nknow because we we view we view privacy\nso so strongly um we do have an option\non all of our plans um which is called\ntemporary mode where we actually don't\nstore um the transcript or any of your\ndata at at all. Like there's no there's\nno retention plan. It literally just\nbypasses our database completely. Um so\nreally designing around that and\nthinking about that is really important.\nUm, but uh, one of the lessons that we\nlearned early on to for voice AI\nproducts is actually making the UX\nreally forgiving. So when a user is\nusing a voice AI product, they're\nactually taking action in something\nelse. Like they could be in a meeting,\nthey could be in a phone call, or they\ncould be in a conversation with\nsomething. The product that you're using\nis almost secondary to them. So if you\nare trying to capture a conversation and\nin order to start that capture it's like\nfour button clicks or different\nconfigurations, the user is just not\ngoing to use it. So it needs to be dead\nobvious like can it be one click or can\nit do it for you? And if there's a blip\nthat happens, can it resolve itself or\ncan it figure things out itself? um just\nremoving that kind of like that uh that\nthose decisions from people while\nthey're using something else I think is\na is a is a huge thing that uh that it's\neasy it's easy to overlook when when\nbuilding voice AI products.\nYeah. and more just general foundry\nexperience. Um, one one thing that uh\nStand and I uh sort of, you know, sort\nof evolved to is is um I don't know like\nthere there's so much uh out there in in\nthe form of best practices and like\nfounder content. Um and there's so much\nuh um\nuh that's the word I'm looking for.\nthere's there's so much dogma, right,\nassociated with just being a founder\nthat um I think I think that the pro tip\nis is like perspectives are nice. Um and\nthere are some frameworks that are\nhelpful, but there's nothing more uh\nhelpful and impactful than just lived\nexperience and just charging through and\njust being a founder and not um uh\nthinking the dogma, you know, is is what\nit really is. You kind of learn that\nlike everybody has an opinion, right?\nBut you have to sort of figure out what\nworks best for you in your organization\nin terms of driving it to success. Um,\nand I think for a lot of folks that are\nsort of like on the precipice of like\nbecoming founders or when they're in it\nreally early, um, they they kind of get\ncaught up in that group think, you know,\nin a way that I think is less than\nproductive. Um, so, uh, yeah. So, the\npro tip there is like advice is great.\nUm, but, you know, chart your own path.\nThat's it's really funny that you\nmentioned that because one of our like\nunofficial company values is beware the\ndogma like um we were going through a\nphase where we like really grew as a\ncompany and we immediately reached for\nlike that enterprise software growth\nplaybook that's common and part of the\nreason why you guys have um stuff like\nunlimited currency with sorry unlimited\nconcurrency with no enterprise agreement\nis because we realize that's not how our\ncustomers pay for and buy AI products.\nThey want flexibility. They want to know\nthat this is a company that they can\nscale and grow with. And kind of like\nletting go of that traditional playbook\nhas allowed us to serve our customers\nand keep them really happy. So, I think\nthat's really great advice. Thank you so\nmuch for sharing that with us, Mark.\nYeah, absolutely.\nWell, that's uh all I really had. Um,\nthanks for your time and uh, I guess I\nwill leave like your contact information\ndown below if if you want to try earmark\nyourself or get in touch with M and send\nand um, everything will be in the\ndescription. Uh, thanks guys for your\ntime. This was really such a great\nconversation.\nYeah, thank you Mark. Really appreciate\nthe opportunity.\nAgreement.",
  "transcript_chars": 25879,
  "ingested_at": "2026-05-15T16:33:29.350712+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 240,
    "like_count": 4,
    "channel_id": "UCtatfZMf-8EkIwASXM4ts0A",
    "categories": [
      "Science & Technology"
    ],
    "tags": []
  }
}