AI4H EP087

[00:00:00] Hey, Merry Shipmas everybody. Open AI is gifting us 12 straight days of product releases and updates. That's right. Sam Altman promised 12 live streams. We also might get Sora video generation, the new reasoning model, and so much more Kevin. Oh, stuff my widow stocking Santa Altman because I've been extra naughty.

That's not what we're doing, Kevin. That's not what we're doing here. You're, you're right. What I meant to say, Gavin, was the holodeck is here? Yes, Google announced Genie 2 and World Labs came out of stealth, both of which are really interesting new ways to imagine prompt to world simulation. We've got all that plus new video, audio, and image generation tools that you can use right now.

It is a packed week and it is only getting crazier here. Yeah, it's packed like my naughty little self. No, no, it is not that it is AI for humans, everybody.

Gavin: Yes, everybody. It is here. 12 days of [00:01:00] shipments. Open AI has gone out and said that over the next 12 days, starting today, meaning that we don't know what the thing is yet, but starting today. Um, they are going to release 12 different live streams and each day are promising a a brand new thing.

Gavin: Sam Altman himself tweeted that , starting tomorrow at 10 a. m. Pacific, we're doing 12 days of open AI. Each weekday we will have a live stream with a launcher demo, some big ones and some stocking stuffers. We

Kevin: I've already got line of sight on the biggest update, Gavin. You're burying the lead. According to some chat GPT users, they saw code that replaces the voice mode button with a snowflake.

Gavin: Oh, can you imagine having to do the live stream for that?

Gavin: We have the most incredible new thing. It is prompt to snowflake button and you are going to love it.

Kevin: And don't worry, Elon's gonna have nothing to say about Sam Altman implementing a snowflake button.

Gavin: I didn't even think [00:02:00] about that. All right. So let's, let's dive into some of the things we expect that we're going to see. First of all, Sora. So Sora has been in the news quite a bit. We, you know, we didn't cover it last week, but there was a big kind of dust up around a bunch of artists, gave access to Sora for the first time through their account, and it wasn't exactly approved but then Kevin, the other big thing that's kind of sitting out there is O1, the full version, which is their reasoning model. Up until now, we've only been using O1 preview and it's very good, but O1, the full Monty, let's call it, is actually quite, , more robust What else do you think we might be seeing out of this?

Kevin: no clue what is actually going to be revealed beyond SORA and probably the full O1 model. We know that OpenAI is working on their own browser.

Kevin: That probably is far away, but would be interesting to see. , they've supposedly been testing a model that is ad supported to make, uh, you know, full fledged OpenAI free for everyone. Now, Sam Altman has said he hates that. And doesn't want that, but then again, trillion dollars to train some of these [00:03:00] models.

Kevin: Let's get some

Gavin: need the money, baby. You need the money.

Kevin: It might not be , the sexiest of updates, not to objectify , the 12 days of ship miss, but I want price cuts. I want the cost of real time voice agents, which they've been pushing and iterating.

Kevin: I want. those costs to go down because as we'll talk about a little bit later in the show, it's clear that sort of voice and agentic behavior are the two next frontiers that we're going to see in 2025. It's just really expensive to build with real time voice, which is something I've been trying to do,

Kevin: I want to see, , less guard railing of those voices, Gavin. I want to get more expressiveness. No, I do. I want, look, when we saw

Gavin: I'm just saying, good luck. I don't know if Santa Altman is going to allow more openness to the voices. We'll have to see.

Kevin: That's, you know, that's a good point. I would like to see that, though. To me, that's a tiny thing. It doesn't require a massive, megaton announcement. Just them loosening it up a little bit so the voices can be more playful and more performative. I think that would open up more exciting use cases.

Kevin: I don't know if you have access to it yet, Gavin, on the Mac [00:04:00] application, the chat GPT app, , where you can give it control over other applications on your device. Do you have that

Gavin: You know, I think I might, but I haven't actually played around with it yet.

Kevin: I do. They give us and they take it away.

Kevin: Oddly enough, sometimes I have access to it and then sometimes it goes away. It's just not there. I would fully expect. We're going to see that rolling out for. The entire user base, and I'd love to see further advancement in there. , insofar as right now, I have a little icon that I can click on my ChatGPT app, and I can tell it, Gavin, go run cursor, , go take control over this terminal window, take a look at this text file and read it.

Kevin: Like you can give it specific access to certain apps, but you cannot let it control a web browser. It refuses to recognize it as an app. , and even still, it is. Reading the apps that you give it access to, but not directly interacting with them or interfacing with them. I would love to see a step towards that

Gavin: so I have two big predictions here. I think one thing we're going to see is, um, a formal AI [00:05:00] agent announcement that comes out of open AI. There's been some rumors around this. And to your point, like really controlling computers, which is, you know, kind of anthropic dropped with computer mode. Like, I think that we're going to see something that's more robust there.

Gavin: I also think. This is more of a hype post kind of announcement, but I would bet that they're going to demo some sort of next gen frontier model and it is not going to be available, but I, I think that that's going to be in these 12 days. I would be kind of shocked if they didn't use 12 live streams and have one of them show off what the next big thing is.

Gavin: I mean, there's a big news here that chat GPT just hit 300 million active weekly users, and that's up 100 million. From just a couple of months ago. So like chat, GPT is crushing. And I think this is open AI saying we are ready to dominate this world. We are ready to kind of move forward into this thing.

Gavin: And they kind of have to put up or shut up pretty soon. Right. Because there's been a lot of people saying that like, well, if GPT 4. 0 is great, but we haven't seen a lot of updates since like, you know, March or April of this year, in fact, one thing that's still missing Kevin, which we probably will get during [00:06:00] this thing is GPT 4.

Gavin: 0 is image capabilities. We didn't think about like, you know, a new Dali. But there's going to be some new image capability too. So I would suspect a new image model, not just Sora, but some, maybe Sora has an image model attached to it, which we've heard rumors of. So maybe that will come. And you know, the other thing is just to kind of transition to our next stuff.

Gavin: I bet that they're going to show off more Sora as world simulator stuff,

Kevin: I thought you were going to mention, Sama Claus's sack is you can see the, the arms and hands and legs kicking against it with all the DeepMind staffers that he

Gavin: that's right.

Kevin: and then got caught in a chimney.

Gavin: that's right. We have to also remember the Sam, Sam and, uh, opening. I just recruited three new members of deep mind. So Google continues to lose members out of its, uh, AI org. And in fact, they, they recruited the person who was in charge of multimodal, training for deep mind, which is a big deal because multimodal training is this whole idea of like how we see the world or how the AI see the world.

Gavin: So to that point, [00:07:00] I think when they talk about Sora, one of these things is going to be about world simulations. And so in that instance, we had two really other giant stories that dropped this week in the idea of AI as world sim. But maybe Kev, for our listeners and viewers, maybe we want to talk a little bit about what AI as world sim even means.

Gavin: Like you, you want to give kind of the TLDR on what that idea is.

Kevin: Yeah, right now, if you imagine, um, Call of Duty, or Fortnite, or Roblox, or any of these big, massive games, they are powered by traditional game engines. These are pieces of software that take 3D objects, they slap a texture on them so a cube can look like a crate, if you will. Uh, they power the sound, they allow you to interact with it using a controller or whatever the input method is, and so these are complex pieces of software that render these interactive worlds that you can journey through.

Kevin: Gross distillation, but here we are, well, instead of a traditional engine, putting all that geometry and handling the lighting and do the user input controls. [00:08:00] What if you used AI to do that? What would that look like? What would that feel like? What new capabilities would that unlock? And while we are very much at the Pong stages of that generation of technology, these AI powered world engines are already producing some really incredible imagery.

Kevin: And what looks like actual interactive behavior based off inputs from the user. Now, Gavin, you lit up when I said we're at the Pong phase. Why are people hallucinating Mass Effect worlds and Halo and these massive deserts? Why aren't we just hallucinating Pong?

Gavin: Well, that's a good question. We could just have, what if we made pong out of giant hot dogs going back and forth in a 3D space? That seems like a perfect thing for us to

Kevin: Guy Fieri's head is the little puck.

Gavin: should make that. Can somebody out there in our, in our fan base make that? So I think you're right. What I was smiling about the pong thing is, is I think we might actually be past the pong stage with what's just come out.

Gavin: So there are a couple of big updates that just came out. Genie two is the update to genie one. If you're a longtime listener, you remember us talking about genie one. This [00:09:00] is Google deep minds, um, a video game simulator or creator tool. Back in the day.

Gavin: They announced genie one because it was not released, but was a side scrolling 2d video game generation. You could put in a prompt to say like, you know, plumber moves through sewer and it would come up with a kind of a version of a plumber through a looking through a sewer. Well, Genie 2 takes that 2D environment and now has made 3D prompt to video games.

Gavin: Now, again, this is not out. This is a blog post and Google DeepMind is very good at shipping blog posts. So we know that they're out there as a lot of these AI companies are. But Kev, when you look at this post, it's pretty. Crazy. Like you can see right away that like, basically there's a UI they figured out like, you know, you can walk through as a robot through a 3d environment, or you can walk through a bunch of things.

Gavin: One thing that really shocked me was the one when it's like a power boat, like there's a power boat on the water and it's, it's getting the physics of the water kind of right. It almost looks

Kevin: the waves and the

Gavin: Yeah. Yeah. to me, that's pretty crazy.

Kevin: a lot of these demos thus far have been mostly [00:10:00] like a matte painting that you can kind of wiggle around and they go, wow, look at this, what we're seeing here. Is coherence as the camera moves about so it's rendering the world, but not forgetting the world that it's supposed to be in and then interactivity with objects within that world.

Kevin: So believe it or not, a character walking up to what looks like a door in any other engine might just be they're walking up to a painting. of a door. It's like the old Looney Tunes, tunnel painted on the canyon, right? It's not a real thing? Well, here the roadrunner can run through the tunnel.

Kevin: The character is interacted with, they move with an animation cycle, there seems to be physics on it, they're casting shadows, they walk up to the door, the door opens, the character goes inside. I still say like pungish phase, cause we don't know if it completely falls apart after that, but we are very quickly going from proof of concepts.

Kevin: Could you even do this to now we're seeing interactivity, physics, , entire worlds hallucinated from a single still image.

Gavin: . That's right. And on top of that, there was [00:11:00] another big announcement, a company that came out of stealth this week called World Labs. And Dr. Fei Fei Li is behind this, who you may or may not have heard of. She is like an AI pioneer. She worked on some of the early ImageNet generations and things like that.

Gavin: Her company basically is doing this exact same thing. But in their mind, what they're doing is they could, you can take a single image and then walk around within that image. And more than that, you can add physics to that image. You can change the lighting of that image in that world. Now, if this one you can demo slightly, if you go to their website, which we'll link into our show notes, you can actually walk through these worlds.

Gavin: They kind of stop you a little ways into it. You can't like go fully because this is a very expensive thing to render. So if you're going to continually render it, It's pretty hard to do, but they have a lot of videos showing people doing that. And Kev, the thing that this one blew me away was like, when you watch it, you can do this in the demo, you can change the lighting of the scenario and the lighting actually follows along.

Gavin: There's a spotlight thing. And then in one of the videos. They throw a bunch of basketballs and you see the physics of the basketballs [00:12:00] move. This is like at the very beginning stages that, that holodeck idea, right? Like that, the idea that you could step into an environment and it's not programmed to do this, but it understands because it knows what the world is supposed to behave like, that as you move forward, this changes, this changes, this changes.

Gavin: And, and that feels fundamentally massive when you think about generating You know, video or generating worlds at large and, you know, not just video games. I think that's an important thing for everybody to know. This is like a bigger deal than just video games. This is like how AI's will probably train going forward.

Gavin: And that's like the secret sauce to all these companies making these, they're looking for more training data.

Kevin: Funny, you know, that's specifically something that the Genie 2 paper checks as well, and their, tweet thread about it says like, oh, we finally unlocked unlimited training data for these new models. And so that, yeah, clearly that's going to go there. I, I liked it. You mentioned that it's not for video games.

Kevin: , specifically because some of my favorite world labs examples that we [00:13:00] saw were full motion video of humans walking and then they replaced the world behind them. It's like an instant VFX swap. If you're listening to the audio version of this podcast, do yourself a favor.

Kevin: Go into the notes or check it out on YouTube because you should see these videos. They are really, really impressive. this is the sort of real time world simulator. And, , I know we're in the middle of ship myths now. Maybe Sora came out this morning from underneath us, but Tencent has been shipping and now there is an open source video model that I think, Some of the cherry picked examples are just as impressive as anything that OpenAI has

Gavin: Your Tik Toks are going to good work. Everybody. Your Tik Toks are being used for everything.

Gavin: Tencent, the company that owns TikTok and a bunch of other, , smaller startups has released a brand new open source.

Gavin: That's the biggest thing. They open source a state of the art video model. It is pretty good. I feel like it's, it's maybe lagging a little bit, but the fact that it's open source is incredible. And you can go try [00:14:00] this right now. , there's a company called Fowl. No, no, no ad here, but that I was able to actually do this on myself.

Gavin: I, I made a generation and I actually asked it for Guy Fieri and, Gordon Ramsay giving a high five and then hugging each other. And you can see Kevin, what came out of it was definitely not Gordon Ramsay or Guy Fieri. But it was a pretty good look at, like, two random guys high fiving and then hugging each other.

Gavin: It did cost me 50 cents for that one generation and it did take, uh, 20 minutes to get it. So like, we are a ways away this, if you, in order, you can run this locally, but you really have to have a beefy setup, like no, one's going to be running this on a 40, 90, but the fact that this is open source and that it is being released into the world is a really interesting sign for where we go for video generation.

Gavin: Again, though, this is a Chinese company and we've talked about this on the show, a bunch, I'm going to talk about cling later on, which I joined and got to play with their motion brush. Chinese video companies are going to be more advanced probably until [00:15:00] we as America either say, okay, train on everything, or maybe a couple of generations down the road, because these Chinese companies have no trouble training on any data that they have.

Gavin: In fact, I joked about tick tock, but it's very likely. That many, many, many TikToks, if not all TikToks are part of this training data set. I would.

Kevin: Oh yeah, I think the only solution is to put a tariff on video generations, Gavin. Imagine if that 50 cent generation actually cost you 50.

Gavin: Yeah, now that would be a big deal and probably make America a lot of money in that way. I bet. Right.

Kevin: I digress. Just a joke. Not politics. I swear. So many are saying that with these tools advancing as quickly as they are, Hollywood is going to be completely out of business. And there's going to be a bazillion different creators out there. There's going to be so much noise.

Kevin: This is going to be so disruptive. There's going to be way too much content. But our sweetheart, dear friend of the podcast, Zack Snyder came out and said, I don't know if you know that he's a huge fan, huge fan.

Gavin: wow. That's great. I'm so glad. Maybe we can get a quote from him at some point. Do you think you can get him on the phone so we can get like a quote [00:16:00] to put on our press release?

Kevin: I, well, I can get him quoted right now through a video at a different panel where he's talking about.

Gavin: We'll put that on our press release. Yeah.

Kevin: Here it

Kevin: is in the same way that when, you know, we all, cause look, every single person has a pretty good camera

Kevin: on their

Kevin: phone, pretty good camera on their phone. Every single person, a pretty good movie camera, like yes, a legit movie camera.

Kevin: Um, and yet, you know, we, we don't have you know, millions of awesome movies, just. Being uploaded out of people's pockets. I, I, I feel like the AI represents, uh, another, another tool that will help us make movies awesome, I hope.

Kevin: So, I mean, that's a point that I think we've had and many others have, but it's great to see professional directors, capital A artists reaching the same conclusion, which is like, having a professional grade camera in your pocket certainly led to everybody sharing their [00:17:00] stories, for better or worse, all across social media, but it didn't fundamentally change.

Kevin: AAA blockbuster movies. it made it cheaper for some to do it, but you still had to be highly motivated and wield it as the tool that it is to make the piece of art, to make the thing. Yes.

Gavin: Are you ready for a counter attack here, Kevin?

Kevin: Fire away.

Gavin: contacted by Skynet and they're ready for a counter attack. Here's the counter attack in my mind, not an attack, but like I think you're absolutely right. And Zach's absolutely right. Like there weren't a million, you know, new triple A blockbuster movies that came out with the phone camera, but guess what did come out?

Gavin: Tick tock. Guess what did come out? Reels people making content that was specifically like camera related content, because it didn't make a lot of sense for a normal person to go out there and shoot like a two hour movie with a camera, I do think these AI tools will make a new sort of generation of people making stuff. Right now, people think of it as AI slop, and that's really happening in a big way like this. When I say AI slop, I mean the idea that [00:18:00] like they're be, we're being flooded with AI content, whether it's AI influencers on TikTok or it's like some of the very funny, weird, uh, AI generations we see all over the place.

Kevin: Yeah, I think it's almost two different things. We know the floor is going to raise and individual creators with very little experience are going to be able to make a certain level of quality of something. So it democratizes creation, something you and I have discussed numerous times.

Kevin: But does it make actual movie creators out of everybody that starts to create content, and I think there's that concern of, well, I'm a professional. Now someone can do what I do. And it's like, I don't, I don't know that I buy that either.

Kevin: I think this goes to sort of Zach's argument is that just cause you have the. Yes, Gavin.

Gavin: Counterpoint 2 is going to be coming up later in the show, Kevin, because in a I see what you did there. We have a 10 minute Batman movie that was clearly lifted a lot of IP, but I would say is a really interesting point to that to counterpoint to this thing.

Kevin: Look, we'll table the conversation [00:19:00] until then. That's totally fine. I'll just say that a team of capable creatives today more often than not, makes something better than a single person, , who is experimenting with a tool. I don't think that's gonna change. As the tools get better, the ability for one person to make Something amazing does rise, but the ability for a group of people or a group of professionals to make something way better, the bar is just going to keep raising along with the floor.

Kevin: I imagine, I guess is my point. Yeah,

Gavin: There are another bar is raising Kevin and that Amazon has jumped into the state of the art AI model release race. This is like, Amazon has come out with a new series of Nova models. So Nova AI, that's our new, uh, we have Gemini, we have, uh, chat GPT. We have anthropic squad.

Gavin: Now we have Nova and Lama, all these different families keep getting bigger. And Kevin, this is like Amazon's kind of big jump out into the world of, of trying to kind of make their own thing. I will say there was a great video that came out where their new AI chips are [00:20:00] called Tranium, the Tranium chips.

Gavin: And by the way, Apple has just said, they're going to be using the Amazon chips to work on some of their stuff. So that's like a big deal, but I want you to play this, this release video and listen to some of the gibberish that is said within it,

Kevin: power and high bandwidth memory are combined in a server with advanced virtualization, enhanced security, and high performance storage and networking. Now imagine connecting these servers with an ultra fast dedicated chip interconnected. And then deploying them in hyperscale clusters with a petabit scale network fabric.

Kevin: What you get is a silicon server and data center architecture that delivers state of the art performance for frontier models.

Kevin: Did you imagine it, Gavin? Did you imagine hyperscaling them into pseudospace so that you could get 40 teraflurps of, petajuice?

Gavin: bed of big fabric that I wrapped around myself and I wore like a robe to Christmas morning. Yeah, so again, those videos are not for normal people and they're not for people like us. But there is a thing [00:21:00] where if you release something, like making it kind of at least understandable by the world at large is a good thing.

Gavin: Amazon coming into this just shows you the actual weight of how big this race is. Each of these massive companies need something. And Amazon has spent a lot of money investing in anthropic, but the fact that they are also releasing their own model, much like Microsoft has released their own models.

Gavin: Might mean that they want a little bit of like freedom from just these startups and they want to do the thing internally as well.

Kevin: Yeah, and they've already taken up the, , Microbrew IPA naming convention, because when you say it's Amazon Nova, you mean there's Nova Micro, Nova Light, Nova Pro, Nova Premier.

Gavin: does Nova light have in it?

Kevin: Yeah, not as many as Nova Zero, uh, you're gonna want.

Gavin: Can it get me wasted off of three of them? That's my question.

Kevin: it's technically four models that came out of this thing, obviously the Micro being super light, super cheap, super fast, all the way to Premier, which says, Coming soon, but it's the most capable multimodal model and it has complex [00:22:00] reasoning,

Kevin: and they also advertise that it's the best for distilling models, Gavin, which is where you take a more capable, powerful model and you poke against it to just train and distill a particular type of knowledge that you want in your model.

Gavin: weren't watching the video, you saw Kevin just, Kevin just did a hand motion that should probably be used for something else. So if you're out there watching the video,

Kevin: It's good for

Gavin: to get that out.

Gavin: Do you know what, Kevin, that might be just the hand motion for come and join our show and subscribe to our podcast.

Gavin: Because this is the important time that we have to say, if you're watching this on YouTube, please subscribe. If you subscribe, it's the only way we grow. If you're listening to this podcast, please share it with other people or also give us a five star review on whatever podcast platform you're on.

Gavin: We grow because y'all listen and share it with other people.

Kevin: I do want to mention that like we, we do get more podcast downloads each and every week, which is mind blowing to me. So thank you. If you're hearing this and you haven't shared yet, know that fellow listeners are, and you need to do your part as well.

Gavin: to

Kevin: How dare you?[00:23:00]

Kevin: No, I

Gavin: you? If you're out there,

Kevin: It really does mean the world to us and it really does help us grow. So if you can take two seconds to share it, post it to your favorite Reddit, send it to a friend, , that's the way we keep this train chugging along.

Gavin: Let's keep this train chugging along. We're going to jump into some announcements from 11 labs. One of our favorite, , voice AI companies.

Gavin: They've done a really good job of shipping themselves lately. The biggest thing that came out of just a couple of days ago is conversational agents. . We've talked a lot about AI agents on the show, and there's a bunch of different definitions of them.

Gavin: But in this instance, it's basically allowing you to create a specific voice to interact with certain people in specific ways.

Hi, what would you like to talk about today? Introducing conversational AI with 11Labs. Build, test and deploy, all in one platform, for the most natural way to communicate with technology. Hey, can I check the status of my order? Yes, of course. Let me check for you now. Your order is due to arrive at around 2pm.

Gavin: Obviously the business applications of this are massive because you could create an [00:24:00] agent that allows you to answer the phone or an agent that allows you to do sales calls.

Kevin: The biggest thing is that even if you have no idea how to create anything, you can go sign up for 11 labs, hashtag, not an ad. You can spin up a conversational agent and with basic English, with basic language, you can give. A voice, a personality, you can give it a rule set. And what's really fascinating about this Gavin is that it's, it's 11 labs, performative, , voice capabilities bolted onto whatever model you want to use.

Kevin: So when you go and you create an agent and you say, here's the system prompt, here's how it greets the person, here's the voice they should use. There's actually a little dropdown for which large language model you want to use. And you can plug in. Claude, Gemini, GPT, or a custom large language model. Maybe you want to go with a Nova, Nova draft, Nova dark, maybe Nova bold.

Kevin: Yeah. Maybe you want to [00:25:00] plug in a new Amazon model. You can do that. And knowledge bases are really important. So whether you're building a character for a video game, or like you said, a customer support agent. That agent needs to know everything about your business and about how your product works or, what your company standards are, your mission statement, you can plug all that in.

Kevin: And when I say plug it in, I don't mean hire an engineer to hike their glasses up and write the code. You can click and drag a PDF and now this agent will know it. And that. Is so powerful. It works over a telephone if you want to activate that. So people can actually receive and make calls with these agents.

Kevin: And then you get full analytics for everything as well. So it will automatically analyze your call and you can plug in. I was going to say KPIs. You can plug in the metrics that you want it to analyze the call with and let you know, like, was this customer happy? Did it lead to a conversion? Did the, did the quest giver give the key, whatever the thing is that you're building, you can build AI agents.

Kevin: Super easy with them. I wish This was an ad. So desperately, Gavin. I [00:26:00] really,

Gavin: Give us, give us money. 11

Kevin: At this point, I'll take credits. Please, dribble your fun credits into my, my Santa stocking, please. Samma's gonna leave room.

Kevin: Gavin, I'm all frothy and excited for these conversational agents, and you are 10 kinds of distracted. What? is going on on your end.

Gavin: the other thing that 11 labs did is they dropped a notebook LM competitor. So, you know, we've talked about notebook LLM. That is one of the best things Google has shipped in the AI space. So you can upload a PDF or anything like that, and it will have two AI co hosts talking about your thing. So, uh, 11 labs has done a similar thing and I uploaded last week's YouTube episode from our show and it says it's picking out the voices and it's like rendering it in the background.

Gavin: So right now I'm waiting for it. It may not come out for a little bit here. But the idea is that clearly there is a use case of this idea of AI explaining stuff that you can upload or at least distilling stuff and voice again, makes it much more accessible for people. In fact, so much so that [00:27:00] Notebook LM themselves has been integrated into Spotify wrapped this year, Kevin, and I can play a little bit of that while this other one's rendering, because what was interesting to me.

Gavin: Was that it comes up on, uh, if you look on the, uh, you may not be able to see this, but it comes up as your wrapped AI podcast on Spotify. So this is Spotify wrap, which just dropped this week. I go into this and what it's done is basically created a five minute podcast. of my rap. So here I'll play this and this is coming straight through the Spotify app.

Gavin: All right, music lovers, buckle up. Welcome to your own personal 2024 Spotify wrapped deep dive visiting from Google's notebook LM here to well unwrap this amazing year of music you've had.

Gavin: We're talking 38, 807 minutes on Spotify.

Gavin: 38, 807 minutes. That's wild. You're in the top 7 percent of listeners globally.

Gavin: That

Gavin: is serious dedication.

Kevin: Get a job.

Gavin: It goes on and you'll kind of get a sense of it. If you do it yourself, I would say it's an okay version of what Google LM [00:28:00] does. It doesn't feel that specific. It's not that far off from just taking basic inputs from your wrapped, like it's not giving me like I, the most song I

Kevin: of Christmas future hath just visited you through the magic rectangular portal in your hand and you're already naysaying it? You just got Two human sounding things having a conversation about your specific listening habits, about how many minutes you consume your favorite artists. You have a gift delivered from the Akashic records plucked from the ether.

Kevin: And you were like,

Gavin: Eh. It's fine.

Kevin: I'm going to sign up for Apple music.

Gavin: It's, you know, listen, I appreciate the fact that first of all, Google was able to allow Spotify to use this and they were able to kind of make it happen so fast and it might be the first kind of big introduction to this product for a lot of people through Spotify, Spotify's AI DJ, we've talked about that on the show too before, and it's interesting, right?

Gavin: Like it kind of talks you through whatever the music is. I thought [00:29:00] that was a really cool thing. All right. So now let's go back. Jen FM is with hope and Chris. So it generated a, a thing with hope and Chris. This is a version of our podcast from last week. And

Kevin: I know Hope. And I'm not kidding. Like, I know, I, you do so much work on Eleven Labs, I'm like, Oh yeah, I've used Hope for so many projects. I already know what Hope is going to sound like.

Gavin: So here we go. Okay. I'm going to play you. Then this is generated in about five minutes in real time.

Gavin: Where's the digital age? Well, not quite today. We're discussing how AI agents may actually be our collaborative partners rather than our replacements.

Gavin: Hmm. That's an interesting perspective. Can you elaborate on what exactly AI agents are and why they're generating so much buzz? Sure thing. AI agents are essentially,

Gavin: I'm just going to skip ahead a little bit. Cause you can slide through it. I'll skip ahead. Let me just skip to a different part. Text to image generators that have been making waves.

Gavin: That's part of it, but it goes much further companies like runway and stability AI are pushing the boundaries of what's possible. So

Gavin: I have

Kevin: [00:30:00] remind, remind folks, What did you put into the magic machine to get this podcast

Gavin: so I literally put in the link to last week's episode of AI for humans in the YouTube. So, so clearly what it's doing is taking our conversation. Not basically talking about who we are, what we are like, the thing about notebook LM is if you put a notebook, this in a notebook LM, it'll often say like, well, we're talking about last week's episode of AI for humans.

Gavin: Are we talking about this podcast? They are for humans where they talked about this. And in this instance, it just kind of jumped right into the stuff that we were talking about. I will also say hearing that for the first time, and I've heard other people do this, and there's people who have said this is like, The voices are not as expressive, and this feels like it might be a little bit more like plug and play, whereas Notebook LM might have had a little bit more tweaking to the sauce that went into it.,

Gavin: Alright, now we're moving on to something we don't normally do on this show, but there's a fair amount of it happening. What's Elon doing, Kevin?

Kevin: Is that the name of the segment?

What's Elon doing this week? [00:31:00] What's he up to, man? What's he up to, man? What's Elon doing? What's Elon doing? What's Elon doing?

Gavin: What's Elon doing this week, man?

Kevin: I like it. I just see you like, uh, uh, the neighbor from,

Gavin: Yeah, I'm Home Improvement, I'm

Kevin: home improvement? Yeah. You're over the fence. What's Elon doing this week? What's Elon got going on over there? , we don't do this often because we don't like to, and we'll keep it quick, but Elon is suing OpenAI for the fourth time now, I believe.

Kevin: A, for going for profit once again, but the other side of the story is that he's claiming some anti competitive, , violating federal law. , nonsense that's going on between OpenAI and Microsoft and other competitors and investors. He claims that someone who previously invested in XAI did not further their investment in XAI because they were explicitly told not to by like a Microsoft in the room saying, Hey, do not [00:32:00] play nice with our competitors.

Kevin: We'll see if this pans out, but meanwhile, while he is trying to Clobber open AI in the courtrooms over here. He's striking deals with Nvidia over there. That's the other hand. If you're just getting the audio version,

Gavin: so he's also struck a deal with NVIDIA, with Jensen Huang's company that is continuing to crush the sell the picks of the AI, uh, digital gold rush. He's going to, for a billion dollars worth of chips in January, delivered in January, so Elon is really going for this.

Gavin: And I do think the one important thing to say here about the Elon watch, that it's kind of important, even though we don't love to focus on this stuff is that Elon now. Sits next to the new president of the United States at his resort in Florida. And I think that is going to be a bigger story in the AI space than we expect.

Gavin: And the fact that Elon is suing open AI is not a great thing from open AI's perspective, because, you know, there's the person that's going to be leading this country, [00:33:00] America. Has a habit of like kind of picking favorites and he's very happy to listen to the people around him, especially those that he dubs like worthy to be heard from.

Gavin: So this is going to be a story to keep track of like, where does the scale get pushed and where does it not get pushed in the AI space going forward?

Kevin: , but listen if you're hearing this and you're filled with panic fear and dread because you're realizing that a handful of Billionaires are going to control our A. I.

Kevin: sentient destiny. Worry not, Gavin, because an exciting new 15 billion parameter research model was trained by the plebs! Sorta.

Gavin: Yeah, that's right. This is a really interesting company and this is a little nerdy for those people out there who are more kind of normie and not as deep in the AI space, but there's a company called news research. I think as you sell it, news, N O U S research, which has been doing a lot of really interesting research work on their own.

Gavin: And have announced that they were able to train a model in a decentralized fashion, which means that many different [00:34:00] computers around the world were used to come together and train this model. And it's 15 billion parameters, which is not, you know, state of the art, like giant model, but still, this is a step in the right direction to see a world where open source models could be trained outside of individual labs.

Gavin: Some AI safety people might be like, this is frightening as all hell because the idea that you could train AI models outside of a controlled space really freaks them out. Whereas at the same point, this is to your point, like, um, if the government really does start to control and crack down on this stuff, this does open the door to having it be a little bit freer in different ways.

Kevin: I love that you wanted to talk about this because this tickles my heart, even though it might make the broader eyes roll. , it's called DEMO, or DEMO, Decoupled Momentum Optimization. It allows for a heterogeneous mixture of hardware, meaning all the

Gavin: Wait, are you Amazon social? That sounds a lot like the pet, pet, pet fabric, uh, wrapping around

Kevin: No, it just, it just means the hardware, the environment is mixed, which is important. So you don't have a standardized machine in [00:35:00] different rooms in different locations. I mean, they did use similar GPUs in their official training run to keep things consistent, but this is the plot to what Terminator are you up to?

Kevin: Terminator 5? Are we at five? I know they had three, wasn't there a fourth that no one

Gavin: There, there are, they've started, you stopped using the numbers. So we're probably at least at five at this

Kevin: This is the Skynet scenario, imagine everybody, because the skies will be nuked, everybody will be in trouble, it'll be Elon, you know, punching, I guess, Cyber Putin, and there's only like three major people with their AI machines, and the people, Us, Gavin, everybody listening and watching. We have to band together, but we don't have supercompute clusters.

Kevin: So we go to our storage sheds and we go, , to our derelict malls and we blow the dust off of our PS2s and our Dreamcasts, which have just enough juice left, Gavin, we plug them in and we, we get this network of all these machines. The Keurig machine is plugged in as well. And it's talking to the Blendtec blender and all of the power combined will train a model that'll be just a little bit smarter and more capable.

Kevin: [00:36:00] And that. It's how we defeat the machines,

Gavin: Wow. I had no idea that my Blendtec blender, which has been in my closet for about four years and, uh, is no longer useful is going to be worthwhile. So that's great. I'm

Kevin: makes juice and it has the juice and don't shade the ninja air fryer. That thing is going to come into play in the near future. Actually, I think it's really cool to do decentralized training like this. I think it's massively important. And I do think it's a big deal for like an open source slash research community moving forward.

Kevin: So I'm glad we discussed it.

Gavin: All right, everybody. It's time for us to discuss some of the stuff we saw you or other people out there doing with AI this week. It is time for AI. I see what you did there.

Gavin: Kevin. I'm so excited about this. This is one of the coolest things I've seen be done with AI to date. And it was done [00:37:00] by three people. Uh, over a couple of weeks with 200 in Kling AI credits, , what this is, is a 10 minute Batman movie, and it is a, essentially a fan fiction, right?

Gavin: But it is 10 minutes. It is a compelling and incredibly watchable movie. I encourage everybody to go find this. This was uploaded by a guy named Kevin the kid k a v a n dash the dash kid and He is somebody that's done a fair amount of work in the at least has uploaded a lot of stuff to the AI video Reddit, but this he said this was done.

Gavin: It was pretty intense. He said it's been done with 1600 clean credits Workflow is pretty much using every available AI tool, starting with image gen, then animating video, cloning up resing took about three weeks to create with three people. Um, so this is an example of what is possible right now in the world of the AI video space.

Gavin: And, you know, we've talked about prompt to Hollywood forever on this show, that the idea at some point you'd be able to prompt something and make a movie out of it now. This is also a good example of prompt to Hollywood [00:38:00] is kind of a myth because these guys did more than just prompt. They prompted, then they edited, then they added voices, then they prompt for the voices.

Gavin: They did all sorts of stuff, but I do encourage everybody to go watch this. It is, again, 10 minutes long. It's like, it's, you know, it's not like the best thing you've ever seen in the world, but it is very compelling and you get a sense of what it is.

Kevin: I don't disagree with anything that you just said, but like, even us in our position being acutely aware of the space a year ago, I mean, we were saying like, and then if you just look back two years ago, impossible and unheard of, let's just all goalposts keep slowly getting nudged as we're looking at different directions.

Gavin: This, five years ago, if somebody uploaded it to YouTube, would have been one of the best Batman, fan fiction recreations ever made.

Kevin: Netflix would have signed them in a minute. It would have been a big deal. UTA would have been in a bidding war. All the agencies about this ragtag team of three that made insane fanfiction. And yet, here we are. And [00:39:00] along those lines, Gavin, you know, we talked about the Coca Cola commercial, which sort of set the internet ablaze, for better or worse.

Kevin: A lot of people shading that one. Well, , just today, as we're recording this on Wednesday, December 4th, A Vodafone commercial. They are the client. The agency was new commercial arts. They released what looks to be a minute long commercial that at a passing glance, I think if this were on my television, I go, Oh yeah, that's a commercial for Vodafone.

Kevin: And it's a spot about, moving to the, , the rhythm of our lives. And it's mostly AI

Gavin: is the rhythm of my life, Kevin? You should know this at this point. What is the rhythm of my life end up at?

Kevin: I imagine like, uh, like a, a toaster pulsating with, , hot dogs. Coming out of it like hot dogs and buns and

Gavin: I don't want to hear any more. I don't want to hear any more. Let's keep going talking about Vodafone.

Kevin: But no listen if you watch the commercial it is these mini vignettes which we know AI can do very well [00:40:00] You know a baby floating in space and then in water being birthed and then you know people taking photos of it It's basically telling the journey this almost of growing up and then it shifts to the perspective of others around it.

Kevin: But it's, rather beautiful slice of life vignettes that look like they were shot on professional cameras. It's very well edited and To what you just said, it's not prompt to commercial, right? If you look at the credits list for this, there's an EP and a producer, a creative director, multiple AI artists, VFX producer, VFX supervisor, the list goes on and on, but if this were done, let's say three and a half years ago, Gavin.

Kevin: The list of credits would be 50 times longer. Oh, easily, easily and so this is the interesting thing. And I think this is what I was trying to get to earlier with like, Hey, the floor is going to raise the bar is also going to raise as well.

Kevin: So we say it every week, but I don't mind being a broken record. Get in and start experimenting with these tools now, because when you look at the credits for this commercial, which is beautiful. , [00:41:00] the spot with the most names on it happens to be AI artists. That didn't exist three years ago, and here we are.

Gavin: And shout out to, uh, Uncanny Harry on X, who's a guy we've been following for a long time. Speaking of new AI art and individuals, Kevin, there is a really cool.

Gavin: Tool called retro diffusion that is now available. It's been available for a while, but a new version of this and what this is, is a, it's a plugin that basically allows you to make pixel art, uh, with AI. And this has always been a tricky thing for AI is that as cool as pixel art is, like, it always has little kind of funky things with this and this guy at real Astro pulse, who I've gotten to know a tiny bit through discords and stuff like that.

Gavin: Has continually worked on this product and it is a paid product, if you're looking to make any sort of AI pixel art, I encourage you to go try this and play around with it.

Gavin: Looking at like what it's been able to pull off is pretty amazing.

Kevin: Gavin, I've got a mini game for you. You ready to play? Yeah. Okay.

Gavin: Oh, I'm ready to play. What's the mini game

Kevin: Listen carefully, because I don't know if this is going to work at all through our [00:42:00] recording system, but, , see if you can identify this.

Kevin: That's all I'm giving ya.

Gavin: Sounds like a guy is with a chainsaw and he's running after a, so a human guy with a chainsaw is running after a little rabbit who rushes away really

Kevin: Wow, we, we would have accepted the answer. We looked at the judges and they got human with chainsaw. And that's actually all we needed, Gavin. What about this? What about this, Gavin?

Gavin: That sounds like a car driving away really fast.

Kevin: Oh, it does! And you don't need any rabbits or anything else. It does sound like it. You're two for two. Gavin, what's this?

Gavin: that's me hungover playing the xylophone for my fourth grade class. Is that right?

Kevin: You know, two out of three ain't bad. It was very close. [00:43:00] So what do all these sounds have in common, Gavin? Well, they're not real. This is the future of video to audio generation. Now, , Tencent, I believe, also does this with their new video model, but this is Adobe's new multi foley AI tool.

Kevin: And basically, you feed it an input video, and it generates the sound effects. For you and

Gavin: Now I see that, that would, that sound you're playing at the end was a woman playing a harp. Is it actually picking up the notes that she's playing? And that would be crazy.

Kevin: It is relatively sinking back the plucks to the fingers, right? So it is, it is letting the video action motivate the sound that comes out of it. I don't think it's playing the actual notes on the string, but if I,

Gavin: it's

Kevin: I would, I would, I would really have to approve it. Well, we don't know if the source video, if she was even capable, she was using Seven of her AI fingers to play. And

Gavin: Come on, man, you're disinherit playing without even knowing what she was[00:44:00]

Kevin: yeah, I don't, I don't even know if she's real. I don't know if that's AI or actual video, but if you look down at the other examples, Gavin, it's clear that they shot a lot of stuff with like, , GoPros or, body cam perspective, but they have people doing stuff like washing things and prepping food in the kitchen.

Kevin: They've got barnyard animals. They've got all this weird footage of it, but it is fairly accurately reproducing the sounds from those videos. And that is just. The next unlock, as we talk about this prompt to Hollywood, it's going to be using all these video generators, giving it a thing, and having it automatically generate a soundscape to match your video.

Kevin: That is going to happen.

Gavin: the world simulator thing, right? Cause that's the other thing, which is really fascinating. Imagine when you're walking through those video game worlds or the stuff we were talking about at the top of the show, one of the things that sound adds so much to the believability of an environment, right? If you can, in real time, start actually foliating the world at large, like we're just getting closer and closer to being stuck inside our goggles forever.

Gavin: Kevin, how are we going to stop this? How are

Kevin: I don't want to stop it, Gavin. I've been outside. It's overrated. I don't [00:45:00] care. There's no reason to go out there. You know what's not out there, Gavin? The AI for Humans newsletter. You can only get that

Gavin: It almost wasn't out there today either.

Kevin: That's right. It wasn't

Gavin: We were a day late. We were a day late this week with our AI for

Kevin: Oh, because we had to delay it to pack it full of nutrient deliciousness. So if you have not signed up for the AI for humans newsletter, you are missing out, , the line is going in the proper direction there as well, and we sincerely appreciate everybody sharing that with their friends and family.

Kevin: Go to AI for humans dot show, and you can sign up for our newsletter. Again, that's AI for humans dot show. Get the newsletter. It is free. It is weekly. And what I love is that it's, it's a breezy, uh, little read. You can kind of scroll it in one flick .

Gavin: Also, you can sign up for Kevin and I to come give you a talk at our website if you ever want to do

Kevin: A stern talking

Gavin: Let's

Kevin: Gavin brings a

Gavin: about what we did with AI this week.

Gavin: I have a couple of really fun things. First, I want to just drop something very fast that everybody can go do, and if you want something to impress people at Christmas, this might be a fun thing to [00:46:00] do.

Gavin: Google dropped a thing called Gen Chess, which is actually a very cool, dumb thing that you can use to basically create a chess set and then play chess. With specific images, you can basically prompt to chess set. And like, what's cool about this to me is I made one that was like hot dogs versus sushi at

Gavin: , the pawns were all little hot dogs and the queen had like a hot dog hat and all this sort of stuff. And it's just a very simple, fun way to use AI to kind of make something. But then you can literally go play chess or share it with other people. So it's a fun thing to try. Have you tried this at all yet?

Kevin: I'm trying it literally right now because when you go to the lab site you can choose from make a classic or a creative chess set and then you can give it any inspiration. I said give me a creative chess set inspired by Nintendo games and it delivered me a fantastic rendition with Mario. Like a full Mario with the M on the cap, wearing his wearing the crown as the king.

Kevin: Princess Peach is the queen.

Gavin: wow. Really?

Kevin: Yeah,

Gavin: that they were [00:47:00] able

Kevin: I'll send

Gavin: all the IPs in. I'm kind of shocked by that.

Kevin: And I've been banned from the internet. quick. Yeah, take a look at this. And then I clicked generate opponent, which is something that will auto suggest based off of your first suggestion, and it says Nintendo versus And sure enough, I've got Sonic, Robotnik, and I think that is the character from Knights.

Kevin: I gotta turn in my, uh, my, my, my nerd card, if

Gavin: for this one. They maybe didn't get enough publicity so that they haven't turned off the IP set on it yet.

Kevin: I'm fully in, I can see the game, and

Gavin: Wow. Cool. Oh, Robotnik looks amazing.

Kevin: Robotnik looks

Gavin: Robotnik as the Bishop. He looks incredible there.

Kevin: Yeah. So very cool tool.

Gavin: So Kev, the other thing I did this week, I saw some really cool videos about Cling's motion brush. So Cling, if you're not aware, is another one of these Chinese AI video models. And their motion brush was getting a lot of pickup because you can literally take up to five subjects and make them do different things.

Gavin: . And it really does give you a lot of different things to be able to do and action to be able to do when you're actually making your AI [00:48:00] videos. A couple of things about Kling, I got a Black Friday special, so I, I paid for this myself at 60 bucks for the year.

Gavin: You get like about, you know, it's not a lot of generations for that, but you get like, I think about 15 to 20 and then you can get a free couple of free ones every day that come in. , it takes a very long time to render these and I'm not sure if it's just the motion brush, but like each one of these takes about eight to 10 minutes to get back, which makes it really hard to work with,

Gavin: and so first I had this picture I made of, of, uh, Shrek holding a little toy. Santa. Hey, I'd love to see if I can get the Santa. What I want to Shrek to throw the Santa into the air. So I took Shrek's arms and I made them go up and then the Santa I made go up as well. And you can kind of mask each section of a person and decide what you want to do and then you can make the background stay similar.

Gavin: So Kevin, if

Kevin: I

Kevin: jumped the gun.

Gavin: So you can look at the first one, it kind of made the Santa like explode at upwards and

Kevin: It gave him a quaddo from Total Recall. The Santa's shirt lifts up, [00:49:00] and there's a full menacing face with teeth on the

Gavin: Oh, you're right. I never even saw that. That's interesting. So like, there's a second like face underneath and then I tried to do it again and it didn't really fully get it.

Gavin: It got Santa going up. But not to Shrek throwing it up So I tried that a couple times and then you know One of the other things I tried was of course because you see all these videos the door brothers or anybody I tried to have there was a picture of Macron the president of France and Trump where they were kind of like Trump's kind of touching his collar I tried to have the two of them hug and then kiss and And one thing that was interesting, Kevin, about this is in the prompt I put for this one, two men embrace and then kiss.

Gavin: And it wouldn't let me do that. But when I put two people embrace and kiss, it did let me do it, but you can see this is pretty good, right? This is one of the benefits of the Chinese models. Like you can do things like this. And again, A. I. Uh, artists like the door brothers use this to make their videos where they have famous people doing stuff.

Gavin: Um, and again, this was just me kind of pushing the motion brushes towards each other. So you can see it's like a, [00:50:00] it's a pretty good way of kind of controlling that motion. And then finally, Kev, I took our thumbnail from last week and I just kind of went crazy with it. So our thumbnail was like, you know, uh, a robot terminate robot holding a turkey.

Gavin: You could doing your. T temple hands and me just kind of in the background. And as you watch us all kind of go around, you can kind of see it figure out 3d in its own way. Is it struggling with a couple of things? Yes. My face definitely changes over the course of it. The, like the kind of graphic floats away, but overall a really super interesting way to use AI video.

Gavin: I wish that. It was faster. And I think it's, it's really powerful. I do think that's more powerful than runways tool of this same sword. Um, and I'm excited to spend more time with it.

Kevin: Well now, having spent time with this, Uh, even a limited amount of time, plus using Runway and Luma, all of these things. If Sora is in fact delivered during Shipmas, and I say this knowing full well that it might be out right now, as people are listening to this discussion, because of the way [00:51:00] time works, but if Sora is just a text to video model, will that be exciting enough now, Gavin?

Gavin: I don't know. And my other thing is like, how much generations will they give you? How long will it take? And there was a lot of rumors around a Sora turbo model that came out when that artist thing happened. And if that's really good and fast, maybe it could be interesting. And, and I don't know. I mean, it's at least.

Gavin: Another tool to play with. It really will also depend on how many generations we get as a paid chat GPT user, like, will we be limited to like two a day? And if that's the case, like it gets really annoying to try that stuff.

Kevin: Or, or will they be announcing a new paid tier, by the way? 20 bucks gets you basic voice chat and text chat, but if you want advanced image generation and Sora, well, that's GPT Plus Premium Max.

Gavin: 50 a month or whatever for it, by the way, which I think if they dropped something like that, probably could get a lot of people to sign up for it. So it's not the dumbest thing for a company that's spending a lot of money.

Kevin: Agreed. I'm still using 12 different services piecemeal. I, if they can give me just good enough across [00:52:00] a multi modality. Yeah, you win. You still have my money.

Gavin: Thank you so much for listening and we wish you all a happy shipmas out there.

Kevin: Merry Shipmas! Ha ha

Gavin: Mary shipmas,

Kevin: ha ha!

Gavin: everyone.