Tim Hwang: How good are you as a prompter on a scale from 1 to 10, with 1 being totally amateur and 10 being world class? Kate Soule is a Director of Technical Product Management for Granite. Kate, welcome back to the show. Prompting. How are you at it?

Kate Soule: Prompting is never something I want to be known for, but I do think I’m pretty good at it, so maybe like an 8.

Tim Hwang: Okay, cool. Nice. Chris. Hey, Distinguished Engineer, CTO, Customer Transformation. Chris, welcome to the show. Your prompting score as a Large Language Model expert?

Chris Hay: I could not possibly answer that question.

Tim Hwang: Got it. And last but not least is Aaron Baughman, IBM Fellow and Master Inventor. Aaron, your prompting skill, please.

Aaron Baughman: Does prompt engineering really exist? I’m not quite sure. I always ask LLMs to produce a prompt for me.

Tim Hwang: Okay. Everybody’s fighting the question. All that and more on today’s Mixture of Experts, a Think podcast. I am Tim Hwang, and welcome to Mixture of Experts. Each week, MoE brings together the sharpest team of researchers, engineers, and product leaders you’ll find anywhere in the world of podcasting to discuss and debate the biggest news in artificial intelligence. As always, there’s a ton to talk about. We’re going to talk about Rick Rubin’s collaboration with Anthropic, Jony Ive with OpenAI, Microsoft’s new agent factory theory. But first, I really wanted to start by talking about the Claude 4 system prompt.

You may have heard our emergency episode where we did a quick review of the release of Claude 4, and true to form, pretty soon afterwards, the system prompts leaked. I think that’s kind of just almost standard practice now. It was pretty interesting. Simon Willison did a super interesting blog post where he annotated the system prompt, and I think in general, I wanted to kind of get this group together ‘cause we haven’t talked about prompting in some time, but it’s also just an interesting document as kind of a state of the art on where prompting is at the moment.

Chris, I’ll start with you. Curious if there’s anything that kind of stuck out to you reading this prompt that you felt was different or really indicated where the state of the practice was in prompting.

Chris Hay: I always find the Claude system prompts super interesting because, one, they’re very transparent about it—they publish it. I mean, there is some stuff they don’t publish, but they’re pretty transparent about that. But it’s long. I mean, this is not a short system prompt. So if you think about the question, “How good are we at system prompting?” this thing is pages and pages long. So Anthropic is really giving you an education themselves on how to prompt properly. I think it’s pretty good.

There are a few things I think are super interesting about it. The first one is probably just simple things like guidance on how it wants to answer—like, if it’s a short thing, please just answer there, don’t use artifacts in this case, et cetera. There’s a lot of guidance on perspective and how to deal with personality as well, and you know, if it’s a sensitive topic, blah, blah, blah. But I think the thing that probably makes me laugh the most is how it always talks to Claude in the third person. You know, “Claude, you should do this; Claude, you should do that.” And I think the AI is gonna have an existential crisis already, thinking in a third-person form. But I think it is worthwhile everybody checking out that system prompt because you can learn a lot from it.

I remember last year, when Claude 3.5 came out, one of the videos I did was I took the Claude 3.5 system prompts and I put them on top of the Llama models. And I’m gonna be honest, even though those system prompts were designed for the Claude models, they actually improved the Llama models as well. So I honestly think it’s something everybody should really read up on.

Tim Hwang: Yeah, for sure. So there’s a lot there. And Kate, maybe I’ll turn to you. I mean, taking Chris’s first point, I thought one of the most interesting things in the prompt was the degree to which it really feels like in prompting we’re trying to figure out how much we need to specify versus leave up to the knowledge of the model. So there’s an interesting quote where it’s like, “Claude should be cognizant of red flags in the person’s message and avoid responding in ways that could be harmful.” And part of Simon’s annotation is like, it just has a notion of what red flags are. And I’m curious about how you think about that. I know Chris is saying that these prompts are very long, but it almost kind of presages a world where we can increasingly rely on model knowledge and keep prompts almost sort of short. Curious how you think about that.

Kate Soule: Yeah. You know, I think what surprised me most was just how much of the Claude experience they’re leaving up to a single prompt versus breaking some of these things down into more granular steps. You mentioned red flags, saying, “Alright, Claude, pretty please don’t respond to red flags, whatever red flags might be.” And you could easily envision a different experience that Anthropic could have built where first there’s a step with literal screening by a model whose only job is to screen for red flags or any other risks, harm, and biases. They might still be doing this behind the scenes, but where I see a lot of the world starting to move—and where I would’ve expected Anthropic to go a little bit more with Claude 4—is dividing this up into more steps, running more inferences, and leaving less to kind of a really long essay that you have to maintain and do basically security on a prayer. Like, “Pretty please will you not respond to harmful content?” Instead, have more verifiable checks and balances that you can articulate via software and more programmatic functions.

Tim Hwang: Yeah, for sure. And Aaron, I guess to take the other side of kind of Chris’s response to that first question, you opened up the round-the-horn question by saying, “I don’t actually really do much prompting at all.” And I think Chris is kind of almost taking the view that it’s good for us to read and understand what’s going on here. But I don’t know if you’d say this is maybe a little bit too aggressive, but is it worth it for us to kind of study these prompts as someone who just kind of gets models to generate them for him?

Aaron Baughman: Yeah, I mean, there’s sort of two schools of thought here, right? Should these prompts be released or not? And if they’re not released, then they’re potentially gonna be leaked anyway. I think one school of thought is we should release the prompts because it’s proof that AI can be incredibly smart but can still completely misunderstand the assignment unless you understand that manual of how to use the LLM. On the other hand, maybe you don’t wanna release the prompts because AI could be like this new intern—eager, unpredictable, but somehow already running the company. So we have to be very careful about releasing too much. And the leaking part of this—from what I saw, it looked like Anthropic did release some of the system prompts, but what was really leaked were the tools part, which could be very dangerous. So the notion of whether people need to read these manuals to understand how to use LLMs—from the expert level, if you’re like an 8 to 10, then I think it’s good to study it. If you’re down on the lower end, one to three, maybe not. But I do think that whether or not these prompts are gonna be released is sort of up in the air, and there are a lot of inherent risks about exposing these prompts, but there are also benefits.

Chris Hay: I think it’s not a bad thing, though. To sort of come back to Aaron’s point, it’s more of a handbook and a guide for the model. The model’s gonna learn loads of things over time and be put in different situations. But like us as humans, we’re in different situations. How I act at a party is gonna be different from how I act on this podcast. Before we came on this podcast, a wonderful producer was like, “Tim, make your bed. Chris, sit up straight. Put your camera down.” Here is the guide for how you should behave in this scenario. And that’s different in other scenarios. So I think it’s okay for them to say, “You are now acting as a general chatbot. You’re answering general queries, and that means average human beings don’t want to hear you waffling on. They want it in a couple of paragraphs, and they don’t want you hallucinating things. They want you to use the web tool and come back with answers.” So I think it’s okay to have that in a system prompt to guide, like a handbook of how it should behave in that case. That’s how we deal with things as well—in different scenarios, we have different guides.

Tim Hwang: And I think one of the most interesting things here—and it goes back to what Kate and I were talking about a moment ago—is that originally, I think the idea of these prompts was to specify in detail what you wanted the model to do. I always remember the joke I had with a friend: “Are we just rebuilding programming? Where you have to say really specifically what you want the computer to do?” But there’s another quote I had written down. One of the elements in the prompt is: “If thinking mode is interleaved or auto, then after function results, you should strongly consider outputting a thinking block.” That’s a very funny thing where you’re like, “Okay, now the model has thinking mode.” But rather than saying, “Under these specific conditions, engage it,” it’s just like, “You should strongly consider it.” And it’s sort of interesting—the degree to which these prompts are actually giving us control over what the models are doing versus us just giving it vague rules. I don’t know, Kate, if you want to respond there?

Kate Soule: Well, what might look like control—I think the other thing is, how much have we really tested? If you don’t release system prompts, it’s really hard for the academic community to do research and validate some of this. But how thoroughly have we really tested if every single line of that system prompt actually has the intended effect? What is the degradation in performance? How often does the model produce thinking if that line is there or not? I see prompts all the time where people write them based off of one weird edge case, so they add a line and that one weird edge case disappears. But do they really impact the model behavior as a whole across everything you’re trying to impact and study? So I think there’s also some degree of wishful thinking with system prompts where the model’s been trained for a lot of these behaviors already, like when to do thinking and all sorts of stuff. So we’re trying to nudge and steer, but it also makes it seem like, “Oh, well, if I told the model X, Y, and Z, then X, Y, and Z will happen because I gave it this nice little playbook.” And I think it gives us a false degree of security that that is actually gonna be followed. I think a lot of these system prompts are probably way too long. If you actually want something that really is—there should almost be standards of: Is this system prompt certified to impact this type of behavior and the degree that it specifies?

Aaron Baughman: I do think there’s a balance. Going back to tool calling, function calling—there’s a huge inherent risk, I think, of leaking those types of prompts because, depending on the use case—for example, on the extreme, if you’re doing robotic surgery, and somebody could have a tool call and hack the tool call and bypass different types of refinements, they could do jailbreaking, bypass content moderation, force different types of searching, which could have catastrophic impacts on the patient. So those types of system prompts could be obfuscated. They could be encrypted within fragments such that they’re not there to be used, because I don’t think some behavior should be enabled and released—like if you’re filing taxes or sending an email. I certainly wouldn’t want it to go, “Oops, sorry Kate, I sent an email on your behalf because I hacked this certain tool call or function call.” So those types of more extreme exploitations just need to be carefully thought of. I thought Anthropic was taking that into account by not releasing some of those system prompting elements within their original manual, but then they were leaked anyway. So there’s always this risk and balance that we all need to think about.

Tim Hwang: Yeah, for sure. And I think the layers of obfuscation here will get very interesting because at the end of the day, it’s just tokens. So you can imagine constructing a prompt where a human reads it and thinks, “Oh, well, these are the rules that guide the system,” but actually imposing certain other kinds of not-written behavior on the model, which I think will be a really interesting next development if it hasn’t already happened, because all these companies know that the system prompt’s just gonna get leaked within hours of the model coming out.

So I’m gonna move us on to our next segment. A really interesting collaboration dropped between legendary music producer Rick Rubin and Anthropic. They dropped this document on thewayofcode.com, and what it appears to be is a rewrite of the Tao Te Ching but about vibe coding. This is both a very funny collaboration in some ways and made me think a little about this famous interview that Rick Rubin did with 60 Minutes, where he said, “I have no technical capability, and I know nothing about music.” He took a lot of criticism for this, being the legendary music producer that he is. But I kind of love this because it sort of asks the question for vibe coding about just how far vibe coding will go and whether or not in the future we really will have Rick Rubin-like producers for code in the same way that we have for music, where it’s really unclear what Rick Rubin’s skill is—he just appears to be really good at getting number-one hits. I don’t know. Maybe Aaron, I’ll throw it to you first: Do you feel like in the future of vibe coding, we’ll see people with zero technical ability be able to do incredible things with computers just given where things are going with code generation?

Aaron Baughman: Yeah, I mean, there’s a continuum here. As an engineer and scientist, I do believe that the mind gets into these different patterns and constructs pathways as one develops and codes and builds. You can think of it as a flow state, and if someone just walks into your office when you’re in the middle of it, your flow state collapses and you’ve got to start all over and rebuild those constructs. That to me is kind of like this vibe coding. But I think the way that Rick is approaching this is more of an art form or a cultural phenomenon. I visited his “Way of Code” site, and it looked like you could go in and actually personalize some of the graphics that he already seeded with a vibe coding element. So in short, if you are building a production application that needs to be at scale, I think pairing vibe coding with good engineering is very important. But if you’re just doing it for a prototype to build an experience that doesn’t have to be so precise, maybe this kind of style of vibe coding is the way to go.

Tim Hwang: Okay. Any responses to this? I know it’s always this kind of push-pull. I think Aaron’s response has a lot in there: “Well, this is good, but we might really need real engineering at some point.” But curious about what you thought reading through “The Way of Code.”

Kate Soule: Yeah. You know, I think in many ways, coding can be viewed reasonably as an art form—it’s creating, the act of creation is inherently artistic and creative. So from that perspective, I think there is something interesting about how we unlock future developers who don’t have the same backgrounds, who bring different experiences to find new ways to solve thorny, challenging problems. I think that’s kind of the spirit that Rick is coming from. But I also think if we talk about critical infrastructure and what the world runs on, there’s a big difference between art and mainframe systems that run all the financial transactions in the world. There are different degrees of reliability and trust. So I think it’s important to make sure there’s a balanced approach. It’s not saying the world is going to be vibe coding and only vibe coding, but how do we use this as a tool to engage more with the community, with people who come from less traditional backgrounds that traditionally don’t know how to code but could bring really new, unusual, and powerful ideas that could be implemented—if they’re going to be implemented in some sort of critical capacity—with more knowledgeable, traditional means.

Tim Hwang: Yeah, I love that. Basically, in an earlier era, computer code sort of couldn’t be approached in an artistic manner, but we’re now living in a world where the boundaries of that are a little expanded. So you can approach it as if you were a music producer or just kind of vibing with it. Chris, responses? I saw you went off mute here.

Chris Hay: Yeah, I love it actually. I do think programming is an art form. I know we want it to be a science, but I do think it is art. So I love the idea of exploration and being able to figure things out. I don’t think we always need to take an engineering approach. If I think of architecture—not computer architecture, but people with pencils and beards and flip-flops—if somebody came in and said, “I want to design a new house,” and they drew a picture and said, “Here’s your new house,” you’d be like, “Huh, should I give that to the builder?” But that’s fine. But if you got the technical schematic architect who just built that, they’re gonna follow the process—this joist needs to connect to this. I don’t know any building terms; “joist” is the only one I know, and then roof. But where’s the creativity? That’s not gonna create the Guggenheim or something like that. So I think you’ve got to have that mix. It’s almost the same as music production. In Rick Rubin’s case, vibe coding allows you to break things down into their individual elements and then recompose them. That’s okay to then take to an engineered state. I think that whole process of creativity is a good thing. So I’m a big fan of vibe coding because you can test out ideas really quickly and explore, and then you can go engineer the parts you need and get a little more process-oriented. But why kill the creativity? So I love it. I’m a huge vibe coder, and I love the collaboration.

Tim Hwang: Kate, this makes me think a little about how vibe coding is gonna evolve within an organization or within an enterprise. In all the companies I’ve worked for, there’s always been a little bit of a class system between the designers and the engineers. The designers say, “Here’s a mockup you should build,” and the engineers are like, “We have to build it. Ah, all these people with their crazy designs.” It kind of feels like what vibe coding is gonna allow is that designers can suddenly build workable prototypes. So there’s a whole degree to which this allows a group of people within a company to kind of seize the means of production in a way that might be deeply disruptive to the natural state of affairs. That feels like it’s gonna be really interesting to watch.

Kate Soule: Yeah, I think it can go both ways. Designers or whoever is trying to test the waters always say, “Oh, go build this; it should be easy. Just put this button over there, and it should do all these other things, and oh, it also needs to be compliant.” And they’ll try it, and undoubtedly it will fail if they just vibe-code and throw it out into the world when it hits real production, and learn some pretty nasty lessons that it’s actually really complicated and there’s a lot of important work developers are doing behind the scenes. So I think it’s just gonna be really important as a communication tool to help better articulate vision, explain what you’re looking for or the target goal, and iterate faster on proof of concepts and experiment faster. So I think it definitely will disrupt from those perspectives.

Tim Hwang: Yeah. And it actually occurs to me as you’re talking that the annoyance will work both ways because suddenly engineers can be like, “Oh, I generated this picture of the website I wanted you to create.” It’s like everybody’s gonna be in everybody else’s business.

Aaron Baughman: Yeah, this whole notion of vibe coding to me is very similar to inventing. You get lots of people together, and you need different perspectives. You need the artfulness of creating novelty, but you also need the engineering to make sure it’s implementable and can be used in some kind of embodiment. Vibe coding is very similar: you get the creatives together, it becomes more of a blur where the scientists and creative become one because you’re vibing to do this vibe science or vibe engineering to have these alternative hypotheses. It’s like exploring different branches very quickly. And then when you need to get into an embodiment, then you build and implement. So I think some of the white space here would be: How do we connect vibe coding to the actual build implementation and deployment of something that’s practical, usable, and can handle high scale and load—some of the really hard challenges we face every day. So I’m pretty excited about that area, which I think is just beginning to emerge.

Tim Hwang: So for a third segment, we’re actually gonna do another design and AI story. The biggest business story of the last week or two in AI has been this enormous USD 6 billion-plus acquisition of Jony Ive’s secretive startup, io. Jony Ive, if you don’t know, was most famously the chief architect of the iPhone and the design mind behind Apple during a whole era of its history. The announcement is that Jony Ive himself is gonna collaborate with OpenAI on hardware through a design collective that he owns. This is a huge transaction—billions of dollars. Chris, maybe to turn it to you: Is it worth it? There’s not even a product here, and they’re putting USD 6 billion down. How do you think about why OpenAI would do this and if it really is gonna pay out for them in the end?

Chris Hay: I hope for USD 6 billion, he does more than collaborate. That seems a huge bill for collaboration. I’m collaborating with you guys just now, and I’m not paying USD 6 billion. Sorry about that, Chris. I would be more worried if they paid USD 6 billion and Jony Ive went, “You can have my company, but I’m outta here; you’re hearing nothing from me.” You’d be like, “What am I buying?” I mean, Jony Ive is incredible. You’re buying his talent, his brand. So I think it’s gonna go beyond collaboration, and I think it’s really gonna be about shaping the ideas that form what the future of AI is gonna look like. We’re now in this multimodal world. AI is becoming cheaper, able to run on-device. So there are new form factors that need to be discovered to have AI in the right place. How do I want to interact in that world? How is the world of agents gonna behave? How does the future of the web look? How does the future of mobile devices? I think there’s a lot to work out and discover. Does that mean how we interact today is gonna change? I think it will. So being able to bring together AI companies and design companies to figure out what that future looks like and experiment—I really think that is a smart move. Having somebody like Jony Ive who’s been through those transformations before is very sensible. So I think it’s an exciting collaboration, and I look forward to what this next wave of experience design for AI is gonna look like.

Tim Hwang: Yeah. And Kate, to give them a little more credit, this is more than just a vibe acquisition. There have been some details leaked or rumored about what they’re working on. As far as we can tell, it’s a kind of AI device with no screens. That’s pretty interesting. We’ve built a whole digital paradigm on screens, so the idea that we’d go completely no-screen in the future thanks to AI is pretty surprising, don’t you think?

Kate Soule: Yeah, I think it’s very surprising. It also gives vibes of some of the AI companion things we’ve seen, like... nobody wants just basically a tamagotchi where you’ve got a tiny little screen companion that you have to feed, otherwise it dies. So I think they’re probably gonna lean into some sort of life assistant route that doesn’t need eyes. If it doesn’t need eyes, it doesn’t need a screen to communicate with you. We’ve got better tools now that they’re working on. It’ll be interesting to see what they come up with. I’ve struggled to see that there won’t be some sort of phone app experience as well that connects to whatever device they’re also working on.

Tim Hwang: Yeah, it’s hard to untether from that completely. Aaron, how do you size it up? The most obvious precedent is the Humane Pin, which we talked about a year ago—a screenless device you wear that’s always on, an AI assistant. One point of view is no one wants that, and that’s why it didn’t work. Another is the technology wasn’t there, and we might finally be there. I don’t know if a year later is enough time, but things are changing quickly.

Aaron Baughman: Yeah, I’m a bit stuck on that this is one of the largest deals for 55 employees—at least that we know of. If I do the math right, that’s about USD 118 million-ish per employee. That’s pretty good. It’s a high-stakes bet on this talent because the valuation is very speculative—this company hasn’t created a user base or any devices. So it’s basically a high-stakes bet on design talent. But if it goes right to creating these AI companions—I saw Sam Altman wanted to release a hundred million of these AI companions—if they can pull it off, they can sell these very cheaply to get back their USD 6.5 billion investment. But I just wanna see something tangible very quickly. I think they can pull it off. Their mission is in the right place. I would just say, Apple, watch out. Apple Intelligence needs to get going quickly because if OpenAI works with Ive here, these AI companions could really be a nice bet to understand what’s happening in one’s life without having a screen, or maybe extending to an already existing screen. These different form factors are gonna be really interesting, combining cutting-edge AI experiences.

Kate Soule: One thing Apple does well—obviously they need to catch up—but as we’ve talked about in the past, assistants have failed. What I think OpenAI will struggle with is still this notion of privacy and trust with data. There’s just still this shadiness factor: Why are my life’s moments being recorded and beamed up to some machine and AI intelligence? I don’t know that OpenAI is best suited to crack that. So it’ll be interesting to see if the new design team can help think through new ways to design for trust. That’s something Apple does have as a better starting position if they can figure out some of their Apple Intelligence work.

Tim Hwang: Yeah, for sure. The paradigm shift implied for OpenAI to get this right is really hard—more than just devices, it’s consumer trust and how you ensure that from a technical standpoint. It’s a whole other way of thinking.

Chris Hay: I don’t know, I think we overthink trust sometimes. It’s a trade-off: here is the functionality I’m gonna get; how better is my life gonna be? Think of the hundreds of millions of people using ChatGPT every day. Everybody knows you’re giving away your data, but you’re getting utility. So everybody’s prepared to make that payment or not. Some things you’re not, and you’ll lean into something else. Personally, I find it very unlikely I’m gonna give up my iPhone. I love my iPhone, my iPad; everything is connected. This thing doesn’t have a screen—what am I gonna play my movie on? Things don’t exist within islands. The thing Apple does very well is they have a very good ecosystem of platforms and devices where everything connects well. If they’re making a move into that space, they’ll do very well because you have to bring the ecosystem along. Back to the point about that pin thing—it didn’t connect into anything; it sat on an island. So that’s really gonna be the problem OpenAI has to think about: What ecosystem are you gonna plug into? The only two choices are Apple and Google. So you’ve gotta start figuring this out because if you can’t plug into that ecosystem, you’re gonna have a problem.

Tim Hwang: Alright, so Chris already beat me to it by saying the word “agent,” but we’d be remiss if we didn’t do a story about agents. I’m gonna close up today with our last segment. A super interesting Verge interview popped up with Jay Parikh, former engineering lead at Meta, now at Microsoft working on all things agents. We haven’t heard from him in a while. I thought it’d be useful to check back in on what he’s been working on and talk about Microsoft’s strategy. The most interesting quote: “I want our platform to be the thing they turn into their own agent factory.” So the idea is whatever you’re building, you’re gonna be able to turn it into an agent using Microsoft tools. We’ve talked about how “agent” means everything. One way of thinking is these companies are all battling for what an agent even is. For Google I/O, their version of agent is search—not surprising ‘cause they’re a search company. Microsoft is articulating its vision: every enterprise being its own manufacturing facility for agents. Kate, maybe I’ll turn to you: It assumes a world where these things become really commodified and democratized. Do you see that happening soon? Is that realistic?

Kate Soule: Yeah, I think we see a lot of other industry players putting pressure on Microsoft to do similar—Agent Force from Salesforce, watsonx announced a bunch of agents at Think just this past conference. So I think Microsoft is just trying to better speak the language of what all our enterprise users and customers have been trained to speak, which is, “I need agents. I need agents now. Everything I can build can be built as an agent.” And trying to make sure that they’re hyper-targeted towards this kind of modality for how people are trying to build. And I think it is very much being democratized as we start to see performance for useful enterprise tasks converge. Any model can do a lot of the things that drive 80% of the value for these companies. So the ability to build your own, to swap out parts, to customize, I think is gonna be critical as people continue to look to how to avoid getting locked into just one endpoint and ultimately continue to innovate within their own four walls and how to use their data to create value.

Tim Hwang: Aaron, there’s almost a question here about the ceiling on commodified agents. We talk a lot on this show about how complex it is to orchestrate agents to work properly. You need the right protocols, tasks done in the right way, fine-tuning, evals. The skepticism I’ve always had is it just seems like not every enterprise has people who know how to do that out of the box. But I guess Kate’s kind of arguing that there’s enough common tasks that the sort of out-of-the-box agent will be something most enterprises can play with. How do you think that market’s gonna evolve? It sort of feels like it’s gonna go in two directions almost over time.

Aaron Baughman: You know, whenever I think about agents, the first thought that pops in my mind is James Bond 007. He’s the ultimate agent. And we need to watch out for double agents and make sure that we can ensure that they don’t go rogue. I was looking at this, and you know what this Agent Factory has—it’s like it has this service, it uses agent identity and governance where I can provide identification for each of the agents such that you can’t go get a fake ID and do maybe doppelganger another agent to go do something else. It’s got observability management, low-code, no-code tools. But I mean, I think everybody in industry is trying to get in the game of AI agents, what they should be. But I think for Microsoft, one of the biggest differentiators I see—I happened to look two weeks ago at the Top500.org, this website that tells you the fastest supercomputers in the world. I was curious, was cloud on there? I think the number four ranked one was called Eagle, and it runs and is built on Azure. So it’s a cloud-based supercomputer, which I didn’t think I would see happen so quickly. So to me, the compute power that Microsoft has on Azure really can give them a nice opportunity here. They have data sources, they can integrate with Windows, they already have Azure AI Copilot pieces they can expand into consumer markets with like Windows Copilot. So I think they have sort of the bread and butter elements to make this AI agent factory happen. It’s just hopefully they can release some of these features to map to their vision of how they’re gonna do it so we can avoid these double agents.

Tim Hwang: Yeah, for sure. And Chris, it looks like you’re about to jump in. If I can maybe prompt you with a question: We’ve talked a lot about who’s gonna win in the agent market. There’s almost a part of me that thinks about Aaron’s comment and is like, maybe actually over time the agent market is gonna divide up. That it’ll just turn out that if you have a task that really requires search, you’ll be using Google’s agents, but you may not really need all those capabilities, and so maybe you’re more married to the Azure infrastructure and use Microsoft. It may not be winner-take-all in this market. I don’t know if that was what you’re gonna address.

Chris Hay: I don’t think it will be winner-take-all, and I’m happy to kind of say that because one of the big things really happening in the market at the moment is commoditization. So if we really think about what’s going on here, all of the major providers have hooked onto Model Context Protocol (MCP) as the standard for remote tool calling. And I think that’s a good thing because we’re gonna move into this world where we want to be built on composition. So if everybody’s at least standardizing on tools, then there can be a marketplace of tools, and it also means the models can be trained to work with those tools very well. And therefore, if you want to shift to a different agent for whatever reason, then guess what? You can bring your tools along with that. And I think in the factory context, this makes sense as well. From a factory perspective, I’m gonna want to build something, but actually maybe 80% of the tools already exist and maybe 80% of the agents that can work with those tools already exist. And I really need to do the 20%. Whereas in the previous world, I would’ve had to do all of that. So I think that becomes important. And then with things like AIO-A and ACP, having agent protocols where you can have a standardized way of having agents be able to talk to each other, and again, whether it’s Salesforce, Microsoft, et cetera, they’re all landing on protocols for that interoperability. So I think that moves us into a marketplace again. So I think as soon as you start to get in this world of marketplaces and you have this area of standardization, then I hope that means we get away from this winner-takes-all market and folks can specialize on the things they’re really good at and their differentiation. The good news and probably the bad news at the same time is actually, I think this brings us back into the discussion we had at the beginning, which is about vibe coding, because actually if I’ve got the engineering of agents that do tasks really well and I’ve got tools that do things really well and models have done that, and then agents know how to talk to each other and we all know how to talk to models, et cetera, then actually vibe coding becomes quite interesting in the world of factories because then I can sort of vibe up what I want and then I can hand it across to some agents who are gonna do a productionized version and use productionized tools, and it completes that circle. So I know we were talking about vibe coding being a toy, but actually I want you to think about that factory model for a second that Microsoft’s discussing, and I think those two worlds blend over time.

Aaron Baughman: I can also envision a world where we have these AI agent skills marketplaces. If we use these new approaches—so we just released what’s called aLoRA, I think it’s activated low-rank adaptation, where you have like these weights that can influence the attention. So your weight matrices that project and create, whether it’s your keys, your queries, your values, but they can be fine-tuned to what kind of skill you would like. And then you save those weights and you can dynamically on the fly import that skill so that now that same model topology of which you created your LoRA weights now has a different behavior. So this decentralization of skills is there, and you could vibe-do some vibe skill to create what kind of skill vibes with you and then put it up on a marketplace to share with your friends or create these emergent skills. But I think that might be where it’s going. And then last thing, I could talk about this for a while, but model distillations could play in that as well.

Tim Hwang: Okay. I’ll let you have the last word here. I know on the first conversation, you were maybe the strongest on, “Look, you’re not gonna use vibe coding to build a bridge.” I think Chris is maybe ending on a note of optimism: maybe agents are the bridge that gets you there. Do you buy that story, or are you still a little bit skeptical about how far Rick Rubin can get?

Kate Soule: I think humans are gonna have to be in the loop more than just in the vibe-coding step. So I completely agree. I think vibe coding to create something, kicking it over to an agent to iterate, build it out a little more detail—all fair game and is gonna be pretty exciting. But I’m not ready to totally just kick out the human-in-the-loop part of the process there, where they start at the beginning and then you just see what bridge pops out on the other end and walk across it blindly.

Tim Hwang: Seems like a fine bridge. All my agents are telling me it’s the best.

Kate Soule: Yes, every agent agrees.

Tim Hwang: Yeah, exactly. All right, well that’s all the time we have for today. Kate, Chris, Aaron, always great to have you on the show, and thanks to all our listeners for joining us. If you enjoyed what you heard, you can get us on Apple Podcasts, Spotify, and podcast platforms everywhere. And we will see you next week on Mixture of Experts. Great job, everyone.