Tim Hwang: The White House has released its AI Action Plan, which is sort of a national strategy for Artificial Intelligence.

Ryan Hagemann: The short version is: this document basically lays out the Trump administration’s policy agenda as it relates to Artificial Intelligence. And part of the reason I think you haven’t seen so much action in Congress is this is also the starter pistol for legislative action in the future.

Tim Hwang: All that and more on today’s Mixture of Experts. I’m Tim Hwang, and welcome to Mixture of Experts.

Each week, MoE brings together a sharp group of thinkers working at the very cutting edge of Artificial Intelligence to discuss, debate, and distill the week’s news. Today I’m joined by Kate Soule, Director of Technical Product Management for Granite; Gabe Goodhart, Chief Architect, AI Open Innovation; and Mihai Criveti, Distinguished Engineer, Agentic AI. Later in the show, we’re going to be joined by Ryan Hagemann, who’s the Global AI Policy Issue Lead. Per usual, we’re going to talk about ChatGPT agents, Mihai’s MCP Gateway Project, and we’re going to have Ryan on to talk about this newly released AI Action Plan.

But first, I really want to start by talking about IMO, which is the International Math Olympiad. It’s the world’s foremost annual mathematics competition for high school students, and we’re talking about it today because both DeepMind and OpenAI have claimed their systems are achieving a “gold standard” in competing in that competition, which would basically put their technology within a comparable performance to the top 8 to 10% of high school mathematicians.

Now, this is a big competition. Over 110 countries send teams to it each year. I think some mathematicians actually took a step back on this news breaking and said, you know, we think that this is a “Lisa Dole moment”—this is as big as AlphaGo from a number of years back. And I think that’s the first kind of ‘round the horn’ question I want to prompt you guys with: at the moment, is this a really big deal, or is this kind of just another benchmark? Gabe, maybe I’ll start with you.

Gabe Goodhart: Well, as a former mathematician who discovered that computers can do math a lot better than I can, and then turned to become a computer scientist... this is both extremely comforting and not surprising. But I think seeing the depth of the logic and reading some of the techniques they used, it’s a really cool piece of technology change. I don’t know that it’s going to flip any tables over today.

Tim Hwang: Cool. Got it. Kate, what do you think?

Kate Soule: I mean, I think it is a similar moment to AlphaGo in that we’ve cracked a new benchmark, just like AlphaGo cracked the game of Go. Similar to AlphaGo, though, I don’t think it’s going to have a tremendous real-world, tangible impact in the next couple of years. We saw AlphaGo beat Go, and then all of a sudden we were impacted in AI in our daily lives every day with all these different value drivers and applications. I think this is kind of similar in that this is an impressive challenge that was beaten. I don’t think this is like, “Oh, we’ve now unlocked AGI and all these other applications are going to fall into place.”

Tim Hwang: Mhm. Got it. And Mihai, what do you think?

Mihai Criveti: I think it’s really cool. I think it’s cool because it demonstrates a lot of the techniques we use in Agentic AI as well—things like computer use, building calculators, and using those functions to solve the problems. It’s not just relying on what the large language models have been inherently trained with. From that perspective, it just demonstrates growth where agentic techniques are becoming more and more fine-tuned and specific to the type of task or workload they’re being applied to.

Tim Hwang: Yeah, that’s great, and I definitely want to get into that. But first, maybe Gabe, I’ll go back to you because I hadn’t realized you had a mathematics background. Do you want to just give our listeners a flavor of how big of a deal the International Math Olympiad is? Is this pretty significant? How difficult is this test? Could I do this test? What are we talking about?

Gabe Goodhart: To be fair, I don’t think I have a good answer to that one, because frankly, I wasn’t that kind of mathematician. I was a liberal arts student that was good at math and needed to pick a major, so I started with math, and then I discovered computer science and said, “This is much better.” But no, I mean, I went far enough in math to really get to the point where there were some hard problems.

The beautiful thing about math is that it is a well-defined ecosystem with strict rules. That’s kind of the whole point of it. And the higher you go in math, the more you’re exploring the boundaries, the edge cases of those rules, and the esoteric rules you might not have thought of when you’re looking at a simple arithmetic-based or geometry-based space. So when you get into chaotic dynamics or multivariate calculus and things I never even got to, you’re really starting to take those rules and go as far as you can.

I think that’s one thing that’s really fascinating about this: it’s basically showing that with these additional techniques Mihai referenced, and with some inference-time compute that, Kate, I know you’ve talked a lot about, they were really able to push the model’s reasoning capabilities a lot further down the boundaries of this very nicely defined space of mathematics, to explore portions of it that typically you have to go pretty deep into and, as a human, have some good intuition about. So I think that’s the really interesting thing here.

You said it really well: it’s basically getting to the point where we’re no longer just sort of throwing a “guess and hope” type of depth where every step in the chain is going to have errors that compound, and you might get lucky. The fact that it could do it consistently enough to get a good score on this many difficult problems shows that the techniques are really starting to reach a point where they have consistent applicability, such that you could imagine applying them to a more difficult challenge in the real world and actually relying on the results without needing to check every step of the way.

Tim Hwang: Yeah, for sure. And that’s something I really want to get to. I think, Mihai, to kind of pick up on a comment you had in response to the opening question: one of the most interesting things about AI is we say, “Oh, AI does this, AI does that,” but we don’t often talk about the fact that AI itself is sort of changing as we go. You highlighted that there’s a bunch of different techniques here. Do you want to give us a little bit of flavor of what’s new versus how we were maybe attacking these problems a few years back when we were more in “stochastic parrot” land? It seems like you’re almost suggesting there are more tools being used to get these types of results.

Mihai Criveti: I think what’s changed is how we approach these problems. To your point, Gabe, we’re no longer just throwing a simple prompt or even a simple chain of thought at the problem. We’re spending time beforehand to build tools, define how those tools will be used—either to generate the answer to a mathematical equation or to verify those results—and we’re executing these tools in parallel, massively.

Think about the problems you used to try to solve with AI before: it cost you a nickel and a couple of tokens to ask a question, you’d get a result, that result was probably terrible, and you’d say, “I’m going to work with it and try again and try again.” The approach here is massively parallel. You’re firing off millions, tens of millions of tokens, and hundreds of different tools, verifying them every step of the way. It’s similar to how researchers work, and you’re going to get really good results.

And as a business, you have to ask yourself: is it more cost-effective to ask a question ten times and get the wrong answer, or to ask it a million times? Probably the latter is going to be more expensive, but you’re going to get the correct answer. So finding that balance is going to be key. I don’t think we can use the same approach for every problem, because that approach requires you to first engineer your tools, design a system, and execute a very expensive query. I don’t suspect this cost a dollar or $10 or $100; it probably cost tens of thousands to ask all the questions to solve this problem.

But if we can get to a reasonable level within software development use cases, for example, where for $1 or $2 you can do massive refactoring of a code base, a class, or a function with the same level of tools, I think we’ll be in good shape.

Tim Hwang: Yeah, and it leads quite nicely actually, I guess, to... Kate, you had a good comment, which was: look, I think almost like AlphaGo at the moment is actually a great parallel because you’re like, “Okay, you cracked Go—a thing we didn’t think we could do.” But it’s also just like, “Okay, then what? Then what do we do?” I guess, you know, this is obviously an impressive technical achievement, but it’s kind of what you were saying a little bit earlier: you feel like the actual practical impact is going to be limited in the near term. To Mihai’s point, it’s not actually a practical approach to most of the problems people are trying to use AI for today.

Kate Soule: Yeah, I’m skeptical that this is going to all of a sudden just totally change the calculus on how we approach all math problems and unsolved problems. I think it’s going to be a really helpful tool; I think it’s going to be incremental change, though, and not a step change.

The thing that I think is most interesting is that if you look... this isn’t the first time Gemini or Google, at least, has been in the news around the Math Olympiad. Over maybe a little over a year ago, if we look at AlphaProof—which is their specialized model for math—it achieved silver performance on the Olympiad. But what was different? They used a specialized model, but their model took days, sometimes, to solve a single question. And what’s really exciting, and what I think is a bit of a breakthrough, is that they are able to solve these problems now, all of them, within the time limit that all the other competitors have to observe, which I think is something like 4.5 hours total for six questions.

So as we talk about these general tools as they’re advancing, and techniques to Mihai’s point, I think this is a really great demonstration of how those tools are starting to enable much bigger changes. I don’t think it’s something that, math-specific, all of a sudden we’re going to have this crazy breakthrough in the field. Math is still going to be really complicated, and there’s going to be a lot of really thorny, cool research problems to explore—hopefully faster with the help of AI. But I think it’s really exciting as we talk about how these techniques and capabilities are evolving to be much more practical and able to run in a much more real-time manner.

Mihai Criveti: I’d really like to see one of these benchmarks have a time limit like this one, but also a budget and a time limit for the prep work you do ahead of time. So, what can you do with three engineers and four hours to prepare? Kind of like Robot Wars, right? You only have four hours to write your tools, you need to get it done, you have a budget of 10 million tokens or however many dollars on whatever platform. Now let’s see who’s the best.

Tim Hwang: Yeah, I want to do like a cooking reality show where it’s like, “The secret ingredient today is this dataset, and you have four hours to create a math-solving robot.” I don’t know, my tastes are pretty specific, I guess. I’d watch that.

So, the last thing I want to address before we move on to the next topic is: I was thinking about IMO, and you know, what do I think about IMO? It is an eval, right? One thing that occurred to me is: a lot of people spend a lot of time getting the International Math Olympiad test together each year, and it’s a really expensive eval to build, but it is, as a result, kind of a gold standard in some ways. I think the reason DeepMind and OpenAI are here is that it seems to be a really strong test of the technology.

I was talking with a friend recently about this and just wanted to test this group on it: it feels like as capabilities expand, trying to get good evals is going to get more and more expensive. You want to know whether or not it can do expert graduate-level math. Well, that’s a little different from getting a bunch of simple arithmetic problems together. I’m curious if anyone’s kind of seeing that in their work—seeing evals become more and more expensive for us to produce in a way that introduces new problems. How much time do you need to get an expert-level evaluation? The number of humans you need to put that kind of eval together gets more and more limited. Gabe, you’re smiling.

Gabe Goodhart: Well, I mean, I think it’s a really interesting point you’re pointing at here: the more the AI is trying to tackle problems that are already specialized to a very small subset of humans, the harder it is to actually have a rigorous evaluation, because statistics stop really applying. There’s not a standard Gaussian distribution of people that would solve this thing. If there’s a small handful of humans that can solve this, that’s not a very well-formulated statistical distribution to say, “Hey, look, you’ve got this score with this variance.” That math just doesn’t apply.

I think one thing... this is going to be a very non-scientific answer, but I feel like we are going to hit a point with these models in general. I’ve made this analogy before: there was a time when we all probably were in our learning phases where it was verboten to cite a Wikipedia article in a research paper. You can’t cite Wikipedia because you just cannot trust its validity. And eventually that just kind of eroded. Everyone just was like, “Well, yes, technically there are some error bars. In my mind, if I see a Wikipedia citation, I have pretty darn high confidence that I can go over to Wikipedia, read the article, and then maybe click through to their citations and say, ‘Yep, the article is accurate.’ Okay, we’re fine. I can just skip that second part; I can probably trust the Wikipedia article.”

I think we’re going to probably start hitting that the higher up we go. In some ways, to your question, the more specific and narrow the population that could solve this as a human becomes, the more we’re going to have to start relying on “just I’m going to choose to trust the model here.” It’s built enough cred on other things that I trust that, in my mind, translates to probably believing that it’s good at this thing, because it just becomes much, much harder to apply rigorous statistics and evaluations to a much smaller population of data.

Tim Hwang: Yeah, for sure. There will just be like a “hypothetical math or like... big, if true” math, where we’re like, “Wow, the model has produced all these proofs.” And yeah, no one’s really verified whether or not it’s right. But if it were, then this is...

Gabe Goodhart: Well, and there are plenty of techniques in math—and again, it’s been a long time since my mathematician days, so I’m not going to use the right words here—but there are a lot of techniques where you do validation against one another, where neither is a source of truth but you have a way of cross-validating. And I imagine we’ll get to that point, where you’ve got a model trying to tackle problems that only a small handful of folks can do, and so rather than trying to produce a rigorous benchmark for those tasks, you instead say, “Hey, expert that could evaluate, do this on your own, evaluate what the AI did.” And now we’ve got a small sample size.

In some ways, that is what the Olympiad is doing here. You’ve got probably a small subsample of expert judges that are able to actually qualify what these mathematicians are doing. So this is already kind of an example of that, where you’re not using a standard benchmarking approach, but instead using expert judges to evaluate what the model has done. And those judges are theoretically fallible as well. But especially as you push further into the frontier, you just kind of have to trust the humans, and then the humans have to work with AIs to trust each other.

Tim Hwang: A lot to discuss here. I’m sure now that they’ve gotten gold, they have to find a new way, a new eval to do, so we’ll keep tracking this into the next year.

The second topic I want to get to is maybe the big product announcement of the week, which is that after a long period of speculation and rumors, OpenAI has finally released ChatGPT Agent. This is a feature whereby you can ask things into the model and it has kind of full-on agentic behavior as its own little browser; it can do all sorts of things for you. It’s only, I think, available at the Lux tier.

Mihai, maybe I’ll toss it to you because I know you work with agents day in, day out. Have you played with it? Any impressions? Strengths, weaknesses? Just kind of curious about your quick, off-the-cuff review.

Mihai Criveti: You probably also know I have the Lux tier for both GPT and Claude and all these things, and now I’m in trouble, probably, because when I see the bill at the end of the month, it’s getting quite expensive, especially in Europe.

That said, I don’t think this is necessarily a new thing. I think ChatGPT has had these kinds of agents internally, and many of the tools that it was using were agentic in nature. For example, it used the code execution sandbox. So if you say, “Write me... build me a diagram,” it internally generated some Python code inside an execution sandbox, used matplotlib, generated that diagram, and then gave it to you. This just increases the number of tools it makes available, makes them a lot more customizable, and gives the same kind of tooling support that DeepSeek researchers used, with internet browsing and that virtual computer use idea. So from that perspective, it’s building on the same concepts it used to have before.

But the market is pushing towards the agentic, especially with Anthropic releasing the Model Context Protocol, everybody building agents in the open-source community. I think they’re seeing the need to give their platform the same first-class experience for an agentic system.

Tim Hwang: Okay. Kate, do you agree with that? I guess the cynical view of what Mihai just said is, “Well, this is kind of an incremental improvement. Really, we should see this as marketing more than anything else.” Is that the right way of thinking about this?

Kate Soule: I definitely agree with that from a technology perspective, like an algorithmic perspective of what’s going on. But I do think this is a tremendous leap forward in a UI and user interaction with AI perspective, because, at least from what I can tell, this seems to be the first major asynchronous workflow enablement with agents and users for OpenAI. A lot of their marketing materials—I haven’t played with it, but just reading about and watching some demos—really heavily focus on: start a task and then close your laptop, walk away. The agent’s going to run, do different things for you, and you can come back whenever it’s done, having made more productive use of your time elsewhere. So they really seem to be heavily indexing on this kind of asynchronous deal, which is kind of the first I’ve seen. I think it’s long overdue; we’ve been waiting for this for a while, so really excited to see some of that start to come to fruition.

I do wonder a lot: they talk about all this value of being able to walk away, but then they also say, “Don’t worry, if the agent’s going to do anything, it’s going to ask you for permission. You’re going to have to enter your credentials yourself.” And so it’s kind of like, “Okay, well, how much can it do on its own if I’m also not trusting it to go and do a bunch of stuff without me giving approval every step of the way?” So I’m curious to see how some of that plays out. But I do think from a UX perspective and an interaction perspective, it is a very interesting leap forward.

Tim Hwang: Yeah, it reminds me actually of stories from the early days of PCs where people had these PCs and they’re like, “What do we do with this?” And so for a period of time they’re like, “Oh, you can use it to keep recipes.” Like, they’re clearly trying to invent uses. It’s like there’s an effort to have to teach people what they do. And it’s like, “Okay, yeah, that’s right.” You’re almost describing a very similar situation where it’s like, “Agents. Okay, that’s really cool. What now? How am I supposed to do this?” And it’s kind of funny to me that the big kind of UX part of this is actually you just can walk away from your computer. Like, the UX is no UX, I guess, in some sense.

How far do you think that’s going to go? Do you think the public is ready to trust these models in this way?

Kate Soule: I think it’s going to be a really engaging and interesting consumer-facing tool. I do not see this being ready for any sort of enterprise deployment. I mean, I mentioned already the security, and that’s just for trusting the agent with my OpenTable login to be able to make reservations where there’s a non-refundable deposit. Those are very small stakes, depending on the restaurant, I guess. But I think we’re still really far off, and there’s so much to work out from a security perspective to get this into enterprise use.

Tim Hwang: I mean, it feels like it’s a little bit delicate, because I don’t know if you agree with this, Kate, but it’s like the two worlds are connected, right? You can imagine that not just OpenAI, but a number of these companies don’t get the consumer experience right, and I think everybody’s general impression is, “Well, if I can’t even book a restaurant reservation, I’m definitely not going to use it for this.” Do you think there’s actually some risk that if we don’t do this agent thing well on the consumer side, it almost closes off the path on the enterprise side?

Kate Soule: I think the pattern that we’ve seen emerge so far, and that will continue to be true, is: iterate and work on getting the workflow right for consumer and flush out all the bugs, flush out all the kinks, then bring it to enterprise. And I think OpenAI is following that exact same playbook here. So they’re going to figure out the kinks, iterate and evolve while it’s relatively low stakes—planning an itinerary, minor purchases at the grocery store, that type of thing. And hopefully that will give them some experience to learn what could go wrong and then be able to address that for when the risks are far greater.

I don’t know how their incentives... if that will be perfectly aligned with where they’re trying to go. Again, they’ve always strayed a little bit more towards the “AGI at all costs” versus focus on making really, really great enterprise-specific tools. So I think there’s going to be a little bit of friction between those two priorities with OpenAI that other companies might not have, with a clear alignment of trying to get faster to enterprise readiness, even if it means going a little bit slower on the more general-purpose intelligence frontier. So I think that’s going to be a big question: can other providers get to enterprise-ready faster?

Tim Hwang: Gabe, maybe I’ll end with you. What’s your trust level with agents like? I don’t know if you’ve played with ChatGPT Agent or any agent. What’s the most important thing you would trust it with?

Gabe Goodhart: Um, yeah, great question. I am very trusting as long as there are no stakes. Which is to say, I’m not very trusting at all. So no, I am very happy to experiment, try things out, and generally use an agentic system any time it could accelerate what I’m doing, as long as I am the final arbiter of the output. And that’s just sort of my comfort level with using these tools for my own personal use. I think eventually, if and when they keep getting better at those things, I’ll gradually step it up. But I’m a fairly slow adopter of things that just have magic behind them, and I suspect that’s true of a lot of people that want to know how things work.

But I do think—I want to second what both Mihai and Kate said about this—that on the one hand, this is an incremental technology change, and on the other hand, this is a major step function in UX. To me, it brings to mind the difference between GPT-3 and ChatGPT. Fundamentally, the technology was all there in GPT-3, and it was literally just the UX of the instruction tuning that went into ChatGPT that made it explode. So even though these agentic patterns of tool usage and long-term inference scaling with DeepSeek research—all of these things have existed, and the individual building blocks have all been there, I think the idea of a single entry point that doesn’t require the user to have knowledge of what tools are appropriate for the task, and the idea of something that interacts more in the way you would with a colleague where you delegate a task, you just hand something off and sort of wait for feedback, and have potentially a mechanism for interactive updating... you know, Kate, to your point, I can imagine my phone pinging me to say, “Hey, my agent needs permission to do this. Do you want to give permission?” I’m happy to be interrupted for a quick context switch to say, “Oh, yeah, this looks good. Check. Go ahead, keep going.” I think the UX pattern is really going to change with this to be the central agent entry point that hopefully will make this much more accessible to folks to help build that trust in these systems.

Tim Hwang: All right, I’m going to move us on to our third segment of today. Mihai, I think because we’ve got you on the show, we want to give you the opportunity to plug your projects. Just to quickly set up the context: we’ve talked a lot about MCP over the course of many shows at MoE, and I think one of the reasons I like the topic is it’s this fascinating question of how new standards are going to emerge in this space and how adoption occurs in open technologies. Mihai, you’ve been working on a specific project I understand called MCP Gateway. Do you want to give our listeners a quick overview of what it is and why you think it’s important?

Mihai Criveti: Yeah, sure. So first, let me give you a bit of an idea of how the project started. We like to treat AI agents as insider threats. So every time an AI agent interacts with the system through a tool, we believe that’s actually a potential insider threat because it is an input: you’re giving it a text, that text goes to your tool, and if that tool just happily executes the input from the user, that can drop your database, delete a database record, delete bits of your code. So we want to have a way to provide observability, guardrails, monitoring, security, authentication, authorization, even things like user impersonation—saying, “Hey, do you want to access this?” and then it goes on your behalf.

We’ve been building a similar system for the last year, year and a half, but we haven’t done something very important: we didn’t go open source. We said, “Hey, we believe this is the standard for how an agent should interact with tools.” Anthropic did, and they did a great job with the Model Context Protocol, and they released it as a standard way to decouple your AI agent from your tools.

However, there’s a couple of interesting things in the mix here: the protocol came out, there’s already version four of it, there was a draft, there’s multiple implementations, there’s like 15,000 open-source servers somewhere in the community all implementing different standards, different versions, or incomplete implementations of the MCP protocol. Some of them don’t have things like authorization and authentication and accounting and all the other things.

So we’ve created the MCP Gateway and Registry as an open-source project, which gives you the ability to first federate multiple servers in the same gateway. So if you have, for example, resources, prompts, tools from multiple servers, you can combine them into a virtual server with its own authentication, its own authorization, retry mechanisms, observability, monitoring, rate limits, health checks, and a plugin system so you can plug in, for example, pre- and post-hooks for every operation. Before a user input, you could trigger Open Policy Agent, or you can trigger a PII filter, or after a specific input you can again do the same thing. So it’s meant to be a centralized point that can give you control over your context, whether that is tools, resources, prompts, but also a mechanism that lets you convert between different protocols. So if your tools aren’t already written as an MCP server—maybe you have a REST API—you can connect that REST API to the gateway, it will turn it into an MCP server and give you the same control over it.

Tim Hwang: So is it right to say—I don’t know if this is putting it too simply—but it almost kind of feels like MCP is a standard, but we kind of know that the world is going to be really, really messy when we actually put MCP into action. And so what you’re attempting to do is give people a lever on controlling that craziness, right? There’s going to be so much variance that you will want checks at every step.

Mihai Criveti: I think to some extent, yes. We also want to give a point where you can plug in your plugins, add your own spin to control or even change the input and output going to these MCP servers.

Tim Hwang: So, Gabe, you think a lot about open protocols, is my understanding. Do you want to think about or respond to Mihai’s project? I’m sort of interested in your take on the backdrop of everything we know about open source, open tech. Is it just the right time? Do we see MCP Gateway-type projects in other domains? Like, how does this all look situated in history?

Gabe Goodhart: Yeah, I mean, I think this is an excellent time for a project like this because we’re at this blossoming of a new open-source standard for a novel interaction pattern, which in this case is fetching context for your AI models. And if you think about historically how other protocols have emerged and the technology implementations that have arisen around those... how much bad HTML is out there on the internet, right? And that put the onus on the browsers to be able to just be wildly robust to all the things that could go wrong. How many HTTP servers out there that occasionally just don’t return, occasionally just randomly spit out a 500, or return some malformed stream of packets that everything on the client side has to be robust to.

So to your point, Mihai, we are very much in these early days. I imagine what we’ll see, just like with HTTP servers, is that for every favorite programming language, we’ll eventually see the de facto standard community edition and potentially enterprise edition of the MCP server and client library emerge, probably one or two small handfuls of them that have their passionate followers and their differentiators. And there will start to be some coalescence because people will stop being interested in implementing the server layer of the MCP and start being interested in what’s sitting behind it.

But I think for a while right now, we’re going to be in the Wild West of actually getting the bits to flow correctly and staying on top of the spec as it evolves. Because also with anything like this, there’s sort of an exponential decay in the volatility of the spec itself. I think, Mihai, you pointed out in the blog post you wrote that they’ve already deprecated one of the primary transport protocols for MCP—SSE (Server-Sent Events) is going away and being replaced by Streamable HTTP. But so many people have already implemented their streaming MCP servers on top of SSE. Like, “Huh, what do we do with that?”

So I think having a piece of, for lack of a better word, middleware in the MCP domain where you can actually coerce standards and isolate—basically manage the chaos in a single central place—is an excellent implementation tool for engineers trying to build this ecosystem themselves, and a good thing to maybe help shake out some of these inconsistencies across implementations, where we can actually start to clearly identify, “Hey, look, every time somebody attaches an MCP server written with this random MCP library in Elixir, it turns out that I have to enable all this glue in the gateway to make it work. So maybe we should not use that one, or get that community to step up their game,” etc. So I think there could be some really strong benefits both at the leaves of this graph, where the developers are trying to build things, and at the connective tissue of this graph, to start isolating patterns.

Tim Hwang: Okay. Kate, I’ll give you the last word here. It strikes me that this topic actually weirdly goes back to what we were talking about a moment ago with ChatGPT Agent, which is that sometimes the fault of the agent failing is that the model wasn’t smart enough or the agent wasn’t smart enough. Sometimes it’s just because the internet’s really messy and the technical implementation is really bad. How do you think about that from a UI/UX standpoint?

Kate Soule: You know, we’re talking a lot about trust, and it’s almost like the agent’s going to take the blame for all of this messiness if we don’t find a way around it. No, absolutely. I think this speaks to a much bigger trend, which is: obviously 2025 is the year of the agents. Everyone’s really excited about what agents can do, but we’re finding that maintaining these agents and maintaining these systems is really difficult, especially given how quickly everything’s moving. And there’s so much performance that’s tied up, as you said, Tim, in how this is all architected and built—that’s beyond the pure LLM weights behind the scenes. That’s obviously going to dictate performance too.

I think, Mihai, this is a great example of emerging classes of middleware that augment existing agent frameworks and protocols in order to better improve how we scalably build agents, how we maintain these, how we build these in a secure manner. And I think this is just the tip of the iceberg of this kind of class of projects that’s going to have to emerge.

Tim Hwang: Mihai, any final thoughts? And if people want to learn more about the project, where should they go?

Mihai Criveti: Well, look, if you want to learn more about the project, go to github.com/ibm/mcp-context-forge and you’ll be able to get started with our MCP gateway implementation. If you want to contribute, we also have a detailed roadmap as well as an issues page where you can bring in new features or say, “I’m missing this feature,” or “I’m having this issue.”

I think, Kate, to your point, decoupling this logic from the agent and letting a piece of middleware handle things like your retry logic and all of the filtering is going to simplify your agentic framework as well. And there’s going to be a lot of duplication between all of these agentic frameworks: LangChain, LangGraph, AutoGen, ChatGPT, IBM... all of these frameworks have their own specific way of doing things. But if you manage to decouple the logic, then you’re going to be able to consolidate that work into one library.

Tim Hwang: All right. Well, I think that’s the time that we have for this panel today. Kate, Gabe, always good to see both of you. And Mihai, we hope to have you back on the show sometime.

Mihai Criveti: Anytime. Thank you all.

Tim Hwang: All right. Thanks, everybody. We’re going to go ahead and move to Ryan. So today we’ve got Ryan Hagemann joining us. He’s the Global AI Policy Issue Lead. Ryan, welcome to MoE.

Ryan Hagemann: Great to be here. Thanks for having me, Tim.

Tim Hwang: So I know we want to scramble this segment because this news just broke yesterday: the White House has released its AI Action Plan, which is sort of a national strategy for Artificial Intelligence. I know there’s a lot going on; I was looking at the document this morning—just a lot of different recommendations. Do you want to just walk us through what this exactly is, and is it important? Should we be paying attention to it?

Ryan Hagemann: Yeah, sure. I’ll walk you back backwards. The answer to the second question is absolutely yes. This is something that not only IBM, but pretty much everyone in industry and here in D.C., and frankly around the world, has been anticipating and looking forward to for basically the last six months since it was originally announced. It’s in a way... yeah, this is the policy Super Bowl for policy nerds and policy wonks.

But the short version is: what this document basically does is it lays out the Trump administration’s policy agenda as it relates to artificial intelligence. And part of the reason I think you haven’t seen so much action in Congress is this is also the starter pistol for legislative action in the future—probably not by the end of this year, but probably moving into next year.

But what is the plan? In short, the plan is basically how the administration wants this Congress and the existing agency apparatus in D.C. to approach thinking about AI. So there’s something on the order of about 134, 135 individual actions that are recommended agencies take. This plan was also accompanied by three executive orders from the president, which he signed yesterday, which provides a little bit more clarity on what exactly some of the agencies are supposed to do with respect to some of the more important features of the plan.

But basically, the plan as outlined goes a little bit something like this: there’s three pillars. There’s accelerating AI innovation, there’s building out American AI infrastructure, and then there’s leading an international AI diplomacy and security. And what this basically boils down to for IBM is a lot of positive momentum.

I think the big thing for us is there was a specific call-out on the value and the importance of open-source and open-weight model development and deployment, which is frankly something that we’ve been asking for from the Administration and from Congress—one of our major pillars that we advocate for here in D.C., the need for policymakers to make sure that the open space and the open community kind of remains hands-off from policymakers.

Tim Hwang: That’s like a big shift, right? It kind of feels like for a little bit there, there was sort of a debate over, “Well, we’ve got these powerful new technologies; is it kind of unsafe for them to be open?” And it feels like here they’re kind of very much affirmatively saying, “No, like we want open. We actually think that’s a really important thing to happen.” So it feels like almost there’s a shift in that discussion, and it’s landing very firmly on the side of open, which I think I’m personally very excited about.

Ryan Hagemann: Yeah, I mean, me too, and I think a lot of us at IBM would view that as a very positive development. I would say where it kind of moved us from was a space of uncertainty, because no one really knew how the Administration was going to come down on this. The last Administration never really took a super strong stance. There was a great report from the Department of Commerce that looked at open model weight development; didn’t make any super strong statements, but it also didn’t say “open source bad,” which was kind of a win at the time because of the uncertainty. So it’s a little bit of a sea-change shift, if only from moving from neutral to positive. But it’s a big positive, and it’s a big signal for us in industry that kind of this direction that we’ve been taking is now essentially getting a little bit of kudos from policymakers, which will be good for the next few years of administrative action.

Tim Hwang: And so you mentioned that there were these executive orders that were signed. I think one of them had to do with energy. I know IBM—we all frequently on MoE talk about the compute buildout and infrastructure side of this. Sounds like there’s been a clear path to really build a lot more. It seems like.

Ryan Hagemann: Yeah, I mean, that was the gist of not only the section on energy and data center buildout in the plan, but in the executive order as well. It basically... it doesn’t touch IBM so much because we’re not really in the data center buildout game, but we definitely benefit from more data centers, more energy capacity on the grid. That executive order basically just streamlines a lot of regulatory approval processes. It gets kind of in the weeds, references a lot of different statutory authorities and existing legislation. But the short version of it is basically: to the extent that there are opportunities for us to be building more capacity on the grid, to be building more data centers, the federal government shouldn’t be standing in the way of that. They should be finding opportunities to help expedite that and make it happen quicker, because America’s going to need a lot more energy if we’re going to be doing a lot more LLMs in the future.

Tim Hwang: Yeah, for sure. Anything else that you were pleased to see? It sounds like you focused a lot on the open side. Anything else that you think is in this action plan that folks should focus on or even pick up the PDF and read?

Ryan Hagemann: Yeah, I mean, too much to mention in a couple minutes. But I do think the one other thing that really struck a chord with me was under the AI diplomacy bucket. There’s essentially a dictate for the Department of Commerce to figure out, with partners and industry, how to kind of create a larger export package of a full American AI tech stack—everything from data center build-out to model developers, model deployers, get everyone together in an industry consortium, essentially package that all together so that the Department of Commerce can then use those export packages to push out American tech to the rest of the world.

Something that we’ve said a lot that this administration can do in AI is really promote that idea of exporting American AI and American technology. So that’s an opportunity for IBM; we’ll see what comes of it. But that, I would say, is the other really big call-out here that IBM, I think, can and I hope stands to benefit from over the next year or two.

Tim Hwang: That’s great. Well, I know we just have a few minutes left. Action Plan is out, executive orders have been signed. What comes next? I assume you’re not saying, “Well, we’re all done with this AI thing here in D.C.”

Ryan Hagemann: Yeah, I mean, I kind of wish I could take vacation for the rest of the year, but I think the reality is there’s a lot of requests for information that are going to be... this AI Action Plan basically says, “Here’s what we’re going to do.” Now they actually have to do it, which means we’ve got a lot of responses to provide, a lot more education and engagements to do on the Hill as they think about legislative packages to help make some of this a reality. So, like I said at the outset, this is really just the starter pistol for the start of the race, and I think now it’s really heads down, books open, pens and paper in hand—we’ve just got to get to work doing it.

Tim Hwang: Well, Ryan, look, I know there’s a lot going on. We’ll have to have you back on the show as this all kind of unfolds. It’d be really good to have your voice in here as we kind of track what’s happening in D.C. on all this stuff.

Ryan Hagemann: Always happy to stop by for a chat.

Tim Hwang: Cool. Thanks, Ryan.

Ryan Hagemann: Thank you.

Tim Hwang: Thanks for joining us, listeners. If you enjoyed what you heard, you can get us on Apple Podcasts, Spotify, and podcast platforms everywhere. And we’re going to see you next week on Mixture of Experts.