Shobhit Varshney: I think double WWDC would start being called, why was design changed?

Chris Hay: I really want Apple to focus on having a good platform, right? The mobile devices, the iPad, the Mac all coming together, and I, I actually think that’s the bigger story of WWDC.

Kaoutar El Maghraoui: So I think Meta is really betting on securing its foundational AI supply chain with this acquisition, which I think is, is very strategic and it’s the right move for them. Infrastructure matters as much as models, training, data evaluation, human feedback, and I think this is where the AI wars are being fought right now.

Chris Hay: Tim gives his homework every time we appear. He is like, you need to read the Apple Intelligence one. You need to read, but Sam Altman’s blog post and you’re like, okay. I’m gonna stick it through o3-pro. I might as well test that out. 13 minutes later. 13 minutes later for it to read the paper. You’re like, come on, dude. I could have read that myself.

Tim Hwang: You’re saying like quantum’s like already here. It’s kinda what you’re saying.

Oliver Dial: We’re saying you’re probably gonna show quantum advantage, so doing something better, faster, cheaper than a classical computer. By 2026 and actually think it’s gonna be even sooner than that.

Tim Hwang: All that and more on Mixture of Experts, a Think podcast. I am Tim Hwang, and welcome to Mixture of Experts. Each week, MoE brings together the sharpest team of researchers, engineers, and product leaders you’ll find anywhere in podcasting to discuss and debate the biggest news in artificial intelligence. Today I’m joined by a rockstar cast: Chris Hay, Distinguished Engineer and CTO of Customer Transformation; Shobhit Varshney, who is head of Data and AI for the Americas and Kaoutar El Maghraoui, Principal Research Scientist and Manager for Hybrid AI Cloud.

Tim Hwang: As always, we have a ton to talk about. We’re going to talk about a bunch of news out of OpenAI, a huge deal for Scale AI, and you should stay tuned for the end, where we’re gonna have a special segment focusing specifically on a very interesting announcement that just came out about quantum. Before we get to all that, I want to talk about Apple’s WWDC, their annual developer conference—the big showcase for what Apple is going to do. There were a bunch of announcements that came out. But first, I just want to go around the horn and get everybody’s opinion on what, coming out of the keynote, we’re still going to be talking about in six months. Shobhit, I’m curious what you think.

Shobhit Varshney: I think “double WWDC” would start being called. Why was design changed? I think this may end up being a Windows Vista moment. I’ve been rocking the new Glass OS for a while now, and it’s just not quite ready yet. And I am very confused: when the glass design is not quite ready, they’re comfortable shipping that, but they’re saying they can’t ship AI because AI is not quite ready. They have to pick a lane.

Tim Hwang: Yeah, for sure. Kaoutar, what’s your reflection? Anything that we’re gonna be remembering this WWDC for?

Kaoutar El Maghraoui: I think the still highly competitive pressure from OpenAI, Meta, Google, Samsung—especially kind of emphasizing open—Apple’s late AI entry puts this under a lot of pressure. Questions around: Did Apple’s partnership with OpenAI evolve or backfire? How does Apple’s AI stack compare to what Meta and Google are doing with their Llama, Gemini, and so on? Especially another point of concern and disappointment for many is the overhaul around Siri and the AI advancements they’re claiming here, which seem to be still lagging behind the competition out there. So I also agree with Shobhit around the liquid glass interface. I think it’s a change, but it’s also a final dig. There is that—really making its pseudo-reception and any potential flaws more immediately apparent, and maybe even more controversial.

Tim Hwang: Yeah, for sure. And last but not least, Chris, what did you think of the show?

Chris Hay: Even darker dark mode. In fact, I didn’t realize how dark my dark mode was, and then I was like, “Oh, I can get dark mode even darker. That’s what I want.” So I thank you, Apple, because I love my dark mode, and the even newer, even darker, darker dark mode is gonna be awesome. So I’m excited about that. Apple, I’m just welcome news. But on a more serious note, I actually think that the ability to access LLMs on-device is gonna be huge as part of the framework. Otherwise, there would’ve been a risk of loads of apps trying to install small models onto your phone, et cetera. And I know some people have been doing that already. So actually, just ubiquitous access to on-device LLMs and being able to easily hook that into your own applications—we can argue about how good the AI on those devices is, but the fact is Apple does have the best hardware. Apple silicon is incredible. So I’m excited to see new applications via these SDKs, and I think that’s a little bit of a glimpse into the future because I think we’re a little bit too low-level at the moment, and Apple’s gonna lead a little bit of that way on framework access.

Tim Hwang: Yeah, for sure. And actually, that’s a great place to kind of pick up because I think that, you know, I was looking at, say, The Verge coverage of WWDC, and there’s a long list of announcements, but in pretty stark contrast to where they were on the last WWDC, the AI stuff is actually weirdly not as prominent or not as talked about. Certainly people are complaining a lot more about the UI/UX design. But if there’s anything that Apple seems to be doing on the AI side, it’s opening up new surfaces for the AI to interact with. So, one, any developer can go and play with their models. There’s also this new idea that Apple Intelligence will be able to see what’s on your screen and interact with it, which I think is also pretty interesting. So, Shobhit, maybe a question for you is: Apple seems to almost be admitting, “Look, we’re not necessarily going to be a leader in the model game, but we might try to use our unfair advantage in terms of our ecosystem.” Is that the right way of thinking about what they’re trying to do here?

Shobhit Varshney: So I think it’s their only play left. They had a lot of one-on-one sessions post-event announcements with different YouTubers explaining and rationalizing why they’re missing out this year on AI. They went through two different architectures, and the first one could do a lot of the stuff they wanted to announce, but it was not quite primetime ready for them to roll it out. So their only other choice at this point was to make sure we do subtle things across the entire ecosystem, having Apple Intelligence move away from becoming the oxymoron of the year to actually starting to deliver some value. And that is the future direction they have to take. They need to come to a point where if they champion the small models running on-device—and hence they can tout a lot of security and stuff like that—in the current phase, the Google Gemini Gemma models, the 3-billion parameters that run on-device, 4-billion parameters, those are actually pretty good right now, and they’re actually better than what Apple just released. So Google is still ahead in the game, even on-device. It seemed like a lot of the stuff they were announcing was catching up. I was like, “Ooh, I can change the wallpaper in my iMessages, like WhatsApp did like a decade back.” They have to get away from these small incremental changes to something that adds some real intelligence. There are very few companies where I would trust all of my personal data with, and Apple has definitely won that trust for me. So I’m very comfortable with my kids having access to an Apple phone—the parental security restrictions and stuff like that. They’ve done a really good job around it. So when we have, as a family, trusted Apple with so much trust, they have all the data, and I would rather have them deliver personalized experiences to me versus all the competitors. It’ll be a nightmare for me to even think about, say, DeepSeek or some of the other competitors having access to all of my data. So we have made our choices: Gmail, right? I’m okay with Google having images of my passport and my medical records—all that goes through my emails and stuff. Both Google and Apple have an extraordinary opportunity to hyper-personalized intelligence that OpenAI or AWS and stuff just cannot get close to. But I think Apple, in this case, has to really double down. They could not have afforded a gap year in AI, and that’s exactly what they’re doing right now. If I was in that, like, heads will—heads should roll on these kinds of announcements.

Tim Hwang: Yeah, for sure. Kaoutar, I think one of the most interesting parts of this discussion—and the last time we really talked about Apple was the Daring Fireball Gruber takedown of Apple—and I think the narrative through all of this is it’s almost like what Google was maybe six to 12 months ago, where everybody was like, “How can you not be winning in this space?” Chris just talked about this incredible hardware they have access to; Shobhit just mentioned all the incredible trust they have access to. What’s your diagnosis of Apple? What’s going wrong? By all rights, they should be crushing it. I’m curious from your vantage point, how you diagnose the issue.

Kaoutar El Maghraoui: Yeah, I agree. I’m also a bit disappointed in terms of this underwhelming AI progress. So for Siri, I think they delayed their Siri overhaul, which is a big significant point of disappointment. I see that across many reactions online. They had promised a much more versatile Siri, but comments say it’s not ready. I think they’re trying to be very cautious or maybe very conservative in their approach. Is it because they’re worried about privacy-first approach, where they’re trying to be really careful and don’t want to mess up things with AI and the privacy aspects that they take so hard, as Shobhit mentioned? I also trust their platform. I love all the integration they have, the control they have over the entire stack, all the way from the apps down to the silicon. It’s a very strong position they have. But I feel they’re overly cautious here in terms of their AI strategy, especially with all the hype happening elsewhere. What’s holding them back? I also don’t understand what’s going on here, why they’re taking a very conservative approach. And especially if you look at the paper—I think we’re going to talk about it—maybe the paper that they released was also timely, maybe the timing of the paper right before WWDC where they had all this skepticism about AI, maybe to justify why they’re very incremental and cautious, taking a slow move. In WWDC, they focused on a lot of other things other than AI. They say, “Okay, there is AR,” but there’s all this other stuff like productivity, visualization, integration, and the liquid glass—a lot of focus on that. But all other stuff that I think they’re positioning is also important, but AI is also going so strong elsewhere, and we need a strong position from Apple on this.

Tim Hwang: So this might have been just regrettable timing, who knows? But basically, Apple also released a paper that got a lot of chatter this same week entitled “The Illusion of Thinking.” In some ways, it’s a critique of what reasoning models are doing. And the narrative online at least was, “Well, it’s weird that these researchers don’t even have confidence in the technology they’re pushing.” I don’t know, Chris, how you’re going to respond, but I think there’s a question of: Does the company culture even really believe in the technology they’re pushing here?

Chris Hay: I like Apple’s approach to this, and I have to say we are in a hype cycle just now—I know we’ll get to the paper a little bit later—but I really want Apple to focus on having a good platform. The mobile devices, the iPad, the Mac all coming together, and I actually think that’s the bigger story of WWDC, which is that they’ve unified, at least on the numbering system for the operating system, but actually I think they are trying to make this a stack where I can easily go from one device to another, and they have control of the full vertical device and the platform, and they all connect. And we can mock the liquid glass, but actually they’ve done a very clever thing here, which is they’ve built a design system that allows you to have consistency regardless of the form factor. Now, that’s going to be important in the future because we are going to want to switch from our iPhone to our iPad to our Mac and not feel as if we’re on a different operating system. So I weirdly think they’re setting themselves up for the future at a platform level. And I also think they’re setting themselves up from an AI perspective because they’re focusing on the SDKs, the hooks, and how people are going to hook into that. Yes, their AI clearly is not matching up to the ecosystem and the platform that they’re building, but they’ll catch up on that. So I would rather they fix all of these things anyway and prepare for the future, and then we’re going to have great developer experiences. I don’t feel as if I’m missing any AI stuff on my phone. If I need to go speak to an LLM, I’ll just bring up the ChatGPT app. It’s fine. So I’m okay with the approach they’re taking. But full disclosure, I’m a full-on Apple fanboy, so, you know, it’s fine.

Tim Hwang: So, Chris, I’ll summarize what you just said: iPad is more like Mac; Mac is now more like iPhone; and iPhone is more like Android. Is that fair?

Chris Hay: No, I am not taking your Google-centric view on this show, but with your love for Gemini—and if you ask Gemini to code, then you might as well get a 3-year-old bashing at a typewriter because you’re going to get equivalent code coming out of it. And I get that you love Gemma, and that fact, and I do love Gemma as well. But the reality is, these smaller models—Apple will catch up on that in time. So I just think there are two approaches: you can chase the AI part, and everybody is chasing that and that’s important, but actually they’re chasing their platform and trying to make that consistent. And I don’t think that’s a bad move.

Shobhit Varshney: So I think Windows has done a better job at opening up the Mac OS with more MCP-based things that I can actually tap into. So my apps that I’m building are actually able to take actions. I was hoping that Apple will create an ecosystem where we can start to build apps on top of that. I mean, we’ll see what comes out of this. I think there’s huge potential we can do there. Second, my challenge right now is if I use a different app, say ChatGPT on my phone, ChatGPT does not have the rich data about me. I don’t want ChatGPT to follow me and track my location preferences and emails and all of that quite yet. So instead of replicating the trust I built in Apple or Gmail with Google and stuff, or Google Photos, I don’t want a third-party app to give me a service that’s not hyper-personalized to my needs. Then there’s a huge missed opportunity by Apple. Hopefully next year.

Kaoutar El Maghraoui: Yeah, hopefully. And I think also, even in their image playground, for example, the features like the photorealistic image generation or these advanced generative features, I think it would be nice for them to have that. And of course, on Siri, I agree with you, Shobhit, because of their hand on the data, the customization is going to be a big leap if you can integrate that all together with their Apple Intelligence with more advanced conversational capabilities and generative AI features. But hopefully, I think they’re delaying because they want to get it right. So that’s what I’m thinking why it’s taking. But I think Chris’s point on getting the platform right is also a very important point.

Tim Hwang: Yeah. Well, we’ll have to wait and see. There’s more coming, and I mean, I think with Apple, they can just keep taking more and more shots on goal, right? I think that’s one thing people frequently miss: they can get it wrong for a very long time before they get it right and still largely be okay.

Alright, I’m going to move us on to our next segment. Some interesting news coming out of OpenAI this week, both of a product nature and also of a philosophical nature, and I kind of want to talk about both stories together. So one announcement, not unexpected, is OpenAI announcing the availability of O3-Pro, which is basically the more advanced version of their reasoning model that has taken the world by storm, and also some pricing updates about how cheaply they’re able to offer a lot of their existing models. And secondly, Sam Altman published this essay called “The Gentle Singularity,” where he makes the argument that we’re already living through the singularity—it’s not a future thing, it’s happening right now, and the world is going to feel very different over the next decade. Kaoutar, maybe I’ll throw it to you first: When you play with O3 and O3-Pro, I kind of want to ask the question, do you agree with Sam Altman that we’re already living in the singularity? Basically, that it’s wild that we have technologies that can do what O3-Pro can do. How do you think about that? How much do we buy what Sam is telling us?

Kaoutar El Maghraoui: Yeah, a very good question here. I think of course the O3 model has brought a lot of advanced features, especially on the reasoning capabilities, the multimodalities—really nice features there, especially focusing on the reasoning aspects. But if you look at Sam’s essay, what I think is, it’s a very optimistic—he’s over-optimistic. He’s softening the idea of this runaway AGI, saying that this won’t be a Terminator; it’ll be an assistant helping you solve hard problems, the super-intelligent that’s going to exceed our intelligence. So it’s a smart narrative shift. I think it’s more politically and socially acceptable than machines can take over. But maybe he’s overall optimistic. There are still a lot of challenges that we have to solve, and I think he’s trying, especially to align with OpenAI’s vision—they’re trying to deliver on the vision of reasoning tools that help humans think; it could reshape science, economics, and education. But I think there are still a lot of issues around misalignment, the impact of these things on jobs, equal access and democratization of AI, because this might also intensify the division globally—who gets access to these technologies, my privilege, certain people versus or certain nations versus others. So there are a lot of other issues that are much deeper than the nice, utopic vision that Sam painted. So I feel he’s over-optimistic, but there are a lot of issues that still need to be resolved.

Tim Hwang: Right. You’re almost like it’s still ahead of us in some sense. Shobhit, thoughts? I don’t know if you’ve played with O3-Pro. Any impressions?

Shobhit Varshney: So the first thing I did was look at Apple’s paper that said, “Oh, these models can’t think,” they had the full prompt for the Tower of Hanoi problem. I took the exact same thing, gave it to O3—slam dunk, like you just crush it right away. So within a few days, OpenAI just came swinging. I was like, “Guys, stop making excuses. AI is working really, really well.” And if you think about this from an enterprise perspective for us, it doesn’t matter that the world has somebody who won Nobel Prize or somebody who’s like Einstein-level IQ and stuff like that. As an enterprise, it’s very important for me: Can I take that capability and apply it to a particular system? Can I solve a problem? Can I deliver economic output? And that requires us to have a hierarchy of intelligence. Generally in our organizations, you’re an obscenely overpaid CEO at the very top who’s an expert in multiple areas. But by the time you get to the accounting department, the procurement department, HR, and so on, you have a really small model that went to a community college that has been doing accounting really well. And that’s the right price point and capability intelligence we’re expecting for that particular job role. So I don’t think O3 coming out with a brilliant model is something that’s going to drastically change the narrative within enterprises quite yet. We can deliver an insane amount of value with the existing intelligence we already have, and it has to be more ROI-driven intelligence versus O3-Pro. It’s quite expensive. But the fact that it just came and rained over Apple’s parade on one side—somebody saying, “Hey, AI ain’t quite ready yet,” the other one saying, “Hey, just come on, stop talking about this”—crushing through this very quickly. Two ends of the spectrum of the same story.

Tim Hwang: That’s right. And I think, Chris, I’m really glad you brought up that Apple paper again. It definitely looms large in my mind. It offers a challenge to what we’re seeing with the reasoning models. I don’t know, Chris, what’s your take? Is Apple just wrong in what they’re arguing here, given everything we’re seeing out of OpenAI?

Chris Hay: Well, I did exactly what Shobhit did as well. I took the paper and gave it to O3. Because I was like—that’s what I love about this show. Tim gives us homework every time we appear. He’s like, “You need to read the Apple Intelligence one. You need to read Sam Altman’s blog post.” And you’re like, “Okay, I’m gonna stick it through O3-Pro. I might as well test that out.” Thirteen minutes later. Thirteen minutes later for it to read the paper. You’re like, “Come on, dude, I could have read that myself.” So I love O3-Pro, but it does take a long time. On the Apple paper, I think it is really interesting. Actually, if you ask O3-Pro about it, it’s kind of like, “Yeah, the core of the paper’s about right, but actually they’ve only gone with a single prompting strategy, et cetera. Is that really going to pan out over time?” So I think even the O3-Pro was a little skeptical of the paper. I think there are some relevant points though, which is that puzzle-based approaches—so if you look at the Apple paper, they were talking about doing things like the Tower of Hanoi, et cetera—I think these puzzle-based approaches are important because there’s a lot of contamination within these models; they have seen things before. And therefore, how do you distinguish between regurgitation and true thinking in that sense? But the flip of that is, I mean, if you take some—especially let’s take code, for example, and I’ll go back to my favorite model, which is Claude 4 Opus and Claude 4 Sonnet—I mean, even without thinking, these models are able to just spit out unique code, end-to-end thousands of lines of code, and it’s just perfect. Oh, okay, you’re going to have to change some stuff, et cetera, but it is incredible. So I think even if it’s simulation and they’re saying the reasoning isn’t going that far, the reality is it is able to do the tasks that I wanted to do to some extent. And maybe that’s the thing to focus on: Is it providing value? And the answer is yes. Now, the other thing on that paper that I think is useful is they’re saying that the models sort of collapse in on themselves right after it generates enough tokens, for example. So when it’s reasoning about something, after a while it will sort of go in a bit of a spiral because it doesn’t know the answer. And there’s a bit of a fundamental flaw to this, which is it doesn’t have access to things like tools in this case. So there’s not a freshness of information coming in, so it’s sort of going into a spiral. So I understand the paper and I get where they’re going with this, and I get what they’re saying about that. Actually, if you take multiple samples, you can get to, you know, “fire the pass” side of things very similar to what you get from reasoning. But I think the reality is diversity of data in that case becomes more of a thing. So I am somewhere in between on that paper.

Tim Hwang: Hmm. Yeah, that’s actually a really interesting response. And yeah, I think that’s like—I’m just reviving a phrase that I don’t think we use anymore nowadays, which is like people used to say, “These are stochastic parrots.” All they do is copy. And I guess my response to this is kind of like: I don’t know if debating what intelligence really is is very helpful. Because in some respect, it’s like, “Okay, say it is copying; it’s incredibly useful for all the tasks that I need and can have a huge impact even with that in mind.” And I know, Kaoutar, it looks like you might want to jump in, but I feel like that’s one of the interesting tensions we have with this era of language models: yeah, they might just be copying in some respects, but it really feels pretty strongly like it’s a kind of reasoning that solves problems, and I almost can’t get too fussed about it.

Kaoutar El Maghraoui: Yeah, and if you look at this paper, “The Illusion of Thinking,” some of the things that I thought: the methodology they used, they cherry-picked these tasks. So, for example, the logic of the puzzles that was chosen—even I think humans struggle sometimes with these things. So does that represent the full spectrum of reasoning tasks? And another thing: these artificial constraints—things like, it did not, for example, in this paper, Apple did not allow the models to use coding as a tool to solve the problems. For humans and also for powerful AI models, writing code is a fundamental way to tackle complex logical problems. So restricting this capability could artificially limit the model’s performance and misrepresent the true reasoning potential. There were also things like the output token limits that they had—some tasks assigned reportedly exceeded the model’s token output limits, essentially setting the models up for failure. And it’s kind of like an all-or-nothing grading thing—the strict, complete accuracy collapse metric—I think might be too harsh. So I think the approach—I feel there are some flaws in the methodology they use. Also, some of these exaggerated claims or misinterpretation of the reasoning. So, for example, the semantics of reasoning—some people, from the analysis I looked at, argue that the paper’s title and conclusions are kind of inflammatory and misrepresent what LLMs are actually doing. While they may not reason in a human conscious way, their ability to perform complex calculations and generate coherent logic and use tools is a form of reasoning. Even if it’s based on pattern matching, it is also a type of reasoning, and really that’s going to evolve and improve over time.

Shobhit Varshney: So, Tim, I’m going to start a whole campaign around “AI Rights Matter.” Like, we should not discriminate against AI. And the reason I say that—

Tim Hwang: I thought we started with O3-Pro, we talked about the Apple paper, and now we’re on Shobhit’s campaign for AI rights. Here we go. Hermione Granger here. Go on, Shobhit.

Shobhit Varshney: So here’s my take on this: I recently we had a client combine the CHRO and CIO function into one. So one of my CHRO clients got spooked, and she and I were sketching up what a trusted employee is in a company today—human employee. And I was trying to extrapolate from there where a trusted AI employee would be. So if you think about it, all of a sudden when we are hiring people, newcomers into the company, fresh hires from the industry or interns, we do a really good job of carving out the right set of tasks that I can delegate to an individual intern or a new hire without having to give them step-by-step instructions and handholding them all the way through. We already have mechanisms of thinking about it in that way, but somehow when we get to an AI model and if it’s performing at 95% accuracy, we are still not satisfied. I think we have to do a better job of appreciating what are the strengths and weaknesses of a new hire and give them, allocate them work accordingly, and the same thing should extrapolate over to an AI model as well. There will be a place to get a very high-end PhD-level O3-Pro; there’ll be a lot of places where we need a much, much smaller model. I’m doing some multi-agent frameworks in production for some clients. I need the orchestration agent to have the logic and think through how to delegate this down to smaller tasks. Right now, anything less than a 70-billion parameter model is not working out well for me. So I’m struggling with how to influence the reasoning model to do things a certain way. And right now, the bigger models are just completely slamming this through. So we’ll get to a point where smaller models in the enterprise environment, we would be able to influence the reasoning and make it work. There’ll be much, much smaller models like intern, and we’ll have a better position for what accuracy and how should we measure the model into.

Tim Hwang: And I, yeah, I think these kinds of discussions we’re going to increasingly get asked, right? It’s this kind of weird line between, “Well, what is this that the model’s actually doing?” But I think to your point, it’s this battle between pragmatism and then kind of having to know. And I feel like that’s a classic theme in the AI space.

I’m going to move us on to our third segment. I think one of the things I keep looking for every time one of these AI headlines comes out is the fact that the transaction dollar amount is just getting bigger and bigger. And this week was no exception: a huge USD 15 billion transaction announced between Meta and Scale AI, which, if you’re not aware, is one of the big players in the data annotation space. So this is a massive transaction. Chris, maybe I’ll throw it to you. I guess the question with all of these is: USD 15 billion—so why is Meta so interested in this?

Chris Hay: I think probably they want to try and get ahead in some way, shape, or form. So the reality is, I love the Llama models; I think Llama 4 is great, et cetera, but I think there’s a little bit of frustration that they’re not maybe ahead of Anthropic, they’re not ahead of Google, and they’re not ahead of OpenAI at the moment. However, with that being said, they are great models, and they’re some of my go-to models all the time. So I think this is, let’s put another bet in the ring there and run something else in parallel and then hopefully get to where they want to go. And I think the reality is, this is being touted as a winner-takes-all market—I’m not convinced it is—but the reality is that you have to spend that type of money to be in that game. It is fierce competition. So if that means attracting new talent and bringing that in and being able to set up a lab focused on superintelligence, I don’t think that’s a bad idea because there are probably two competing things also going on at the moment, which is they need to get AI into their products, they need to get AI out to their consumers, et cetera, and that’s different from actually, “How do I set myself up for the future?” So by actually splitting this up a little bit and saying, “Okay, you’re going to be focused on superintelligence, and we’re going to continue to get Llama out to folks and get it into our platform so people can use it and sort of gain productivity benefits,” then you don’t have competing goals and intentions, and the folks can be focused on different things: one on the research element, one on the product. And so I don’t think it’s a bad thing.

Tim Hwang: Well, and on that, I think Shobhit, you want to jump in? I mean, I think what’s curious about this transaction is, in the past, some of the really big deals have been around technical leaders in the space—you know, Noam Shazeer, big acquisition built around that—like obviously the kind of key leadership of OpenAI has been involved in all these big deals. What’s interesting about Wang is that he’s not necessarily a noted machine learning expert or pioneer. I think that almost suggests there are other things leading Meta to this acquisition. I don’t know if you agree with that analysis.

Shobhit Varshney: So Meta is trying to make sure that LlamaCon does not become a WWDC. Sure. But Alexander and Lucy co-founded Scale AI; both of them are brilliant, both of them the youngest billionaires. Like, he is worth like USD 4 billion plus at this point; he is like 28 years old. What a phenomenal story. He has—I would argue that he has done an exceptionally good job. He himself has been a math prodigy, coding genius, and whatnot. So I would say he has had the street cred comes from a very talented family as well. So I think he has done it the right way. And we’ve worked with Scale AI quite a bit—phenomenal assets. They do a really good job at creating synthetic data. They’ve done a lot around how to get this data ready for training these models, and there are very few competitors who can actually do what Scale AI is doing. There’s like a dozen more people that we look at, work with, but Scale AI has definitely done a really, really good job. And this was definitely an acqui-hire. There’s no other way for you to get Alexander to come and join. And Meta has recently gone through a couple changes in their org structure in their machine learning, generative AI, and AGI groups and stuff like that too. So clearly, they usually do not hire high-profile people in leadership roles; this is an exception, and it’s a very senior role that they’re bringing an external person for. It just, I think, doubles down on what Chris was saying in terms of where Meta is with respect to the market today. With OpenAI on one end, you have Apple saying, “Hey, this is all hype,” on the other end, they’re like, “Hey, we are already passing singularity.” So Meta somewhere in the middle is saying, “Which side should I go? I’m going to go follow where the market is going.” So I think there’s a lot. The USD 15 billion—if you look at the overall CapEx that Meta has committed to spending this year, they’re looking at about USD 60 billion-ish is what they’re going to spend this year. They’ve been spending half of what AWS and Microsoft are spending in CapEx on this every year for the last couple of years. So this year, Meta is doubling down and catching up. Even if they spend about USD 60-70 billion, this USD 15 billion will come from that particular portion. Companies are getting very, very creative with how they’re handling talent and IP and data. Scale AI, I think, is a really, really good acquisition, or at least 49% acquisition, for Meta as a company. They need it. We’re still waiting for the behemoth, still waiting for a really good reasoning model from Meta. There’s a long way they need to go, and they have to shake things up a bit.

Tim Hwang: Yeah, for sure. Kaoutar, our final thoughts: if you had USD 15 billion, would you use it to recruit Alexander Wang to your company?

Kaoutar El Maghraoui: Well, maybe I might not be as bold as Meta, but overall I think it’s a strategic move, and this also shows a growing trend. Infrastructure matters as much as models, training, data evaluation, human feedback, and I think this is where the AI wars are being fought right now. So infrastructure playing in the AI race is becoming very important. These investments, I think, highlight the broader industry trend where controlling the underlying data and the infrastructure for AI development is becoming as important as, if not more important than, just developing new models. So I think Meta is really betting on securing its foundational AI supply chain with this acquisition, which I think is very strategic and the right move for them. So of course, they’re betting a lot, and I think they’re also saying they want to secure this high-quality data for AGI, and AGI is one of their big bets for the future. And this investment they’re doing is kind of trying to secure for them that future.

Tim Hwang: Yeah, that’s a great response, Kaoutar. I don’t know if I would be as bold as Zuck to spend USD 15 billion. We’ll keep an eye on how this deal goes and how this partnership evolves.

As always, this was so good. Kaoutar, Shobhit, Chris, thanks for joining us, and stay tuned for our next segment. We will have an interview with Oliver Dial, who I recorded an interview with earlier today, focusing on some recent announcements from IBM Quantum.

So, as I promised at the very top, we wanted to make some time at the very end to talk about a super exciting announcement coming out of IBM on Quantum. And in some ways, we have the perfect person to talk about it: Oliver Dial is joining us on the show. He’s the CTO of IBM Quantum. Thanks for taking some time today; really appreciate it.

Oliver Dial: Absolutely, no problem.

Tim Hwang: You know, MoE, we don’t usually talk too much about quantum, but we keep an eye on it because our scope is just emergent technologies in general. And unfortunately, most of the airtime is always taken up by AI, but we always want to keep an eye on what’s happening in quantum, in part because there’s really exciting things happening. And so, Oliver, really excited to have you on the show. You know, the thing I always have with quantum is that people are always like, “Ah, it’s never going to happen. People have been talking about quantum forever.” I think what’s really interesting is I always push back, and I’m kind of like, “Look, in the last 24 months, there have been some really major leaps in quantum that make it feel like, even though everybody’s complaining about how long it’s taking, it might actually finally be around the corner.” And so I wanted to bring you on the show to talk first about the announcement that you guys have that came out today from IBM on the idea of fault-tolerant quantum computing. So maybe let’s start with the problem. Oliver, I’m curious if you can walk us through: Why has fault tolerance been such a big problem for getting quantum to actually work practically?

Oliver Dial: Yeah. Well, at the end of the day, you’re trying to do computation with some of the most sensitive systems anyone’s ever made in the world. You’re trying to manipulate single quantum states and use them to store information and compute. And so the error rates that we have with today’s hardware are orders of magnitude larger than what you would have on a classical computer—many orders of magnitude. In fact, for our two-qubit gates, which is what we usually talk about when we talk about error rates, our current failure rate is about one in a thousand. And so imagine trying to do computation where the computer one time in a thousand makes a mistake.

Tim Hwang: Yeah. And I think those errors are caused by just—like, it’s too close to an electrical field or something, right? It could be anything.

Oliver Dial: So the biggest one for our qubits is we actually store the one or the zero in a single microwave photon. And so anything that can absorb microwave energy can suck that one away and turn it into a zero.

Tim Hwang: Which is kind of like everything, right? Like everything.

Oliver Dial: Yeah, everything on the planet. The worst thing for us are surfaces because anywhere there’s a surface, there’s a place where you can have a thin layer of oxide, you can have contaminants, all kinds of things that’ll absorb microwave energy. So you’re trying to do computation with this fantastically high error rate. And the really amazing thing to me is even today, despite that high error rate, we’re already doing computations that a classical computer can’t run. That’s what we talk about for the era of utility. And the reason is we have a lot of ways that we can sort of statistically remove the impact of those errors. So today we play a game where we run a circuit, we run a quantum computation many, many times, we statistically remove the errors to get an accurate answer out. And using that, we’re starting to be able to tackle some really interesting problems in chemistry, optimization, and material science. And we think that—

Tim Hwang: You’re saying quantum is already here, is kind of what you’re saying.

Oliver Dial: We’re saying you’re probably going to show quantum advantage—so doing something better, faster, cheaper than a classical computer—by 2026, and actually I think it’s going to be even sooner than that. And you were talking about this cadence of news releases; I mean, that is going to be such an “all boats rise” moment because that’s going to be saying, “Okay, wait, this isn’t something that’s going to be next month or next year; this is actually really happening today now.” But the big thing with all these techniques we use today is they have an overhead that scales exponentially with the size of the problem. And so it’s a little bit of a, in the long term, losing proposition. You know, we have this computer that’s exponentially faster for some jobs, but we’ve added an exponential overhead.

Tim Hwang: And so, it’s a real horse because I think, is it right to say that the bottleneck is computation? You’re basically saying we’ve got this noisy process and we kind of de-noise it using computation, but it just can’t scale the way we want it to.

Oliver Dial: Exactly. So when we talk about fault tolerance, we bring in another set of tricks, and it’s error correction. Like if you’re doing communication between a classical computer, you might include parity bits so that you can detect if the data got through correctly and retransmit it. We can pull the same kinds of tricks with a quantum computer. These error-correcting codes are really nice because instead of correcting the errors after the computation is done—which is where that exponential scaling came from—you’re actually correcting the errors on the fly as you do the computation. And so the overhead is no longer exponential. So if you can crack error correction in a way that lets you get to this fault-tolerant regime, then now you can really do any computation you want that the capability of the computer is logarithmic in how large you want the circuit to be. Now, not all the error-correcting codes that we can talk about are the same. A lot of people have talked about using something called the surface code, which is a really neat code. You’ve got to remember, at the end of the day, these qubits aren’t virtual objects; they are physical things on a chip, and there’s actually little superconducting wires connecting these together. When you actually look at these devices, they’re really pretty-looking. So those superconducting wires connecting together—if you’re doing an error-corrected chip, you really want them to exactly match the parity checks that you want to be able to run, because that means you can run those parity checks really efficiently without introducing additional layers. For the surface code, the layout that you want for the qubits looks like a checkerboard, and the qubits are on a square lattice with nearest-neighbor connections. So that’s really neat. As a semiconductor manufacturing person, you can kind of look at this and say, “Yes, I can see how to build this; it’s not too bad.” The problem with the surface code is you’re talking thousands or tens of thousands of physical qubits per logical qubit. And so although it’s easy-looking, it’s not practical. It’s too expensive, too large. To put that in perspective, the largest devices that we’ve ever made are about a thousand qubits.

Tim Hwang: So you’re kind of priced out of doing this very quickly.

Oliver Dial: Exactly. And those are also, by the way, the largest devices anyone has ever made. So part of this announcement is we have this new code that we’re calling the Gross code. And we’re calling that not because it’s disgusting, but because dozens keep on showing up whenever you talk about it, and a dozen dozen is called a gross. So the Gross code breaks the rule that we had in the surface code. You can no longer lay the qubits out in a checkerboard; in fact, each qubit, in addition to that checkerboard, has two long-range connections that go to other qubits that are far, far away on the device—think of it like a highway overpass across the chip. And what those let us do is create a code that is much more efficient. With the Gross code, with 300 physical qubits, we can encode 12 logical qubits. So the qubits come in 12-packs, which is kind of neat. But it’s an order of magnitude fewer qubits you need than for the surface code to build devices like this.

Tim Hwang: That’s fascinating because I think the way you described the problem initially, my layman’s view is, “Well, just try to shield it so you don’t get so much noise.” And kind of what you’re saying is the solution is a physical solution, but it’s for the purposes of making the computation process much more efficient. But in some ways, you’re not trying to deal with the fundamental problem that these are really delicate systems that are subject to all sorts of interference from the environment.

Oliver Dial: I mean, we do want to solve the fundamental problems too, as well as we can. What these error-correcting codes do is they sort of exponentiate your error rate. And so the Gross code, we call it a distance-10 code. Basically, the error rate goes like your physical error rate to the fifth power. And that’s really neat in the sense that if we make our physical error rate 10 times smaller, the logical error rate will get a hundred thousand times lower. You get this enormous lever arm on that physical error rate. And so actually, your incentive to continue to make the underlying physical hardware better is still there; if anything, it’s even stronger. But there are fundamental limits that we think are going to be really hard to exceed. You’re never going to see a physical error rate of 10^-10 coming out of these systems; there’s just too many sources of error.

Tim Hwang: Well, and so I think one way of thinking about this is: we’ve got existing computers, traditional computers, that can solve a certain landscape of problems. And then quantum is now kind of catching up. You look across this landscape, and more areas of the map are filled in by, “Oh yeah, quantum could do that,” or even new places on the map: “Oh, you can never solve that problem with a traditional computer, and quantum can do that now.” Do you want to give our listeners a little intuition for, with this kind of efficiency gain, I assume you have certain problems in mind where you’re like, “Oh, suddenly we could start to crack ABC problems”? And I think maybe it makes it more tangible to talk a little about where this goes. Suddenly you can approach someone and say, “Hey, maybe quantum is an application for what you’re doing.” I’m curious what you think those are.

Oliver Dial: So today, we’re just barely beginning to be able to do small chemistry problems—like a couple of atoms, a key part in a molecule—in a way that’s compelling. With these fault-tolerant computers, really complex chemistry problems in catalysis and organic chemistry become possible, as well as a lot of really interesting problems in material science, studying superconductivity, studying exotic states of matter become possible. One of the original motivations for quantum computers was ultimately simulating quantum systems. But I think the thing that excites most of our clients the most is optimization. And that’s a place where having a wider logical register, more qubits that work better, is just a huge benefit because, of course, optimization problems—nobody really cares if you can solve a small optimization problem.

Tim Hwang: Because in practice, they’re not small. Yeah, exactly.

Oliver Dial: And so, even as we’re working on better algorithms so that we can solve these problems on near-term hardware, it’s this fault-tolerant hardware that is really going to begin to unlock this space. And so when we’re talking to financial institutions, when we’re talking to logistics companies, that’s really where we get a lot of interest.

Tim Hwang: Yeah, I think that’s one of the wonderful things about the current generation of all these emerging technologies. I joke about it all the time in the AI space: you’ve built a machine intelligence, and then the primary application is we use it to optimize customer service. I guess in some ways, quantum almost has a very similar quality: you’re literally playing with the fundamental building blocks of reality, and then it’s like, “We’ve got to solve the traveling salesman problem.” It’s kind of where you end up.

Oliver Dial: Yeah. I’m not sure the traveling salesman problem has that much money behind it. There are really good approximate answers there.

Tim Hwang: Yeah, that’s right. Well, that’s great. I guess maybe just the last question: I’m curious, now that you’ve kind of worked on this, what’s next? What do you think is the next big challenge the community’s focusing on? Maybe give us a flavor for what we might expect in the next six to 12 months.

Oliver Dial: So I think, well, first big one is quantum advantage. We are going to see it, and like I said, it’s just going to be the gunshot that gets heard around the world on quantum. The other one is now that we’ve announced this roadmap to fault tolerance, I think we’re going to start to see a lot of other competitors in the field updating their roadmaps to use some of the same ideas. And so we’re really going to begin a second stage: this race to fault tolerance.

Tim Hwang: It’s like game on, basically. For you guys, it’s game on.

Oliver Dial: And remember, we have a plan. It’s really neat that in principle we have solutions to all the problems you need to build a fault-tolerant quantum computer—to the extent that we’re actually starting to build a data center in Poughkeepsie, New York, to house these things. Like, that’s where we are in the planning. But every stage of that can get better. We have a bare minimum; we are continuing to do research. We expect every aspect of our plan to still continue to improve and for these machines to only get more capable than what we’re anticipating today.

Tim Hwang: I think maybe just one thing to end where we started: I’ve been hearing a lot from quantum people about how this is very soon and going to come in the next 12 months, and this is going to be a shot heard around the world. I don’t want to be too mean, but kind of like, why should we believe you guys that this time is different? What about your roadmap makes it an actual practical thing?

Oliver Dial: Well, if you think about the way we build really complicated machines, the key is to be able to build them out of an individually testable and composable unit cell. That if we needed to build Blue Jay—it’s going to be about a hundred thousand physical qubits—if I needed to build you a hundred-thousand-physical-qubit chip, we would need aliens to come down to Earth and give us that technology. But the great thing about the Gross code is the module size is actually pretty small. Once we’ve added all the extra qubits we need for computation, for communication, we can build a module of about 500 qubits, and that module we can then step and repeat and connect together to build the system as a whole. And so the great thing about this design is that we don’t need to build a hundred-thousand-qubit system; we need to build a 500-qubit system. We need to be able to manufacture it, test it, and connect a lot of those together. And it’s really that modularity that is bringing this into reach, and the fact that the module size is no bigger than what we’ve built in the past. We built Condor; we have built a thousand-qubit device that was only a little bit less complex than what we need to build to make this code happen. And that’s why we’re saying it’s practical.

Tim Hwang: That’s awesome. Well, great. I’m really looking forward to having you back on the show, Oliver, and thanks for taking the time today.

Oliver Dial: No problem at all. Thank you.

Tim Hwang: Thanks to all our listeners for joining us. If you enjoyed what you heard, you can get us on Apple Podcasts, Spotify, and podcast platforms everywhere, and we will see you next week on Mixture of Experts.