After Talking to 15 Silicon Valley AI Companies, I See the Open vs. Closed Debate Differently

Let me start with three conclusions:

Yesterday, Google reported its Q2 results, which provide a timely footnote to this piece. Google Cloud revenue grew 82% year over year to $24.8 billion, while backlog rose to $514 billion. One number was particularly striking: throughput on Google's first-party model APIs had reached roughly 22 billion tokens per minute, up from 16 billion the previous quarter.

We've been talking about Token Optimization for six months, yet total token consumption is still accelerating. This is Jevons Paradox playing out in real time.

And yet the stock sold off after hours. Capex had doubled to $44.9 billion, and Google posted its first quarterly negative free cash flow since going public.

No one is questioning the growth anymore. The market is starting to price in the cost of that growth.

Growth hasn't slowed. The logic behind the growth has changed.

Last week, I attended a closed-door gathering in Menlo Park and spent two days talking with more than a dozen AI-native companies, including Cursor, Fireworks, Cognition and Sierra.

Combined with conversations I've had at Stanford and one-on-one interviews with founders, I've been hearing the same questions again and again. And I've become increasingly convinced of one thing:

For the past two years, the question was who could train the strongest model. Today, enterprises are asking a different question: which tasks actually need the strongest model, and which ones can be handled by something cheaper?

Two concepts capture this shift: Token Optimization and Model Routing. The first means enterprises are starting to manage AI costs systematically for the first time. The second means companies no longer have to rely on a single model — they can dynamically allocate traffic across closed-source, open-source and even internally developed models depending on the task.

The conversation around open-source models in Silicon Valley has also changed. It is no longer simply: "Is it smart enough?" It is increasingly: "Can it handle more production traffic?"

And what sits underneath that question could have a real impact on the revenue growth trajectories of OpenAI, Anthropic and other frontier labs over the next several years.

Growth is still extraordinary

Let me first correct one common misconception: enterprises optimizing tokens does not mean AI investment is cooling.

If anything, the opposite is true.

The AI-native companies I spoke with are still growing at extraordinary rates. Three examples are enough to show just how fast the market is moving:

Three things are driving this growth.

First, model capabilities have improved meaningfully over the past six months, and enterprises are starting to see real ROI.

Second, products are moving from Copilots to Agents, which means each task can consume significantly more tokens.

Third, more companies are moving toward hybrid Subscription + Usage pricing, allowing token consumption to translate directly into revenue.

AI spending hasn't slowed. It has simply entered a more disciplined operating phase. Companies are still willing to spend. They just don't want to pay for low-value tokens anymore.

Every customer conversation is about cost

Token Optimization was probably the most frequently discussed topic during those two days. Cursor put it very directly: every customer conversation is about this now.

Companies are redesigning workflows and asking which tasks genuinely require the strongest model, and which can be pushed down to something cheaper.

But not everyone is hitting the brakes. ClickHouse is an interesting counterexample. They told us their spend on Claude went from roughly $20,000 per month in January to $1 million per month in June — a 50× increase, equivalent to about 5% of their own revenue.

Even at that level, the company continues to encourage employees to use AI more, with spending expected to reach 10% of revenue, because they believe the productivity gains are worth far more than the cost.

ClickHouse is clearly an outlier. Across the broader group, most companies are already putting on the brakes.

One company described the recent optimization wave as being triggered in part by software vendors collectively moving toward token-based pricing. Microsoft, for example, changed GitHub Copilot to token-based pricing on June 1. Once the cost became visible, companies started managing it.

This reminds me of Cloud Optimization in 2022–2024. Companies didn't abandon the cloud. They simply became much more focused on resource utilization.

But one thing is worth remembering: at the time, the bulls also argued that optimization would ultimately lead to more spending. AWS, Snowflake and Datadog subsequently went through four consecutive quarters of growth deceleration.

Will the same pattern repeat this time, with the object of optimization shifting from cloud resources to tokens? That is one of the questions I'll be watching most closely in the second half of the year.

Enterprises are no longer buying a single model

If Token Optimization is the goal, Model Routing is the method.

We've already moved beyond the stage where AI simply means giving users a dropdown menu of models. Cognition, for example, can break an inference task into multiple subtasks and assign them to different models based on cost, reasoning ability, latency and accuracy. The user may not even know which model is being called underneath.

What enterprises are really buying is no longer a particular model. They are buying a system that can intelligently coordinate multiple models.

How quickly is this happening? Two data points stood out to me.

Decagon, an AI customer-service company, disclosed that roughly 30% of its token requests are already being routed to open-source models.

On Factory, open-source models represented only 0.1% of token volume last December. That rose to 1% in February and is now around 10–15%. That's more than a 100× increase in seven months. The founder even believes it could reach 90% within 12 months.

I'm skeptical of the 90% number. But the direction — and especially the slope — is clear.

And routing to open-source models isn't simply about saving money. Several companies brought up another motivation: they don't want to be locked into any single model provider.

Either Great or Cheap. The middle gets squeezed.

After talking to all these companies, my view of the model landscape can be summed up in one word: Barbell.

Both ends will also see consolidation. At the cheap end, new models have been appearing constantly over the past six months — Kimi K3, GLM 5.2 and Qwen have all taken turns grabbing attention. But the model that continues to set the floor on price-performance is DeepSeek. It remains the largest single model supplier on OpenRouter, ahead of any individual U.S. provider. One investor described it as the market's "price-performance floor."

DeepSeek has effectively set the floor for value across the market. Any model that costs more has to be meaningfully better. If it's only marginally better, users may simply use DeepSeek and retry more often. A model that merely matches DeepSeek, or is only slightly better, will have a hard time surviving.

At the high end, the real question is: will cheaper models eventually cross the "good enough" line?

The risk is straightforward: once cheap models become good enough, most users may simply be satisfied, and the premium commanded by frontier labs could start to erode.

I don't agree with that view. I don't think demand for frontier intelligence will ever be completely satisfied. Every meaningful step up in model capability unlocks another set of tasks that previously could not be done, or could not be done economically. There will be durable demand at both ends of the barbell.

A little over a month ago, Gavin Baker offered a useful framework:

At the time, I wondered whether that framework might become outdated quickly. Kimi K3 and DeepSeek V4 were still ahead of us. Looking at the latest data, my conclusion is that the token half of the framework is playing out even faster than expected, while the value half is still holding up.

OpenRouter routing data showed that Chinese models had already surpassed U.S. models in token volume by early June. By mid-July, Asian models accounted for roughly 60% of all tokens on the platform, with their share tripling since the beginning of the year.

Looking back over a full year, the shift is even more striking. Google, OpenAI and Anthropic together went from roughly 70% of token share last June to around 30% this June. DeepSeek alone accounts for more than 16% and has become the largest model supplier on the platform. Its share doubled within six months of the V4 release, with agentic workloads being a major driver.

But the other half of the framework is holding up as well. Anthropic represents only around 12–13% of token volume, yet captures far more revenue than its token share would suggest. In coding-related spending, the Claude family has consistently captured more than 60%.

The market is increasingly separating into two lanes: the commodity lane for volume, and the premium lane for revenue.

There is, of course, an important caveat: OpenRouter's user base is disproportionately made up of developers and price-sensitive traffic, so it does not perfectly represent the enterprise market. But as a directional indicator, the signal is clear enough.

A concrete example is Harvey. I spoke with their technical leadership last week. They used proprietary legal data and worked with a third party to perform post-training on an open-source model, then added a router. The result was reportedly better than directly using Opus 4.8 — while costing less.

This may be what the future looks like: frontier models continue to be heavily used, but an increasing share of tokens flows toward task-specific open-source models.

List prices are losing their meaning

Another change worth watching is how little model list prices — input and output token prices — actually tell us.

Enterprises don't ultimately care about the price of one million tokens. They care about the total cost of completing the same task.

Cache hit rate, inference stability, retry frequency and token efficiency all affect the final economics. Sometimes, a model that looks four times cheaper on paper ends up costing much less than that in practice. If one model succeeds 60% of the time and requires an average of 1.7 attempts, the apparent price advantage can quickly disappear once retries and cached tokens are included.

The feedback I'm hearing from builders in Silicon Valley reinforces this. A cheaper model that needs two or three additional attempts can quickly give back its token-price advantage. And the additional retry traffic can generate cached tokens that further erode the headline savings.

DeepSeek has pushed the price floor extremely low. But one of the most common complaints I hear from production users is not price. It is stability. Outputs can be inconsistent across runs. The same task may require multiple attempts. A cheaper model that fails more often may not actually be cheaper in production.

So the metric I want to leave you with is:

Cloud vs. on-prem: the same math applies

The same math applies to the debate over running models in the cloud versus on-prem. People on X have been doing the math. GLM 5.2, for example, has been quoted at around $1.40 per million input tokens and $4.40 per million output tokens. If you buy the hardware yourself, the entry point is roughly $20,000, with output around 20 tok/s.

Even if you ran the hardware 24/7 at full utilization, it would take roughly 5.5 years to break even.

In an environment where compute is constrained and hardware generations are moving so quickly, the economics increasingly favor the cloud.

The most underestimated constraint

This is one constraint I think the market underestimates.

Models such as GLM are still heavily constrained by chips. They are primarily distributed through inference platforms such as Together and Fireworks, and are not yet broadly available across every major public cloud — although AWS and Azure have started offering some Chinese models. Even Kimi has temporarily paused new-user subscriptions at points because of insufficient compute capacity.

Chinese models are absolutely worth watching. But I think it's still too early to make a heavy investment call based on token share alone.

I'll keep tracking two questions:

1. When will Chinese models achieve large-scale, reliable availability across public clouds?
2. Is the quality gap versus OpenAI and Anthropic closing, or stabilizing?

Those two questions will tell us far more about the real impact than Twitter sentiment.

What does this mean for investors?

Frontier Labs

The question we need to rethink is not whether OpenAI or Anthropic will be replaced by open-source models. It is whether we need to reassess their ARR growth assumptions.

My straightforward view is that some of Anthropic's very strong first-half growth may have come from using the most expensive models for tasks that didn't necessarily require frontier-level intelligence.

In the second half, some of those workloads will either be eliminated through Token Optimization or routed toward cheaper models.

Expensive models will continue to handle the highest-value tasks. But future revenue growth will increasingly need to come from high-value reasoning, rather than simply having every category of token grow at the same rate.

Semiconductors

I am actually more bullish here. Investors who worry that Token Optimization will suppress GPU demand may be missing Jevons Paradox. Lower unit costs have historically expanded total consumption rather than reducing it. Cheaper models can push AI from early adopters into the mainstream.

And don't forget: China has a compute shortage. These open-source models ultimately need to be hosted somewhere with sufficient compute capacity.

The industry's center of gravity will gradually shift from training toward inference, but GPUs, HBM, high-speed networking and advanced packaging remain some of the highest-conviction parts of the AI infrastructure stack. One detail I find particularly interesting: giant open-source models such as Kimi K3 require extremely high HBM density to fit within a rack. The bigger open-source models become, the happier memory suppliers may be.

Cloud Infrastructure

Cloud providers will increasingly compete on more than GPU availability. The question will be: who can help customers complete inference at the lowest total cost?

Inference efficiency, model deployment, scheduling and TCO will all become more important. The deployment of Chinese models could actually increase rather than reduce demand for cloud compute. Databricks has publicly made a similar point.

AI Software

The biggest opportunity may increasingly sit above the model layer. As models become more commoditized, the platforms that help enterprises automatically select models, optimize costs and improve ROI become more valuable. Routing, inference infrastructure and agent orchestration are all areas worth watching.

This is one reason companies like Fireworks continue to interest me. They are not competing primarily on the model itself. They are competing on the efficiency of the enterprise AI system.

The risk is equally clear: if frontier labs respond to a more competitive model market by vertically integrating further into the software stack, infrastructure software companies could face increasing pressure. That is a competitive dynamic I will be watching closely.

Bear case, bull case, and the four signals

I am personally more tilted toward the bull case. But a thesis should be falsifiable. Over the next two quarters, I'll be watching four things:

AI competition hasn't ended. The function has changed.

AI competition isn't over. The function being optimized has changed.

For the past two years, the industry optimized for Intelligence. Increasingly, it will optimize for:

I'll come back to this in 12 months and see where the thesis held — and where it didn't.