I'd like to preface this post by saying that I am not a subject-matter expert in the LLM / machine learning space. These are merely my high-level thoughts as someone that works in tech and likes to think about where the industry trends are headed.

There's quite a bit of noise around the recent GPT-5 announcement from OpenAI. I've played around with this latest model a bit, and for the most part my sentiment matches what many others have had to say: it's mostly underwhelming.

Whereas a couple years ago it felt like every new announcement was leading to big leaps in language models' capabilities, mid-2025 can be best summarized as... meh.

Hitting the limits of scaling

Back in 2020, OpenAI released a seminal paper describing the scaling laws they observed for language models. In essence, they observed loss decreases predictably with a model's size, the training set's size, and amount of compute used during training for transformer-based language models.

With every passing day, it seems this loss curve becomes less and less predictable. We seem to be hitting an inflection point where the scaling that used to result in large qualitative jumps in model output is starting to wane. Factors contributing to this dynamic include:

• running out of high-quality training data: there is no second internet worth of information to train foundational models against.
• compute budgets have already reached exorbitantly high levels, so increasing compute for new model generations is becoming harder to justify.
• running inference for the larger foundational models is not a trivial expense, and the budget for inference also becomes harder to justify as more users onboard to ChatGPT, Claude, etc.

Whereas GPT-3 to GPT-4 felt like an earth-shattering update, GPT-5 feels more like a murmur at best. GPT-4 saw improvements such as the ability to pass the bar exam, but GPT-5 is not taking the place of any human lawyers. While the discourse has shifted away from AGI, it's important to reflect and remember the expectations that were set by industry leaders just a couple years ago.

Reading the tea leaves on what's next

My thesis: while players like NVIDIA have benefitted immensely from the current AI hype cycle with the biggest tech companies splurging billions (to trillions) on capital expenditures to fuel their GPU-based datacenters to train bigger and bigger models, the GPT-5 announcement is writing on the wall for the scaling paradigm being over. The same GPU superclusters that have been big tech's major assets these past couple years may very well become a potential liability.

My gut sense is that over the next couple of years we'll see a lot more activity in the AI space dedicated to experimenting with different architectures beyond transformers. I believe we'll also see many of the hyperscalers and flagship AI research labs dedicate more resources to the optimization of current LLMs to reduce costs as pressure builds from investors to see a return on investment on the many, many billions spent so far. Arguably, we're already seeing this take shape as GPT-5 uses a "router" to direct requests to the appropriate model based on query complexity.

Perhaps there's something missing in my assumptions and I'm totally off base in my predictions. This space is so fast moving it's hard to tell where we'll be even six months from now. It will be interesting to see what of this post holds as developments in the AI space continue to progress. Thanks for reading!