"Scaled Mediocrity": The counterintuitive AI Strategy that's delivering ROI

My Learnings from Talking to LLM Leaders as hype settles

I’ve spent the last few months in conversation with other LLM leaders, from those at big tech to founders at scrappy startups. My goal was simple: to get past the public-facing hype and understand what’s actually working.

As leaders in the LLM-native data engineering space, our job is to cut through that noise, find what's real, and build on it to get ahead.

Here are the key takeaways from those honest, behind-the-scenes conversations.

The real win: the transformative power of "scaled mediocrity”

The first thing that came up in almost every discussion was this: the real enterprise win isn’t superhuman AI. The most transformative applications come from what one leader perfectly described as "scaled mediocrity."

It’s about doing less with less, at scale. The magic isn't in one perfect output; it’s in the massive aggregate value you get from applying a "good enough" model to tasks that were previously uneconomical to tackle, like classifying millions of customer support tickets or generating thousands of first-draft product descriptions from a spec sheet.

This principle isn't a limitation; it's the largest untapped opportunity in AI. The goal is to democratize 'good enough' solutions to drive tangible results at an unprecedented scale. This democratization is already breaking down old barriers and empowering new workflows across the organization:

• The Marketer: No longer needs to wait in a creative queue for five ad variations. They can now generate fifty 'good enough' versions themselves in minutes, enabling massive A/B tests that were previously impractical and leading to better, data-driven results.
• The Junior Analyst: No longer has to wait weeks for a data science team. They can now get an 85% accurate summary of thousands of customer surveys in real-time. The insight is actionable today, not next quarter.

This isn't just about +50% efficiency; it's a strategic shift. The value comes from empowering the person closest to the problem with 'good enough' tools to get immediate, tangible results.

Beyond RAG vs. Gen: the future is hybrid and multimodal

The early debate of "safe" RAG versus "creative" generation is already feeling dated. Large organisations prefer “Retrieval” because this is grounded in fact while generation is more for the fast and loose black hat marketers. The consensus among actual practitioners is that it’s all hybrid anyway, and the best systems are architected from the ground up to blend retrieval, generation, and tool use in complex workflows.

This hybrid approach is now expanding beyond text. We’re seeing the rise of multimodal storage - unified systems for representing text, images, audio, and structured data. Pioneering technologies like the LanceDB and their Multimodal data lake and new capabilities in platforms like Snowflake are making this a reality. This allows us to build systems that were science fiction a few years ago.

For example, imagine an insurance platform that can retrieve a customer's text-based claim, pull up photos of the vehicle damage, analyze the recorded audio statement, and then generate a comprehensive summary and risk assessment. That's the power of building for multimodality, and it’s where serious enterprise architecture is heading.

The view from the trenches: surviving the hype bubble

This pressure to innovate is something every leader I spoke with is feeling. From our vantage point in data engineering, the hype cycle often feels disconnected from the on-the-ground reality. We're all in a constant state of exploration where our technical learnings have a three-month half-life. The shared feeling is that while the code we write is often temporary, the training our minds get (learning to ask new questions and rethink old problems) is the real, lasting asset.

The devil's advocate: the 10x future is here (just not where you think)

Despite the challenges, none of us are bearish. The 10x future isn't an "if" anymore; it's a "where and when." While the conversation often defaults to developer productivity, we find the most profound shifts are happening elsewhere, especially since the act of coding is often the shortest part of the engineering lifecycle.

Instead, look at areas like drug discovery and materials science. AI models are now generating and validating novel molecular structures in weeks, a process that used to take labs years and millions of dollars. This, combined with breakthroughs like DeepMind's AlphaFold completely solving protein folding, shows that AI isn't just accelerating workflows; it's unlocking entirely new scientific and industrial possibilities.

Our shared frustration: "AI-washing" vs. real 10x innovation

A common frustration is the trend of "AI-washing", vendors adding a thin AI veneer to a legacy product. As data leaders, we compare it to building a "penny-picking Roomba." It’s a clever automation of a low-value task, but it doesn't move the needle or create fundamental value.

Our shared belief is that true innovation comes from rebuilding the core platform, not just adding a smart button. While these small features can sometimes act as a "gateway drug" to get organizations comfortable, we can't let them be the final destination.

Our role as data engineering leaders is to be the bridge between the incredible potential of LLMs and the practical reality of building the secure, scalable systems that deliver real value, grounding the dream in realistic expectation.

Our journey from 1.1x to 10x

Our journey with LLMs at dltHub has progressed through four distinct workflows, each teaching us a valuable lesson about the true nature of AI's impact. Our initial steps, the docs assistant and slack support bot, were clear 1.1x improvements; they successfully made existing processes faster and lowered cognitive load but didn't change the fundamental work.

Even our third workflow, a deeply integrated IDE assistant, which could double development speed, represented the peak of the old paradigm - a powerful enhancement to the classic code-and-debug loop, for data engineers who know what they are doing.

The true 10x transformation only arrived with our fourth workflow: creating a truly LLM-native development environment. By introducing dltHub Workspace, we are bringing these massive gains beyond the core data engineering persona. We are no longer just assisting a specialist; we are changing the entire creative process to one that generates, debugs, visualises and maintains data pipelines in an integrated, intuitive way, where the LLM has access to code, data and semantics end to end.

Of special importance in this AI-first, "doing less with less" paradigm is data quality testing. If humans must test their own work, AI-generated pipelines require even more validation. For this purpose, dltHub’s unique dataset interface separates the compute engine from the code (can run same code on snowflake, duckdb, spark etc). This enables you to run robust tests, written in either Python or SQL, on data in transient, isolated environments before it ever reaches production. We have some exciting ideas about how this will become even more useful in LLM native workflows.

You can dive into this new reality by exploring our starting point of over 2,000 LLM scaffolds - or watch them evolve over time into fully running pipelines through our future community sharing feature.