The Trough Of Disillusionment Has A Name

There is a pattern in how technology categories die. They do not collapse all at once. They get quietly bypassed.

The pattern was best described by Andy Grove, the former CEO of Intel, who watched the personal computer arrive and immediately understood what it meant for the mainframe business. Grove's framework was simple. A technology gets adopted when it is roughly ten times better or ten times cheaper than the thing it replaces. Anything less and switching costs hold the incumbent in place. Anything more and the incumbent gets bypassed regardless of how entrenched it appears to be. The personal computer was ten times cheaper than the mainframe from the first day it shipped, and the mainframe business has been losing ground ever since.

We are in the same kind of moment right now. The thing being bypassed is the entire stack of infrastructure that enterprises spent the last fifteen years assembling to manage their data. The thing doing the bypassing is the architectural assumption that data should not be copied in the first place. The numbers are not ten to one. They are larger than that, and the gap is widening every quarter.

This piece is about what the bypass actually looks like, where each layer of the legacy stack currently sits on the hype curve, and why most enterprises are about to learn that the substrate underneath their AI strategy is the substrate that determines whether the strategy works at all.

What the data is actually saying.

Start with the failure rate of enterprise AI, because that is the most visible signal that something foundational is broken. The RAND Corporation's 2025 analysis found that more than 80 percent of AI projects fail to deliver intended business value. MIT's NANDA initiative, in its 2025 State of AI in Business report, found that 95 percent of generative AI pilots produce zero measurable profit and loss impact. S&P Global Market Intelligence reported that the share of companies abandoning most of their AI initiatives jumped from 17 percent to 42 percent in a single year. Global enterprises invested $684 billion in AI initiatives in 2025, and more than $547 billion of that investment failed to deliver value. These are not seasonal fluctuations. They are the dominant outcome.

The instinct is to blame the models. The data does not support that. The MIT research is clear that the failures are not about model capability. They are about everything sitting underneath the model. Informatica's 2025 CDO Insights survey identified the top obstacles to AI success: data quality and readiness at 43 percent, technical maturity at 43 percent, skills at 35 percent. Every one of those is a substrate problem. None of them is solved by a smarter language model.

Now look at what the substrate is actually costing. Fivetran's 2026 Enterprise Data Infrastructure Benchmark Report found that enterprises spend an average of $29 million per year on their data infrastructure. They run an average of 328 data pipelines and employ 35 to 60 full-time engineers to maintain them. Data teams spend 53 percent of their engineering time on maintenance, with $2.2 million per year per organization spent specifically on pipeline upkeep. Pipeline failures cause an average of 60 hours of downtime per month at a cost data leaders estimate at $49,600 per hour. That is roughly $3 million per month, or $36 million per year, in lost business value from broken pipelines alone. McKinsey found that a typical mid-size financial institution spends $60 to $90 million per year just on data access.

This is the cost of moving data. It is not the cost of using data. It is the cost of getting data from where it was produced to where it can be analyzed, in a form that the analysis layer can actually consume. And that cost exists because the entire architecture is built on the premise that data must be copied before it can be useful.

The most damning statistic is what happens after all of that copying. IDC's research found that organizations have an average of 13 copies of every database and file, and that 60 percent of enterprise storage is dedicated to managing those copies. The cost of storing duplicate enterprise data alone was estimated at $55 billion per year. Approximately 90 percent of the data in any given enterprise is a copy of data that exists somewhere else. The ratio of unique to replicated data globally has been worsening for a decade.

And the artifacts at the top of this stack? Industry research and Gartner-cited data show that only 29 percent of employees actively use the analytics and business intelligence tools their companies pay for. Between 60 and 80 percent of dashboards go unused. Two hundred dashboards in a typical enterprise deployment, maybe forty viewed regularly, maybe twenty viewed more than monthly. The rest are digital ghost towns. The companies that built them spent six and seven figures to produce them. They are looked at less often than the company's internal phone directory.

Where each layer sits on the curve right now.

The Gartner hype cycle moves through five stages. Innovation trigger. Peak of inflated expectations. Trough of disillusionment. Slope of enlightenment. Plateau of productivity.

The cloud data warehouse sat at the plateau of productivity for most of the last decade. It is sliding off. The architecture was designed for batch analytics on historical data, which is exactly the workload that AI does not require and cannot wait for. The category leaders built their entire revenue model on the premise that data movement and storage would keep growing. AI inverts that premise. AI needs to act on live data, in source systems, in the moment a decision is made. A snapshot from last night is not live. A copy from this morning is not source. The warehouse cannot become what AI needs without ceasing to be what made it profitable.

The Modern Data Stack, the constellation of ingestion, transformation, orchestration, catalog, and reverse ETL tools that grew up around the warehouse, sits squarely in the trough of disillusionment. The category attracted tens of billions of dollars of venture capital on the premise that better tooling would solve the data problem. It did not. It produced more tooling. Every tool in this stack exists to manage the consequences of the original architectural decision to copy data out of source systems. Fivetran exists to copy data in. dbt exists to transform the copies. The catalog exists to track where all the copies went. Reverse ETL exists, and this is the part nobody likes to say out loud, because the industry quietly admitted that the warehouse was the wrong place to drive operations from. Hightouch and Census are a category that emerged because everyone realized data needed to be pushed back into the source systems it came from. That is a confession in product form. The customer paid to move the data out, paid again to move it back, and paid a third time for the tools that managed the round trip.

iPaaS is in the same trough. Workato, MuleSoft, Boomi, Zapier. Middleware exists because the systems underneath do not federate. Once the underlying systems can be queried in place, the middleware compresses to a thin orchestration layer.

The dashboard sits at the end of its useful life. The category that defined business intelligence for two decades is being eaten from above by conversational interfaces and from below by the realization that decisions made on yesterday's data are decisions made too late. Dashboards measure what is easy to measure, not what matters. They present lagging indicators. They require users to leave their workflow and interpret visualizations rather than receiving answers. The 29 percent adoption rate is not a training problem. It is a category problem.

Underneath all of this sits the AI initiative, which Gartner's 2025 hype cycle for AI puts squarely in the trough of disillusionment for generative applications. Eighty percent failure rates do not produce slope of enlightenment without a foundational architectural change. The change is not better models. The change is the substrate.

Why this is a 10x moment, not a 2x moment.

Andy Grove's rule says ten times. Let's see if the math actually gets there.

An enterprise running the legacy stack spends roughly $29 million a year on data infrastructure, loses 53 percent of its data engineering capacity to maintenance, sustains roughly $36 million in pipeline downtime cost annually, maintains 13 copies of every database, stores 60 percent of its data as duplicates of other data it already has, watches 80 percent of its AI initiatives fail, and gets 29 percent adoption on the dashboards it pays for. The fully loaded cost of getting one decision made on accurate live data, when you back out every layer of duplication and rework, is staggering. McKinsey's $60 to $90 million per year for a mid-size financial institution, just on data access, is the conservative version of it.

The replacement architecture removes the copying. No pipelines to maintain. No warehouse to provision. No reverse ETL to install. No catalog to chase. No dashboards required because the answer comes back live. The cost compression is not 2x. It is not 5x. It is structural, because every layer of the legacy stack exists only to manage the consequences of a decision the new architecture does not make.

Add the speed dimension. The legacy stack operates on batch refresh cycles measured in hours or days. The replacement architecture operates in the moment the question is asked. For any decision where the underlying data has changed since the last batch ran, the legacy stack does not just lose. It produces the wrong answer with confidence, which is worse than producing no answer at all.

Add the risk dimension. Every copy of permissioned data that exists outside the permissioning system is a governance failure waiting to happen. Every embedding generated from enterprise data and stored in a vector store is the same problem, in a new form. The replacement architecture inherits permissions from the source rather than reinventing them. That is not a feature. That is a structural difference in how risk accrues.

This is more than 10x. The category leaders cannot pivot to match it because their revenue depends on the architecture being preserved. They are in the position the mainframe vendors were in when the PC arrived. Their customers will not all leave at once. The customers who do leave will compound faster than the ones who stay.

What to do about it.

The decision is not between vendors. The decision is between architectures.

A useful diagnostic. Look at your AI strategy and ask one question. Does the strategy depend on data continuing to be copied out of source systems into a centralized destination, or does it depend on data staying in the systems that produced it and being reached in place? If the answer is the first version, the strategy is built on the same substrate that produced the 80 percent failure rate. If the answer is the second version, the strategy is built on an architectural posture that almost no enterprise currently has.

The second question. Look at the cost line for your data infrastructure and isolate how much of it exists to manage the consequences of copying. Pipelines. Warehouse compute. Storage for duplicates. Reverse ETL. Catalog. Reconciliation. Quality tools that catch errors introduced by movement. If you remove all of that, what is left. The answer is almost always: very little, and what remains is the actual work of producing decisions from data.

The third question. Look at how your security and governance posture handles AI access to your data. If your model is sitting on top of a vector store, ask who can see what is in the vector store, how the permissions in the source system are reflected there, and what happens when an employee's access changes in the source. If the answer is uncomfortable, that discomfort is the architecture telling you something.

The enterprises that get this right in the next eighteen months will compound advantages for a decade. The ones that wait for their warehouse vendor to ship a better product will spend that decade explaining to their boards why the AI initiative did not work.

This is what the new architecture looks like. Federated. Live. Permission-aware at the source. Engineered to act, not just to report. Built on the assumption that copying data was always the problem, and that the right answer is to stop. A few companies are starting to deliver on this. Adaly is one of them. The architecture itself is the thing that matters, regardless of who builds it.

The trough has a name. The window is open. The decision is now.