There are two opposing forces on the earth of information: an general consolidation inside the trendy knowledge stack & an enormous enlargement pushed by AI capabilities. AI is rewriting each rule about what’s doable with knowledge in 2025.
Listed below are Concept’s Prime Themes in Information in 2025 with the total presentation on the backside.
-
The Nice Consolidation. After a decade of increasing complexity within the trendy knowledge stack, firms want to dramatically simplify their architectures to drive higher outcomes. Consumers we communicate to say, “Don’t promote me one other device.”
Because of this, we’re seeing consolidation on particular person cloud knowledge warehousing platforms Snowflake & Databricks, the place most enterprises have picked their dominant structure. There’s additionally a wave of consolidation inside BI favoring collaborative BI instruments that steadiness centralized & decentralized management like Omni. There’s a race to command extra & extra compute inside these consolidated platforms as a result of nearly all of income & in the end income resides there.
The workplace of the CFO continues to use stress to drive extra ROI on core knowledge & AI, which is a shock given 2010-2022 knowledge budgets grew unabated & the fervent curiosity in AI. Particularly, this has pressured the cloud knowledge warehouses, & clients are on the lookout for novel architectures to dramatically scale back cloud knowledge warehouse spend the place doable. 50% price financial savings are doable with newer transformation architectures like SQLMesh / Tobiko Information.
-
Scale-Up Architectures. New cloud knowledge storage codecs are growing in significance though slower than anticipated by way of adoption as a result of they lack the related enterprise instruments. Nevertheless, in the long run, this creates an increase in workload-specific question engines like MotherDuck & Datafusion.
These question engines are sometimes scale up fairly than scale out. This implies builders can begin on their native machines & benefit from the outstanding computing energy of their MacBooks to deal with all of the overwhelming majority of workloads.
-
Agentic Information. If the IT division sooner or later is the HR division for AI brokers, we must always anticipate that knowledge can be remodeled simply as a lot as each different workforce. Traditionally, there was a divide between software program engineering & AI/ML groups, & that can change. Lots of the software program design rules, like digital environments, will come to knowledge.
We must always anticipate the overwhelming majority of SQL queries to be executed by AI. To make these queries correct, knowledge modeling turns into a completely important know-how to remove hallucinations & assure high quality. As well as, knowledge observability instruments like Monte Carlo will turn out to be more and more essential as knowledge not solely feeds the core BI & analytics layers but in addition the AI programs which can be key elements of each manufacturing utility, each inner & exterior.
That AI ought to drive some fairly vital efficiencies that parallel the 25% to 50% productiveness features throughout Google, Microsoft, & ServiceNow, or the $275 million in price financial savings of Amazon migrating from one model of Java to a different. This could free budgets for brand spanking new initiatives.
Smaller fashions will dominate inside the enterprise with the candy spot someplace between 10 billion to 70 billion parameters, due to a 600X distinction in inference price & very comparable ranges of accuracy. If the wave of innovation from DeepSeek is any indication, there’s considerably higher efficiency & a continued deflation & inference price over the brief & intermediate time period.
Additionally on a enjoyable observe, this presentation was the one I synthesized utilizing AI. If in case you have suggestions, let me know, & the total AI speaker notes can be found right here.
