Data Platforms: Our Perspective on Getting It Right Without Overcomplicating
In 2 July 2025
I’m not a data architect. I’m not a data engineer. But I’ve led teams that included both. And for the last four years, working with dozens of clients, I’ve seen the same story play out again and again:
How do we design a data platform that delivers real business value… without drowning in complexity?
A Journey Back: When the Warehouse Ruled the World
Let me take you back about a decade. Back then, the corporate data warehouse was the shining star—the single source of truth for decision-making. Or at least, that was the dream. The reality? A central warehouse surrounded by dozens of departmental silos, each quietly thriving on their own.
Life was clean, predictable, and organized—but painfully slow. You wanted a new KPI? A new data source? A report slightly different from the last? The answer was always the same: “Sure… in three to six months.” No wonder business teams started building their own spreadsheets, databases, and shadow systems.
The Avalanche Arrives
Then came the data lake. And then the lakehouse—promising the best of both worlds. We started talking about data mesh, data fabric… and how to govern this new, sprawling chaos.
But just as we were beginning to digest that, Generative AI exploded onto the scene. Almost overnight, everyone was talking about LLMs, foundation models, AI agents, and agentic frameworks.
And soon, the phrase “Modern Data Platform for the AI Era” was everywhere—whatever that actually meant.
The Noise Becomes Deafening
I’m sure I’m not the only one who feels it. The newsletters. The LinkedIn posts. The webinars. The “ultimate guides.” Each one selling the “best” approach to data platforms.
Just look at this landscape and only in a perspective of AI governance tools —an image from the book “AI Governance Comprehensive: Tools, Vendors, Controls & Regulations” by Sunil Soares (2024). It’s overwhelming. Across hyperscalers, privacy tools, observability, governance, labeling, and AI infrastructure, the options seem endless.
If you’re not at least a little paranoid about whether you’re making the right platform choices—you’re probably not paying attention.
But here’s the trap: When technology becomes the focus, we risk losing sight of the actual goal—solving real business problems with data.
Refocusing on What Matters
Your data ecosystem must serve three things:
- Data Products That Deliver Value
Whether it’s predictive maintenance, customer churn models, operational dashboards, or AI copilots—it’s about delivering real business outcomes that solve today’s challenges and enable tomorrow’s opportunities. - Foundations That Make Data Trustworthy
Governance, data quality, lineage, privacy, security, and literacy. These are non-negotiable. Without them, your AI ambitions will collapse under the weight of bad data. - A Platform That Supports Both—Without Becoming the Problem
A platform that is scalable, adaptable, and sustainable. It should support your business—not distract from it.
What a Modern Data Platform Really Needs
Regardless of vendor, architecture, or cloud, the fundamentals are the same:
- Real-Time & Streaming Capabilities
Fresh data, low-latency pipelines, event-driven architectures, and live AI models. - End-to-End MLOps Integration
From feature stores to deployment, monitoring, and continuous learning, AI workflows must be connected to the data backbone. - Support for Embeddings & Vector Databases
LLMs, recommendation engines, and semantic search rely on this. Your platform must handle vector-based workloads natively. - Governance & Observability By Design
Lineage, data contracts, quality checks, access controls, and comprehensive monitoring are built-in—not bolted on later. - Scalability & Cost Efficiency
Cloud-native, elastic, serverless where possible—but with governance over costs, ensuring flexibility doesn’t become financial chaos.
A Word of Caution (From Gartner, Not Just Me)
In the May 2025 Gartner report (Emerging Tech: Data Management Evolves From Stand-Alone Tools to Converged Platforms), they make it clear how the data management landscape is reaching a critical tipping point. Organizations are overwhelmed by an increasingly fragmented ecosystem of specialized tools—each addressing slices of governance, quality, observability, privacy, integration, and AI enablement. This tool sprawl not only inflates costs but also slows agility, creates data silos, and undermines governance.
“Organizations are inundated with data management products. Vendors must unify capabilities into converged platforms or risk losing customers.”
In other words, businesses are demanding more integrated, future-proof solutions—data ecosystems that operate as a cohesive platform, not a pile of disconnected tools. If your current architecture feels like assembling IKEA furniture blindfolded—constantly fitting mismatched pieces together—you’re doing it wrong. The future belongs to unified, interoperable, and scalable platforms that adapt with your business.
Ironically, AI Is Helping Simplify It
AI isn’t just another workload—it’s becoming a key enabler of modern data platforms. From AI-assisted metadata discovery and automated data pipeline generation to intelligent data quality monitoring, AI is embedding itself into daily platform operations.
Use cases like generating transformation logic for ingestion pipelines, automating lineage tracking, and anomaly detection in data streams help keep the platform reliable with less manual oversight. AI-driven suggestions for governance—such as access controls or detecting schema drift—support compliance and integrity.
The tools are moving toward democratization, empowering not only engineers but also analysts and operational users. This shift helps organizations reduce bottlenecks, improve platform operations, and accelerate the adoption of AI-powered services enterprise-wide.
My Simple (Not Simplistic) Advice
- Start With the Business Problems
What’s the value? What do we need to solve? Focus on the data & AI products that will drive outcomes today—and be relevant tomorrow. Look beyond immediate pain points and think about how the business is evolving, how AI, automation, and digital transformation will shape operations, products, and customer expectations. - Build the Right Foundations
Governance, MDM, privacy, data literacy, and quality. No shortcuts. But also, design these foundations not just for current needs—but for scale, for complexity you don’t yet have, and for use cases that will emerge. This is about future-proofing your data ecosystem as much as possible. - Design (or Redesign) the Platform
Whether you’re building from scratch or evolving what you already have, the platform conversation comes last—but it’s critical. It must serve the business value path and the foundations you’ve defined. And it must be designed with flexibility in mind, to scale, adapt, and grow as your data and AI ambitions evolve.
The Apgar Way
This is exactly how we work at Apgar. Our Apgar Pathfinder Methodology is built as a comprehensive framework that blends strategic vision with operational execution.
The Pathfinder journey is structured into three interconnected paths:
- Business Value Path
Define the data & AI products that solve real-world problems and deliver measurable business outcomes. - Data Foundation Path
Establish the governance, quality, privacy, and data management capabilities that ensure trust, compliance, and scalability. - Data Platform Path
Design or redesign the data platform architecture that supports both the business value and foundation requirements—flexible, scalable, and future-ready.
The Pathfinder journey is not a linear checklist; it’s a collaborative, iterative process that starts by deeply understanding the business context and challenges. From there, it’s about analyzing the current landscape, identifying gaps and opportunities, and then designing a tailored roadmap. This roadmap connects the business goals with the right data products, governance foundations, and platform architecture—all with a focus on making the ecosystem scalable, future-ready, and aligned with long-term objectives.
And while every client’s needs are different, we have the benefit of a rich ecosystem of technology partners—hyperscalers, MDM providers, catalog tools, governance solutions, and AI platforms. Our approach is always vendor-agnostic: we recommend what fits the client’s context, maturity, and goals.
The Bottom Line
This isn’t an article about whether to buy, build, or combine your data platform. The truth is—it depends. It depends on your context, your challenges, your maturity, and where you want to go.
But one thing is universal:
Start with the business value. Build trust through strong data foundations. Enable that with the right platform—designed not just for today but with tomorrow in mind.
Get those right, and the platform becomes an enabler—not a problem. Everything else? Just noise.
Would you like to get in touch with our experts?
If you agree, disagree or have something to add to these views on corporate strategy, please contact us.