I talked with Alistair Croll, Chair of FWD50 and coauthor of Lean Analytics and the upcoming Just Evil Enough, about one of the most pressing challenges in enterprise IT today: application modernization and real-time analytics. Alistair will speak on Day 1 of Prodacity 2025 in a session titled “MVP for the Enterprise,” where he will explore how individuals and organizations can innovate within constraints.
Our research shows that modernization is no longer optional; it’s imperative. Based on the responses to our latest survey data, a majority of CIOs name application modernization as their top priority. Yet organizations face a persistent skills gap and operational complexity that impact progress. On the other hand, developers are often stuck maintaining legacy systems rather than driving innovation. Modernization remains a balancing act between balancing resources to maintain operational stability and leveraging new technologies to drive growth.
Alistair provided a unique perspective on the role of real-time analytics, the challenges of bridging heritage systems with modern platforms, and the evolving role of AI. While we agree on many points, there’s room for debate – particularly around how quickly generative AI can resolve modernization challenges. Here’s what I took away from our conversation.
Real-Time Analytics: Fueling Innovation in Modern Applications
Real-time analytics is often said to be the backbone of modern applications, enabling operational intelligence that static reports can’t match. However, adoption still faces hurdles, with complexity and skills gaps topping the list. I asked Alistair how he sees analytics driving the next wave of innovation and what advice he has for organizations looking to embed real-time capabilities.
Alistair’s Perspective
He broke down analytics into four distinct timeframes:
1. Machine Real-Time: Machines can act on data immediately; the app is the consumer. This applies in military scenarios (e.g., safety or attack opportunities) and civilian services (e.g., suggesting relevant resources).
2. Human-in-the-Loop Real-Time: Humans can parse high-bandwidth data to make informed decisions. For example, merchandisers decide which products to feature, or operations managers identify resource needs.
3. Customer Engagement: These are tactical responses to data-driven insights that can help organizations decide when and where to engage customers in real-time.
4. Strategic or Legislative: Long-term trend analysis (seasonal, geographical) can inform policies and strategic decisions. This timeframe is slower and bumps into political and cognitive resistance like confirmation bias.
For Alistair, the focus should always be on behavior rather than vanity metrics. He believes the question should be, “What behavior are you hoping to change? Don’t start with the data you want to collect; work backward. Define the desired outcome, identify mechanisms to influence it, and measure the right metrics.” Vanity metrics, he explained, might look good on paper but fail to drive meaningful change.
My Thoughts
While Alistair’s frameworks offer a powerful approach, the adoption of real-time analytics remains challenging. Our research shows that 54% of enterprises struggle with complexity, and 70% face critical skills shortages when focused on modernization efforts. Tools like automated observability platforms, data pipelines, and AI dashboards can alleviate these hurdles.
Key Takeaway: Real-time insights are critical for CIOs to improve operational agility. For developers, ensuring data is actionable and understood is essential for success.
Bridging Heritage Systems with Modern Platforms
In our surveys, modernization consistently tops CIO priorities, with 20% citing application portability as critical and 67% stating it is very important to their success. Yet the modernization journey remains a struggle for many organizations. I asked Alistair for his take on building a roadmap to bridge heritage systems with modern platforms.
Alistair’s Perspective
“The economics of modernization have changed massively,” he stated, explaining that generative AI now allows organizations to build or migrate applications at 10x to 100x lower costs than before. Every CIO, he argues, needs to revisit past cost decisions in light of this shift.
At the same time, Alistair emphasized that modernization shouldn’t be about the app itself but the outcome it enables. “Don’t modernize the app; modernize the outcome.” He pointed to the reduced cost of building apps as an opportunity for organizations to innovate incrementally without massive disruption.
My Thoughts
While Alistair makes a strong case for generative AI lowering costs, this doesn’t eliminate the skills and operational challenges we still see across organizations. As mentioned, our data shows that 70% of organizations note skills gaps as one of their top modernization hurdles. Even further, balancing the desire for modernization with the risks of disrupting operations remains a concern.
For CIOs, it’s necessary to quantify the cost of maintaining legacy systems versus the long-term benefits of complete modernization. For developers, tools like containerization, APIs, and hybrid cloud frameworks may provide lower-risk paths to incremental modernization by building new applications in modern frameworks and connecting data rather than migrating data from legacy systems.
Real-Time Analytics in the Age of AI and LLMs
With over 54% of production applications now leveraging AI, organizations are exploring ways to train their LLMs while navigating the challenges of heritage data lakes. I asked Alistair about the most significant challenges and how real-time analytics might evolve as AI scales.
Alistair’s Perspective
He said the primary challenge is hallucination. The liability question is a key barrier to corporate adoption, even though insurers are actively working to quantify risk. Alistair advised organizations to train models with low temperatures to ensure predictable outputs and verify responses before production use.
Case studies like LexisNexis demonstrate how organizations can successfully integrate AI with heritage data by maintaining consistent, human-validated outcomes.
My Thoughts
AI scalability and real-time analytics rely on stringent data validation processes and human oversight. Organizations that can successfully integrate LLMs with heritage systems should focus on:
- Governance: Clear accountability for AI decisions.
- Reliability: Caching correct outputs and reducing false positives/negatives.
- Context-Aware Applications: Real-time insights feed into AI to enhance accuracy and relevance.
Governance frameworks will be essential for CIOs to ensure clean, relevant, and accurate data. For developers, training models prioritizing predictability and consistency over creativity provide worthy outcomes.
Emerging Trends and MVP for the Enterprise
Alistair’s upcoming talk at Prodacity 2025, MVP for the Enterprise, promises insights into innovation within organizational constraints. I asked for a sneak peek into emerging trends in analytics and modernization.
Alistair’s Perspective
He explained that the key to driving change is recognizing the system you’re in. Systems are built for stability and to reinforce the status quo, which makes change hard. Organizations can:
1. Work within the system.
2. Build parallel systems.
3. Subvert the system to behave in unintended ways.
He also emphasized the need for outcome-driven innovation where accountability shifts from the processes to measurable results.
My Thoughts
Many of the emerging trends we see align with Alistair’s views:
- AI-Driven Observability: Managing complexity through real-time, AI-powered insights.
- Outcome-Driven Modernization: Aligning innovation initiatives with business goals to deliver measurable results.
- Digital Literacy for Leaders: IT is no longer an optional skill.
Key Takeaways and Alistair’s Prodacity Talk
Alistair’s insights remind us that application modernization isn’t about the apps but about enabling desired outcomes. Whether it’s embedding real-time analytics, bridging heritage systems, or scaling AI, the challenges are clear, but so are the opportunities.
For CIOs and developers alike, the path forward requires balancing technological innovation with governance, accountability, and skills development.
Alistair will expand on these ideas at Prodacity 2025, where his session, MVP for the Enterprise, will offer strategies for innovating within constraints. If you’re attending, it’s a session you won’t want to miss! And if you haven’t already, check out Just Evil Enough – it promises to be a great read.