Why digital transformation projects fail and how data architecture can help

Every year, large corporations spend trillions of dollars on digital transformation projects, designed to help them firmly embed digital technologies in their business operations. These technologies include predictive analytics and decision-support systems, and have the potential to transform the traditional focus of IT from saving costs and solving technical constraints to proactively creating value and opportunities across the organisation.

However, implementation challenges mean many of these digital transformation projects end in failure, leading to huge financial losses for the companies involved. In an empirical study published in KeAi’s Journal of Digital Economy, researchers from the US’ Harvard Business School and Stockholm School of Economics in Sweden, explored the origins of these implementation challenges, and examined how enterprises can improve their chances of driving business value creation from new digital technologies, especially big data and cloud computing.

The study drew on detailed survey data. This was collected by Keystone Strategy and Microsoft via in-person interviews with senior technology executives at more than 100 Fortune 1000 corporations, with combined revenues of over $4 trillion. The survey covered a comprehensive set of technical frameworks and advanced tools developed by digital natives in data analytics and AI adoption. Each corporation surveyed was chosen for its traditional operating model, which can negatively impact the adoption of new digital technologies. These companies are often constrained by complex legacy enterprise systems that have been created and modified over long periods of time.

The researchers identified ’data architecture‘, in other words, the way in which an organisation’s logical and physical data assets and data management resources are structured, as a crucial element of successful digital transformation. Sam Cao, an Assistant Professor in the Department of Entrepreneurship, Innovation and Technology at Stockholm School of Economics, explains: “Data architecture broadly captures the technological architecture’s capabilities around processing data streams through multiple layers and pipelines, combining data sets across various sources and locations, and developing and deploying machine learning models at scale. A coherent data architecture allows firms to integrate large amounts of data in an automated and timely manner. And it helps traditional corporations translate technical investments into user-centric co-inventions. Such co-inventions include machine learning applications and predictive analytics embedded across the organisation in various business processes, which increase the value of work conducted by data users and decision-makers.”

The researchers also found that legacy systems negatively affect a firm’s ability to align their data architecture and commit to transformation. Cao says: “Outdated technology components within existing IT systems can encumber digital transformation efforts. Complex software systems further exacerbate the problem. Mission-critical software applications are embedded in many parts of the organisation, making legacy technologies even harder to remove.”

She adds: “The study points to the fact that firms should carefully evaluate these challenges before committing to digital transformation initiatives, and it highlights the importance of a practical architectural framework for managing AI.”

Image: Legacy servers and data architecture coherence. CREDIT: Harvard Business School  

###

Contact the corresponding author: Sam Ruiqing Cao, sam.cao@hhs.se