As early as 2014, a McKinsey study found that companies that are data-driven are 23X more likely to acquire customers, 6X more likely to retain them, and 19X more likely to achieve profitable results. These are fantastic numbers. So it’s no surprise that organizations over the past decade have enthusiastically amassed data, invested in analytics talent, and set up data infrastructure.

Yet, only a few businesses are truly able to put data at the heart of their operations and unleash its power. In most cases, tons of data is collected, monitored and reported—but very little is actually put to use. In a Forbes essay, Jarret Jackson, strategy expert and former Head of Data-driven Insights at Fidelity, points out these common data misses:

• Leaders monitor performance assiduously but rarely use data to find new insights for differentiation or growth.
• Businesses myopically rely on past data to make future decisions without considering external factors and inputs. (“We all know stories about the marketing exec who became CEO only to be fired because he didn’t focus enough on operations — and vice versa.”)
• Business users who don’t get unrestricted access to data end up making ineffective decisions (“if digital product managers don’t have usage statistics for everything customers can “do” on the pages they manage, how can they tell if the new functionality that they put on a particular page works?)

How do a few businesses get it right, and why do so many get it wrong?

The difference lies in the data environment they create and nurture.

What is a data environment?

Simply put, the ecosystem — i.e. the technology, systems, people, and practices — that helps an organization use its data to produce actionable insights.

Building a data environment is never a one-and-done exercise. The best data environments are nimble ones that keep pace with the business as it evolves. Because when the data ecosystem moves in lockstep with the organization’s journey, magic happens! 

In this deep-dive, we’ll look at the various components that make up a data ecosystem, and how they evolve in tandem as the business matures. If you’re already familiar with the basics, skip ahead to our six key recommendations to build a strong, agile data environment.

The five pillars of a data environment

When we say data environment, we are usually talking about a confluence of entities - the data itself, the tools that aggregate and analyze it, and the people who use it. Let’s look at the evolution of each through the journey of an organization.

1. The Data

The first pillar (no points for guessing) is the data itself. More specifically, how your organization’s data is stored, how easy is it to access, and how connected your data systems are. 

Often, in the early days of a business, what little data there is exists in an ivory tower - accessible only to CXOs and a few data analysts. But as the business evolves, there should be a conscious effort to break down barriers that prevent people from gathering, accessing and analyzing data - with the right guardrails for security, of course. In short, there should be a move towards democratizing data.

This also includes the transformative process of breaking data silos and merging data sets. When you string together the beads of information lying in CRM tools, billing tools, product usage tables, and so on, you have a superset of data that will unlock insights you were never able to see before.

2. The  Producers

These are the people who make sense of your data. While your business is in its initial stages, you may be working with a barebones business intelligence team to crunch numbers for your strategy meets. 

As your business increases in complexity and the volume of data swells, you will need data engineers and analysts, to extract actionable information from the data. 

Next come the data scientists, typically brought in when your enterprise requires predictive models, personalized recommendations, or more sophisticated analysis. 

And finally, as the organization's reliance on data grows, and it becomes imperative to have a strategic approach towards data governance, compliance, and aligning data initiatives with business objectives, a CDO - Chief Data Officer - becomes essential. For perspective, a study by NewVantage Partners suggests that in the past ten years, the number of Chief Data Officers within organizations has increased sevenfold, going from 12% in 2012, to 82.6% in 2023.

3. The Consumers

Who are the chief consumers of data in your org? At a nascent stage, it may be limited to the CXO team and some BU leaders. As data maturity grows, product management, sales, support, and marketing heads etc begin to use data to take strategic calls.

At its most evolved level, data-drivenness is in the very DNA of the company. In such a setting, it is second nature for every employee to ask, where's the data to support this?

‍

4. The Insights

A firm with a budding data environment might be using data largely to answer the question of ‘what happened’. A simple example is the development of a dashboard for top management. At this stage, business intelligence comprises basic insights such as business metrics reporting, trend analysis, making sense of anomalies etc. 

This undergoes a palpable change as the environment matures, and the question is extended to  ‘why did it happen’. This evolving data environment understands the interplay between different metrics. It starts looking with intent at return on investment, attribution of success to various underlying factors, and channel efficiency. Leaders begin keeping an eye on lead conversion rates, account health, and how data can improve these matrices. 

At a more advanced level come predictive analytics and prescriptive analytics - what is likely to happen/what is the best course of action. 

For example, data assets to equip the sales team with predictions of customer behavior, revenue forecast for opportunities etc. With the democratization of machine learning, data analysts can now quickly operationalize initiatives that would have until recently been prohibitively expensive.

This is not all. Beyond the realm of quantitative data, lies the richness of qualitative text data. Today, few organizations efficiently analyze the corpus of text data generated in customer interactions and on social media. This analysis, which is now possible through powerful text analytics engines, can be a powerful tool to understand consumer motivations and inform decisions about product development and customer experience. 

5. The Software Tools

A business typically starts with simple tools and adds layers of sophistication later. Tools like excel sheets and PowerPoint are the starting points for basic data crunching and sharing of insights. However, this system remains manual, and there is a significant lag between an event and its reporting. Analysts depend on developers to provide them with data dumps. This infrastructure starts creaking pretty quickly as the volume of data increases. 

Next, more complex data platforms begin to develop -  with ways and means for data ingestion, data discovery, and processing, along with a query console. This system now enables the data teams to answer questions quickly and have set procedures for repetitive tasks. At this stage, there is a relay race between data engineering teams and data analysts to turn around and answer business-related questions using data. 

Over time, systems keep getting more automated, with lower response times, as the data team becomes more experienced and starts understanding the business context. 

In a mature data environment, all kinds of structured and unstructured data can be handled, using tools such as Hadoop, Snowflake, or Databricks environment, and data visualization tools like Sisense, Power BI, Tableau etc. While this evolution can take years, executive sponsorship and a careful consideration and reckoning of the right software tools can leapfrog your business along this journey.

Evolution of Data Environment in a Firm

Note: Elements within the same row of the table may not happen simultaneously.

The data environment must evolve with the business.

‍

The Virtuous Cycle of Data Analytics

For the best return on investment, all the pillars we spoke about have to grow in tandem. There isn’t much point to investing in an expensive, sophisticated software when you’re sitting with patchy, insubstantial data silos. Nor is there a justification for bringing in data scientists or a Chief Data Officer when your data machinery is still nascent and figuring out its way.

That said, there is a natural tendency for these pillars to evolve together, because of a virtuous cycle at play. More structured, comprehensive data leads to better insights. This in turn leads to better understanding from the consumers and consequently, more incisive questioning. This leads to adoption of software with better answering capabilities, and more granular presentations of data, leading to even more sophisticated questions - and so on and so forth.

The Role of Leaders in Data Evolution

When it comes to setting up a data environment, executive sponsorship can cut through and clear many of the initial hurdles - skepticism, data quality issues, and resistance from business stakeholders. So how can we, as stewards of the data journey, enable this? Here are some recommendations to build a data environment that evolves with your business. 

1. Embrace Data Evolution:  Plan for a data environment that follows the trajectory of the company. Continuously assess the data environment's needs and capabilities, staying adaptable to changing demands. Update tools, technologies, and skill sets regularly to keep pace with evolving data requirements.
2. Foster a Data Culture: Secure executive sponsorship to back the data team against initial hurdles and foster a data-driven culture within the company. Ensure easy access to data analysis for sales and marketing teams by creating assets like pipeline evaluation and buyer personas. Enable outbound sales with attribution models to measure the impact of various outreach efforts. You could also consider positively reinforcing data use by factoring it into performance evaluations. 
3. Select the Right Tools: Invest in appropriate software tools that suit the organization's needs and budget. Make a concerted effort to break data silos, thus ensuring clean, trusted, ready-to use data and a more holistic view of data for decision-making. 
4. Invest in Analytics Talent: Build a diverse team of skilled data analysts, scientists, engineers, and architects to drive the data agenda across the organization. Offer training and mentorship to improve their data proficiency. 
5. Harness Qualitative Data: Utilize powerful text analytics engines to analyze qualitative data, such as text data from customer interactions and social media, for better product development and customer experiences.
6. Stay realistic: While your data vision might be sky-high, define clear objectives for the data environment in the short and medium term. This can include developing dashboards for top management, equipping sales teams with predictive insights, and measuring ROI for sales and marketing efforts, among other things. 

As you try to orchestrate or influence your organization’s data evolution, you may find it a Herculean undertaking. But it is one well-worth investing time and resources into, because having a coherent data environment is not a nice-to-have—it is mission-critical.