by Leon Tchikindas
Republished from InformationWeek, July 13, 2017
Being a data-driven organization is an ongoing process — you need to invest in the right team, infrastructure, software, and processes that can promote lasting change.
More and more companies are describing themselves as “data-driven,” but are they really? When we talk about being data-driven, we mean more than just passively collecting data and reporting on end-of-month financials. Data-driven organizations inform their decision making with key insights drawn from multiple data sources and nuanced analytics.
In my experience, I’ve learned that a company can’t become data-driven in a vacuum. When non-technical or non-analyst teams start to dabble in data collection and self-serve processing, there’s a high risk of analytical bias and erroneous interpretation, which can lead to misinformed decision-making. Gartner predicts that through 2017, 60% of big data projects will fail to go beyond piloting and experimentation. To ensure this transformation sticks, you’ll need a qualified and specialized team supporting your efforts.
Identify the core team
To get a team up and running, you’ll want a data engineer, an analyst, a product manager (this is a plus, not a strict requirement), and an executive sponsor. Generally, this group will sit in marketing or finance at first. As the team begins to deliver value, it may start to support product and sales before spinning out into a group all its own.
Your data engineer is responsible for moving and propagating access to data. Rather than analyze and interpret the data, their chief mandate is piping it to the right places. How technical they need to be will depend on the analytics stack, which we’ll get to shortly.
Your data analyst should be focused on answering business questions using data. They know SQL, and are comfortable in a few other languages such as Python or R. This is a critical role, as your analyst will effectively be a bridge, translating data into business insights and coding business insights into meaningful reports and metrics. This role will grow to be even more important in the future.
To help this team effect change, you’ll also need an executive sponsor, preferably someone at the VP level or above with some technical background. This person needs to believe in the team’s mission and can help tackle inevitable roadblocks from engineering, legal and IT, as well as concerns from stakeholders.
Believe me when I say that this role is crucial. A company I worked at earlier in my career had a particularly convoluted data stack. There was no redundancy, the databases frequently got corrupted, and concurrent work among multiple analysts was challenging. Despite the analytics team’s advice, the stack was not upgraded until a power outage caused some data loss. Having an executive sponsor could have prevented this.
Choose the right analytics stack
Your team needs to precede your tool investment. Prematurely opting for a self-service solution often leads to data misinterpretations and misdirection. Our confirmation biases only compound the problem as we look to find insights behind the numbers. These false insights can instill a distrust in data, putting the kibosh on your data team’s ambitions before they’re able to get started in earnest.
Let your data team drive the requirements gathering and RFP process for analytics products. They’ll be more attuned to how the data will need to be piped in, as well as data shape and volume.
Choose a tool — or a suite of tools — your organization can grow with. But once you’ve found a tool, don’t plan to rest on your laurels. Your team should spend about 10% of their time exploring new technologies, as data is constantly evolving.
Be prepared for challenges
A company doesn’t become data-driven overnight or even in a quarter. You’re dealing with fundamental organizational shifts and change in mentality; even if people are philosophically on board, when you’re asking them to change how they’ve always done things, you may run into some pushback. But persevere, and the benefits will be tremendous.
Here are the five major challenges you should anticipate.
- Technology: Plan to run into a few bumps in the road when it comes to deciding how to record, store, and process relevant data.
- Budget: Early on, ROI can be difficult to rationalize. This is where you’ll need your executive sponsor to go to bat for you.
- Perception: If the pain of blind decisions or the need for accurate data isn’t immediately apparent to key stakeholders, organizational penetration can be difficult.
- Hiring: Your first few hires will have to be multifaceted — e.g., analysts with some engineering experience, engineers who can interpret data, clean data and come up with business insights, etc. These people are both valuable and hard to find, let alone hire.
- Bandwidth: Once your team is up and running, and the organizational appetite is there, expect to be flooded with requests. It’s easy to find yourself prioritizing and re-prioritizing requests. In this case, scaling is not at all linear; the next few hires will likely firm up your foundation, not produce more throughput.
Where do you go from here?
Building the right data team is a critical first step in becoming more data-driven. Such organizational change can be fraught with friction and you’ll often be tempted by shortcuts and compromises. Resist. Being a data-driven organization is an ongoing process — you need to invest in the right team, infrastructure, software, and processes that can promote lasting change.