It’s an interesting time to be in analytics for sure. Maybe things look bleak and scary as more and more of the difficult and tedious work can get automated away with AI. It’s hard to see the skills that took you years to master accomplished by a machine within a few seconds. However, depending on where you work can make all the difference in the world.
See, the secret to successful AI implementation isn’t just getting the product or engineering team to plug into your database. There’s one key thing that needs to happen.
Treating data like an asset.
In most of my working career as an analyst, there was a similar common thread that I experienced. Data was evidence not an asset. Even at companies that attested to be “data first” or “data driven” the leaders treated data as merely the citation that backed up their claims of needing more headcount.
I hear you now, “What do you mean take care of data. It’s just data?”
Again, data is an asset. Much like your house is an asset or your car. Hear me out.
When you decide to buy a home there’s a lot to consider, even before getting to the purchase. You will consider if the area is likely to develop and increase the value of your home. You will consider the location and ease of access to the places that you want or need to go. You think about your future and whether the house has enough rooms for all your future children that you plan to have. You go into buying a home with a strategy and a decent understanding of not only what you need now but what you will need in 5 years.
This is the data strategy. I’ve seen organizations approach data management with basically zero strategy. Often, they don’t even think about handling data points until they have stood up a new process or product then have a shocked face when told they need to wait two more months to even get the answer to if the process is working or not. Going back to our house analogy, they bought the house without looking at it. Now they are shocked that the toilets are missing, and it needs some renovation work to even be habitable. Now you have to explain to exasperate partner that the house was a great investment despite the work. The sewage plant next door is actually a good thing because the smell will deter potential burglaries.
As far as I can tell, what is happening is these individuals make choices and establish their ideas without ever thinking about a few key questions.
1. What is success for this project?
Some leaders today seem to be convinced that success is just making something happen aka productivity theater. They were asked to roll out a new campaign or a new project and never consider what the value is. In many cases, this is the result of a poor C-Suite who have anxiety anytime something isn’t “happening”. Sometimes this is just a manager or director who wants to pad out their yearly review accomplishments.
As an analyst, this is a struggle because without the hypothesis (aka metric of success) built out before the product you may find yourself cobbling together data points to insinuate the desired metric. Additionally, there’s a tendency of leadership to change their actual measure of success once they have already invested their time, energy, and reputation on a project. They have a vested interest in the “success” metric being something successful. The “I’ll know it when I see it” phrase is a very clear indicator that you have this kind of stakeholder.
One personal experience of this was working on a report to understand if new processes were improving the business. Sounds simple until you ask, so what does that mean? Are we wanting this to be done faster, with less errors, they are just done on time, it is generating more money, it is more efficient? Now this got even more sticky as the team was not sure what their primary purpose was to begin with. Was it to bring in more profit, generate less issues, who knew… they sure didn’t. To be fair, their leadership’s leadership didn’t really know what their purpose was either. They had a specific task, but no one could answer what doing their task well even meant.
I come into meetings asking what key metrics we need to be able to show only to be met with a response of “all of them.” No, no this can’t be real. But yes, it is very real. After a grueling several months, the metrics are built. There are pages and pages to the report but no coherent story. Does this table explain this chart? Not at all, it only makes the whole situation muddier as any questions to why X metric was so bad was then references to how Y metric explains it away.
Honestly, it was brilliant if it didn’t cost me so many sleepless nights to just create.
Additionally, the report required multiple data sources that all needed to be managed with many conflicting entries that needed to be reconciled. The calculations made generating it take hours and often “tweaked” right before sending them out because, “well I don’t think that should really count towards our numbers,” but last week it was fine because we were in the green zone.
2. How will this key information be stored, maintained, and protected?
The other part of this situation is the importance of managing the data that should inform these success metrics. In practical applications, just the way the data is stored can affect the effort needed to extract what’s needed. There may be extra steps needed to extract data from an application. Maybe there are limitations on what can even be obtained that need to be accounted for in the process of development.
An example of this is salesforce reporting. The general reports have a limitation of so many records being displayed at once or only a certain number of calculations can be created with a single report. These limitations need to be communicated with stakeholders. If they cannot tolerate the limitations, then time, effort, and resources need to be taken to find a solution to those limitations. It’s a very simple calculation that I have seen stakeholders react to like I just tried to force them to learn linear programming.
Now, there are many simple and low-cost methods to solve these problems, but they generally create bigger problems down the line as they are unsustainable and time-consuming. Going back to our salesforce example. I have in my career asked to solve the record limitation by manually downloading a csv file every day to a specific folder. I was expected to do this in perpetuity. It would be a task I would pass on to those who would come after me like some golden pocket watch that made it through the war.
I did a cost estimate of the time it took for me to do the whole process, and it cost the company about $25 dollars a day for me to just download a file, look at it, and move it to the right place which roughly equates to $6,250 a year.
This leader chose that solution as opposed to any other such as:
1. Just deal in aggregates as no one was doing anything with the record level data they requested.
2. Limit the process to once a week instead of daily even though from the usage metrics alone, I could see they were only accessing the report right before the big senior leadership meeting.
Backing up a bit, we also have to consider maintaining the data and ensuring the accuracy. So again, in this salesforce example (same person and place actually), they wanted to track a specific response in a certain field that was entered into salesforce. This would have been simple, except they continually changed this field’s picklist. Repeatedly. Imagine nailing Jello to the wall but the house is also on fire and therefore the Jello is also melting.
I would pull the report daily, update the report and review and then report back that the frequency of the key response was in decline. YAY! Everyone cheers and the teams celebrate. Two months later I noticed that all the responses are in decline, which triggers the QA process. This isn’t normal. I wandered into the report builder and toy with the filters only to find that new responses have been added and the key response we were tracking was just ever so slightly reworded. I completely redo the report and find that, actually, our key response metric is increasing and dramatically so!
I let the bad news fly and in swoops top level leadership to grill me (the analyst) on how I let this happen! To which I inform them, without proper management of data points such as a change control process, these things will happen and continue to happen. People are mad, feelings are hurt, but still no one wants to lose “flexibility” and “agility” to just agree to a process to properly inform the data team of these changes.
Ok, so that was a lot. I really liked most if not all the people I have worked with (so if they read that and know I am talking about them, just know that I still adore you) but those experiences really demonstrated a fundamental problem in analysis which is the lack of data strategy. Instead, data was just the evidence or the afterthought. Neither leader came into the situation planning out how data was a key part of the project.
So how would I change the situation as a leader?
Require that all projects, campaigns, or whatever create a data strategy as a part of the proposal. Think before you do.
Your people need to know what matters to their objectives and agree to what they will hold themselves accountable to. This doesn’t mean they need to map out the exact location in the database but they should be able to give a laymen’s version of their metric along with any important caveats such as working days only or if they want the median as opposed to the average. Changing this metric or any data point should be a big deal and require justification that should be documented. These are your business rules. There should be a clear understanding of how your metric aligns to the goals of your team. If you don’t know your goals, then someone somewhere needs to get you on the right page. Sure, the analyst can keep up with the documentation but it’s up to leadership to enforce compliance.
Data should be protected as an asset with consideration given to what value it provides, if the data you want is actually helpful, do your requirements even make sense when you do ask for data or are you wasting precious time and resources.
So I opened this whole thing with AI and then never talked about it. That was intentional as the problem with AI implementation won’t necessarily be if it can make the bar chart or the plot you need without paying a data analyst tons of money. But it’s not going to think that critically when your leadership team wants to skip data as a key part of the strategy. Sure there are the few unicorns out there that get it but most don’t.
If you are an analyst reading this, take away some key points here. The things that we used to do daily such as manipulating excel sheets or making dashboards may go away but eventually companies will realize that they need someone to keep the data strategy in line and hold people accountable. My guess is that we all lose our jobs for a few years and slowly the positions as “data strategist” or “data governance manager” will pop up.