Over the past couple of years we’ve been hearing about big data. Discussions have emerged online about pigs and hives and new jobs have appeared called “pig architects” which is an excellent job title by the way – imagine meeting a customer for the first time, “Hi I’m a pig architect”, but I digress.
Big data has emerged as companies need to find ways to manage massive amounts of data. By massive I mean really massive amounts, like gargantuan amounts of data, yottabytes for instance which sounds a bit like something out of star wars but equates to 1 quadrillion (however big that is) gigabytes. These numbers of magnitude have been created and one day the Internet will reach those levels, it’s simply a matter of time and then they will need to make up something even bigger. According to the Guardian the Internet was estimated to have 500 billion gigabytes of ones and zeros or 500 exabytes for instance.
So yes big data is well…. big. But in my opinion the challenge has never really been about getting data. As i’ve just demonstrated by throwing a few big numbers around we can talk about yottabytes of data and the human race is using 500 exabytes of data today. Getting data today is not difficult. Getting data today is almost too easy. In fact in my job it would be easy to become overwhelmed by data if it weren’t for one thing.
Key Performance Indicators (KPIs)
I’m sorry it might sound boring but KPIs are what save me from going insane. My advice is if you don’t have a business question in your head, don’t look at data because without a business question you can’t form a KPI.
Looking at data for the sake of it is like looking at the desert and wondering how much sand is there. Let’s expand that a little.
If you’re driving through a desert I guarantee that anyone with half a brain would make damn sure they have enough fuel to get them to the other side. The KPI here is fuel and the amount it takes to get you from point a to point b. The ‘business question’ is “how much fuel will it take to travel over 200 miles of desert?”. The data is your cars mile per gallon and how much it can hold. There are lots of other things you can measure, revolutions per minute, engine speed, time taken, engine heat, oil consumption, but the main question is do you have enough fuel for your journey? The fuel is the important focus point, the key decision point, the main driver so to speak. Fuel was the KPI that dictated your strategy.
Now in the world of big data this doesn’t change. Just like with the scenario above there are lots of things you can measure but there are some KEY ones which drive your strategy.
One big data project I worked on a year or so ago was about using big data to to send an advertising message when a mobile phone subscriber was near an advertisers businesses. Turns out that the business in question had data on everything imaginable about the subscribers, the real time geo location of their phones, how old they were, gender, what their interests were etc etc. But by zeroing in on what was important we were able to focus their efforts to do 1 thing, send the right ad to the right person when it was most relevant to them. We advised they design a target segment of people (pre-defined via their preferences and demographics) be served ads when the right geo-location flag was triggered. Then an ad was sent from one of the advertisers in the area. So we ignored mountains of data and got very specific to the business problem at hand with a much more meaningful amount.
Google does this every day on a gigantic scale with adwords (geo-targeted ads). So does Facebook. So does Microsoft/Yahoo! (albeit in lower volumes). So the secret of making big data manageable is to make it small by focusing on KPIs. The process is then exactly the same no matter what size of data you’re dealing with. Set KPIs. Measure. Optimise. Repeat.
Your Data Strategy
It doesn’t matter whether you’re dealing with big data or not you need to have a well designed business process behind what your doing that sets you up for success. Develop solid KPIs that are rooted in making you more profit.
Big data makes me smile. It’s the latest buzz word. But at least the hype is helping the C level understand more about the field and importance of doing good business analysis. For that at least I am thankful, but if you are the C level beware of the hype as well.