Key performance indicators (KPI’s), data driven decision making, these are notions that proliferate throughout the modern business world. KPI’s are measured and recorded to track performance and business decisions are rendered with the use of this supporting data. The intent behind this movement is to provide managers and leaders with more information and drive better decision making.
Digital marketing has always been a key contributor to this rise due to the proliferation of data usage throughout its relatively brief history. From the beginning, measurement of digital media and website activity has been readily accessible and experiencing a steady rise in the availability of more detailed data. Managing digital campaigns, website traffic and content has always been reliant on metrics as an indicator of performance.
From the digital marketer perspective, being data-driven in your decisions is arguably the same as breathing to live. However, this engrained reliance on data to make decisions can pose significant problems as well, even in an arena with so much experience and success in the use of data insights. While these issues can apply to any industry or situation reliant on data, digital marketing and its ease to record individual outcomes, track actual users and more make it extra susceptible. In addition, it often lacks the wealth of more qualitative data options that more mature industries are used to relying on.
The first and most important issue relative to use of data in decision making is its relationship to the actual goals of the organization. Key performance indicators are meant to bridge this connection between data measurement, the tracking of performance and achievement of these objectives. When properly designed, KPI’s record direct results related to achievement of goals if not the actual completion of the goals themselves. The best examples of these are KPI’s tracking online sales for an ecommerce website.
Business organizations exist to generate revenue and online sales directly contribute to this goal. Product performance, traffic conversion rates and other similar KPI’s also showcase best practices when designing, measuring and acting on key performance indicators that directly tie to and support the achievement of organizational goals (like revenue). Despite having a clear understanding of this concept and full intention to measure and act on things that best support the business, digital or otherwise, data can still lead managers astray.
Here are a few examples where KPI’s can prove to limit or distract from goal completion and where data reliance can lead well-intentioned managers astray.
Too Much Data Available
Today, there is more data available to managers and leaders than they can possibly process. Technological advancement has greatly expanded what can be effectively measured and stored. The problem this creates is an abundance of information that is more likely to overwhelm than assist in making better decisions. In digital marketing, this is especially poignant given the secondhand nature to utilize data to manage operation and performance. As more and more in-depth details are made available, more of these metrics get incorporated into the decision process. It is often assumed that more detail paints a clearer picture of what is actually happening.
This, however, is a trap that digital marketers can easily fall into. As more metrics are utilized, the objectives one was originally measuring for get increasingly clouded. Too much data quickly becomes noise that obscures and distracts from the objectives KPI’s were meant to measure against. This is where the old adage “what gets measured, gets managed” turns into a warning. Key performance indicators that are further removed from goal achievement still get “managed” creating that increase in noise just described rather than serving to make better decisions that achieve organizational aims.
Data Quality and Interpretation
Sometimes, when there is an overwhelming amount of data at ones disposal, the issue of quality can be exacerbated. Relying on data to make decisions assumes that what is being utilized is accurate. This is the underlying premise behind data-driven decision making and its ability to lead to better conclusions and subsequent actions. In most cases, data collection is done correctly but it is never fully guaranteed. When analyzing numerous metrics simultaneously, keeping abreast of quality across all of them becomes much more difficult.
When seeking answers to only one or a few key questions, one can delve more into understanding the data being collected to answer them and better ensure the accuracy of KPI’s. Generally, though, the accuracy issue lies more with a misinterpretation of what the data is actually showing. It might include recorded outcomes that are not relevant or misrepresented like bot traffic. This is an obvious issue that leads to wrongful interpretations of the data due to a misunderstanding of what the metric is actually demonstrating. Interpretation can also be impacted through understanding in how the different metrics are tabulated and in turn, what insights can be formulated.
For example, bounce rate can be interpreted several different ways. Google analytics counts bounce rate as a single page visit. When no further action is triggered, Google is led to interpret the single page visit as a bounce. If a particular page offers exactly what the user is looking for, say an address or article, a bounce is not necessarily a bad thing. When the intent of a page is to drive users to another via a link click, the bounce occurrence is negative. Sometimes, an auto-play video that launches upon arrival might trigger an action that forces Google to interpret it as a non-bounce and count a normal visit. Other, potential analytics tag setup errors could cause similar results as well.
Regardless, a clear understanding of exactly how the metric is calculated coupled with its context taken within the page in question has an impact. Taken with the other accuracy and interpretation issues just discussed, the importance of clarity between KPI and objective is clear. Fewer, more concise questions being answered by more direct KPI’s help to mitigate or avoid these kinds of data issues and ensure better guidance when using data to judge performance and make decisions.
KPI’s not able to Measure Goals Directly and the Use of Proxies
Key performance indicators should be measuring the actual objective itself and any other key actions or outcomes that will lead to that objective. Ecommerce website sales have already been mentioned as a prime example where this can be done in a clear, concise manner. Other cases where alignment of KPI’s to overall business objectives are strong include websites that sell ad space or those that generate leads in support of offline sales.
Websites that drive revenue through advertising sales do so through generation of impressions and clicks for ads across the site. KPI’s measuring ad performance (impressions and click) tie directly to revenue as does the amount of traffic and level of conversion recorded from the audience. Content performance relative to its contribution of impressions and conversion are effective as well. Lead generation measurement offers a similar degree of effectiveness in measuring a well aligned KPI to an offline objective. In these cases, the website does not actually perform a sale or directly account for revenue but does contribute in a measurable way. What is important to note however, is that for the lead KPI to truly be effective, one must also measure lead quality via an offline conversion rate.
This exemplifies the potential danger inherent within KPI’s that do not measure direct goal achievement outcomes. In this case with leads, it is very easy to focus solely on volume which can misdirect limited resources away from whatever needs to be done to ensure higher quality and that, ultimately, is more important to an offline outcome completion. So, despite the online lead KPI being a very direct and measurable means to track performance against an offline goal (sale or contract), it is still laden with potential issues relative to the direction of online/digital efforts toward organizational objectives. If this is the case here, then what happens when such a clear connection to an offline outcome does not exist?
Brand oriented websites can present the best example of situations where no clear connection between online activity and offline outcomes exist. They offer no online sale opportunities and lead forms or their generated leads cannot be connected to an actual offline sale. Instead, they must rely on alternative metric options that serve to stand in place of direct KPI measurements and these can be referred to as proxies. In short, a proxy is a figure used to represent the value of something else. Or, in other words, it is a metric measured as a stand-in for more direct options like leads or sales. Managers relying on proxies as KPI’s likely utilize various engagement metrics that are believed to be indicators of a potential sale further in the cycle.
The issue lies in the fact that the proxy metric is merely an indicator and not a measure of the actual goal achievement. Taking this a step further, the selection of the proxy metric represents an assumption that the action or engagement it measures leads to a final, offline sale. Earlier, the danger of focusing on lead volume versus lead quality was mentioned and the proxy metric presents the same problem. Its basis as merely an assumption poses a danger in the misuse of resources toward maximizing a KPI outcome that might not actually assist in completing the objectives it was meant to measure against. Again, the importance of alignment with overall objectives is established but the idea of proxies requires a bit more explanation.
Proxies and Spurious Correlations
The use of a proxy metric as a KPI, while likely necessary, is largely based on an assumption that the outcome it records does drive offline goal achievement. While true, their selection and inclusion is not random and likely involves a great deal of thought behind them. Many times, regression analysis is used to determine the correlation of various engagement actions to the ultimate outcome the organization is striving for. Actions with a high correlation rate are deemed to be strongly associated with the final, offline outcome and are selected as the proxy KPI to track performance. They are assumed to have a causal relationship that justifies their selection to measure performance.
What gets lost over time is the fact that they do not directly tie to said outcome, like a sale. They only “indicate” that the user that performed them “might” make a purchase or sign a contract, not that they actually will. Further, the use of correlations itself is fraught with potential issues. Correlation rates should be at least 95% or higher to be considered statistically relevant. Rates that are lower may appear to be strong indicators but in reality cannot be relied upon. What’s more, high correlation rates, even those pushing 99 to 100% can occur in the most ridiculous of circumstances. These are called spurious correlations and can demonstrate high correlation rates in inexplicable, non-causal relationships. Examples include drownings and ice cream sales or murder rates and Microsoft Explorer usage. These high correlation rates supposedly demonstrate a causal relationship in situations where they simply do not exist.
A key performance indicator’s purpose is to measure against the achievement of overall goals. The information they provide is meant to be used to improve decision making and lead to better or more numerous outcomes. While the use of proxy metrics as KPI’s may be necessary, it is very important that the true nature of their relationship to the overall goals is always recognized. Focus can easily shift to maximizing the proxy KPI and that may very well be a misdirection of resources away from other efforts that might benefit objective completion more. Their analysis can be useful as an indicator as to how a website is contributing to an offline goal but they should never be set above the goal themselves.
Conclusion
KPI’s are a quantifiable measure used to evaluate the success of an organization, employee or website in meeting performance objectives. They are central to the data-driven decision movement and have been a big part of digital marketing since its inception. However, a KPI’s usefulness is directly associated with its relationship to the objectives it was meant to support. Several examples were discussed demonstrating how a key performance indicator can misalign with important goals or misdirect decision making.
Alignment with an organization’s overall objectives requires diligence in the digital marketing arena, especially today. The proliferation of more data sources and the reliance on them to make decisions offer challenges to this as well as benefits. Management needs to be keenly aware of what their KPI’s are actually telling them to ensure that their digital marketing efforts and resources are directed toward outcomes that will help their organizations achieve their goals. Culling of data utilized in decision making and review of KPI’s to ensure alignment is a best practice that will lead to better fulfillment of the promise data-driven decision making offers.
novelspin.com 