8 KPIs to Measure Big Data Business Impacts - From KPIs to Benchmarks

DataBench selected the metrics shown in the figure to measure the business impacts of Big Data.

The business KPI definitions are based on business and marketing literature, but these definitions have been simplified and operationalised to allow measurement through business surveys. This approach is one of several options for the measurement of technology business impacts, an approach chosen for its applicability to an objective of the project – namely, the need to estimate business-impact-related industrial benchmarks that are valid for European industry and differentiated by sector and company size. The data collection process is illustrated in the next paragraph.

Since IDC focuses on emerging technologies and market forecasting, we have developed a methodology based on business surveys that enables us to collect data about the overall average impacts of technology investments based on companies’ own evaluations. Since companies do not carry out investments without an economic or business rationale, this data has a sound basis, even though it is technically a result of the opinions of respondents. To ensure these opinions are valuable, and fact based, we have employed several methods, including:

  • The careful selection of the role and responsibility of the survey respondent (who must have the relevant knowledge)
  • The careful quality control of survey data, discarding incoherent and unbelievable answers, as well as the careful management of the survey itself (for example, rotating answer options so that no ranking bias exists)
  • Statistical elaboration techniques, discarding outliers and extreme values, by checking the maximum and minimum data points
  • Long experience in survey management and a reliance on experienced and well-known interviewers
  • Comparative analysis of the resulting data with literature and other sources about the business impacts of technology innovation

All these methods have been employed in this project to define and collect data about the business impacts of BDA and to calculate industrial benchmarks. Table 1 and Table 2, below, provide details of each KPI, its metrics, and the measurement results.

Thanks to our methodological approach, the business KPIs selected for the project are valid metrics and can be used as benchmarks for comparative purposes by researchers or business users for each of the industry and company-size segments measured. These indicators are:

  • Benchmarks, because they represent the average improvement achieved by business users and can be used for comparative purposes, as a target or as a best performance metric
  • Of industrial significance, because they apply to the actual and emerging needs of specific industries and specific company-size segments
  • Of European economic significance, because the benchmarks are measured for all the relevant European industries and company-size segments in which Big Data can have the highest impacts
  • Useful for linking technical and business benchmarking, because they are also measured for the main use cases, consisting of the application of Big Data technology to particular business processes and/or application domains, thus enabling the user to match the expected business improvements with the type of technology performance needed to achieve the business goals

We differentiate the KPIs into two different categories, according to how they are evaluated and what they measure. The first batch measures the quantitative business KPIs, which are profit increase, revenue increase, and cost reduction; the second batch comprises soft qualitative KPIs that cannot be easily or straightforwardly calculated, so a range of expected improvements is provided. Within this second set of KPIs, we include time efficiency, product/service quality, customer satisfaction, new products/services launched, and business model innovation. 

The business KPI definitions are based on business and marketing literature, but have been simplified and operationalized to allow measurement through business surveys and case studies.
They have been measured by industry and company size class. They measure the average/median improvements achieved thanks to Big Data by European business users in each industry and company size class. Therefore, each KPI can be used for comparative purposes, as a target or as a best performance metric.

More details can be found in DataBench deliverable D2.4.