Last Updated November 26, 2018
The Decision-Based Valuation method, proposed by J.B. Stander in 2015, seeks to estimate the relative contribution of data (similar to the Business Model Maturity Index method) while also accounting for data attributes (such as quality and frequency of collection) relative to the decision being made.
The Decision-Based Valuation method is similar to the Business Model Maturity Index method in that it assesses data’s value in terms of the contribution data make to the outcome. The Decision-Based Valuation method also adjusts the value of the data based on how fit-for-purpose the data are by accounting for
This method takes a top-down approach by identifying the use case first and then determining which data are needed to inform the decision (Figure 1).
Figure 1: The Decision-based Valuation method takes a top-down approach to estimate the contribution of data to achieving the desired impact.
The Decision-Based Valuation method progresses from (1) identifying a desired outcome and its potential impact, (2) developing a series of use cases, with the data required, to achieve the desired outcome, (3) adjusting the value of the data based on how fit it is for this purpose, and (4) estimate the costs, (5) calculate a return on investment, and (6) using expert judgment to estimate the relative contribution, and value, of data to each use case.
Clearly articulate a desired outcome that can be quantified in terms of time savings, cost savings, water savings, lives saved, etc. The estimate does not have to be precise and can rely on expert judgment.
Use cases refer to decisions and strategies that can achieve the desired outcome. For each use case, estimate the implementation cost and the potential impact. The value of the decision should be accounted for over the full life-time of the project.
The value of the use case is adjusted based on the frequency of data collection and their quality relative to what is needed to make a good decision.
Frequency compares how often the data must be collected to inform decision-making. The frequency tolerance is the window around which decision-making will be impaired if data are collected less frequently. The adjustment is:
Data accuracy is compared with the required accuracy for a use case. The adjustment is:
When the quality is not known, expert opinion can be used. The recommended categories are:
The Quality Modifier ranges from 0 to 2 and is the sum of the Frequency and Accuracy score. The estimated value of the data is adjusted by multiplying it with the quality modifier.
Stander uses five distinct categories to estimate costs:
If the data are currently used for a single decision point, then the full cost is weighted against the benefit of that decision. However, if the data are used for multiple decisions, then the cost can be equally divided between those decisions. Additionally, infrastructure can be depreciated or amortized annually.
The Return on Investment (ROI) is calculated as:
Note that while the economic ROI might be less than one, the reputational, social, or environmental benefits may still justify the cost of the project. Initially the potential ROI is calculated, but as the project occurs, the actual ROI can be estimated by adjusting according to the Performance Value.
The value of the data can be estimated based on their relative importance to the use case. The importance ranges from 0 (none) to 1 (critical).
The value of the data can also be weighted by the level of difficulty to transform the data into usable information, such that data are more valuable with lower processing time.
The value of the data is multiplied by its relative contribution and divided by its processing value:
The ROI for data is estimated by dividing the adjusted value by the data’s cost.
IoW Alfalfa grows alfalfa and is exploring ways to save money while maintaining productivity. Specifically, they are attempting to reduce energy costs by 10 percent.
IoW Alfalfa wants to reduce energy costs by 10%. Energy costs have increased as groundwater levels have declined. Currently, it costs IoW Alfalfa $0.21 to pump each AF of water one foot in elevation. IoW Alfalfa spends $2.52M annually to pump 60,000 AF from a depth of 200 ft. If they reduce energy costs by 10%, IoW Alfalfa could save $252,000 annually.
IoW Alfalfa developed several use cases to help them achieve a 10% energy reduction (Figure 2).
The data required for making these decisions include:
Because this method is more intensive than the Business Model Maturity Index method, we will only apply the Decision-Based Valuation method to the groundwater recharge use case.
Figure 2: The first steps are to identify a desired outcome and potential use cases. For the purpose of this example, IoW Alfalfa will only explore the groundwater recharge use case.
IoW Alfalfa’s next step is to document the ability of the use case to meet the 10% energy reduction goal. Here, groundwater recharge has the potential to raise water levels by 1 to 10 feet per year for a total of 10 to 100 feet over the 10 year period. This would result in between $12,600 (for 1 foot rise in groundwater levels) and $126,000 (for 10 feet) in energy savings per year (1-5% of the desired energy savings), resulting in between $0.693M and $6.93M savings in pumping costs. By year 10, annual energy used for pumping would be reduced by 10 to 50% (Figure 3).
Figure 3: The estimated value of increasing groundwater level per year (top) and across years (bottom). Lower values are realized if groundwater levels only rise 1 foot, higher values for 10 feet, per year.
IoW Alfalfa would like to revisit groundwater recharge decisions every 5 business days. Data more than two days old would impact decisions, meaning their tolerance for decision-making is 2 days, while the actual data collection frequency is daily. Because the Collected Frequency ? Required Frequency, the Data Frequency = 1.
If the data were collected every 14 days the Data Frequency score would be zero because data frequency is below zero:
IoW Alfalfa knows the accuracy of their groundwater sensors are ± 0.2 ft. However, the decision only requires an accuracy of ± 1 foot. For this decision, the accuracy is 0.8 because Accuracy = (1 Ft – 0.2 Ft)/1 Ft. If the required accuracy were 2 ft, then the data accuracy score would be of 0.9.
Quality Modifier
Since the frequency and accuracy of the groundwater data are fit-for-purpose, the Quality Modifier is 1.8 (Frequency Score + Accuracy Score). Assuming the average annual groundwater level rises 5 ft per year, the average cost savings is $63,000 (2.5% annual savings). Once adjusted, the estimated value of the data increased to $113,400 per year ($63,000 * 1.8) (Figure 4).
Figure 4: Adjust the value of the data based on collection frequency and accuracy
IoW Alfalfa estimated the costs of the groundwater recharge project:
The first year is the most expensive at $39,000. Each subsequent year costs an additional $3,800. By the end of year 10, total costs will amount to $77,000 (Figure 5).
Figure 5: Estimated costs to implement the groundwater recharge project.
The ROI for this use case ranges from 2.91 in Year 1 to 15.49 in Year 10 (groundwater levels now 50 feet higher). The overall ROI over the 10 year period is 85.2 ($6.24M/$0.077M) (Figure 6). IoW Alfalfa revisited this estimate after groundwater levels rose by only 3 feet in Year 1 with energy savings of $68,040. The realized ROI decreased to 2.33 in year 1 and 51.12 over the entire 10 year period (groundwater levels now 30 feet higher).
Figure 6: The estimated potential ROI ranged from 3.49 in Year 1 to a 97.2 after 10 years. The realized groundwater levels produced an ROI of 2.33 in year 1 and 52.13 after 10 years.
The relative importance of each data source were estimated using expert opinion. The sum of the relative importance was 5.25 (Table 1). The relative contribution of groundwater level data, for example, was 0.19 (1.0/5.25). The realized value of groundwater level data in year 1 was $68,040*0.19 = $12,960 (Figure 7). It took IoW Alfalfa 3 hours to collect data from various sources and 10 hours to clean, combine, and convert the data into information (Processing Value = 3.33; 10 hours / 3 hours). This decreased the value of raw groundwater level data to $3,892. The total adjusted value of all data types in year 1 was $20,432, for an ROI of 0.52. However, by year 10 groundwater levels had risen by 30 feet (saving $1.12M in pumping costs) with ROI of 15.35.
Figure 7: ROI for data used in the groundwater recharge use case.