Researchers worked with MnDOT technical experts to develop a method for identifying the financial and other benefits of MnDOT research projects. They developed a seven-step process for quantifying benefits and applied the process to 11 recent MnDOT research projects. Results showed that these projects were yielding significant financial benefits.

“We have very high expectations for the research dollars we spend,” said Hafiz Munir, Research Management Engineer. “MnDOT Research Services & Library. Following this project, we now ask investigators to tell us upfront what benefits their research could achieve, and we have improved our internal process for tracking and assessing the quantifiable benefits.”

“A lack of before-research data on the transportation activities being studied may be the biggest challenge to quantifying the benefits of research on Minnesota transportation needs. Other states are also trying to do this, but they use informal or ad hoc processes,” said Howard Preston, Senior Transportation Engineer, CH2M Hill.

What Was the Need?

MnDOT Research Services & Library manages more than $10 million in research each year, with 230 active projects covering everything transportation-related — from subgrade soils to driver psychology. Communicating the value of these research investments is an important component of transparency in government, a core interest in Minnesota.

Quantifying the benefits of research projects that lead to innovations such as new and improved materials, methods and specifications is important to MnDOT and its customers. However, because MnDOT conducts such a wide variety of research projects, it is challenging to assess the benefits that will, when applied in practice, result in quantifiable savings of time, materials or labor, or that will lead to safer roads and fewer traffic crashes.

What Was Our Goal?

MnDOT undertook this project to develop a more systematic method for identifying and measuring the financial and other benefits of its research in relation to the costs. The goal was to develop an accessible, easily applicable process that could be pilot-tested on a selection of MnDOT research projects from recent years.

What Did We Do?

MnDOT provided researchers with documents about benefits quantification practices to review and with the results of a survey of state departments of transportation on their approaches to quantifying research benefits. This review identified few states that had developed formal guidelines for assessing research benefits, and none were easily applicable to MnDOT procedures.

After reviewing the findings and consulting with MnDOT technical experts, investigators recognized that any procedure for quantifying benefits should be rooted in current MnDOT research processes. Researchers worked with a number of MnDOT offices to identify research projects that were suitable for assessing financial and other benefits from research results.

In addition to identifying projects for benefits analysis, investigators and MnDOT identified categories of benefits and developed a seven-step process for gathering and organizing cost data for various project types, applying a benefits assessment process and comparing benefits to research cost.

What Did We Learn?

The research team performed benefit-cost assessments for 11 projects. Six of the assessments had high confidence levels. One challenge in developing a uniform process included refining the complex range of cost input categories, input data options and research objectives associated with the research projects. Assembling and organizing before-research data, even for fairly simple maintenance activities, proved particularly challenging and impeded the development of benefits assessment processes.

Investigators developed a user guide, a training presentation and a quantification tool — a complex set of spreadsheets for inputting data and calculating comparative benefits. The quantification tool should eventually develop into a user-friendly software package or Web interface.

Based on the analysis of cost and savings data, the 11 research projects showed significant benefits. In one 2012 project, investigators developed an inexpensive baffle that is inserted into stormwater sumps and slows the flow of water in and out, allowing more contaminated sediment to settle rather than being carried into streams and lakes. Re-search to develop the baffle, at the University of Minnesota St. Anthony Falls Laboratory (SAFL), cost $257,000. The cost to purchase and install the baffle is about $4,000 in Minnesota compared to $25,000 for more traditional stormwater mitigation solutions. Use of SAFL baffles in Minnesota is projected to save the state about $8.5 million in equipment, installation and environmental costs over a three-year period.

In total, the research cost of $1.98 million for the 11 projects analyzed is expected to save an estimated $68.6 million for MnDOT and Minnesota cities and counties over a three-year period, for a benefit-to-cost ratio of about 34-to-1. The expected savings will be enough to pay for the research budget for six or seven years.

What’s Next?

MnDOT has added quantification-of-benefits elements to its research proposal evaluation process, and since late 2015 has asked potential principal investigators to supply information on the current costs of the activities they propose to study and improve.

Since 2016, research project awards have included a request that investigators develop quantifiable data resulting from their research activity. The awards offer additional funds for that work. Investigators now provide a brief memorandum within the first 90 days of the project describing how they will quantify benefits, and in some cases presenting preliminary data. At the end of the project, these investigators describe their quantification process and results. MnDOT has tracked this information in a database, finding that about three out of every four projects show potential to yield quantifiable benefits.

This post pertains to Report 2017-13, “Development of a Process for Quantifying the Benefits of Research,” published July 2017.