Tapped potential - evaluation of the Efficiency Studies programme 2006–2017
Since 1999, the Netherlands Organisation for Health Research and Development (ZonMw) has funded research on the efficiency and effectiveness of healthcare through the Efficiency Studies (DO) programme, developed on behalf of the Ministry of Health, Welfare and Sport (VWS). After VWS asked ZonMw to evaluate the programme’s process and impact, ZonMw appointed an external evaluation committee to oversee the evaluation. Consisting of stakeholder representatives (patients, researchers, policymakers and healthcare providers, institutions and insurers), this independent committee commissioned SiRM to conduct the evaluation.
We based our evaluation on insights from interviews, a focus group, desk research and an online questionnaire with lead researchers previously supported by the DO programme, consulting over 60 experts, lead researchers and other stakeholders in total. Health gains and economic returns – both part of the impact evaluation – were calculated for 24 select high-potential projects and their lead researchers all consulted.
The scientific returns of the DO programme are high. Three-quarters of subsidised projects resulted in a scientific publication, 90% of them in international scientific journals primarily belonging to the top 25% impact factor in their scientific field. The programme’s social returns are also considerable: public attention to healthcare efficiency significantly increased in the last ten years, and around 30% of project results are implemented in guidelines.
Projects approved by the DO programme yielded substantial health gains and economic returns. Based on the 24 high-potential projects we selected, the programme generated nearly 7,500 quality-adjusted life years (QALY’s) and a total estimated financial return of €1.1 billion (with €0.3 billion as monetised QALY’s). A total of €480 million were cost savings, including €280 million saved on healthcare spending and the remaining €200 million saved on social costs.
Our evaluation suggests insufficient implementation of research findings, despite ZonMw’s increasingly definitive implementation conditions and assessment criteria. Interviewees report that market players and regulatory authorities need to take a more active role in increasing implementation, the importance of which is evident from the DO programme’s considerably higher potential yields had positive research findings been implemented more broadly. Based on ambitious yet realistic levels of implementation, we calculated that the health gains could have been as high as 13,000 QALY’s (almost double) and the economic returns €4.1 billion (nearly quadruple).
