Evaluation of the AgriInnovation Program Stream A:  Research Accelerating Innovation – Summary Report

March 30, 2017

Purpose

To provide a summary of the AgriInnovation Program Stream A: Research Accelerating Innovation (referred to as Stream A) evaluation report and management response and action plan.

Evaluation Scope and Methodology

The Evaluation of Stream A was included in Agriculture and Agri-Food Canada's (AAFC) Five-Year Evaluation Plan (2014-15 to 2018-19) and fulfils evaluation requirements under the Financial Administration Act, as well as the Treasury Board Policy on Evaluation (2009) and subsequent Policy on Results (2016).

The AAFC Stream A program is managed by the Science and Technology Branch and consists of internal research, development, and technology and knowledge transfer projects.  The program’s budget was $150 million from 2013-14 to 2017-18.

Methodology:

• The evaluation was undertaken by the Office of Audit and Evaluation, with support from management consulting firm Ference and Company.
• The evaluation used multiple lines of evidence (including document, file and database review; a benchmarking analysis of similar programs; interviews with internal/external representatives; site visits to six AAFC research centres; and case studies on ten projects) to assess program activities undertaken between April 1, 2013 to March 31, 2016.

Evaluation Findings

Relevance:

• The evaluation found that there is a continued need for Stream A scientific research that helps the agriculture and agri-food sector to optimize productivity, identify and mitigate various production risks, capture market opportunities and keep pace with sustainability considerations. Scientific research is a key driver of product and process innovation required to address the competitiveness and profitability needs of the agriculture and agri-food sector. Studies suggest that long-term research has the potential to yield some of the highest returns available from public investments and producer groups recognize that investment in research and development is key to the success of the sector.
• The objectives of Stream A are well aligned with the priorities of the federal government and the strategic outcomes of AAFC, particularly with respect to the priority to invest in agricultural research^{Footnote 1} and AAFC’s Strategic Outcome 2: An innovative and sustainable agriculture, agri-food and agri-based products sector.
• While there are many stakeholders involved in agricultural research activities, AAFC has an important role in delivering the type of research and development funded through Stream A because it has a public good focus (for example, improving sustainability of agricultural practices) or is too far from commercialization for private sector to realize a return on investment in the near term, yet it provides a critical base for more targeted adoption and commercialization projects. In other research organizations, such as some universities, scientists tend to focus on theoretical discoveries without a clear application in the markets.  Private sector organizations are inclined to focus on applied research that can generate faster returns.

Effectiveness:

• The evaluation found that, three years into the five-year program, Stream A has made considerable progress towards the achievement of its intended outputs and outcomes. As of March 31, 2016, Stream A has committed $59.3 million in funding to 159 research and innovation projects. Most of the first round projects are complete (82%), with the majority starting in 2013-14 and ending in 2015-16. As of March 31, 2016, these research projects has resulted in:
  • 278 peer reviewed articles and 419 presentations at conferences events and annual meetings.
  • 37 new products, processes, and technologies relevant to the minimization of catastrophic threats to crops, optimization of livestock efficiencies, and food health and safety; and
  • 15 innovations related to environmental sustainability such as new sustainability metrics tools, new or adapted beneficial management practices, and new or improved decision-support tools.
• One example of these innovations is a national project that resulted in the development of a new wheat variety that is higher yielding and resistant to Ug99 (an emerging fungus with devastating consequences) and other diseases. The project could result in economic benefits to the sector of up to $700 million annually from reduced losses and reduced fungicide use, if Ug99 were to arrive in Canada.
• Several projects have also resulted in additional opportunities for further collaboration and research. According to project reports, 30 projects resulted in 71 follow-on research grants, awards, honours or prizes at the time of the evaluation. Most of the projects reviewed as part of case studies resulted in (or will likely result in) follow-on projects or investments such as investments by AAFC and key industry stakeholders, and new areas of investigation or collaboration.

Efficiency and Economy:

• The major factors contributing to the effective delivery of Stream A projects include:
  • Stream A builds on other Science and Technology Branch investments, thereby taking advantage of complementary science activities and existing infrastructure, such as highly qualified scientists and technicians, research centres across multiple eco-zones, and leading edge scientific technology and equipment.
  • The project selection process promotes science excellence, accountability and directs resources to areas where there is the greatest need and projects which are most likely to have significant impacts.
  • Collaborative research teams with multiple areas of expertise have leveraged internal and external scientific expertise and capacity, and facilitated knowledge dissemination and transfer to the scientific community. On average, Stream A projects involved nine AAFC scientists and three AAFC technical staff. Over one third of Stream A projects involved external collaborators.
  • The program supports activities, such as regular outreach with industry stakeholders, hosting regional meetings with industry and research stakeholders, and distributing publications that highlight major research achievements, that have facilitated the transfer and adoption of innovations.
• The evaluation found that Stream A projects are sometimes constrained by the following factors:
  • The current three year timeframe for funding projects. A longer funding timeframe for projects (for example, 5 years) would allow more time to focus on longer-term research goals, demonstrate program results, and assess the incremental value of continuing the activities in subsequent rounds of funding.
  • Data storage and computation capacity constraints. The evaluation identified some constraints with respect to available data storage and computation capacity, as well as challenges in procurement timelines associated with acquisition processes. This is particularly affecting areas of research related to genetics which is producing more and more large and complex datasets.
• Science and Technology Branch (STB)’s performance measurement and data collection tools are useful for communicating project results, but some issues were identified with the performance measurement data and it is difficult to measure the intermediate and end outcomes of Stream A projects based on the existing performance indicators.

Recommendation and Management Response and Action Plan

Recommendation

In line with the new Policy on Results and the Government of Canada results and delivery agenda, AAFC should continue to refine the program’s approaches to performance measurement to ensure that the indicators used and the performance data collected can accurately and consistently report on the impacts of scientific research activities.

Management Response and Action Plan

Agreed

1. STB is revising all its sector strategies to include logic models to better align with the results and delivery agenda.  Target Date: June 30, 2017. Responsible Position: Director General, Partnerships and Planning Directorate, Science and Technology Branch.
2. STB will create Program Information Profiles (PIPS) for all programming as per the new Results and Delivery Agenda for the Government of Canada. Target Date: November 1, 2017. Responsible Position: Director General, Partnerships and Planning Directorate, Science and Technology Branch.