Evaluation of the Minor Use Pesticides Program

March 29, 2018

Abbreviations

AAFC

Agriculture and Agri-Food Canada

FTE

Full Time Equivalent

IR-4

Interregional Research Project Number 4

MRAP

Management Response and Action Plan

OECD

Organization for Economic Co-Operation and Development

O&M

Operations and Maintenance

Executive summary

Purpose

The Evaluation of the Minor Use Pesticides Program (the Program) was undertaken by Agriculture and Agri-Food Canada's (AAFC) Office of Audit and Evaluation (OAE) as part of the Five-Year Integrated Audit and Evaluation Plan (2017-18 to 2021-22). This evaluation examined the relevance, effectiveness and efficiency of the Program. The results from this evaluation are intended to inform future program, policy and funding decisions.

Methodology and scope

The evaluation assessed activities and results achieved by the Program from 2012-13 to 2016-17. To assess relevance, effectiveness, and efficiency of the Program, the evaluation used multiple lines of evidence including document review, case studies, secondary data analysis, surveys, and interviews.

Background

The Program was launched in June 2003 as a joint initiative between AAFC and Health Canada's Pest Management Regulatory Agency. The Program aims to increase grower competiveness by improving access to new and effective crop protection tools. A “minor crop” includes crops other than wheat, canola, barley, soybean, and corn. A "minor use" of a pesticide refers to the crop-protection treatments, such as herbicides, insecticides, fungicides, and nematicides, usually used on low acreage, high-value crops, or, where pest control is only needed on a small portion of the overall crop acreage. The objectives of the Program are to provide benefits to Canadian producers, the environment, and consumers by making minor use pesticide products more readily available; and providing Canadian producers with access to new pest-management technologies to improve their competitiveness domestically and internationally.

Findings

Minor use pesticides are important to the competitiveness and environmental sustainability of the agricultural sector. The Program addresses an ongoing need for grower access to pesticides that is not met by the provinces or industry. Without support for minor use pesticide registrations, the Canadian sector would be at a competitive disadvantage, particularly with the United States where a similar government program has existed since the 1960s. The Program is aligned with federal priorities and the AAFC strategic outcome of a competitive agriculture, agri-food and agri-based products sector that proactively manages risk. Specific findings of the evaluation include:

• Minor crop production is an important and growing sector. The Program fills an ongoing need for grower access to minor uses of pest control products to support the competitiveness and environmental sustainability of this sector. This need is not addressed by any other stakeholders.
• The Program is aligned with federal government priorities and departmental strategic outcomes. The Program is consistent with federal responsibilities for the regulation of pesticides and a national perspective supports Canada's participation in international trade and regulatory harmonization activities.
• New pesticides for minor uses are enabling growers to adapt to changes in the technological, regulatory, and trade environment. The Program's participation in international fora and agreements supports harmonization of regulations with other countries. This activity is not clearly articulated in the Program's logic model.
• Economic analyses indicate that the incremental economic impact of the Program is substantial. Since its inception, the Program is estimated to have contributed to the prevention of crop losses in the range of $653-million to $998-million. This is estimated to be a return of $42 of net benefits for every $1 invested by the government.
• The Program is well-regarded with a sound design, and clear and adequate governance; no significant changes are required to the Program's key components.

Recommendations

• Recommendation 1: The Assitant Deputy Minister, Science and Technology Branch, should update the logic model for the Program to ensure:

  • International efforts are linked to intended outcomes; and
  • Performance targets reflect current activity levels and future program aspirations.
• Recommendation 2: The Assistant Deputy Minister, Science and Technology Branch, should take steps to improve flexibility in the setting of priorities by category.
• Recommendation 3: The Assistant Deputy Minister, Science and Technology Branch, should conduct an assessment of in-house versus contracted laboratory services, taking into account direct and indirect costs, to determine the most efficient and effective use of resources to deliver the Program.

1.0 Introduction

This evaluation examined the relevance and performance of AAFC's Minor Use Pesticides Program. The Program aims to increase grower competitiveness by improving access to new and effective crop protection tools. The results of this evaluation are intended to inform future program, policy and funding decisions.

2.0 Methodology and scope

The evaluation was conducted in accordance with the Treasury Board of Canada Policy on Results, and AAFC's 2016-17 to 2020-21 Integrated Audit and Evaluation Plan.

This evaluation reports on the program relevance, effectiveness and efficiency, focusing on results achieved by the Program from April 1, 2012 to March 31, 2017. While the Program involves both AAFC and Health Canada, the evaluation was led by AAFC and focused on the components of the Program delivered by AAFC's Pest Management Centre. Some data collection was conducted related to Health Canada's regulatory decisions and feedback from Health Canada officials are included, where appropriate.^{Footnote 1}

The evaluation used multiple lines of evidence including document review, case studies, secondary data analysis, surveys, and interviews to assess relevance, effectiveness, and efficiency of the Program.

The detailed evaluation methodology is in Annex A.

3.0 Minor Use Pesticides Program background

3.1 Objectives

The Program was launched in June 2003 as a joint initiative between AAFC's Pest Management Centre and Health Canada's Pest Management Regulatory Agency (hereon after referred to as the Regulatory Agency). The Program aims to increase grower competitiveness by improving access to new and effective crop protection tools and technologies for minor crops. A “minor crop” includes crops other than wheat, canola, barley, soybean, and corn. A "minor use" of a pesticide refers to the crop-protection treatments, such as herbicides, insecticides, fungicides, and nematicides, usually used on low acreage, high-value crops, i.e. minor crops or, where pest control is only needed on a small portion of the overall crop acreage.

Specifically, the objectives of the Program are to provide benefits to Canadian producers, the environment, and consumers by making minor use pesticide products more readily available, with an emphasis on reduced-risk products; and improving Canadian producers' competitiveness domestically and internationally with access to new pest-management technologies.

The intended outcomes of the Program are:

• New minor uses of pesticides are available to growers.
• Stakeholders implement strategies and use tools to manage changes associated with regulatory modernization and crop protection.
• Agriculture, agri-food and agri-based products sector is able to adapt to a changing regulatory environment.

For more details, see the logic model in Annex B.

3.2 Activities

The Program is comprised of several major activities delivered by the Pest Management Centre (AAFC) and the Regulatory Agency (Health Canada). Stakeholders include grower representatives, minor use registrants, Provincial Minor Use Coordinators, Regulatory Agency officials and other specialists. They provide input to AAFC as AAFC identifies sector needs for minor uses of pesticides and conducts an annual prioritization process. AAFC then develops a pre-submission request for the Regulatory Agency, which responds to AAFC by providing the data requirements for the final submission. AAFC generates the required data through field trials and lab analysis, the results of which are used to complete the final application package for submission to the Regulatory Agency. The Regulatory Agency reviews the submission, delivers a regulatory decision, or multiple decisions, and, if the submission is approved, the eventual outcome is the registration of a new minor use (or multiple uses) of a pesticide product. Figure 1 illustrates the process between AAFC and Health Canada's Regulatory Agency.

Description of above image

Figure 1 illustrates the process between AAFC and Health Canada's Regulatory Agency.

• 1. Input from:
  • Growers
  • Regulatory Agency
  • Grower representatives
  • Registrants
  • Provincial Minor Use Coordinators
  • Specialists
• 2. AAFC
  • Identification of needs
  • Prioritization (Minor use Priority Setting Meeting)
  • Pre-Submission Request (sent to the Regulatory Agency)
  • Data Collection (Data Requirements identified by the Regulatory Agency)
  • Data Generation
    • Field Trials
    • Lab Analysis
  • Submission Packages to the Regulatory Agency
• 3. The Regulatory Agency
  • Identification of Data Requirements
  • Review of the Packages submitted by AAFC
  • Regulatory Decision
  • Registration (Product availability)

Source: Pest Management Centre: Registration Process Presentation

Identification and prioritization of minor use pesticide needs (AAFC)

The Pest Management Centre facilitates a grower-driven process to identify and select priorities for minor use pesticide projects. Each provincial government has designated provincial minor use coordinators who are responsible for consultations with growers/grower organizations in their province and represent crops without organizations during the priority setting process. There is also a coordinator for forestry uses. Provincial minor use coordinators, together with local producers and producer groups, identify and prioritize the major pest problems in their regions. The Pest Management Centre then combines provincial priority lists into a national list, which is used at an annual national AAFC Minor Use Pesticide Priority Setting Workshop to select national priorities according to specific categories. Approximately 45 new projects are selected each year, which include 10 projects for each discipline (pathology, entomology, and weed science), plus five projects for regional uses, two for the organic industry and six product screening trials (i.e., projects for cases where there is no known pesticide solution to a pest problem).

The Pest Management Centre compares national priorities selected through the workshop with those of the parallel minor uses program in the United States, the Interregional Research Project Number 4 (IR-4), which undertakes its own annual prioritization process. Where there are common priorities, the Pest Management Centre and IR-4 identify opportunities for collaboration and work-sharing. An additional 15 to 20 joint Canada-United States projects are selected annually.

Data collection and analysis, preparing submissions to the Pest Management Regulatory Agency (AAFC)

For Health Canada to register a new pesticide use, there must be sufficient information to assess its safety and value. Through the Program, AAFC generates the scientific studies needed by Health Canada to support the registration of new minor uses of pesticides. Data requirements to obtain registration, if required, are determined in consultation with the Regulatory Agency. AAFC then generates the required data through field trials and lab analysis. The Pest Management Centre obtains final product manufacturer support and prepares the submission package for the Regulatory Agency to support the registration of new pesticide minor uses. The Pest Management Centre's service standard for project completion is five years, from priority selection to submission to the Regulatory Agency.

Review of submissions (Health Canada)

Health Canada's Pest Management Regulatory Agency is responsible for administering the Pest Control Products Act, which regulates pesticides in Canada. The decision to register a product for minor uses is based on whether the product demonstrates merit and value, and whether the risks to human health and the environment can be appropriately managed. If the submission is approved, the eventual outcome is the registration of a new minor use of a pesticide product. The Regulatory Agency waives the evaluation fees for minor use pesticide submissions received from AAFC and the provinces.

3.3 Resources

The Program is resourced through AAFC's Growing Forward 2 funding framework, largely from the AgriCompetitiveness program, Stream C: Facilitating and Supporting a Modern Regulatory Environment. From 2012-13 to 2016-17, $37.9-million was allocated to the Program. Based on the 2013 Memorandum of Understanding between AAFC and Health Canada, $4-million is transferred annually from AAFC to the Regulatory Agency in the Supplementary Estimates process for the Program.

There are on average 49 full-time equivalents (FTE) dedicated to the Program each year, with several FTEs located at seven research centres across Canada to conduct field and greenhouse trials and residue analysis. The details by fiscal year can be found in Table 1.

Table 1: Minor Use Pesticides Program Resource Allocation (2012-13 to 2016-17)

Category	2012-13	2013-14	2014-15	2015-16	2016-17	Total
FTE	49	49	48	50	49
Salary	4,017,839	4,569,676	4,162,582	4,298,635	4,154,315	$21,203,047
O&M	2,622,941	2,880,715	3,281,988	3,242,141	3,539,335	$15,567,120
Major Capital	-	-	-	-	189,919	$189,919
Total	$6,640,780	$7,450,391	$7,444,570	$7,540,776	$7,883,569	$36,960,086
Source: Program financial data

3.4 Governance

The Pest Management Centre, located within the Science and Technology Branch at AAFC, is responsible for delivering the Program. The Program is led by an Executive Director who reports to the Director General of the Coastal Directorate. The Memorandum of Understanding between AAFC and Health Canada (April 2013) outlines the program governance structure, including roles and responsibilities, reporting structure, and a performance measurement strategy. An AAFC-Health Canada Interdepartmental Working Group oversees the general objectives of the Memorandum. The Working Group reports annually to the Director General-level Joint Management Committee which, in turn, reports at least annually to Assistant Deputy Ministers at both AAFC and Health Canada. Annual progress reports are submitted to the AAFC/Health Canada Assistant Deputy Minister Joint Management Committee.

The Program’s Technical Working Group provides operational expertise on scientific and technical issues, and promotes information exchange in areas related to minor uses of pesticides. In addition to the Pest Management Centre and the Regulatory Agency, members include two representatives from each of the following stakeholder groups: Provincial Minor Use Coordinators; the pesticide industry; and growers.

4.0 Program relevance

4.1 Continued need

Pest management is an important component to protecting and increasing crop yield and quality, as well as contributing to grower economic viability and competitiveness. However, for horticultural and other specialty crop growers, access to pest control products can be challenging. This is due to the expense and limited economic return for companies registering minor use pest products in Canada.

The low acreage, high value minor crop production in Canada includes fruits, vegetables, and some pulses. Increasingly, it now includes rice, forage crops, and crops for pharmaceuticals and oils. The Program was established to increase access for Canadian horticultural and specialty crops producers to pest management products, which were available to their American counterparts. At the time, it was argued that without Government support for registering minor uses of pesticides, the Canadian agricultural sector would be at a competitive disadvantage internationally, especially with the United States^{Footnote 2}. The originating rationale for the Program is still valid as market incentives for companies have remained unchanged, while growers continue to require access to a variety of minor use pesticides to address changing pest conditions and the introduction of new crops.

A 2016 report conducted by AAFC's Research and Analysis Directorate, found that farm receipts for minor crop production in Canada have increased at an average rate of 4.4 percent over the eleven years covered in the study (2002 to 2013). This represents approximately 34 percent of total crop receipts. This sector is estimated to have directly contributed $4.9-billion to Canada's Gross Domestic Product in 2013 and created 105,600 direct jobs in the same year.^{Footnote 3}

Evaluation evidence supports the continued need for the Program given increased minor crop production, the evolution in growing techniques (e.g., crops grown "under cover" in greenhouses or hoop houses) and pesticide application methods. Further, the increased interest in biopesticides or new products that harmonize with Maximum Residue Limits in export markets; the arrival of new invasive species and pests; and gaps in pest control when existing pesticides are withdrawn from the market as a result of the Regulatory Agency's regulatory review process are supported by the Program. Almost all surveyed stakeholders (96 percent) indicated that it is very important for Canadian growers to have access to new minor uses of pest control products, while 86 percent noted they did not have access to all of the minor uses of pest control products that they need.

4.2 Alignment with Government and AAFC priorities

The alignment of the Program with federal priorities dates back to its program design to address gaps and stakeholder concerns outlined in the Report to the House of Commons Standing Committee on Agriculture and Agri-Food, Registration of Pesticides and the Competitiveness of Canadian Farmers (2002), and the Report to the Standing Committee on the Environment and Sustainable Development, Pesticides: Making the Right Choice for the Protection of Health and the Environment (2001). These concerns included the availability and access to minor use and reduced-risk pest management tools.

There is a demonstrated link between the Program, objectives and departmental strategic outcomes. The Program was renewed in 2013 under the Growing Forward 2 framework. A 2017 AAFC Evaluation found that the framework appropriately focused on priorities related to increased market share, improved productivity and environmental sustainability, improved resilience, and supported the overall enhancement of agricultural Gross Domestic Product growth.

The Program is aligned with AAFC's Strategic Outcome: “A competitive and market-oriented agriculture, agri-food and agri-based products sector that proactively manages risk.” The key program rationale for the Program is to support the competitiveness of Canadian growers, including addressing trade barriers with export markets.

4.3 Alignment with federal roles and responsibilities

Formal responsibility for the regulation of pesticides is shared between the federal and provincial/territorial levels of government. The federal government has the authority to regulate the import, manufacture, and use of pesticides, which is exercised under the Pest Control Products Act, a responsibility of Health Canada's Regulatory Agency. All pesticides must be registered under this Act before they can be used in Canada. Provincial governments may place additional requirements on products, related to their sale and disposal.

While applications to the Regulatory Agency are typically submitted by companies, provisions were made under the Canadian pesticide regulatory framework to allow AAFC and the provinces to apply on behalf of Canadian growers for minor use registrations.

The federal government is well-positioned to facilitate grower access to minor use pesticides through the Program. Given the limited capacity of other stakeholders, such as provincial/territorial governments or grower associations, the federal role in the Program is appropriate in supporting the development of minor use registration submissions that industry would not undertake on its own. To meet Regulatory Agency requirements, the residue data provided in minor use registration submissions must be collected through Good Laboratory Practices, developed by the Organization for Economic Co-Operation and Development (OECD). These practices are a set of quality control principles designed to “ensure the generation of high quality and reliable test data related to the safety of industrial chemical substances and preparations”.

Prior to Good Laboratory Practice compliant residue studies, submissions to the Regulatory Agency for minor use registrations were primarily developed by grower associations like the Canadian Horticultural Council, and provincial governments. Federal funding had been available through other programs like the Canadian Adaptation and Rural Development program, but this was sporadic and limited. In line with other OECD partners, the Regulatory Agency harmonized requirements for Good Laboratory Practice compliant studies in 1998 to facilitate the mutual acceptance of data. Provinces and grower associations did not have the capacity and resources to undertake these compliant studies. This, in turn, had the effect of widening the existing gap in minor use pesticide registrations with the United States. To fill this gap, the federal government launched the Minor Use Pesticides Program in 2003 under the Agricultural Policy Framework. While provincial governments continue to develop submissions to the Regulatory Agency, these submissions are those that do not require the collection and analysis of residue data.

Evaluation evidence demonstrates that the role of the federal government in minor use pesticides is beneficial given the international dimension of regulatory frameworks and trade. The Pest Management Centre has established effective partnerships with international jurisdictions (notably the United States) to foster regulatory harmonization and address trade barriers. However, this activity, while viewed as important, is not within the formal objectives of the Program, nor represented within the Program's logic model.

Canada is considered by stakeholders to be a leader in minor use pesticides, hosting in October 2017 a well-attended Global Minor Use Summit meeting. Since 2009, Canada has signed agreements with Brazil, China, and India to foster scientific and technical collaboration around the registration of minor use pesticides. These agreements and other international efforts of the Program (joint reviews with IR-4, participation in CODEX^{Footnote 4}) were viewed by stakeholders as an important future thrust of the Program. With its international activities, the Program hopes to build on the efficiencies realized through its collaboration with the American IR-4. Agreements with other countries aim to create conditions that reduce trade barriers for Canadian growers such as through harmonization of Maximum Residues Limits, creation of allies in developing bilateral trade agreements and setting of international minor use pesticide policy, standards and regulatory requirements.

5.0 Performance

5.1 Availability of new minor use pesticides

Minor use submission preparation

From 2012 to 2017, the number of new minor use pesticides available to growers increased and exceeded internal performance targets. Between 2013-14 and 2016-17, 259 projects (160 was the target) were submitted to the Regulatory Agency for registration of minor uses, with the majority of these being submitted directly to the Regulatory Agency by the Pest Management Centre and the remainder indirectly through registrants (see Table 2). About 40 percent of projects were joint projects with the IR-4. Harmonization efforts are resulting in new minor uses being registered for Canada and the United States at the same time.

The Pest Management Centre and IR-4 work cooperatively with companies to include new minor uses during the initial submission resulting in new minor uses being registered at the same time as the initial product registration.

Submissions far exceed targets established in the AAFC-Health Canada Memorandum. In particular, reflecting the priority on collaboration and harmonization, the number of joint submissions annually exceeds the target by four times. This disparity is most likely the result of low performance targets combined with increased program efficiencies such as crop grouping, and harmonization efforts with the American IR-4 program which, in turn, have led to an increase in the number of submissions.

Joint submissions during the current study period (106) are also considerably higher than recorded during the four year period of the 2012 evaluation (28).

Table 2: Minor Use Pesticide Projects 2013-14 to 2016-17

	2013-14	2014-15	2015-16	2016-17	Total	MOU Target
Regulatory Agency - Domestic	24	36	34	26	120	-
Regulatory Agency - Joint	15	14	24	25	78	-
Registrants - Domestic	13	8	8	4	33	-
Registrants - Joint	17	7	4	0	28	-
Total Domestic	37	44	42	30	153	-
Total Joint	32	21	28	25	106	7/year
Total	69	65	70	55	259	40/year
Projects Withdrawn	15	24	12	8	59	-
Source: AAFC, Pest Management Centre, 2013-17 Report to the ADM Joint Management Committee.

Review and approval of submissions

Between 2013-14 and 2016-17, the Regulatory Agency made 376 regulatory decisions regarding minor uses submitted by AAFC (domestic and joint) and by the provinces (see Table 3). The number of regulatory submissions evaluated by the Regulatory Agency has met the Memorandum target of 75 per year, except for 2016-17, where 72 regulatory submissions were reviewed. In terms of new minor uses, 2,314 new uses for minor crops were registered during the period. Annually, between 458 and 745 new minor uses were approved, far exceeding the target of 200 new uses available each year. As noted under 3.2 Activities, there may be multiple regulatory decisions that result from each project and multiple uses may come out of a registration.

Feedback from program stakeholders confirms that the Program has significantly contributed to the availability of new minor uses pesticides; 87 percent of surveyed stakeholders indicated that the Program has had a substantial positive impact on improved access to, and adoption of, minor use pest control products.

Table 3: Availability of New Minor Use Pesticides 2013-14 to 2016-17

	2013-14	2014-15	2015-16	2016-17	Total	MOU Target
Regulatory Decisions Made - Provinces	45	63	66	34	208	-
Regulatory Decisions Made - AAFC	22	30	39	14	105	-
Regulatory Decisions Made - Joint Reviews/ Workshares	17	13	9	24	63	7/year
Sub-Total	84	106	114	72	376	75/year
New Uses - AAFC	291	186	162	315	954	-
New Uses - Provinces	167	435	583	175	1,360	-
Total	458	621	745	490	2,314	200/year
Source: AAFC, Pest Management Centre, 2013-2017 Report to the ADM Joint Management Committee.

Availability of minor use pesticides

A key strength of the Program is that the availability of pesticides for minor uses is evidence-based. Stakeholders expressed confidence in the Program's calibre of research and quality assurance. Interviewees and survey respondents indicated that the minor use pesticides offered the best match between grower needs and industry solutions, and created availability of reduced risk products for minor uses.

Stakeholders are generally less positive about the timeliness of the availability of new minor use products; only 12 percent indicated availability of minor uses is timely and this proportion was lower among external stakeholders. As well, improving timeliness was the most common suggestion for program improvement among surveyed stakeholders.

The availability of new minor uses pesticides reflects grower needs, owing to an effective prioritization process (described in more detail in section 6.4). Notably, the annual priority setting workshop can increase the availability of pesticides for minor uses outside of the formal Minor Use Pesticides Program process. Some growers noted that the relationships built and cultivated through the priority setting workshop can result in tangible benefits for growers by identifying crop protection solutions outside of the Program's formal mechanisms (e.g., growers and companies directly sharing information).

A concern going forward is the ability of the Program to support growers affected by the Regulatory Agency's re-evaluation process. The Regulatory Agency conducts a re-evaluation of products every 15 years to ensure they continue to meet modern scientific standards, and that the risks and value remain acceptable. The re-evaluation may result in a regulatory decision to withdraw a product if it is not compliant with modern environmental and health standards. This has resulted in many older uses being de-registered or entire products being removed from the market. In some cases, there are no alternative uses available for growers. As a result, there is an ongoing need for new minor use registrations to replace solutions being eliminated through re-evaluation.

While the Pest Management Centre works closely with the Regulatory Agency to anticipate potential gaps in product availability due to re-evaluation, the time required to prepare and evaluate a submission for an alternative product may exceed the timeline for de-registration, resulting in a gap in product availability for growers.

5.2 Managing change associated with regulatory modernization and crop protection

The evaluation data does not provide definitive information on the extent to which growers are adopting newly registered products for minor uses (see limitations in Annex A). Stakeholders are confident that adoption is occurring, in large part because the Program's priority setting process is grower-driven. While the survey sample sizes are small, adoption is confirmed by almost all growers and grower associations who indicate that they or their members had used at least one minor use pesticide and most reported use of multiple products. Similarly, across all six case studies, new minor use label extensions permitted growers to access products that provide protection against pests which can cause devastating damage to crop yields.

Growers adopt new minor use products for a number of reasons: efficacy of a new product or absence of other solutions for crop protection; access to a robust selection of pesticide solutions for the development of integrated pest management strategies to manage pesticide resistance; and health and environmental benefits related to safer application and lower-risk products. The availability of lower risk products that consider international Maximum Residue Limits is a significant incentive for growers who export their commodities.

Almost all surveyed stakeholders indicated that the Program improved crop protection practices, and most indicate that the Program has improved management of pest control product resistance. These positive views have steadily increased over the last three evaluations of the Program (Table 4).

Table 4: Stakeholder Reporting on the Program's Achievement of Outcomes, 2009-17

Proportion of Surveyed Stakeholders Indicating the Program had a Substantial or Small Positive Impact or Contribution	2009	2012	2017
Improved access and adoption of minor use pesticides	75%	97%	98%
Improved management of pesticide resistance	60%	83%	86%
Improved crop protection practices	68%	88%	92%
Improved competitive parity with the United States	56%	88%	86%
Source: Stakeholder Surveys for the 2009 Evaluation of the Building Public Confidence (BPC) Initiative, 2012 Evaluation of the Agricultural Regulatory Action Plan under Growing Forward, 2017 Evaluation of Minor Use Pesticides Program.

Where barriers to adoption exist, these most often relate to the cost of the product, including the product price, as well as use conditions (e.g., complex applications, re-entry intervals, rates, and frequency of applications). Certain product characteristics (e.g., low distribution properties, short shelf-life) can make some products less appealing. These issues can be exacerbated by poor or inconsistent product labelling. Program staff indicate that while cost is not a formal criterion in the priority setting process, the priorities proposed lead growers to informally take cost into consideration. Adoption can be impacted by the cyclical nature of pests and their localized nature.

Another potential barrier to adoption is awareness. Awareness of new minor uses is fostered through a number of channels, including the Pest Management Centre listserv and website, communications through Provincial Minor Use Coordinators and provincial extension, as well as through companies supplying the products. However, the case studies suggest that the robustness of provincial channels to raise grower awareness of new pesticides for minor uses may be declining in some jurisdictions. Respondents indicated challenges with the functionality of the Centre's listserv; limited communications to growers about the efficacy of new products and integrated pest management methods; and weaknesses related to the website including the absence of standardized templates and search capacity based on crop.

5.3 Adapting to a changing regulatory environment

The Program has contributed to growers' ability to adapt to changes in the broader technology, regulatory, and trade environment. One of the originating rationales for the Program–the ‘technology gap', the gap in availability of minor use pest control products, between Canada and the United States–continues to be addressed through registration of priority products for minor uses. The period under study has seen an expansion in collaboration between the Program and the parallel IR-4 program in the United States leading to an increase in joint submissions. The Program has also devoted efforts to increasing regulatory harmonization with other countries that trade with Canada through a series of Memorandum agreements.

International cooperation and coordination is widely perceived by stakeholders to be providing advantages to growers such as lowering trade barriers, the timely access to products, a diversified pest management toolbox, and safer options for workers. Most surveyed stakeholders indicated that the Program has had a positive impact on competitive parity of the Canadian agricultural sector with regards to pest management (i.e., with the American sector). Similarly, 78 percent of stakeholders indicate that the Program was at least somewhat closing the gap in availability of minor use pesticides between Canada and the United States.

Feedback alone from growers and grower associations does not permit a reliable estimate of the financial impacts of the Program on growers. However, the majority of growers and grower associations surveyed indicated that having access to newly registered minor pest control uses was making a significant difference to crop production/yield.

In 2015, the Pest Management Centre commissioned a study to determine the impacts of the Program on minor crops in terms of increased productivity (yields) and revenues. The internal study estimated that the Program has contributed to the prevention of crop losses in the range of $653-million to $998-million since implementation in 2003, resulting in societal benefits of $3.4-billion. From the government perspective, this would mean that for every $1 invested, $42 of net benefits is accrued to society.

5.4 Economy and efficiency

Management of resources

A comparison of allocated and expended program funds indicates that there was marginal lapsing of 3 percent of program funds during the period under study, which is much lower than the 21 percent lapsed funding by the Pest Management Centre during the years covered by the 2013 evaluation of the Program. During the current period, stable staffing has led to a fully operational program which is a key reason for the reduction in lapsed funds. The small variance during the current period under study is typically due to delays in field trials due to weather, pest conditions, or capacity limitations among contractors or private laboratories commissioned to conduct work for the Program.

Table 5: Comparison of the Program's Budget and Expenditures 2012-13 to 2016-17

Fiscal Year	Budgeted	Expenditures	Variance
2012-13	6,627,550	6,640,780	0%
2013-14	7,805,551	7,450,391	−5%
2014-15	7,670,096	7,444,570	−3%
2015-16	7,775,737	7,540,776	−3%
2016-17	8,119,159	7,883,569	−3%
Total	37,998,093	36,960,086	−3%
Source: Program Financial data

The evaluation found that the Program has been able to function effectively with its current funding, but may encounter increased demand, and challenges relating to the following issues: keeping up with innovations in the field; addressing emerging issues such as climate change; and increases in registrations related to the introduction of new crops requiring minor use registrations (e.g. legalization of cannabis). Furthermore, AAFC interviewees noted persistent quality and timeliness issues with private laboratories are driving an interest internally to build internal laboratory capacity (the Good Laboratory Practice certified lab capacity being a current strength of the Program), as opposed to contracting this type of activity. Funding for the Program has remained stable since 2003, while the cost of contracted research has increased.

Another potential challenge to the efficiency of the Program is the resource intensive nature of the Priority Setting Workshop (three-day process conducted in-person on an annual basis). However, there was no consensus that the process could be streamlined (e.g., conducted every second year or conducted virtually) as the current annual in-person process has benefits for the Program's priority setting, as well as ancillary benefits associated with direct interactions and sharing of information.

Timeliness

Program efficiency has increased since the 2013 evaluation due to the following factors:

• Increased collaboration with other jurisdictions on minor use projects, including increased number of joint projects with the American IR-4 program, which combine resources for data collection, has led to the development of standard operating procedures and data collection templates; and complementary work with the provinces, which lead rationale-driven submissions using existing data, thus conserving the Program's resources for projects requiring new residue and efficacy data.
• Joint Canadian-American reduction of field trial requirements allowing the Program to leverage reliable data generated in other jurisdictions to support submissions.
• Refinements to the Regulatory Agency's data requirements to streamline regulatory submissions, including the use of crop groupings and adjustments to regulatory requirements as new scientific data becomes available.

The 2013 evaluation of the Program noted issues related to program efficiency. The report pointed to project delays, which undermined the relevance of some minor uses to the sector by the time they were registered.

In response, the Program introduced service standards in 2015. The service standards target the various phases from project planning to report writing and submission, with submission targeted for completion by the end of the fifth year. The Program goal is that 90 percent of the activities are processed within the applicable timeline. Due to the five-year timeframe, an assessment of achievement of these standards will be determined in the next evaluation.

While the evaluation found that the Program has improved its timeliness in the preparation of minor use submissions, challenges still exist. This is a function both of the time for the Pest Management Centre to prepare submissions to the Regulatory Agency and the time for the Agency to review submissions. As most minor use pesticide research is conducted externally by consultants instead of in-house, there is a potential source for delays in the product registration process, and interviewees noted that there may be a possible limit to the analysis quality.

Program design and delivery

The Program is delivered as designed and intended, with no fundamental challenges in delivery. During the period under study, the Program has matured in its effective management of the preparation of submissions. While the Program was originally created to close the technology gap with the United States, it is now working more towards simultaneous registrations and reducing trade barriers. Increased collaboration with IR-4 and other international jurisdictions around harmonization was assessed as appropriate by stakeholders and is consistent with the growth in exports of minor crops.^{Footnote 5} From 2004-2013, the proportion of minor crops that was exported increased from 13 to 33 percent

The foundation of the Program and a key strength is the annual national priority setting process. Both internal and external stakeholders are satisfied with this process and confident that the selected priorities merit investments by the Program. Selected priorities are grower-driven and priority solutions require the pesticide company's approval which ensures the feasibility of selected projects. Most stakeholders commend the collegial, collaborative, grower-driven, consensus-based approach and feel that the criteria are sound to answer high-priority needs and lead to successful projects.

The evaluation found that the priority setting process could be improved by increasing flexibility in the number of priorities allocated by discipline. Informants explained that it can be difficult to find 10 priorities for weeds, for instance, and that some of these priorities could be re-allocated to address needs in other categories. This could enable the Program to address urgent needs in some categories and/or to increase emphasis on biopesticides, in light of consumer demand, efficacy concerns and the complexity of registering these products. The evaluation noted that the IR-4 program has already taken steps to increase flexibility in the priority setting process.

The governance of the Program is widely viewed as appropriate, clear, and effective. The relationship between AAFC and Health Canada is generally good, as is the engagement of other AAFC internal partners (dealing with international trade).

The stakeholder consultation and involvement, such as through the priority setting workshop, are strengths of the Program. The annual Priority Setting Workshop provides an opportunity for communications with stakeholders.

6.0 Conclusions and recommendations

6.1 There is a continued need for the Minor Use Pesticides Program

Minor crop growers lack ready access to pest management products due to the high cost of registration and comparatively low return for pesticides manufacturers. As technology is evolving and the need for pest management and pest resistance management is ongoing, the Program remains relevant to address these needs.

The Program is aligned with priorities endorsed under the broader AAFC Growing Forward 2 framework. Both AAFC and Health Canada play an appropriate role that is aligned with strategic objectives and legislative authority in supporting the registration process to benefit improved grower productivity and competitiveness.

6.2 There is progress toward achievement of intended outcomes

During the evaluation period, program results significantly exceed performance targets, most likely due in part to low initial performance targets and, in part due to an increase in the number of submissions as a result of the harmonizing of efforts with the American IR-4. Stakeholders generally regard the availability of new pesticides for minor uses as a positive (evidence-based, meeting grower needs), although timeliness of availability is a concern.

Evaluation evidence indicates that adoption of products registered through the Program is occurring, supported by a grower-driven process to identify and prioritize projects and grower need for crop protection and resistance management, while meeting international standards.

The availability and use of new minor use pesticides is contributing to crop protection and resistance management, adaptation to new technologies, and meeting regulatory requirements in export markets. The minor use sector is growing and economic analyses suggest that the incremental economic impact of the Program is substantial.

The Program's work related to harmonization and pre-emptively addressing potential future trade barriers through joint projects with IR-4 and collaborations with other jurisdictions is viewed as effective and welcomed by stakeholders. However, this activity, while viewed as important, is not within the formal objectives of the Program, and is not represented within the Program's logic model.

Recommendation 1: The Assistant Deputy Minister, Science and Technology Branch, should update the logic model for the Program to ensure:

• International efforts are linked to intended outcomes; and
• Performance targets reflect current activity levels and future program aspirations.

6.3 Administration of the Program is effective and efficiency is improving

The evaluation evidence points to a generally strong program design and satisfaction among stakeholders in the delivery of the Program. The priority setting process to select projects annually is viewed as grower-driven and collaborative, although some priorities are less well-substantiated due to issues such as a lack of grower preparation in some instances. The evaluation indicates that there may be room for more flexibility in the number of priorities assigned to each category (region, discipline, organic).

A key aspect of the Program is its ability to conduct Good Laboratory Practice compliant residue studies. Provinces and growers do not have the capacity or resources to develop minor use registration submissions based on Good Laboratory Practice compliant studies. To meet demand, and deliver on the Program's objectives, the Pest Management Centre is contracting out large portions of its laboratory research work externally. This was a potential source for delays in the product registration process, as well as a possible limit to the analysis quality.

Recommendation 2: The Assistant Deputy Minister, Science and Technology Branch, should take steps to improve flexibility in the setting of priorities by category.

Recommendation 3: The Assistant Deputy Minister, Science and Technology Branch, should conduct an assessment of in-house versus contracted laboratory services, taking into account direct and indirect costs, to determine the most efficient and effective use of resources to deliver the Program.

7.0 Management Response and Action Plan

Evaluation of the Minor Use Pesticides Program

Recommendation	Management Response and Action Plan (MRAP)	Target Date	Responsible Position
1) The Assistant Deputy Minister, Science and Technology Branch, should update the logic model for the Program to ensure: International efforts are linked to intended outcomes; and Performance targets reflect current activity levels and future program aspirations.	Agreed The Pest Management Centre will revise the logic model to include performance targets as approved under the Canadian Agricultural Partnership and clearly linking program international activities to outcomes.	August 31, 2018	Director General, Costal Region, Science and Technology Branch
2) The Assistant Deputy Minister, Science and Technology Branch, should take steps to improve flexibility in the setting of priorities by category.	Agreed The Pest Management Centre will: Conduct an analysis of scenarios to determine capacity to undertake changes in project flow by discipline; and Determine if changes to priority selection are needed. The target date takes into account the timing of the Priority Setting Workshop, which takes place in March.	March 31, 2019	Director General, Costal Region, Science and Technology Branch
3) The Assistant Deputy Minister, Science and Technology Branch, should conduct an assessment of in-house versus contracted laboratory services, to determine the most efficient and effective use of resources to deliver the program.	Agreed The Pest Management Centre will do an analysis of contract versus in-house laboratory services to determine the most efficient and cost effective program delivery.	August 31, 2018	Director General, Costal Region, Science and Technology Branch

Annex A: Evaluation methodology

The evaluation is based on five sources of evidence. Where possible, at least two sources were used to generate findings for each evaluation issue. Sources of evidence were as follows:

• Document Review. Documents were reviewed including: program foundational documents, Departmental performance and other reports, documentation on program activities, international agreements, internal studies, previous evaluation and performance monitoring of the Program, and documentation on the US IR-4.
• Review of Administrative Data. Program data were reviewed related to financial expenditures, and program activity data (submissions, regulatory decisions, joint submissions).
• Key Informant Interviews. AFFC provided the list of potential informants, who were chosen from various stakeholder groups for their knowledge of the Program. In total, interviews were conducted with 25 interviewees.^{Footnote 6}
  • Internal informants
    • AAFC staff (n=6)
    • Health Canada staff (n=4)
  • External informants
    • Growers and Representative of Grower Groups (n=8)
    • Provincial representatives and experts (n=3)
    • Representative of the pesticide industry (n=2)
    • IR-4 representatives (n=2)
• Stakeholder Survey. An online survey of stakeholders was conducted with the aim of collecting information on stakeholders' satisfaction with the Program; their views on program impacts; achievement of outcomes and suggestions for improving the Program. Survey respondents included representatives of federal and provincial governments, grower associations, growers, biopesticide product registrants or manufacturers, academia and consultants. In total, 203 individuals responded to the survey questions about the Program. Of the 203 respondents, 48 (33.3 percent) worked for the federal government and 96 (66.7 percent) worked for other organizations.
• Case Studies. Case studies were conducted of six priorities selected by the Program during the period under study, drawn from a variety of regions and disciplines, and including joint IR-4 submissions. Each case study involved a review of documentation and interviews with AAFC officials, industry representatives and/or producer representatives.

The case studies included:

• Beleaf 50SG for Greenhouse Peppers
• Cyazypyr 10SE for Caneberry
• Oberon 240 for Caneberry
• Dual II Magnum for Celery
• Ranman 400SC for Greenhouse Cucumber
• Orondis for Greenhouse and Field Herb (basil)

Methodological limitations

Methodological limitations were taken into account in interpreting the data:

Limitation	Impact on Evaluation	Mitigation Strategy
Empirical evidence related to intermediate outcomes (i.e., adoption of products registered through the Minor Use program) is limited	The evaluation offers some insights into the progress and impacts of the Program to date, but is limited in its ability to identify intermediate and longer-term impacts.	Adoption, including incentives and barriers to adoption, were examined through qualitative lines of evidence.
The representativeness of the stakeholder survey data is not known and the sample size for sub-groups such as growers is small.	The evaluation ensured that all Stakeholder survey results were triangulated with other lines of evidence.	The views of all stakeholders may not be represented.

Annex B: Minor Use Pesticides Program logic model

Stream C: Facilitating and supporting a modern regulatory environment – Minor Use Program

Objectives	This program will address the needs of Canadian growers for new minor uses of pesticides and will assist stakeholders to be sustainable and remain competitive in the market place. The program will provide tools that will enable stakeholders to modernize crop protection within Canada.
Activities	AAFC Program management and oversight AAFC Stakeholder engagement and collaboration: Consultation to identify stakeholder priorities for minor use pesticides Establishing and managing an MOU with Health Canada AAFC regulatory research AAFC-Pest Management Centre (PMC) regulatory research activities (data generation and compilation, preparation of regulatory submissions)
Outputs	AAFC Program management and oversight products MoU with Health Canada’s Pest Management Regulatory Agency AAFC Stakeholder engagement and collaboration products Documentation of stakeholder priorities for minor use pesticides AAFC Regulatory research products Regulatory submissions for new minor uses of pesticides Health Canada – Regulatory Agency review of AAFC and provincial regulatory submissions of new minor uses of pesticides Regulatory decisions issued (address the prevention of the growth of the technology gap)
Immediate outcomes	New minor uses of pesticides are available to growers
Intermediate outcomes	Stakeholders implement strategies and use tools to manage changes associated with regulatory modernization and crop protection.
End outcomes	Agriculture, agri-food and agri-based products sector is able to adapt to a changing regulatory environment.