Evaluation of AgriRecovery

Abbreviations

AAFC

Agriculture and Agri-Food Canada

BRM

Business Risk Management

Executive summary

Purpose

The Office of Audit and Evaluation of Agriculture and Agri-Food Canada (AAFC) conducted an evaluation of the AgriRecovery Framework to assess its relevance, efficiency and effectiveness.

Scope and methodology

AgriRecovery activities from 2015-16 to 2021-22 were evaluated using multiple lines of evidence: a review of documents, data and files, transcriptions of parliamentary or legislature debates, literature and media; an environmental scan; interviews with internal and external stakeholders; and 6 case studies.

Background

The AgriRecovery Framework is a disaster relief framework. AgriRecovery sets out a 6-step process for the federal, provincial and/or territorial governments to follow when a natural disaster impacts agricultural producers and short-term assistance beyond what is available through existing AAFC, provincial or territorial programs may be needed. The objective of AgriRecovery is to help producers recover from a natural disaster and to resume operations as quickly as possible by providing funding for the extraordinary costs incurred by producers.

The AgriRecovery process begins when the federal government receives a formal request for an AgriRecovery assessment. Assessments are jointly conducted by the federal government and relevant provincial or territorial government(s), with collaboration from other stakeholders as required. All potential initiatives first undergo a preliminary assessment. If the preliminary assessment criteria are met, a formal assessment begins. If the formal assessment concludes that a response is warranted, the federal and relevant provincial or territorial government(s) then obtain the necessary authorities to design and fund the initiative. Initiatives are implemented by the relevant province or territory, who is required to report to AAFC on the timeliness, reach and impact of each initiative.

The federal, provincial and/or territorial governments work together to deliver programming under the AgriRecovery Framework as part of the suite of Business Risk Management (BRM) programs.

Findings

• AgriRecovery is aligned with the federal and departmental priority of helping agricultural producers manage risks that threaten the viability of their farms.
• AgriRecovery is part of the BRM suite, designed to cover extraordinary costs necessary to recover from a disaster over and above what would be covered under the other BRM programs. Potential support from other programs is considered when determining eligible expenses under AgriRecovery initiatives to avoid duplication between program payments.
• There is evidence that AgriRecovery may not be prepared to respond to the increased demand for support due to climate change. AgriRecovery is not designed to encourage producers to implement actions to increase their resilience to natural disasters.
• Federal and Provincial officials generally understand AgriRecovery's processes but a lack of clarity persists for producers and industry associations.
• Targets for timeliness of assessments and program payments were generally met. There are however some challenges faced in meeting these targets, principally related to availability of data to support the assessment process.
• Initiatives are reported to reach affected producers, but the characteristics of participants are unknown.
• Information on impacts is positive but limited to self-reported data from initiative recipients.

Conclusion

The evaluation confirmed that AgriRecovery is relevant to federal and departmental priorities, including the departmental core responsibility of mitigating sector risk. AgriRecovery is part of the BRM suite of programming and covers extraordinary costs (those not covered by the other BRM programs) that are necessary to help farmers recover from natural disasters.

There is currently no comprehensive procedure manual bringing together all the necessary procedures and guidance materials related to the federal component of the AgriRecovery Framework; the process currently relies on the knowledge and experience of long-term staff.

The role of AgriRecovery in the BRM suite, as well as the relationship of AgriRecovery to the other BRM programs is not well understood by producers. This contributes to pressures placed on AAFC and provincial and territorial governments to implement AgriRecovery initiatives when the criteria to do so are not met.

AgriRecovery does not require participation in other BRM programs as a condition of receiving AgriRecovery funding, or subsequent enrolment in BRM programs for those who have received funding. Thus, there are no provisions included in AgriRecovery to encourage mitigating actions for producers to increase resilience to natural disasters. Such resilience is becoming more important with the increase in frequency and intensity of natural disasters resulting from climate change.

AgriRecovery is an efficient framework but has little data to assess the reach and/or impact of its initiatives. Existing performance metrics that relate to impact are based solely on self reported survey data from producer surveys and is limited to asking whether funding was helpful or not. Other indicators relate to the administration of assessment and initiatives and provide no insight into the progress of the Framework or its initiatives in helping farmers recover from natural disasters.

Recommendations

Recommendation 1: The Assistant Deputy Minister (ADM), Programs Branch, in collaboration with the ADM, Strategic Policy and the ADM, Public Affairs and in consultation with provincial and territorial partners, should develop a plan to increase producer participation in other federal, provincial and territorial programs that address business risk management and increase producer resilience to natural disasters.

Recommendation 2: The Assistant Deputy Minister, Programs Branch should develop a comprehensive and evergreen procedure manual for the federal component of AgriRecovery to ensure consistency in processes.

Recommendation 3: The Assistant Deputy Minister, Programs Branch, in consultation with the provinces and territories, should develop performance metrics to assess the reach and impact of AgriRecovery initiatives.

Management response and action plan

Management agrees with the evaluation recommendations and has developed an action plan to address the majority of them by May 2024.

1.0 Introduction

The Office of Audit and Evaluation conducted an evaluation of AAFC's AgriRecovery Framework as part of the Office of Audit and Evaluation's 2022-23 to 2026-27 Plan. The evaluation complies with the Treasury Board of Canada's Policy on Results and fulfills the requirements of the Financial Administration Act. Findings from this evaluation are intended to inform current and future program and policy decisions.

2.0 Scope and methodology

The evaluation assessed the relevance, efficiency and effectiveness of the AgriRecovery Framework between 2015-16 to 2021-22. Activities out of scope for this evaluation include other programs within the BRM suite and other emergency management activities within AAFC.

The evaluation used multiple lines of evidence to evaluate AgriRecovery including: literature and media reviews; an environmental scan of other federal disaster relief programs; document, data and file reviews; a review of federal, provincial and territorial Hansards (transcriptions of parliamentary or legislature debates); interviews with AAFC staff, provincial officials, industry associations and subject matter experts; and 6 case studies. For the detailed methodology, please see Annex A.

AgriRecovery activities were most recently evaluated through the Evaluations of the Agricultural Disaster Relief Program in 2017 and 2011.

3.0 AgriRecovery overview

3.1 Activities

Framework description

In 2006, the federal, provincial and territorial governments agreed to implement the AgriRecovery Framework as a national agricultural disaster relief strategy. AgriRecovery provides a 6-step process for the federal, provincial and/or territorial governments to follow when a natural disaster impacts agricultural producers and short-term assistance beyond what is available through existing AAFC, provincial or territorial programs may be needed.

The objective of AgriRecovery is to help producers recover from a natural disaster and to resume operations as quickly as possible. AgriRecovery provides the basis for a process of conducting assessments of the impacts of natural disasters on producers and if needed, developing and implementing initiatives that provide funding for the extraordinary costs incurred by producers as a result of the natural disaster.

Extraordinary costs are those which producers would not incur under normal circumstances, but which are necessary to:

• mitigate the impacts of the natural disaster; and/or
• resume farming operations as quickly as possible following a natural disaster.

AgriRecovery is part of the suite of Federal-Provincial/Territorial BRM programs. The components of the suite are intended to work together to help producers manage risks that are beyond their control. While other BRM programs are designed to address production and income losses, AgriRecovery initiatives, developed under the AgriRecovery Framework, are focused on helping producers with the extraordinary costs needed to recover from natural disasters, such as for the destruction of diseased animals or crops, replanting, or the purchase and transportation of animal feed. Funding from initiatives developed through AgriRecovery is intended to be a short-term support for producers and does not replace the need for long-term support provided to producers through the other BRM programs. Similarly, AgriRecovery is designed to ensure producers are not compensated twice for the same cost.

AgriRecovery initiatives can be implemented to respond to disease outbreaks, pest infestation, extreme weather events, or events resulting in the contamination of the natural environment. Initiatives cannot be developed to respond to cyclical or long-term market trends; market-driven events or trade actions not taken in direct response to a disease or pest event.

Assessments and initiatives

The AgriRecovery Framework outlines the process that is activated when the federal government receives a formal request for an AgriRecovery assessment. Assessments are jointly conducted by the federal and relevant provincial or territorial government(s), with collaboration with other stakeholders as required. All potential initiatives first undergo a preliminary assessment, which considers whether the disaster event was a recurring event, an abnormal event and whether there were extraordinary costs incurred by the affected producers to recover. If the preliminary assessment criteria are met, a formal assessment begins. The formal assessment considers whether the disaster is a collective experience, has significant negative impacts and has significant extraordinary costs that are beyond the financial capacity of the producers to manage. If the formal assessment concludes that a response is warranted, the federal and relevant provincial or territorial government(s) then obtain the necessary authorities to design and fund the initiative. Initiatives are typically funded 60% by the federal government and 40% by the province or territory. Initiatives are implemented by the relevant province or territory and the province or territory is required to report to AAFC on the timeliness, reach and impact of each initiative. Figure 1 contains a simplified description of the assessment process and Annex B contains a detailed description.

Description of Figure 1

Preliminary assessment questions:

1. Is this event considered to not be a Recurring Event?
2. Is this an Abnormal Event?
3. Are there Extraordinary Costs necessary for recovery?

If any one of the Preliminary Assessment criteria is not met, the assessment process stops and governments cannot provide assistance through the AgriRecovery Framework. If all three criteria are met, governments can proceed to the Formal Assessment stage.

Formal assessment measures:

1. Is it a Collective Experience?
2. Are there Significant Negative Impacts?
3. What are the Extraordinary Costs associated with recovery and are they significant?
4. Capacity to Manage: With the help of existing programs, are Extraordinary Costs beyond the capacity of producers to manage?

The results of the analysis for the four Formal Assessment Criteria will be taken together to conclude whether or not an AgriRecovery response may be warranted.

3.2 Resources

Allocation of full-time equivalents

AgriRecovery full-time equivalents ranged from 1.63 to 7.99 over the course of the evaluation period due to unpredictable variations in workload requirements. When few, small or simple assessments were underway, AgriRecovery workload was lighter and some employees were reassigned to other projects, such as African Swine Fever, Livestock Tax Deferral, Livestock Price Insurance projects and corporate items such as reporting and changes to the BRM suite of programs in the Sustainable Canadian Agricultural Partnership.

Allocation of Funds to AgriRecovery initiatives

AgriRecovery is allocated $125 million annually. Due to the unpredictable nature of natural disasters, the full allocation may not be spent in any given year. Twenty-seven AgriRecovery initiatives were active during the evaluation period, totalling $540.6 million in federal contributions (see Annex C). During the evaluation period, the annual allotment was exceeded once (in 2021-22) (see Figure 2).

Description of Figure 2

	Number of initiatives reporting in fiscal year	Annual Allotment ($ 125 million)	Actual federal payments (million $)
2007	3	125	0.15
2008	5	125	30.11
2009	6	125	12.91
2010	9	125	240.53
2011	12	125	235.28
2012	2	125	11.55
2013	2	125	1.28
2014	1	125	3.23
2015	1	125	4.12
2016	3	125	8.27
2017	3	125	12.27
2018	1	125	0.84
2019	2	125	9.47
2020	5	125	23.69
2021	11	125	481.97

4.0 Relevance

This section summarizes evaluation findings on the relevance of AgriRecovery, specifically the alignment with departmental and federal priorities, the relationship with AAFC's other business risk management programs and responsiveness to challenges faced by the sectors due to climate change.

4.1 Alignment with departmental and federal priorities

AgriRecovery aligns with governmental roles, responsibilities and priorities outlined in the Minister's Mandate Letter and in the Departmental Plan AgriRecovery contributes to the departmental core responsibility of addressing sector risk by mitigating the financial impact of risks beyond producers' control that threaten the viability of their operations. AgriRecovery provides a decision-making structure for federal, provincial and territorial governments to determine when federal and provincial/territorial support payments for disaster recovery are warranted. These payments help producers with the extraordinary recovery costs that are beyond their capacity and enable producers to resume business operations as quickly as possible.

4.2 Integration of AgriRecovery with the core BRM suite of programs

AgriRecovery is part of the BRM suite of programming. In addition to AgriRecovery, the BRM suite of programs consists of 3 core programs that are intended to assist farmers to cover income and production loss risks in different ways. AgriRecovery is designed to cover extraordinary costs (excluding income, revenue and production losses) necessary to recover from a natural disaster, beyond what is covered by these core BRM programs (see Figure 3). The core BRM programs are:

• AgriStability: a margin-based program that protects Canadian producers against large declines in farming income for reasons such as production loss, increased costs and market conditions. Farmers generally must enrol annually and pay a fee, though in cases of significant downturn, federal and provincial governments may allow producers to enrol after the enrolment deadline. Participating producers may receive a payment if their production margin for a given year falls below their historical reference margin by more than 30%. The program is delivered by the federal government in Manitoba, Newfoundland and Labrador, Nova Scotia, New Brunswick, Yukon and the Northwest Territories. AgriStability is delivered provincially in the other provinces.
• AgriInvest: a self-managed producer-government savings account designed to help producers manage small income declines and make investments to manage risk and improve market income. The program includes government matching of up to 1% of Allowable Net Sales. AgriInvest is managed federally in all provinces other than Quebec, where it is managed provincially.
• AgriInsurance: a federal-provincial-producer cost-shared program that stabilizes producers' income by minimizing the economic effects of primarily production losses caused by severe but uncontrollable natural hazards. AgriInsurance currently covers crops in all provinces, maple syrup production in some provinces and has developed directives for provinces to follow in offering livestock production insurance. AgriInsurance is delivered by provincial entities (departments or Crown corporations) in all 10 provinces. The specifics of each province's AgriInsurance program differs, including eligible costs, eligible commodities, deductibles and maximum coverage. Producers pay insurance premiums to participate in the program.

Figure 3: Relationship between BRM Programs with frequency of events and source of risk

Source: Adapted from Antón, J., S. Kimura, and R. Martini's (2011) OECD risk management in agriculture in Canada and Agriculture and Agri-Food Canada's (2012) Evaluation of income stability tools — AgriStability and AgriInvest

Description of Figure 3

Frequency of events is depicted on the Y axis, ranging from rare and severe events to frequent and less severe events. The source of risk is depicted along the X axis, indicating: production, price and input, and extraordinary costs. AgriInvest addresses production, and price and input risks, that are frequent and less sever. AgriStability addresses production, and price and input risks, that are rare and sever. AgriInsurance addresses production risks that are frequent and less sever through its standard layer; as well, as rare and sever events through its catastrophic layer. AgriRecovery addresses extraordinary costs for rare and severe events.

Through the assessment process, the amount of costs incurred by producers is compared to the support they can expect to receive from federal and provincial programs to limit the duplication between AgriRecovery and other programs. Other BRM programs might not (based on eligible costs) cover all types of extraordinary costs incurred by producers; or might have deductibles that are too high for producers to cover in the case of a disaster. These program details can vary by province and territory based on decisions they have made in the implementation of the BRM programs.

There are currently no requirements for producers to participate in other BRM programs to be eligible for AgriRecovery, or to have to subscribe to other BRM programs in the future as a condition of receiving AgriRecovery funding. Evaluation findings suggest that some producers choose not to participate in other BRM programs in anticipation of receiving AgriRecovery support if they need it (at no cost to themselves). Similarly, there is no evidence to suggest that producers receiving AgriRecovery funding are choosing to subsequently enrol in other BRM programs to protect against future risk of a similar type of disaster. However, participation in the core BRM programs can help farmers manage ongoing risks, including risks that are becoming increasingly prevalent due to climate change.

4.3 Climate change

Canada's agricultural sector will be negatively affected by the increased frequency and intensity of extreme weather events due to climate change. This was noted in AAFC's 2017 evaluation of AgriRecovery and re-affirmed in federal publications such as the Emergency Management Strategy for Canada: Toward a Resilient 2030 and the Sustainable Agriculture Strategy: Discussion Document. Recent extreme weather events have decreased crop yields, deteriorated pasture and rangeland conditions, caused widespread feed and water shortages and threatened farming communities with wildfires. AAFC recognizes that agriculture is an inherently risky activity and provides tools that help protect agricultural producers against income and production losses due to external factors such as natural disasters through its BRM suite. As the frequency and intensity of extreme weather events increases, the demand on BRM programs, including AgriRecovery, will increase. Concerns about the ability of AgriRecovery to adequately support producers in recovering from natural disasters have been raised in various fora including a 2017 parliamentary standing committee report. These concerns generally focused on AgriRecovery's provisions limiting eligibility for recurring and multi-year events.

In addition to increased focus on the impacts of climate change, the federal government has also recognized the importance of emergency management in response to natural disasters, with the publication of government documents like the Emergency Management Framework for Agriculture in Canada (2016) and signing on to the United Nations' Sendai Framework for Disaster Risk Reduction. The emergency management continuum consists of prevention and mitigation, preparedness, response and recovery. AgriRecovery's assessments and initiatives focus principally on the response aspect of the emergency management continuum.

AgriRecovery initiatives do not currently include provisions to incentivize producers to mitigate the impacts of future similar disasters and thus be more resilient to future extreme events since AgriRecovery is not designed to facilitate mitigation or promote resilience. However, there are other programs that encourage prevention, mitigation and preparedness activities for producers, such as the Sustainable Canadian Agricultural Partnership Cost-Shared Resilient Agricultural Landscape Program and AAFC's Agricultural Climate Solutions — On-Farm Climate Action Fund and Agricultural Clean Technology Program: Adoption stream.

5.0 Efficiency and effectiveness

This section provides an overview of the extent to which AgriRecovery's assessments and initiatives were efficiently and effectively delivered and a discussion of the key factors contributing to AgriRecovery's efficiency and effectiveness.

5.1 Understandability of AgriRecovery

The evaluation found that there is a clear distinction in understanding AgriRecovery and its processes between those who deliver AgriRecovery and those who may receive funding from the initiatives under AgriRecovery.

Government officials understand their roles and responsibilities, though implementation ability varies

The federal and provincial officials, who are responsible for delivering AgriRecovery by conducting the assessments and establishing and delivering the initiatives, generally understand their roles and responsibilities under AgriRecovery, its 6-step process and the key criteria and their definitions that determine whether a natural disaster is eligible for an AgriRecovery initiative.

While federal and provincial officials understand their roles and responsibilities, there is a distinction between large and small provinces in their capacity to lead the conduct of the assessments. Larger provinces often have the capacity and expertise to lead the assessment, whereas smaller provinces generally have less capacity to lead assessments and are more likely to rely on federal expertise and data in assessments.

Overall, there is widespread agreement that the division of responsibilities between federal and provincial levels of government is appropriate, collaborative and functioning well.

Producers and industry associations do not have a good understanding of AgriRecovery

The evaluation found that producers and industry associations have a poor understanding of both the assessment process and initiatives' eligible costs and their interactions with other BRM programming. AAFC and its AgriRecovery partners have worked to make AgriRecovery more understandable to producers and industry associations by including information describing the 6-step process of AgriRecovery online, by holding stakeholder feedback sessions explaining when initiatives were not required and advertising the initiatives through various channels (see section 5.3 for more details). However, despite these efforts, a lack of understanding persists.

For assessments conducted under AgriRecovery, producers and industry associations reported a lack of clarity on how the key terms in the formal assessment have been applied. For example, producers and industry associations did not understand how the producers' capacity to manage was assessed in the formal assessments. Producers and industry associations also reported a lack of predictability for how initiatives were launched across the country. For example, producers and industry associations have noted when initiatives were launched in other provinces but not theirs for similar natural disasters and did not understand the reasons why initiatives were not launched in their provinces. This perceived inconsistency and lack of predictability has incentivized producers and industry associations to pressure the provincial governments to request an AgriRecovery assessment after each natural disaster in the hopes of receiving support, even if the situation does not merit it.

For initiatives, producers hold 2 beliefs that impact their understanding of AgriRecovery and limit their participation in other BRM programs:

• Despite information published on AAFC's website that clearly notes that revenue losses are not covered by AgriRecovery, producers mistakenly think that AgriRecovery's initiatives cover such losses. As a result, they often put considerable pressure on provinces to submit a request for an assessment when one may not be applicable.
• Producers also have a poor understanding of how AgriRecovery eligible expense amounts are determined. Assessments assume that producers fully participate in other BRM programming and eligible amounts available to producers in the initiatives are set at amounts based on that assumption. Producers may receive less than required to cover their extraordinary costs if they do not participate in other BRM programs, instead choosing to rely primarily on AgriRecovery to support them through natural disasters. Additionally, producers may not implement other risk mitigation strategies (for example, private insurance and/or self-insurance activities on farm) because they assume an AgriRecovery initiative will be created to support them after any natural disaster.

Producer misunderstandings and the impacts those misunderstandings have on the implementation of AgriRecovery is a longstanding issue. The 2011 evaluation found that producers did not have a good understanding of AgriRecovery's eligibility criteria or its role as part of the suite of BRM programming. The evaluation recommended that AAFC work with provinces and territories to provide more effective producer education and communication about the intent of AgriRecovery and its role within BRM. AAFC responded by committing to update the AgriRecovery web page and including an AgriRecovery guide online that could be printed and distributed. AAFC also committed to continue holding sessions to explain AgriRecovery and the assessment process.

5.2 Timeliness of assessments and initiatives

Assessments

AgriRecovery aims to announce initiatives within 120 days of receiving a request for an assessment so that producers have timely information on the type of support available to them to make operational decisions in response to natural disasters. Most of the initiatives active in the evaluation period (19/27) were announced within 120 days.

The evaluation found that there are 2 longstanding issues in the assessment process that continue to impact the timeliness of assessments and the announcement of initiatives: data availability on the impacts of the natural disaster and a lack of risk-based program delivery.

Data availability is critical for timely assessments

One of the key factors that contributes to the timeliness of assessments is the federal and provincial governments' access to information on the impacts of the natural disaster on producers, at the time that the assessment is requested. Assessments are more efficient when sufficient data is available to understand and analyze the impacts of a natural disaster. Starting an assessment while a disaster is still in progress and sufficient data is not available can result in assessments taking longer and initiatives being delayed in their implementation.

AAFC staff and their provincial partners developed informal processes in the evaluation period to increase the timeliness of assessments by either undertaking national level assessments that encompass multiple provinces or using past information in their assessments. For example, AgriRecovery officials used a national approach to assess the impacts of Covid-19 on hog and cattle herds and applied provincial circumstances in the initiative development for the Covid-19 hog and cattle initiatives. AgriRecovery officials also used data from previous assessments in more recent assessments for wildfires in British Columbia and bovine tuberculosis in Alberta and Saskatchewan.

Assessments did not undergo a risk assessment process

The evaluation found that assessments for smaller initiatives took much longer than those for the larger initiatives that were complete in the evaluation period. This is consistent with the 2017 evaluation of AgriRecovery, which found that smaller initiatives were less likely to meet their timeliness targets. The 2017 evaluation recommended that a risk review template be developed and implemented that would reduce the time taken to assess the natural disaster and implement the initiatives, particularly for AgriRecovery initiatives of relatively low risk or materiality. The 2013 Office of the Auditor General of Canada audit also recommended that AAFC assess the risk for each initiative and streamline its administrative efforts for smaller, lower-value initiatives. However, AgriRecovery officials indicated that they had developed such an approach, but found it to not be helpful.

In its current form, AgriRecovery assessments undergo the same 2-stage assessment process, regardless of the number of producers affected by the natural disaster or the level of financial support required by producers following a natural disaster.

Broadly, delayed assessments result in delayed initiatives and support for producers, which can negatively impact their ability to recover from a natural disaster. Given the longstanding nature of this inequitable provision of funds, there is a need for AAFC to identify the sources of the delays and implement measures to resolve them.

Need for procedure manual and additional guidance at federal and provincial levels

The evaluation found that AgriRecovery processes rely heavily on existing expertise of the federal and provincial officials responsible for AgriRecovery as opposed to accessing comprehensive and evergreen guidelines. The federal processes on how to fulfill the federal responsibilities in the AgriRecovery 6-step process rely on the experience of the officials involved and is not formally documented in a comprehensive procedure manual for AAFC staff. In addition, provincial representatives, particularly for smaller provinces, were interested in the development of guidelines and templates to support the assessment process, including further details on the definition and articulation of key terms.

Given the unpredictable nature of when assessments are requested, the volume of assessments that may be required to be conducted and initiatives that may need to be developed at one time, AAFC staff are assigned to other projects and only assigned to AgriRecovery related activities when the workload is high. In flexible workload assignments like this with short timeframes when work is required, procedure manuals are essential to meeting deadlines. Comprehensive, evergreen procedure manuals have many benefits, including:

• standardizing steps in a process
• increasing the efficiency of the project because there is a documented reference for employees to consult before seeking guidance
• increasing the scalability of work because more people can do the same tasks the same way
• assisting with the onboarding of employees to a new project
• increasing the equitable treatment of potential initiatives being assessed
• identifying areas where the process is delayed, from which solutions to resolving the delays can then be determined and implemented

Initiatives

Payments for AgriRecovery initiatives are made in a timely manner to disaster-affected producers

The evaluation found that the majority of the 21 initiatives completed in the evaluation period (14/21; 67%) issued 75% of the program's payments within 300 days of the start of the assessment process making support available to disaster-affected producers in a timely manner. AgriRecovery was generally recognized as being able to provide support quickly once initiatives were in place, especially when compared to AgriStability payments. AAFC's 2022 evaluation of AgriStability found that AgriStability's program payments often took several months and sometimes took up to a few years to be processed.

Tension between timely support and targeted support

The evaluation found that there is a tension in AgriRecovery between providing timely support to producers and having the necessary data available to provide targeted support that responds to the demonstrated need from producers. Producers and industry associations pressure governments to request assessments during a natural disaster in the hopes of receiving support during the disaster. AgriRecovery officials need sufficient data on the actual impacts of the natural disaster on producers, which may not be available until after the disaster is over, to fully assess the impact of the disaster and trigger an AgriRecovery initiative. For example, in the case of weather-related disasters involving crops, it is not possible to calculate the losses incurred by producers until harvesting is completed.

To respond to the uncertainty when delivering an initiative before a disaster event ends, AgriRecovery has designed the terms and conditions of some initiatives to be flexible to accommodate changing situations for producers and remain relevant as the disaster event evolved (for example, the 2018 Canada-British Columbia Wildfire Recovery Initiative and the Covid-19 hog and cattle initiatives). This created a precedent for how AgriRecovery could respond to natural disasters and may have long-term impacts on the volume and timing of requests for assessments and expectations for how initiatives will be designed.

5.3 Reach of initiatives

There were 27 AgriRecovery initiatives in place during the evaluation period reaching 33,859 producers. In 2021, 31,346 (93%) of the recipients received funding when there were 9 active initiatives, accounting for 89% of total expenditures in the evaluation period.

Initiatives in place during the evaluation period supported a wide variety of producers, including but not limited to, cattle and hog producers, beekeepers, maple syrup producers and apple and pear farmers. The initiatives supported producers' purchase of additional feed for livestock impacted by droughts and wildfires, replaced bee colonies destroyed by wildfires, replace maple syrup collection systems and replaced perennial crops.

The evaluation found that the AgriRecovery initiatives were assumed by AAFC and provincial officials to have reached eligible affected producers because of the focused assessments and the demand-driven nature of the initiatives. As part of the formal assessment step in AgriRecovery, AAFC and the relevant province/territory collect data and analyze the impact of the natural disaster on the affected producers in a region. The analysis determines whether those affected have the financial capacity to manage the impacts of the natural disaster. Based on the formal assessment analyses, the federal and provincial/territorial officials may design an initiative that responds to the impacts on and extraordinary costs incurred by, those affected producers. Affected producers are not required to participate and can decide whether to apply for the funding available through the initiative.

AAFC, the provinces and the industry associations have used numerous ways to make the initiatives known to affected producers and to make the initiatives easy to participate in. For some initiatives

• industry associations notify their members of the initiatives when they are announced
• governments notify producers of initiatives through media releases
• provinces notify producers individually when they know who is affected and eligible
• provinces post initiative information on their websites
• provinces have simplified their applications (including instances where only an attestation is required)

Despite these efforts there is no evidence to confirm whether all producers in the area affected by a disaster were aware of the implementation of an initiative. There also remain situations where eligible producers face barriers in participation. For example, producers may face challenges in participating if they cannot afford to pay upfront costs and then submit claims for reimbursement or are unable to submit applications through established application processes (for example due to limited internet access).

AAFC does not have information to determine whether the initiatives reached under represented groups (that is, women, youth/new farmers, Indigenous Peoples, persons with disabilities and other groups who are under-represented in agriculture) in the producer groups who were affected by the natural disasters and eligible for the initiatives. Under the Canadian Agricultural Partnership, the provinces and territories, as the initiative administrators, were not required to collect information on the characteristics of initiative participants, including data on gender and diversity factors and provide this information to AAFC. Provinces and territories were required to provide AAFC with the numbers of participating producers while the initiatives were active through progress reports and after the initiatives had ended through final reports. Progress reports included the number of applications received, the number of applications processed and the number of eligible applicants. The final reports were to provide the number of producers affected by the disaster and the number of applications received under each initiative. Neither type of report included demographic information about the producers receiving support. As a result, the evaluation was unable to assess how well AgriRecovery and its initiatives supported different population groups.

5.4 Impact of initiatives

Initiative delivery and administration was cost effective

The evaluation found that most of the 21 initiatives completed in the evaluation period (16/21; 76%) had a ratio of direct provincial and federal delivery costs relative to the total cost of payments to producers of less than 6%, the threshold set in the performance measurement strategy for cost effectiveness. The average direct delivery cost was 4%. In cases where the 6% administrative cost threshold was exceeded, provinces explained the higher costs were due to, for example, only a small number of producers participating but still needing to set up and implement the initiatives; and using third-party delivery agents which increased the administrative costs for the initiative.

The measurement of cost effective initiatives was included as an ultimate outcome in AgriRecovery's Logic Model (see Annex D). Ultimate outcomes for programs generally describe the desired change in the target population of a program and are logically linked to the objective of the program, whereas the AgriRecovery indicators focus on the process (how the initiative is delivered) rather than the results of those initiatives. The link between the objective of AgriRecovery initiatives (that is, to help producers recover from a natural disaster event and resume operations as quickly as possible) and one of the ultimate outcomes in the Logic Model (that is, cost effectiveness of the delivery and administration of AgriRecovery initiatives) is not clear. Thus, there is a need for the development an appropriate indicator to measure the extent to which the initiatives achieved the objective of AgriRecovery.

Provinces report that AgriRecovery initiatives support producers' in recovering from natural disasters

Almost all of the 21 initiatives completed in the evaluation period (20/21; 95%) reported that at least 70% of initiative participants stated that the payments played an important role in their recovery. One initiative did not collect data on this outcome because it had only one participant. This information was provided to AAFC by the provinces and is based on self-reported data from producer surveys. However, this metric is biased toward a positive response as recipients of funding tend to rate funding that they receive positively and does not enable an understanding of how initiative funding helped farmers.

Challenges in assessing the performance of AgriRecovery against its outcomes has been an ongoing issue and was first noted in the 2011 evaluation. Under Growing Forward, AgriRecovery reported on the percentage of recipients still farming one year after they received initiative funds. This indicator was removed under Growing Forward 2 and was not required under the Canadian Agricultural Partnership. Best practices for collecting performance data include requiring recipients to report after the conclusion of a project, such as through periodic follow-ups with recipients; or options that would support linking AgriRecovery recipients to current measurements of outcomes include collecting the business numbers/tax data of AgriRecovery initiative recipients and including questions on AgriRecovery initiatives in the farm financial survey. To determine the feasibility of various options available, AAFC will have to work with the provinces and territories.

The evaluation found that the provinces and producers consider the initiatives to be important in the response to the natural disasters. In the absence of an initiative, provinces reported that they would have to either seek other support (such as by soliciting ad hoc funding) or provide support unilaterally. Producers reported that they would either obtain private loans, cease operations or take several years to recover from the effects of the natural disaster. Producers and provinces both stated that the initiatives assisted the producers in recovering from a natural disaster or gave them time to make decide how to respond to the long-term impacts of the natural disaster (such as maintain operations at current levels or downsize operations like herd size).

6.0 Conclusions and recommendations

Conclusions

The evaluation confirmed that AgriRecovery is relevant to federal and departmental priorities, including the departmental core responsibility of mitigating sector risk. AgriRecovery is part of the BRM suite of programming and covers extraordinary costs (not covered by the other programs) that are necessary to help farmers recover from natural disasters.

There is currently no comprehensive procedure manual bringing together all the necessary procedures and guidance materials related to AgriRecovery. Pulling existing guidance materials and knowledge in to a comprehensive manual has the potential to help increase the efficiency of assessment and initiative design processes.

The role of AgriRecovery in the BRM suite, as well as the relationship of AgriRecovery to the other BRM programs is not well understood by producers and industry associations. This contributes to pressures placed on AAFC and provincial and territorial governments to implement AgriRecovery initiatives when the criteria are not fully met. Given the impact that pressure from producers and industry associations can have on when assessments are conducted, AAFC officials should explore further actions that can be taken to increase producer and industry association understanding of the intent of AgriRecovery and its place in the BRM suite of programming so that producers and industry associations access the appropriate support from the right BRM program at the appropriate time. Options include, but are not limited to:

• increased education and communication with producers and industry associations that receive AgriRecovery funding to encourage participation in these programs
• including provisions in AgriRecovery initiatives that require purchase of insurance to better support future disasters

AgriRecovery does not require participation in other BRM programs as a condition of receiving AgriRecovery funding, or enrolment in BRM programs for those who have received funding. Thus, there are no provisions included in AgriRecovery to encourage mitigating actions for producers to increase resilience to natural disasters. Such resilience is becoming increasingly important with the increase in frequency and intensity of natural disasters resulting from climate change. Furthermore, there are opportunities for AAFC to consider the impacts that climate change will have on the demand for assessments and initiatives, including the resources that will be required to assess more events and develop corresponding eligible initiatives.

AgriRecovery is an efficient framework but has little data to assess the reach and/or impact of its initiatives. Existing performance metrics that relate to impact are based solely on self-reported survey data from producer surveys and is limited to asking whether funding was helpful. Other indicators relate to the administration of assessment and initiatives and provide no insight into the progress of the Framework or its initiatives in helping farmers recover from natural disasters.

Recommendations

Recommendation 1: The Assistant Deputy Minister, Programs Branch, in collaboration with the ADM, Strategic Policy and ADM, Public Affairs and in consultation with provincial and territorial partners, should develop a plan to increase producer participation in other federal, provincial and territorial programs that address business risk management and increase producer resilience to natural disasters.

Recommendation 2: The Assistant Deputy Minister, Programs Branch should develop a comprehensive and evergreen procedure manual for the federal component of AgriRecovery to ensure consistency in processes.

Recommendation 3: The Assistant Deputy Minister, Programs Branch, in consultation with the provinces and territories, should develop performance metrics to assess the reach and impact of AgriRecovery initiatives.

Management response and action plan

Management agrees with the evaluation's recommendations and has developed an action plan to address each recommendation.

Annex A: Evaluation methodology

Literature review

To assess program relevance, efficiency and effectiveness, the evaluation assessed provincial and federal government debate transcripts, peer-reviewed and grey literature and media articles published since 2015. The review also provided context for the environment in which AgriRecovery operates. This work was supported by staff from the Canadian Agriculture Library.

Environmental scan

The evaluation reviewed of other disaster relief programs available to producers in Canada.

Program document and file review

The evaluation reviewed AgriRecovery foundational documents, as well as departmental and federal government reports, documents and files.

Key informant interviews

Interviews were conducted with internal and external stakeholders to assess program relevance, efficiency and effectiveness. The evaluation involved 37 interviews, including AAFC staff (4), industry associations (7), provincial officials (17), recipients of initiative funding (8) and an academic expert (1).

Case studies

Six case studies were conducted of 5 initiatives and one assessment that did not result in an initiative. Case study selection considered the geography, affected commodity group, nature of the disaster, value of payments to producers and number of producers receiving assistance.

Research for the case studies included document and file review, a review of project reports and other available documentation along with interviews with the relevant province and a selection of funding recipients. The case studies were:

• 2021 Canada-British Columbia Wildfire and Drought Recovery Initiative
• 2021 Canada-Saskatchewan Drought Response Initiative
• 2020 Canada-Ontario COVID-19 Hog Maintenance Initiative
• 2017 Initiative Canada-Québec d'aide aux entreprises agricoles affectées par des épisodes exceptionnels de grêle
• 2016 Canada-Nova Scotia Maple Sector Initiative
• 2018 New Brunswick Excess Moisture and Cold Temperatures Assessment

Methodological limitations

The following methodological limitations were considered in interpreting the data:

Table: Methodological limitations

Limitation	Mitigation strategy	Impact on evaluation
Devolved initiative administration The application of AgriRecovery results in initiatives that are administered by provincial/territorial bodies.	The Evaluation was designed to examine the Framework rather than the initiatives administered by the provinces.	Low
Insufficient Details or Context in Hansards The Hansard debates did not always identify the natural disaster affecting producers or what AgriRecovery initiative members of the government were discussing.	To contextualize the details of the debates, internet searches were performed using the time period of the debate to understand what natural disasters or AgriRecovery initiatives were being discussed. Data from Hansards were triangulated with other lines of evidence.	Low
Interviewee response bias. Key informants who participated in the evaluation may have a vested interest in the continuation of BRM programming.	Interviews with participants from the different stakeholder groups generated a variety of perspectives. Data was synthesized within and across stakeholder groups and triangulated with other lines of evidence where possible to eliminate potential bias.	Low
Interview response rate. There were 5 invited producer groups that did not participate in the interviews and therefore their perspectives on AgriRecovery are not represented in the key informant interview line of evidence. Producer group perspectives varied depending on whether they had benefited from AgriRecovery or not.	Other sources of evidence such as literature and media reviews were examined to broaden the perspectives of stakeholders considered in the evaluation. All sources of evidence were triangulated with other lines of evidence where possible to eliminate potential gaps.	Low

Annex B: AgriRecovery Framework Process

The AgriRecovery Framework includes 6 steps for the federal, provincial and territorial governments to follow to determine whether AgriRecovery related intervention is required and if so, to develop and administer an initiative. The 6 steps are as follows:

1. Request for an assessment: The AgriRecovery Framework is activated when a request for a disaster event to be assessed under the AgriRecovery Framework are made. Requests are typically made by a province or territory, but can be initiated by the federal government.
2. Assessment is performed: After a request for an assessment is made, the federal government and provincial/territorial governments involved form a task team, which is responsible for determining whether additional government support is needed and the nature of the response. The assessment consists of 2 stages.

   Stage 1: The first stage is the preliminary assessment, which assesses whether the disaster event was a recurring event, an abnormal event and whether there were extraordinary costs incurred by the affected producers to recover. If the assessment concludes that it was not a recurring event, it was abnormal and extraordinary costs were incurred, the second stage of the assessment is completed.

   Stage 2: The second stage is the formal assessment, which assesses whether the disaster event is a collective experience, has significant negative impacts and has significant extraordinary costs that are beyond the financial capacity of the producers to manage.

3. Decision and authorities: The findings and conclusions of the assessment form the basis on which Federal, Provincial and/or Territorial governments then decide whether to proceed with an AgriRecovery response.
4. Agreement on the technical details: Participating governments work together to finalize the technical details of the initiative, which are set out in a funding agreement.
5. Launch of the initiative: The initiative is announced, or targeted producers are notified and program materials are made available (for example, the terms and conditions for the initiative, application forms, etc.)
6. Payments are made to producers: Submitted applications are processed and AgriRecovery payments are made by a provincial or territorial entity.

Annex C: AgriRecovery initiatives 2015-16 to 2021-22

Initiative name	Planned federal contribution ($)	Actual federal contribution ($)	Percentage of planned federal contribution spent (%)
2014 Canada-BC Avian Influenza Assistance Initiative	750,000	408,152	54
2014 Canada-Manitoba Forage Shortfall and Transportation Assistance Initiative	3,000,000	2,340,000	78
2016 Canada-Alberta Bovine Tuberculosis Assistance Initiative	9,000,000	8,210,641	91
2016 Canada-Nova Scotia Fire Blight Initiative	1,690,000	783,328	46
2016 Canada-Nova Scotia Maple Sector Initiative	590,000	590,000	100
2016 Canada-Saskatchewan Bovine Tuberculosis Assistance Initiative	618,000	62,968	10
2017 Canada-British Columbia Wildfire Recovery Initiative	12,000,000	5,781,598	48
2017 Canada-Québec Support Initiative for Farm Enterprises Affected by Exceptional Hail Episodes	7,800,000	6,487,200	83
2018 Canada-British Columbia Bovine Tuberculosis Assistance Initiative	600,000	274,339	46
2018 Canada-British Columbia Wildfire Recovery Initiative	3,000,000	839,273	28
2018 Canada-Prince Edward Island Fall Harvest Recovery Initiative	9,300,000	9,198,725	99
2019 Canada-British Columbia Forage Freight Assistance Initiative	600,000	256,859	43
2020 Canada-Alberta Fed Cattle Feed Cost Offset Initiative	25,920,000	16,983,187	66
2020 Canada-Manitoba Finished Cattle Feed Assistance ProgramEndnote 1	2,500,000	854,162	34
2020 Canada-Ontario COVID-19 Beef Emergency Feed Maintenance Initiative	3,000,000	1,899,676	63
2020 Canada-Ontario COVID-19 Hog Maintenance Initiative	3,000,000	783,262	26
2020 Canada-Saskatchewan Livestock Set Aside Program	7,500,000	3,764,662	50
2020 New Brunswick Hog Recovery	75,000	24,632	33
2020 PEI Hog Recovery Initiative	300,000	257,059	86
2021 Canada-Alberta Hog Recovery Initiative	1,800,000	436,408	24
2021 Canada-Alberta Livestock Feed Assistance InitiativeEndnote 1	219,000,000	212,926,382	97
2021 Canada-British Columbia Flood Recovery Program for Food SecurityEndnote 1	120,000,000	23,950,220	20
2021 Canada-British Columbia Wildfire and Drought Recovery InitiativeEndnote 1	12,000,000	7,193,948	60
2021 Canada-Ontario Dry Weather AgriRecovery InitiativeEndnote 1	7,500,000	2,334,302	31
2021 Canada-Manitoba AgriRecovery Drought Assistance InitiativeEndnote 1	93,000,000	54,178,737	58
2021 Quebec COVID-19 Initiative	13,086,000	13,086,000	100
2021 Canada-Saskatchewan Drought Response Initiative	168,498,000	166,725,199	99
Grand Total	726,127,000	540,630,920	74
Source: AAFC program administration data Notes: Table 1 note [1] Initiative was still active as at March 31, 2022; actual federal contribution reflects amounts paid as at March 31, 2023. Return to table 1 note [1] referrer

Annex D: Logic Model

Activities

AgriRecovery Assessment

• Assess disaster events to determine whether the event meets the AgriRecovery criteria and, if so, the level and type of assistance necessary to help affected producers; Set milestone targets and monitor progress against these targets;
• Review risks related to assessment and proposed initiative.

Authorities / Initiative Development

• Draft documents to obtain authorities, negotiate agreement, initiative terms and conditions and other program material (for example, applications, etc.) with province(s)/territory(s)

Initiative implementation and delivery

• Receive and review applications; process payments
• Record data and submit progress reports on program uptake, payments and administration expenses
• Submit Final Reports, participate in Federal/Provincial-Territorial discussions of lessons learned, submit annual report to Treasury Board Secretariat (TBS), participate in Recipient Audits

Outputs

AgriRecovery Assessment

• Assessment report: includes recommendations to participating Federal-Provincial/Territorial Ministers on level and type of assistance; agree to Timeline Milestones; identified risks and mitigation strategies

Authorities / Initiative Development

• Approved Program Authorities, Initiative Announcement
• Signed Contribution Agreement
• Terms and Conditions; Application Forms

Initiative implementation and delivery

• Payments to Recipients; Reimbursements to province/administrator; Progress Reports; Final Reports; annual Report to TBS, Recipient Audits

Immediate outcomes

• AgriRecovery initiatives are announced in a timely manner to allow producers to make operational decisions
• Once applications for the initiative are received and processed, payments are made in a timely manner to disaster-affected producers

Intermediate outcomes

• AgriRecovery initiatives are accessed by disaster-affected producers in targeted regions

Ultimate outcomes

• AgriRecovery initiatives support producers' in recovering from natural disasters
• Program delivery and administration was cost effective

Source: AgriRecovery Performance Information Profile, Programs Branch, 2022