Evaluation of the Agricultural Greenhouse Gases Program

December 2, 2014

Report:
Office of Audit and Evaluation

Acronyms

AAFC
Agriculture and Agri-Food Canada
AESB
Agri-Environmental Services Branch
AGGP
Agricultural Greenhouse Gases Program
AF
Agroforestry
BMP
Beneficial Management Practice
CA
Contribution Agreement
CAAP
Canadian Agricultural Adaption Program
CH4
Methane
CO2
Carbon Dioxide
CMP9
Meeting of the Parties to the Kyoto Protocol
COP19
The nineteenth session of the Conference of Parties
CS
Cropping Systems
DSDS
Departmental Sustainable Development Strategy
DIAP
Developing Innovative Agri-Products
EC
Environment Canada
EQ
Equivalent
FSDA
Federal Sustainable Development Act
FSDS
Federal Sustainable Development Strategy
GF
Growing Forward
GF2
Growing Forward 2
GHG
Greenhouse Gases
GHGMP
Greenhouse Gas Mitigation Program
GDP
Gross Domestic Product
GRA
Global Research Alliance
IPCC
Intergovernmental Panel on Climate Change
LS
Livestock Systems
MRAP
Management Action and Response Plan
MOU
Memorandum of Understanding
MT
Megatonnes
NAHARP
National Agri-Environmental Health Analysis and Reporting Program
NCGAVS
National Carbon and Greenhouse Accounting and Verification System
N2O
Nitrous Oxide
OAE
Office of Audit and Evaluation
PB
Programs Branch
RB
Research Branch
SAGES
Sustainable Agriculture Environmental Systems Program
SSTA
Sustainable Science and Technology Advancement
STB
Science and Technology Branch
UNFCCC
United Nations Framework Convention on Climate Change
WEBS
Watershed Evaluation of Beneficial Management Practice Program
WU
Water Use Efficiency

Executive Summary

Background and Profile

The Agricultural Greenhouse Gases Program (AGGP) is Canada's first domestic response to the Global Research Alliance (GRA) on Agricultural Greenhouse Gases. Canada is one of the founding members of the GRA which was formally launched at the United Nations Climate Change Conference in Copenhagen in December 2009.

The objective of the Agricultural Greenhouse Gases Program (AGGP) is to enhance the understanding and accessibility of agricultural technologies, Beneficial Management Practices (BMPs) and processes that can be adopted by farmers to mitigate Greenhouse Gas (GHG) emissions in Canada.

The Government of Canada originally committed $27 million towards Vote 1 (Operating) and Vote 10 (Grants and Contributions) for this AAFC program from September 1, 2010 to August 31, 2015. The end date was later extended to March 31, 2016.

Key Findings

The AGGP is addressing a need for Agricultural Greenhouse Gas mitigation research. It is aligned with federal priorities and contributes to the AAFC departmental Strategic Outcome (2014-15) of: "An Innovative and Sustainable Agriculture, Agri-food and Agri-based products sector."

The AGGP continues to be relevant since the program's GHG mitigation efforts have the additional benefit of enhancing the environmental sustainability of Canadian agricultural production.

The AGGP is achieving its expected outputs with the development of: scoping studies, technology transfer and research plans, formal and informal collaborative arrangements, papers, journal articles, presentations, information products and technical studies.

The program is achieving its immediate outcomes by developing, verifying and validating new GHG mitigation information and technologies, and by producing technology transfer methodologies and approaches targeted at farmers.

The program is making progress towards achieving its intermediate outcome of making GHG mitigation information and technologies available to farmers. It is also helping to promote the sustainability of agricultural production.

Research projects are producing BMPs in numbers that achieve the anticipated performance measurement targets. A review of the 2012-13 project performance reports indicated that there were at least 12 BMPs demonstrated or under development. This amount was in line with the performance measurement target of 12 for the program duration. At this stage, it is difficult to ascertain the extent to which there will be farmer uptake of BMPs.

The transfer of the AGGP from the Agri-Environment Services Branch (AESB) and Research Branch (RB) to the Programs Branch (PB) led to some challenges including issues with communications and administrative processes in the middle of the program.

1.0 Introduction

This report presents the findings of the Evaluation of the Agricultural Greenhouse Gases Program (AGGP). The AGGP is a five-year federal initiative within the fiscal years of 2010-2011 to 2015-2016. The AGGP supports projects that develop greenhouse gas (GHG) mitigation methods and technology and makes them available to Canadian farmers through beneficial management practices (BMPs).Footnote 1

1.1 Purpose of the Evaluation

The evaluation was conducted by Agriculture and Agri-Food Canada's (AAFC's) Office of Audit and Evaluation (OAE) as part of AAFC's Five-Year Departmental Evaluation Plan (2014-2015 to 2018-2019). The evaluation examines the core issues of relevance and performance including effectiveness, efficiency and economy in accordance with the Treasury Board Policy, Directives and Standards on Evaluation (2009). The evaluation fulfills the requirement of the Financial Administration Act, that all programs of grants and contributions be assessed every five years. This program has not been previously evaluated. Findings from the evaluation will inform planning for future program development.

1.2 Structure of the Report

The evaluation report contains seven sections, including this introduction section (Section 1.0). Section 2.0 describes the context around the program, the program profile, delivery and resources. Section 3.0 provides the methodology used in the evaluation: the scope and approach, a description of the lines of evidence used in the evaluation and evaluation limitations. Section 4.0 details the evaluation findings including issues of relevance and performance (effectiveness, economy and efficiency). Section 5.0 provides the conclusions. Section 6.0 provides the issues and recommendations for the evaluation and Section 7.0 provides a listing of appendices.

2.0 Program Profile

2.1 Program Context

Agriculture, including livestock and crop production, and associated land use such as soil and nutrient management and deforestation is responsible for about a quarter of world greenhouse gas emissions.Footnote 2 Canada's agricultural greenhouse gas emissions occur through livestock or animal production, crop production, manure management, agricultural soils, forest land conversion to cropland and water use practices. The emissions from livestock production are caused by enteric fermentation particularly during the handling and storage of livestock manure.  Emissions of Nitrogen (N2) from agricultural soils consist of direct and indirect emissions as well as emissions from animal manure on pasture, range and paddock.

In Canada, emissions directly related to animal and crop production account for 8.0% (56 Megatonnes (Mt) Carbon Dioxide (CO2) equivalent) of the total 2012 GHG emissions. This represents an increase of 19% (9 Mt CO2 equivalent) since 1990. Twenty-two percent of methane (CH4) and 74% of nitrous oxide (N2) national emissions are related to agriculture. Several factors account for the increase from 1990 such as the expansion and intensification of the beef cattle and swine industries, and increases in the application of synthetic nitrogen fertilizers in the prairiesFootnote 3.

Table 1: Canada's GHG Emissions from the Agriculture Sector (1990–2012) in Megatonnes CO2 equivalent
Greenhouse Gas Categories 1990 2000 2005 2008 2009 2010 2011 2012
Enteric Fermentation 16 20 22 20 19 18 17 18
Manure Management 6 7 8 7 7 7 6 6
Agriculture Soils[a] 25 29 29 31 30 30 29 32
Agriculture Sector Total 47 56 58 58 56 55 53 56
[a] Includes emissions from Field Burning of Agricultural Residues
Source: Reproduced from Environment Canada, 2012. National Inventory Report 1990-2012: Greenhouse Gases, Sources and Sinks in Canada.

Table 1 presents GHG emissions from the Agriculture Sector from 1990 to 2012 including the categories of Enteric Fermentation, Manure Management and Agriculture Soils. It shows that emissions have increased from 1990 to 2012 but decreased slightly from 2008 to 2012.

Emissions trends can vary and change according to a number of factors including the amount of livestock population, crop production and synthetic fertilizer consumption as well as the ability of soils to sequester carbon. For example, from 2005 to 2008, livestock populations decreased, while synthetic fertilizer consumption increased and crop production was high. As a result, the declines in emissions from livestock production were offset by increases in emissions from crop production. These trends have persisted in recent years, influencing overall emissions.Footnote 4

As the world population increases and food demands correspondingly increase, projections indicate that world-wide agricultural emissions will rise by about 30-40 percent above 2005 levels. Rising GHG emissions will impact climates and influence the productive potential of food producing areas since warming climates have a negative effect on crop production.Footnote 5

The Canadian agriculture sector has an important part to play in mitigating greenhouse gas emissions since it is recognized as a world leader in soil carbon sequestration (the process by which carbon is taken from carbon dioxide in the air and transformed into the solid organic matter found in soil) and can draw on its expertise to assist other countries to meet mitigation and adaptation targets through the transfer of knowledge and technology.Footnote 6

Canada has participated in addressing global agricultural greenhouse gases through its membership in the Global Research Alliance (GRA). Canada is one of the founding members of the GRA which was formally launched at the United Nations Climate Change Conference in Copenhagen in December 2009. The Global Alliance on Agricultural Greenhouse Gases Charter was signed by Canada, on June 24, 2011 in Rome, Italy. The GRA is an international network of more than 30 developed and developing countries from all regions of the world. The goal of the GRA is to increase international cooperation and investment in research activities to help reduce the emissions intensity of agricultural production and increase the potential for soil carbon sequestration. The GRA aims to do this in a way that will help improve the efficiency, productivity, resilience and adaptive capacity of agricultural systems, contributing in a sustainable way to overall greenhouse gas mitigation efforts while still helping to meet food security objectives.

As a member of the GRA, Canada joined an international network to focus agricultural research activities on practical benefits for both the environment and Canadian farmers. Canada has contributed to the GRA by participating in two Research Groups: Croplands and Livestock Emissions. Through the AGGP, Canada is concentrating its efforts on four priority areas: Agroforestry, Irrigation and Water Use Efficiency, Perennial Cropping Systems and Livestock Emissions. As well, Canada has contributed to the GRA activities of the cross-cutting themes of soil carbon and nitrogen cycling and is the co-lead, with the Netherlands, for the inventories and measurements theme. The AGGP is Canada's first domestic response to the GRA on agricultural greenhouse gases.

Apart from membership with the GRA, AAFC has had a history of involvement with the monitoring and research of greenhouse gas mitigation. Previous to the AGGP, there were several programs that played a role in greenhouse gas measurement, monitoring and mitigation. They include: the Greenhouse Gas Mitigation Program for Canadian Agriculture (GHGMP), the National Carbon and Greenhouse Gas Accounting and Verification System (NCGAVS), National Agri-Environmental Health Analysis and Reporting Program (NAHARP), the Sustainable Agriculture Environmental Systems Program (SAGES), and the Watershed Evaluation of Beneficial Management Practice Program (WEBS).

In addition to the AGGP, AAFC continues to be involved with GHG research and mitigation efforts through a number of initiatives including: Holos, Sustainable Science and Technology Advancement (SSTA), Cropland Estimates, research projects at AAFC Research Centres across Canada and projects within the Agri-Clusters Program and the Canadian Agricultural Adaption Program (CAAP). (See Appendix C)

2.2 Overview of the Program

The Agricultural Greenhouse Gases Program (AGGP) is designed to provide Canadian farmers with beneficial management practices (BMPs) to manage their land and livestock in a way that will mitigate greenhouse gases (GHG) emissions. The objective of the AGGP is to enhance the understanding and accessibility of:

  • agricultural technologies;
  • beneficial management practices (BMPs) – agricultural practices aimed at reducing the environmental impact of farming activities on the landscape; and
  • processes that can be adopted by farmers to mitigate GHG emissions.

The AGGP supports the development of projects that will lead to the adoption of GHG mitigation technology, and make it available to Canadian farmers. The program's focus is on science and knowledge creation, and knowledge and technology transfer. Mitigation will be achieved by reducing emissions or by increasing carbon sequestration, consistent with the overall mission of the Global Research Alliance (GRA).

AGGP projects are intended to achieve the following:

  • provide science-based results that are intellectually credible, evidence-based and/or action-oriented to advance knowledge and understanding of greenhouse gas mitigation in agriculture;
  • develop new greenhouse gas mitigation technologies and practices that Canadian farmers can adopt to improve their environmental performance and profitability;
  • enable outputs/deliverables to be incorporated directly into technology transfer to agricultural groups and individual farmers;
  • address gaps in the development and/or extension of agricultural greenhouse gas mitigation technology;
  • be incremental or complementary in nature and/or build on existing information and not duplicate existing work; and
  • provide information that can be shared domestically and internationally.

The AGGP is a five-year, application-based contribution agreement program with projects located across Canada. With respect to Canada's membership commitments to the GRA, the AGGP focuses on the following priority areas:

  • Livestock Systems (LS): Activities that reduce GHG emissions and may include manure management, as well as grazing and feeding strategies;
  • Cropping Systems (CS): Developing knowledge and technology for reducing GHG emissions and maximizing carbon sequestration from forage, pasture and rangeland, as well as wetland and zero tillage management, if significant information and technology gaps exist;
  • Agricultural Water Use Efficiency (WU): Quantifying GHG emissions and enhancing carbon sequestration through improved irrigation and drainage practice; and
  • Agroforestry (AF): Developing and enhancing research networks and demonstrations across Canada, focusing on carbon sequestration and sustainable bio-energy production.

The AGGP proposal applications were rated to ensure that all of the proponents had anticipated results, deliverables and performance indicators that included:

  • formal plans for research and technology transfer activities for AAFC and its partners;
  • technology transfer plans for national promotion of GHG mitigation BMPs and technologies; and
  • formal and informal collaborative arrangements for developing and sharing of applied research on GHG mitigation.

Eligible applicants included: corporations, cooperatives, partnerships, charitable organizations, provincial or territorial governments, municipal governments, educational institutions, First Nations bands, and provincial or territorial crown corporations.

2.3 Program Delivery

The AGGP has been managed by AAFC's Programs Branch (PB) since April 2012. Prior to this time, it was developed and managed by Agri-Environmental Services Branch (AESB) and the Research Branch (RB) as part of the Department's commitment to an environmentally sustainable agricultural sector. An AAFC Technical Committee, comprised of content experts, is responsible for the rating and selection of proposals using a proposal rating guide and subsequently makes recommendations to the Minister. Upon approval, contribution agreements are developed and signed.

2.4 Program Resources

The AGGP is a $27 millionFootnote 7 federally funded program running from September 1, 2010 to March 31, 2016.Footnote 8 The total program funding authority over six fiscal years is $1.9 million for Vote 1 and $25.1 million for Vote 10. AGGP considers projects that are larger multi-year or smaller shorter-term, providing that they meet program objectives. A maximum of $10 million per recipient is available over the duration of the program.

Table 2: Program Resources (in Millions of $) Authority and Approved – Vote 10
Balance 2010/11 2011/12 2012/13 2013/14 2014/15 2015/16 Total
Authority 0.5 5.4 5.4 5.4 5.4 3 25.1
Approved 0 3.2 5.3 4.8 4.4 1.1 18.9
Variance (in %) 100 41 2 11 19 63 25
Source: Programs Branch, AAFC

Table 2 presents AAFC program resources for Vote 10 in millions of dollars for each fiscal year (2010-11 to 2015-16) of the total duration of the Program. The total authority (Vote 10) was $25.1 million and the total approved to date was $18.9 million indicating a variance of 25 percent.

3.0 Methodology

3.1 Evaluation Scope and Approach

The evaluation was conducted by AAFC's Office of Audit and Evaluation (OAE) employing internal and external resources to complete the data collection and analysis. The evaluation collected and examined both primary and secondary data from multiple lines of evidence to address the evaluation issues and questions. The evaluation examined AGGP activities between the fiscal years of 2010-2011 and 2013-2014.

As per the Treasury Board Directive on the Evaluation Function (2009), the evaluation examined the program's relevance and performance. Specifically, the evaluation examined: the continued need for the program; alignment with government priorities; alignment with federal roles and responsibilities; achievement of intended outcomes; and, the extent to which the program demonstrated efficiency and economy.

The evaluation covers programming under AAFC's Program Alignment Architecture (2014-2015), consisting of the Strategic Outcome: "An Innovative and Sustainable Agriculture, Agri-food and Agri-based products sector," within Program 2.1: Science, Innovation, Adoption and Sustainability and Sub-program 2.17: Agricultural Greenhouse Gases.

3.2 Data Collection Methods

The evaluation included the following data collection methods:

  • Document Review: A review of foundational documents provided background information and context on the design and delivery of the program and helped to assess questions related to relevance and performance.
  • Project File Review: An analysis of project files was undertaken for 19 approved projects of which 18 had contribution funding agreements.Footnote 9 The review provided information on the projects such as: proposals, screenings and ratings, communications, contribution agreements, site visit reports, work plan progress reports, financial and budgetary reports and annual performance reports. The file review examined two progress reports for each funded project: the first from April 1, 2011 to March 31, 2012 and the second from April 1, 2012 to March 31, 2013. One final project report was reviewed from one project that was completed on March 31, 2013.
  • Operational Data Review: As part of the operational data review, general program records were examined including: case-specific records as well as procedural guides and manuals. This information provided evidence of procedural processes and performance information to gain an overall understanding of the program outcomes and economy and efficiency.
  • Literature Review: A review of literature provided an overview of programs addressing greenhouse gas emissions and climate change in Canada, New Zealand, the United States of America and the European Union. Funding programs were identified through internet searches and searches through government portals. Programs were selected based on their similarity to the AGGP. Comparative information was gathered on approaches to funding research on greenhouse gas mitigation technologies, inventory of current greenhouse gas stores, and dissemination of information to the agricultural community.
  • Key Informant Interviews: Interviews (n=22) were undertaken with program recipients and AAFC staff. A sample of 12 program recipients (2 project managers and 10 project researchers) were interviewed to collect information on results achieved, any challenges experienced, and lessons learned. Ten AAFC staff members were interviewed including: staff involved in current and past program design and delivery, and staff who were also GRA members. The questions addressed both relevance and performance issues for the program.
  • Case Studies: A sample of three projectsFootnote 10 was analyzed at a more in-depth level to assess activities and outcomes. The sample was selected based upon the degree to which BMPs have been developed, the degree to which BMP's have been adopted by producers, the amount of project data available (that is, progress reports, interviews), the representativeness in terms of region as well as the priority needs and the viability of preliminary project information (that is, ability of existing project information to inform evaluation of immediate and long-term outcomes through examining BMP adoption and GHG emission reduction potential). Each case study was based on several lines of evidence, collecting information from a range of sources including: an in-depth structured document review of the project documents (that is, foundational documents, project files, database review) and; in-depth interviews with project stakeholders (n=5). The case studies identified the extent to which the project was implemented as intended, challenges encountered, preliminary impacts, need for funding and the extent to which funding contributed to other complementary GHG reduction activities.

3.3 Evaluation Limitations

There were several limitations associated with this evaluation. Analysis around performance reporting information was limited to two progress reports for each funded project: from April 1, 2011 to March 31, 2012 and from April 1, 2012 to March 31, 2013 inclusive. This was due to the timing of the evaluation and the availability of reports (recipients have 60 days after the fiscal year-end to provide their Annual Performance Reports). In all cases, but one, the information provided in the progress reports was limited as the program has not yet ended and the progress reports available were from the early stages of the projects.

The program transition from AESB and RB to PB also affected the evaluation since PB was not the custodian of some earlier operational records, program data, historical context and information concerning the Program which were not available from AESB/RB.

The evaluation mitigated these limitations by using multiple lines of evidence to fill in any gaps around missing information. Where possible, the evaluation team collected information on the history of the program and longer-term impacts of the program through interviews with early program staff members to ensure the coherence and consistency of evidence.

4.0 Evaluation Findings

This section of the report presents key findings related to AGGP. More specifically, it explores the relevance of the program, its effectiveness and the extent to which economy and efficiency have been realized.

4.1 Relevance

This section provides key findings on the relevance of the AGGP which includes an examination of: continued need for the program; alignment with government priorities; and, alignment with federal roles and responsibilities.

4.1.1 Continued Need for the Program

The evaluation assessed the alignment of program activities to determine whether the program continues to address a demonstrable need and is responsive to the needs of Canadians.

The AGGP addresses Canadian's international obligation to the GRA to work towards agricultural greenhouse gas mitigation. This program also addresses a need based on scientific evidence that agricultural greenhouse gases have been increasing since 1990 (as outlined in Table 1). As such, the AGGP is responsive to its commitment to the GRA and the needs of the agriculture and agri-food sector by working towards mitigating agricultural greenhouse gases through BMPs.

Through interviews, file review and operational data review, the evaluation found evidence that there was a large interest for the AGGP as evidenced by the response to the initial call for research proposals. There were 68 proposals received from a wide array of organizations, including universities, non-government organizations such as industry groups and local conservation groups as well as provincial agencies. The funding request from the initial call for proposals was over $100 million which was considerably greater than the available funding under the program. This response indicates interest in the program and motivation to conduct research in the subject area. The majority of project recipients and program staff indicated that there is a strong need for the federal government to support agricultural GHG projects.

The AGGP addresses the need for Canadians and agricultural stakeholders to have research on agri-environmental sustainability. Through the public consultation process on Growing Forward, agricultural stakeholders indicated that Canadians are increasingly concerned about the environment and expect government to provide leadership in this area. They called for greater government investment towards applied research in the development of BMPs to support agri-environmental sustainability and innovation. Consistent with this, during public consultations on the draft (2013-2016) Federal Sustainable Development Strategy (FSDS), Canadians emphasized the importance of the environment and sustainable development. One of the topics included encouraging more actions to accelerate reducing GHG emissions.Footnote 11 In Growing Forward 2 consultations, adaptability and sustainability were again part of the discussions. It was felt that there should be research and development on intensive agriculture not on extensive energy-heavy agriculture and it was suggested that BMPs should continue to be researched, evaluated, modified and developed.Footnote 12 Federal, provincial and territorial Ministers of Agriculture agreed to increase opportunities for provinces and territories to invest in agri-environmental initiatives. This includes more flexibility for provinces and territories to tailor programs to local needs and increased opportunity for provinces and territories to invest in agri-environmental initiatives and on-farm water infrastructure. Key to achieving this goal is collaboration with industry, academia and the public sector, and maintaining competitiveness and strengthening capacity in science and innovation in the agriculture, agri-food and agri-based products sector.Footnote 13

Prior to the AGGP, research suggested that producers generally lacked an understanding of agriculture's contribution to GHG emissions and climate change. As well, the absence of financial incentives for the mitigation of GHGs, or disincentives for curtailing emissions of GHGs, often resulted in GHG emissions remaining a non-issue for them.Footnote 14 Producer acceptance of environmental sustainability issues is changing however, as indicated by the Canadian Federation of Agriculture's (2012) National Food Strategy's emphasis on an "environment perspective" for its strategic outcomes and recognition of the need for GHG mitigation and BMPs. Specifically, the report indicates the importance of Canada's food chain capacity in meeting future demand in a sustainable manner and that Canada should continue to be a leader in: validating greenhouse gas emission reduction within the food production and processing industry across all countries; having international agreements signed with trading partners on environmental standards for food production, processing and inspection and; the use of renewable resources for food production and processing.Footnote 15

The AGGP meets the needs of producers' growing interest and understanding of beneficial management practices.

The AGGP is meeting the needs of the agriculture sector by its distribution of funding projects across the country. The AGGP does not have projects spanning the entire value-chain or representation from all Canadian bio-geoclimatic zones, but provincial jurisdictions within Canada are fairly well represented. Most provinces had applicants applying to the AGGP and at least one project being approved, with the exception of New Brunswick and Newfoundland.

Table 3: Number of Projects by Province
Province Applied but not Funded Approved and Allocated Funding Total Number of Projects Proposals
Alberta 14 2 16
British Columbia 3 2 5
Manitoba 4 2 6
New Brunswick 2 0 2
Newfoundland and Labrador 1 0 1
Nova Scotia 1 1 2
Ontario 10 4 14
Quebec 9 4 13
Saskatchewan 6 3 9
Total 50 18 68
Source: Programs Branch, AAFC

Table 3 presents the Number of Projects by Province and indicates that there were applications submitted from 9 provincial jurisdictions of which 7 were approved and allocated for at least 1 project.

The AGGP is meeting the needs of the priority areas for greenhouse gas emissions mitigation efforts in Canadian agriculture. The project funding was distributed across four areas specifically: Livestock Systems having 4 committed projects, Cropping Systems having 6, Agroforestry having 5 and Agricultural Water Use Efficiency having 3 (See Table 4). The inclusion of agroforestry and irrigation in the program's research topics was considered by respondents to be an unusual and beneficial aspect of the program.

The AGGP is the only place where we have seen agroforestry identified as an eligible area. Because we combine forestry on one side and crops and livestock, it is a systems approach rather than a commodity; it tends to fall through the cracks.

- Project Researcher
Table 4: Approved Funding Dollar ($) Amounts by Project Type
Project Type Total Applications for Projects Total Projects Allocated Funding Sum of Projects Allocated Funding Dollars
Agroforestry 10 5 $4,302,341
Cropping Systems 28 6 $4,949,616
Livestock Systems 20 4 $6,812,301
Agricultural Water Use Efficiency 10 3 $2,851,121
Total 68 18 $18,915,379
Source: Programs Branch, AAFC

Table 4 indicates the total applications for projects, the total projects allocated funding and the sum of projects allocated funding dollars by the project type. There were 10 applications for Agroforestry projects, 28 for Cropping Systems projects, 20 for Livestock Systems projects and 10 for Agricultural Water Use Efficiency projects. Livestock Systems and Cropping Systems were allocated project funding of $6.8 million and $5.0 million respectively. Agroforestry was allocated $4.3 million and Agricultural Water Use Efficiency was allocated $2.9 million.

Complementary AAFC Greenhouse Gas Related Projects

The majority of AGGP recipients interviewed did not access other AAFC initiatives and viewed the program as having minimal overlap with other federal funding programs.

There are complementary programs that examine greenhouse gas monitoring and research at AAFC. Programs such as the Agri-Science Clusters Program (Clusters) and the national Canadian Agricultural Adaptation Program (CAAP) have had projects that address agricultural greenhouse gas mitigation issues using vote 10 contribution agreements:

  • the national Canadian Agricultural Adaptation Program (CAAP) (2009-2014) had two projects that addressed greenhouse gas mitigation: a boiler-incinerator (for burning biomasses) project and a dairy greenhouse gas pilot project; and
  • the Clusters Program(2008-2013) under Growing Forward had some research projects that addressed greenhouse gas mitigation such as within: the Dairy Cluster, the Canadian Agri-Science Cluster for Horticulture, and the Canola/Flax Agri-Science Cluster.

Even where there were no current projects taking place, many of the Cluster organizations' websites indicated an interest in and recognition of greenhouse gas mitigation and environmental sustainable practices. For example, the Canadian Ornamental Horticulture Research and Innovation Cluster had a "Research and Innovation Strategy" that included decreasing the environmental footprint of horticulture production; the Pulse Science Cluster indicated that pulses are a low carbon food since pulses use half the non-renewable energy inputs of other crops; and the Organic Science Cluster characterized the contribution of organic production to reducing greenhouse gas emissions.

It was found that AGGP projects tend to be distinctive from Clusters and national CAAP projects since the AGGP requires the development of BMPs, and this is generally not the case with Clusters and national CAAP projects. The additional evidence of stakeholder interest in GHG research and sustainable agricultural practices further supports the need for the AGGP.

In gathering information on AGGP and comparing this program with other AAFC GHG initiatives, the evaluation found that a comprehensive repository of information for GHG work that includes scientists and specialists, facilities, networks, BMPs, tools, inventories and producer groups involved does not currently exist. This finding may indicate a need to better coordinate GHG activities across AAFC.

4.1.2 Alignment with Government Priorities

The evaluation assessed the linkages between program objectives and government priorities and departmental strategic outcomes. It was found that AGGP objectives are aligned with federal government priorities and AAFC departmental strategic outcomes.

The AGGP objective of developing agricultural practices aimed at reducing the environmental impact of farming activities on the landscape is consistent with the federal government's commitment to sustainable development reflected in the Federal Sustainable Development Act which received Royal Assent on June 26, 2008.

The purpose of the Act is "to provide the legal framework for developing and implementing a Federal Sustainable Development Strategy (FSDS) that will make environmental decision-making more transparent and accountable to Parliament".

The Act requires each Minister presiding over an FSDS department or agency to prepare a Departmental Sustainable Development Strategy (DSDS). The FSDS was developed by Environment Canada in consultation with 27 federal departments and agencies, to bring together goals, targets and implementation strategies under the following four priority environmental themes:

  • addressing climate change and air quality;
  • maintaining water quality and availability;
  • protecting nature; and,
  • shrinking the environmental footprint - beginning with government.

The FSDS, tabled on November 4, 2013, guides the Government of Canada's (2013–2016) sustainable development priorities for a period of three years, as required by the Federal Sustainable Development Act (FSDA). AAFC has updated its DSDS which was submitted as part of its Report on Plans and Priorities for 2014-15. AAFC supports the implementation of the FSDS through the activities found in the departmental strategy. Specifically, AAFC is committed to an economically, socially and environmentally sustainable agriculture, agri-food and agri-based products sector. The AGGP aligns with this in that its greenhouse gas mitigation efforts were found to be supporting an environmentally sustainable agriculture. Sustainable management of natural resources is a core requirement for an economically successful agricultural sector.

The DSDS describes AAFC's objectives and plans for sustainable development appropriate to its mandate. It articulates AAFC's vision and decision-making process for sustainable development, and sets out the Department's contribution to the goals and targets of the FSDS. The AGGP is included within the Report on Plans and Priorities under sub program: 2.1.7. stating that: "The Agricultural Greenhouse Gases Program (AGGP) provides contribution funding to partners from the agriculture industry, governments and academia for projects to undertake research on greenhouse gas mitigation and make new mitigation technologies and Beneficial Management Practices (BMPs) available to farmers."

When the AGGP was initiated in 2010, the objective was directly linked to the AAFC strategic outcome of "An environmentally sustainable agriculture, agri-food and agri-based products sector" and fell under the "On Farm Action" program activity in the Department's Program Activity Architecture."

During the evaluation period of 2014-2015, the AGGP resides under the strategic outcome of "An Innovative and Sustainable Agriculture, Agri-food and Agri-based products sector."

4.1.3 Alignment with Federal Roles and Responsibilities

The evaluation assessed the alignment of roles and responsibilities for the federal government in delivering the program.

The Federal government has a role and responsibility for delivering programs such as the AGGP within its Federal Sustainable Development Strategy and is committed to international cooperation on sustainable management of agriculture, forest and other lands. The Federal government has outlined a commitment to Agricultural Greenhouse Gas emissions through "Canada's Action on Climate Change." Canada participated in the United Nations Climate Change Conference (COP19 or CMP9) which was held in 2013 in Warsaw, Poland. Canada's priorities arising from the conference were to have a fair, comprehensive, global climate change agreement. The Government of Canada supports the Durban Platform for Enhanced Action, and is committed to ensuring negotiations progress efficiently towards a single, new international climate change agreement. It indicates that such an agreement must:

  • include meaningful and transparent commitments from all major emitters;
  • support constructive and ambitious global action;
  • balance environmental protection and economic prosperity; and,
  • maintain a long-term focus.Footnote 16

Through Canada's Action on Climate Change (2013) the Government of Canada believes that sustainable management of forests, agriculture and other lands is critically important to the global effort to address climate change. Deforestation and forest degradation, through agricultural expansion, conversion to pastureland, infrastructure development, destructive logging, fires etc., account for up to one-quarter of the world's greenhouse gas emissions. The Government of Canada's approach to climate change is focused on delivering environmental and economic benefits for all Canadians through:

  • regulations to reduce greenhouse gas (GHG) emissions;
  • strategic investments in areas such as clean energy technology and climate change adaptation;
  • world-class scientific research to support policy development and decision-making; and,
  • taking a leadership role in international climate change efforts.

The AGGP falls within the roles and responsibilities of the federal government policy of Canada's Action on Climate Change (2013). The federal government's commitments to Action on Climate Change (2013) through the Federal Sustainable Development Strategy, the United Nations Framework Convention on Climate Change (UNFCCC), and the GRA has ensured that there is national funding for scientific research to improve understanding of the impacts of agricultural activities on GHG emissions, as well as the development of tools, practices and technologies to reduce and mitigate emissions. The AGGP reflects these commitments by increasing international cooperation, collaboration and investment in public and private research activities to help the sector reduce GHG emissions while enhancing productivity and resilience to climate change.Footnote 17

Although the AGGP is not a Growing Forward (GF) program, it aligns with GF1 and GF2 agricultural policy frameworks in that it establishes conditions for long-term competitiveness, sustainability and adaptability, with an emphasis on industry capacity and industry self-reliance. The AGGP is federally funded but its projects and BMP are being developed and implemented locally.

4.2 Performance

4.2.1 Achievement of Expected Outcomes

In this section of the report, the evaluation examines the effectiveness of the AGGP. It examines the extent to which the AGGP produced its anticipated outputs, and achieved its immediate, intermediate and long-term outcomes.

Information on the projects' progress was obtained through the project file review, operational data review, interviews and case studies. It was found that in spite of delays with some projects, and the short time frame since the program inception, the projects were beginning to achieve outputs and outcomes.

4.2.1.1 Achievement of Outputs

The evaluation found that the program had produced its anticipated outputs including: scoping studies, technology transfer and research plans, formal and informal collaborative arrangements, papers, journal articles, presentations, information products and technical studies.

An examination of the files indicated that projects generally produced scoping studies, and technology transfer and research plans early in the research project development.

Formal and Informal Collaborative Arrangements

Most of the projects had information in the files on how their collaborations were progressing beyond the original proposals with the exception of two projects. With the 16 other projects, collaborations included working with universities, provincial and regional governments, AAFC scientists, producers and producer organizations, agrologists, industry, organizations, institutes and associations. In general, the types of collaborations included: research expertise, consultations and information sharing, communications and coordination with stakeholders, and field resources. This included financial contributions (such as salary dollars, professional services or equipment and materials) and in-kind contributions (such as professional time spent on projects, lab materials or equipment usage). The maximum amount of government leveraged funds was encouraged not to exceed 85% (but in the Terms and Conditions of the program, the maximum could be up to 100% of eligible expenses and this did occur, where recipients leveraged no funding). The recipients contributed 31% on average towards eligible expenses.Footnote 18

There were a few recipients that interacted with AAFC scientists during their projectsFootnote 19, but these collaborations were limited due to the funding design of the program. Recipients noted that due to the nature of the funding program (Vote 10 funds only) this scientific resource was not fully available and observed that this should be a consideration with future programming.

An important element of the AGGP was enhancing international communications and collaborations. The evaluation found that international collaborations were occurring with scientists and specialists from other countries such as the US, New Zealand, Australia, Scotland and Norway. In many cases, project researchers were attending international conferences and reporting on their findings. The most common method for dissemination of research findings was through journal articles and conference proceedings (posters and presentations).

Canada's voluntary participation in the GRA is also an important mechanism for information sharing on GHG mitigation technologies and beneficial management practices. As the research results continue to emerge, continued membership in the GRA will help these outcomes to be transmitted to other GRA member countries. As well, international collaborations could be enhanced, if AAFC scientists participate in this kind of programming down the road.

GHG Mitigation Knowledge Transmission

The evaluation found that GHG Mitigation knowledge was being produced and disseminated. Approximately 50 journal articles and papers were submitted and/or accepted and 60 presentations at conferences and professional forums on GHG mitigation were made during the evaluation period. The project researchers were submitting their findings to a range of journals across the various disciplines; the majority reported targeting journals with high impact factors for their discipline, or journals of note (for example, American Journal of Soil Science, Forestry Ecology Management and the Journal of Dairy Science).

In addition, other information products included: 2 book chapters, 38 technical reports, factsheets, newsletters and articles, 4 theses completed, 3 on-line courses (e-learning platforms) and 2 web-based educational videos.

Table 5: Knowledge Information Produced
Knowledge Information Produced Related to GHG Mitigation Number
Journal Articles and Papers Submitted and/or Accepted 50
Presentations at conferences and professional forums 60
Book Chapters 2
Technical Reports, factsheets, newsletters, articles 38
Theses completed 4
On-line courses developed 3
Web-Based Educational Videos 2
Total 159
Source: Programs Branch, AAFC (File review examining the September 1, 2010 to March 31, 2013 period)

The AGGP was also promoted, identified and gained visibility through having:

  • media coverage in the Globe and Mail, Canadian Broadcasting Corporation, Western Producer, Hill Times Policy Briefing and Canadian Television Network's "Farm Gate" program;
  • a project selected as one of the regional finalists for the province of British Columbia's (2012-13) Premier's Innovation and Excellence Awards Program;
  • project partners receive the British Columbia Premier's Award for Excellence in Partnership;
  • increased visits to the prairieshelterbelt.ca website;
  • virtual and physical field tours hosted by beef producers; and
  • graduate students winning awards for work and presentations.

The AGGP was designed to develop and transfer knowledge to Canadian farmers and encourage actions to mitigate GHGs. The focus is on both science and knowledge creation and on implementing technology knowledge transfer. The AGGP emphasizes that through the transmission of information and adoption, producers will be able to see the link between GHG emissions and their present agricultural practices, how this link relates to other agri-environmental issues, and how any action they take could affect their competitiveness.

The evaluation evidence suggests that the AGGP increased the volume of collective knowledge regarding agricultural GHGs in Canada. In turn, demonstrable agricultural GHG BMPs further heightened the profile of Canadian researchers as leaders in the field.

The AGGP provides a significant basis of science, especially for Canada in international negotiations and reporting as it provides Canadian agriculture with Canadian studies that can back up Canadian agricultural production practices.

- AAFC Staff

It was found that federal government funding provided credibility to the research and helped to facilitate the dissemination of research findings on an international level.

4.2.1.2 Immediate Outcomes

The evaluation found that the AGGP was making progress towards achieving the two immediate outcomes:

  1. New GHG mitigation information and technologies were being developed, verified and validated; and
  2. technology transfer methodologies and approaches were being developed for targeting farmers about GHG mitigation.
New GHG Mitigation Information and Technologies

The evaluation found that the AGGP was making progress towards achieving its immediate outcome of developing, verifying and validating new GHG mitigation information and technologies. The file review indicated that there were about 30 new GHG mitigation tools, information and technologies being developed, verified and validated during the evaluation period. This was well above the initial target number of 5 indicated in the performance measurement targets of the Program. Each project reported to be producing at least one tool or technology addressing a specific GHG issue. Some examples of information, tools and technologies developed, verified and validated were:

  • soil analysis methodologies;
  • estimates of enteric methane from non-confinement beef production systems;
  • impacts of shelterbelts on livestock and crop production;
  • geochemical techniques to examine field scale process in the soil and groundwater responsible for GHG production;
  • data on the economic and environmental influence of silvopastoral systems;
  • impacts of controlled tile drainage; and
  • impacts of feed production systems on GHGs.
Technology Transfer Methodologies and Approaches

The evaluation found that the projects are achieving the immediate outcome of developing technology transfer methodologies and approaches targeted at farmers. At the time of the evaluation, there were about 13 methodologies and approaches being developed, which was above the initial performance measurement target of 8. Most project researchers were hesitant to state how successful their research had been at transferring technology to farmers as their research was currently underway. The majority of those who did feel that their technology could be shared felt that they have had significant success in transferring their technology to farmers.

The AGGP created the opportunity for additional components to be included in research projects allowing for secondary impacts of technologies that were being developed. This value-added position allowed for more robust research to be conducted and provided a more complete picture of the associated benefits of a BMP.

The way we have to approach it is from a production or integrated management perspective. The AGGP has provided us with the ability to layer in that component which we normally would not have had either the funding, the capacity, or the time to layer in... What AGGP has allowed us to do is to take a management approach for agroforestry/silvopasture... and allowed us to layer in the greenhouse gas component. This becomes part of the package. We report out to industry as we normally would not have covered that.

- Project Researcher
4.2.1.3 Intermediate Outcomes
Availability of GHG Mitigation Information, Technologies and BMPs

The evaluation found that project beneficiaries are making progress towards developing information and technologies for farmers and demonstrating new BMPs at field days, fairs and workshops. The file review indicated that during the evaluation period, approximately 1650 producers had attended sessions where the projects were demonstrated or communicated. This included information sessions with producers at meetings, conferences or farm exhibitions, and visits to web-sourced information on BMPs. As well, during the evaluation period, over 60 producers participated in the research by volunteering their property as research sites. The majority of project researchers had difficulty providing an exact figure for the number of farmers that they have reached via all methods of communication (for example, field days, journal articles, newspaper articles, online portals). They were all able to provide estimates for the number of farmers participating in their field days. They indicated that the number of producers participating in field days ranged from 20 to 200, with the majority reporting just under 50 producers per field day. The use of field demonstrations was reported as being successful as farmers could see the science in action.

Through a review of the 2012-13 project performance reports, there were at least 12 BMPs (demonstrated or under development) identified. This amount was in line with the performance measurement target for the program of 12. For about half the projects, it was too early to identify BMPs. Given the progress to date, it is anticipated that the program will surpass its target number of BMPs.

Some examples of BMPs from AGGP projects include:

  • Silvopasture project:
    1. Debris fencing as a livestock deterrent and
    2. Mechanical site preparation of forest sites for forage-seed bed preparation;
  • Farm-scale Assessment of Greenhouse Gas Mitigation Strategies in Dairy Livestock-Cropping-systems project:
    1. Manure application method (injection)
    2. Anaerobic digestion
    3. Feeding strategies;
  • Soil Nitrogen Supply, Nitrogen Mineralization Function and Nitrate Exposure project:
    1. On-farm tools to assess and manage N2 emissions
    2. Nutrient Management Training Program; and
  • Agroforestry Based Livestock Systems Adoption project:
    1. 31 virtual tours available to inform and share agroforestry based livestock system BMPs with beef producers.

The hands-on approach was cited by the majority of project researchers as the best way of disseminating results to farmers, as farmers need to see the impact of the technology and realize its benefits prior to experimentation with or adoption of the technology.

We are communicating with participating farmers on what they are doing but we are also doing work on their fields. That has greater relevancy and authority to producers. Too often, we are only doing research on research sites and on research plots which producers cannot relate to quite as directly as they can when they see the results in producers' fields. Scientifically conducting research in farmers' fields is a little bit of a challenge, as there are fewer controls in those situations, but we are undertaking these projects on producers' fields to give these technologies exposure and to engage these producers.

- Project Researcher

The following benefits were mentioned by project researchers and program staff as relating to demonstrating technologies on farmers' fields:

  • Provides real world testing of technology;
  • Improves quality of communication of technology's benefits; and
  • Increases farmer access to and understanding of technology.

Project researchers reported a variety of communication strategies that demonstrated GHG mitigation information and technologies. Most of these strategies involved some level of in-person communication. Many project researchers had begun to disseminate preliminary findings to farmers and farming communities. Methods employed included: word-of-mouth communications, field days, fairs, pilot research sites on farmers lands, demonstration sites/farm tours, presentations to Producer Groups and extension staff, and direct communication to schools. Information was made available to farmers through mass media, newspaper articles, websites, and industry newsletters.

While governments and universities often have greater resources for conducting research, smaller organizations have other advantages. The evaluation found that effective communication of research findings was influenced by the level of trust in the source of the information. Smaller organizations such as non-profit conservation organizations have a distinct advantage as they are in a position of trust to communicate directly with producers, whereas provincial governments and universities often require intermediaries (partners) to engage in their communication strategies. This may be a consideration for communication plans in future programming.

GHG Mitigation as Agricultural Sustainability

The AGGP's GHG mitigation research was reported to have the additional benefit of being perceived in the larger context of agricultural sustainability. The AGGP was reported by program staff to be providing policy makers with detailed GHG research within the context of Canadian agricultural production systems. The research was also noted to provide a credible source of information about Canadian agriculture's sustainability during trade negotiations.

By having the AGGP we could talk about agricultural GHGs and advertise the program. We could also talk about our involvement with the GRA and how all of this work would not only support the industry or the applicants to the program - not only support our department or our government and our country, but our collaborators and allies around the globe.

- AAFC Staff

Most respondents indicated that even though the needs of the sector have not changed, the logistics have. The AGGP provides the science required by the sector for the development of BMPs and their adoption. This, in turn, will allow Canadian products to be labelled as being sustainably produced and marketed.

The main change is this emphasis on environmental sustainability and sustainability certification. The sector, the commodity groups, and the marketers are becoming more and more interested and concerned with being able to demonstrate or certify sustainability... Now GHG mitigation or practices that minimize GHGs are seen, within the context of a whole suite of practices, to make agriculture more environmentally sustainable.

- Project Researcher

One of the unintended benefits of the program is that it will enable producers to develop BMPs and products which provide them with a strategic, marketing advantage in a competitive global economy.

4.2.1.4 Long-Term Outcomes
Understanding and Adoption of GHG Mitigation Technologies

The evaluation found that it was too early to demonstrate whether there was a change in farmers' level of understanding and adoption of GHG mitigation technologies. Project recipients reported that they believe that their research will have a positive impact on farmers' production, while at the same time mitigate GHG emissions. However, the actual impacts of the projects could not be measured since the research was ongoing. Project researchers speculated that impacts will be measurable five years after project completion.

This kind of work does not really result in an impact right away. Adoption follows later and impact follows much later. What comes out of the AGGP, we will not really know for another 10 years.

- Project Researcher

The majority of project researchers reported that producers tended to follow up after the field days looking for more information on BMPs. Nonetheless, it was noted that farmers may only adopt new techniques where it is economically beneficial or until a negative event occurs (for example, drought, frost). A few respondents reported that producers were adopting technology and offering their fields as additional research sites, for example:

Farmers are interested in assessing silvopasture as a tool along with some of their other potential management tools… In the southern interior region of BC, there have been many queries for information as they are interested in assessing it as a tool... Ranchers are interested in if they can institute this management practice themselves and conduct them on a 10 year license on Crown land under this model. Farmers have also instituted this practice on their own lands. The project has raised the profile of the system.

- Project Researcher

Adoption of GHG mitigation, BMPs and technologies could be enhanced if the economic rationale was made clearer to farmers. Respondents also suggested improvements to the program's communication strategy to encourage BMP adoption. Suggestions ranged from further examination of the role the Internet can play in technology adoption, development of new communication strategies to raise awareness, and more collaboration with other researchers and research teams.

In general, the uptake of new agricultural BMPs was reported to take extended periods of time, depending on the complexity of the technology and cost of adoption, well after research funds provided by the AGGP have been used. Providing a comparison, project researchers observed that tillage practices in the prairies took a decade to change between their initial introduction and wide adoption.Footnote 20

On these relatively short term projects, it is the lasting effect of the communication strategy after the termination of the project that determines the project's impact.

- Project Researcher

Recipients and program staff also indicated that once the research projects have been completed, a third party (for example, producer group) could communicate and champion BMPs more effectively to facilitate adoption.

AGGP would prepare the BMPs, but the BMPs would be rolled out to producers through the cost-shared vehicle that we have: the cost-shared environment program. If the program gets renewed, I think that is one of the streamlining things that could happen. The cost-shared environment program is a tool that has a track record and I think that cost-sharing BMP type programming should probably be the vehicle to roll out BMPs and there should be a linkage between programs like AGGP that are developing BMPs, and use the cost-shared vehicle to put it on the ground.

- AAFC Staff

The AGGP program will likely achieve some of its long-term outcomes. It is anticipated that most research will be completed and the BMPs will be developed to a level that is consistent with, or beyond, program expectations. Basic questions remain such as the actual value of each BMP, and the motivation that will make producers adopt them. Scientists are well aware that for BMPs to be adopted by producers there needs to be an economic upside for producers. One of the long-term lessons learned from the AGGP will be how to address the economic rationale of BMP adoption more directly. Actual adoption of BMPs by farmers within their jurisdictions is an outcome which will have to be revisited to fully determine the level of success.

4.2.2 Economy and Efficiency of Program Design and Delivery

Several issues were identified by AAFC staff and recipients that had an impact on economy and efficiency. These included: proposal application processes, program duration and timing, claim processes, collaboration and STB and PB communications.

Proposal Application Processes

Proposals for the AGGP were screened and rated according to set criteria and program requirements.Footnote 21 AAFC subject matter experts outside of the AGGP participated in the evaluation of proposals to ensure that proposals strongly rooted in science with the greatest chance for success were selected. To obtain approval, a project had to rate at least 50% in each category and 70% overall.Footnote 22

The selection process was based on scientific capacity and criteria of these projects. Different projects had different angles that made it intriguing... We were looking at the ability to discover science, how do we develop it, and then how to deliver it to producers. There is also a determinant direct aspect of it. While we had science, we also had to be cognisant that there also had to be a tie back to the land and be able to get to the producer.

- AAFC Staff

Sixty-eight proposals were screened in and rated. Of the 68 funding proposals rated, 19 proposals were approved and 18 of these proposals completed the process and resulted in a signed Contribution Agreement (CA). The AGGP had a 28% proposal approval rate which is similar to several other AAFC programs.Footnote 23 The AGGP was unique in that, proposals were approved in three pools: Most were approved on March 25th, 2011 (Pool A), about a quarter of proposals were approved on June 28, 2011 (Pool B) and 1 proposal was approved on April 5, 2012 (Pool C).

A total of 18 projects were allocated $18,915,379 of funding under the Program (See Appendix D).Footnote 24 Project funding ranged from $99,440 to $2,996,451, with an average of $1,050,854. All projects were multi-year.

A review of the rating data indicated that a great deal of time and effort was involved in approving and rejecting proposal applications which impacted on the economy and efficiency of the program. The evaluation found that AAFC could have streamlined the rating process better by working with applicants to allow for the optimization of proposals prior to rating. This would have sped up the approval rate and lowered the cost and time involved with rejecting proposals.

Program Duration

The duration for approvals and completion of CAs was in line with similar AAFC programs. Interviews however, indicate that there was some dissatisfaction with the process, primarily regarding the delay between being notified of project approval and completing a CA.

Based on the program data provided, it took on average 3 months for a proposal to arrive from the start of the program. During the first fiscal year of the program (2010-2011), 100% of the funding was lapsed.

Where projects were delayed (often past the summer of 2011), the achievement of expected results outlined in the original application proposals was affected. The timing of agricultural research projects has an important impact on project success, as a small delay during critical periods (for example, seeding) can result in the loss of an entire year's worth of data and the participation of highly qualified personnel (for example, graduate students).

Longer program durations can assist researchers in their research efforts, allowing for more conclusive results.

Having a five year program is huge because frequently now we have programs that are looking at one or two years. Anything like this that is dealing with biological systems, it is difficult to get meaningful results out of one or two years. The long duration allows you to predict out for the longer term.

- Project Researcher

The delay in the release of funds impacted researchers differently depending on the type of their organization. Recipients from NGOs expressed more concern regarding funding delays, as their research could not be conducted without AAFC funding. In contrast, other organizations, such as universities, had the option of reallocating funding from other sources.

It was found that applicants were not always aware of typical program processes and durations and so they did not always assign their project resources efficiently. The program design did not take into account the time available for projects based on the typical requirements for approvals and CAs. It was not timed to maximize the efficient utilization of resources across the annual growing cycle.

Claim Processes

In general, the efficiency of claim processing improved over the course of the program. Specifically, the average duration for claim processing duration decreased (or improved) over the first 3 years of the program.

There was a mixed response from recipients with respect to claim processing. Some recipients were mostly satisfied with the process, some were neutral and some dissatisfied. Comments with respect to dissatisfaction with the claims process included: a lack of guidelines, a lack of information requirements and templates, delays in repayment, a lack of transparency and a burdensome claims reporting requirement. As well, during the restructuring of the program, recipients reported that they were unable to contact the individuals handling the claims.

AAFC staff indicated that the difficulties faced by recipients were lessening since AGGP claim reporting was now in line with the Department's current standard practices, where recipients are not required to submit receipts with their expense claims. Instead, recipients are asked to provide a selected sample of receipts to the program after the expense claim has been received.

Several factors may explain these findings. New programs tend to be less efficient at first and then more efficient over time as both clients and AAFC staff grow familiar with the systems and procedures. It was also evident that changes within the department were enabling these processes to improve.

Need for Greater Collaboration

In Canada, scientists involved in GHG research are a relatively small community, spanning across industry, academia, provincial and federal governments. The exclusion of federal government researchers from the AGGP was questioned by recipients as they felt their participation would have strengthened the research efforts.

Partnerships between university, government, and industry researchers come with their own challenges, but finding that middle ground really gives you the best chance for success and realizing the larger scale goals.

- Project Researcher

The evaluation found that the program did not take into account existing scientific resources both external to and within AAFC to ensure the most efficient allocation of resources with an inclusive funding model.

STB and PB Communications

The program faced unique issues arising from the transfer from AESB and RB to the PB. PB staff had a steep learning curve during the transitional phase as they became familiar with the program. PB did not have staff on hand with GHG scientific expertise, or prior experience working on the program during its preliminary phases and design. Both the recipients and the PB staff had to spend time establishing relationships and rapport with each other.

As a result of the changeover of the program from one branch to another, the program contact person changed and the researcher did not know who to talk to. The new program contact person does not have a scientific background. The lack of continuity provided a stumbling block for researchers.

- Project Researcher

Clearly identified roles and a system of regular communications between the STB and PB are essential for scientific programs like the AGGP. The STB's participation is important for the proposal review process where individuals with a solid science background in GHG research methods are required to analyze the science within projects and determine the capacity of project researchers. The expertise of the PB is significant for program planning, risk assessment, monitoring and compliance.

5.0 Conclusions

Relevance

The evaluation findings confirm that there is a need for the federal government to support research and projects that provide Canadian farmers with beneficial management practices (BMPs) to manage their land and livestock in a way that will mitigate greenhouse gases (GHG) emission. There is also a need for the federal government to participate in national and international efforts to mitigate agricultural greenhouse gas emissions through projects such as the AGGP.

The AGGP continues to be relevant since GHG mitigation efforts have the benefit of enhancing the environmental sustainability and/or intensification of Canadian agricultural production.

The objectives of the AGGP are consistent with government priorities and AAFC departmental strategic outcomes and align with federal government roles and responsibilities.

Performance

The AGGP has been making progress towards achieving intended outcomes. AGGP Projects have been successful in developing, verifying and validating new GHG mitigation information and technologies. They have also developed technology transfer methodologies and approaches that target farmers about GHG mitigation.

The evaluation found that the AGGP is generally successful in developing formal and informal collaborative arrangements with agriculture, industry, provincial governments and agencies. The Program is also effectively generating knowledge information on GHG mitigation and BMPs through a variety of forms and mechanisms.

The AGGP is making GHG Mitigation information and technologies available to farmers through demonstrations and presentations at field days, research sites, workshops, producer meetings, fairs and farm tours. Information is also made available through the media, industry newsletters and websites.

The program is making progress towards achieving its intermediate outcome of making GHG mitigation information and technologies available to farmers and it is helping to promote the sustainability of Canadian agricultural products.

Research projects are producing BMPs in numbers that achieved the anticipated performance measurement targets. At this stage, it is difficult to ascertain how much farmer uptake of BMPs will occur.

In general, the required duration to process applications and finalize contribution agreements affected the economy and efficiency of program delivery.

6.0 Issues and Recommendations

Recommendations

The evaluation includes the following two issues and recommendations:

Issue #1:

There is an ongoing need for agricultural GHG research due to the national and international priorities of AAFC and the Government of Canada, and the direct impact of GHG on the sustainability of the agriculture sector.

Recommendation #1:

AAFC's Programs Branch with the Science and Technology Branch should:

Discuss the ongoing need for agricultural GHG programming that is responsive to the objectives of AAFC and supports Canada's national and international GHG priorities and commitments.

Issue #2:

The timing around the program start and the finalized contribution agreements had an impact upon program economy and efficiency. Coordinating program processes across multiple branches can create unique challenges related to communication, coordination, and approvals.

Recommendation #2:

AAFC's Programs Branch should:

Review program processes and timelines to improve the economy and efficiency of program delivery to design, deliver and monitor scientific-based programs such as the AGGP to ensure that the value for money in expected outcomes is identifiable and achievable.

Appendix A: Management Response and Action Plan

Recommendation Management Response and Action Plan (MRAP) Target Date Responsible Position(s)

Recommendation #1: AAFC's Programs Branch with Science and Technology Branch should:

Discuss the need for agricultural GHG programming that is responsive to the objectives of AAFC and supports Canada's national and international GHG priorities and commitments.

Agreed. A working group including STB, PB and SPB has been established and is discussing options for agricultural GHG Programming. Options that meet AAFC objectives and support Canada's national and international agricultural GHG priorities and commitments will be developed and presented to senior management for consideration.

December 2015

DG, Innovation Programs Directorate (PB)

DG, Cross-Sectoral Strategic Direction (STB)

Recommendation #2:
AAFC's Programs Branch should:

Review program processes and timelines to improve the economy and efficiency of program delivery to design, deliver and monitor scientific-based programs such as the AGGP to ensure that the value for money in expected outcomes is identifiable and achievable.

Agreed. When program delivery was consolidated in PB in the spring of 2012, AGGP was transferred to PB and benefitted from its standardized processes.  As well, PB reviews its processes and timelines on an ongoing basis with the goal of continuous improvement. In particular, PB is currently reviewing its claims process and its application review process using the LEAN methodology. A number of actions were identified that will be implemented in the next few months. Lessons learned are being applied to existing GHG programming along with other research and development programs, and would be applied to future programs.

LEAN methodology actions to be implemented by February 2015

DG, Innovation Programs Directorate (PB)

For future GHG programming, PB will be working collaboratively across the Department to assess applications and will ensure that the potential value-for-money of proposed projects is assessed.

For the next generation of AGGP, applicants will be asked to include in their proposals, among other things, a plan to facilitate farmer uptake of the Beneficial Management Practices to be developed under the proposed projects. Departmental Technical reviewers will be asked to assess the feasibility of those plans.

April 2016

DG, Innovation Programs Director (PB)

Appendix B: Logic Model

Objective: The objective of the Agricultural Greenhouse Gases Program (AGGP) is to enhance the understanding and accessibility of agricultural technologies, Beneficial Management Practices (BMPs) and processes that can be adopted by farmers to mitigate Greenhouse Gas (GHG) emissions in Canada.

Activities:

  • Enhance partnerships, networking and information sharing (domestic and international) to facilitate and coordinate research and technical delivery of BMPs.
  • Create scientific and applied research to develop new knowledge on GHG mitigation from agriculture.
  • Develop knowledge/information products to develop GHG mitigation technologies.
  • Promote technology transfer approaches that make GHG mitigation management BMPs and technologies available to farmers.

Outputs:

  • Formal plans for research and technology transfer activities for AAFC and its partners.
  • Formal and informal collaborative arrangements for developing and sharing of applied research on GHG mitigation.
  • Peer reviewed papers, journal articles, presentations at professional forums on GHG mitigation.
  • Information products, technical studies, designed to increase the body of knowledge related to GHG mitigation.

Immediate Outcomes:

  • New GHG mitigation information and technologies are developed, verified and validated.
  • Technology transfer methodologies and approaches developed for targeting farmers about GHG mitigation.

Intermediate Outcome:

  • GHG mitigation information and technologies are made available to farmers.

End Outcomes:

  • Increased knowledge by farmers about GHG mitigation BMP's and technologies.
  • Increased adoption of GHG mitigation BMPs and technologies by farmers.

Appendix C: Agriculture and Agri-Food Canada Programs (Past and Current): GHG Research, Monitoring and Mitigation

Program Name Objective Funding Period Eligible Recipients

Greenhouse Gas Mitigation Program for Canadian Agriculture (GHGMP)

To reduce greenhouse gas emissions through improved soil, crop, nutrient and livestock management practices; assess the potential of the selected BMPs to reduce GHG emissions and raise awareness and encourage adoption of BMPs among producers. This was achieved through the development of greenhouse gas-related communications materials, and through the funding of research and demonstration activities designed to raise awareness about BMPs and the economic benefit to producers.

2002-2007 (Action Plan 2000 on Climate Change)

National Industry Groups: Canadian Pork council, Dairy Farmers of Canada, Canadian Cattlemen's Association
Soil Conservation Council of Canada

National Carbon and Greenhouse Accounting and Verification System (NCGAVS)

Provide a scientific basis for monitoring, accounting and reporting on greenhouse gas emissions and removals from agricultural activities.

Support Canada's international reporting obligations under the UNFCCC in accordance with the intergovernmental Panel on Climate Change guidelines for estimating and reporting national GHG inventories.

2000-2005 (Climate Change Action Plan – Environment Canada)
2006-2008 (Agricultural Policy Framework)
2009-2013 (Growing Forward) Environment Canada (EC) was responsible for reporting to the UNFCCC, but allocated the reporting of GHG estimates on agricultural lands to AAFC through an MOU.
2013- (Growing Forward 2) AAFC continues to deliver annual agricultural GHG emissions and removals estimates to Environment Canada for inclusion in Canada's annual National Inventory Reports to the United Nations Framework Convention on Climate Change, through the Sustainable Science and Technology Advancement (SSTA) initiative under GF 2.

AAFC

National Agri-Environmental Health Analysis and Reporting Program (NAHARP)

NAHARP develops science-based models that predict the interaction between agricultural practices and the environment to enable and support the policy development and assessment process. The agri-environmental indicators (AEIs) measure key environmental conditions, risks and changes resulting from agriculture and management practices used by producers. AEIs are science-based indicators used to identify trends with respect to soil, water, air, biodiversity and environmental farm management.

AAFC began developing AEIs in 1993 with the Agri-Environmental Indicator Project.
NAHARP was then introduced in 2003 under the Agriculture Policy Framework (APF). Under Growing Forward, NAHARP was allocated Vote 1 funding to continue developing and reporting on AEIs.

AAFC

Sustainable Science and Technology Advancement (SSTA) Initiative

NCGAVS and NAHARP are integrated into SSTA – SSTA will have annual indicators instead of the previous five year cycle. Databases on sustainability information will be more accessible to clients.

2014 – Growing Forward 2

AAFC

Sustainable Agriculture Environmental Systems (SAGES)

The Sustainable Agriculture Environmental Systems (SAGES) initiative provided science-based research and development projects in water and climate change. The program created new and improved agricultural practices, offered policy options and increased understanding and adaptation to water and climate impacts.

Farmers benefited from the development and availability of new and improved agricultural practices that address environmental challenges such as climate variability and crop, livestock, and water management in an economically sustainable manner. SAGES supported 25 peer-reviewed research and development projects and provided direct benefit to producers through knowledge and development.

2009-2013- Growing Forward

AAFC

The Watershed Evaluation of Beneficial Management Practice Program (WEBS)

The Watershed Evaluation of Beneficial Management Practices (WEBs) program objective was to determine the economic and water quality impacts of selected agricultural beneficial management practices (BMPs) at nine watershed sites (an area of land from which all runoff drains into the same water body) across Canada. WEBs was designed to enhance land-use decision making at the farm and landscape levels. The primary focus of WEBs was examining water quality degradation caused by excessive sediment and nutrient runoff as a critical environmental impact in agricultural watersheds however, other indicators of environmental health – such as soil or riparian health, biodiversity and greenhouse gas emissions – were also examined at several WEBs watershed sites.

2004-2013- Growing Forward

AAFC, Ducks Unlimited Canada and over 70 other government, academic and local watershed conservation groups were also partners in the program

Holos

Downloadable software program for producers for greenhouse gas monitoring. Holos is a whole-farm model and software program that estimates greenhouse gas (GHG) emissions based on information entered for individual farms. The main purpose of Holos is to test possible ways of reducing GHG emissions farms and is available at no cost to users. Users can select scenarios and farm management practices that best describe their operation and then adjust these practices to see the effect on emissions. Holos is continually being updated with new data and improved features. Holos 2.1, released in February 2014, includes updated Canadian data based on new beef and dairy research, more detailed user input such as monthly herd size adjustments and estimates of production.

Ongoing- A base funding

AAFC

Cropland Estimates

AAFC generated estimates from Cropland remaining Cropland (CLCL) using two models: The Canadian Regional Agricultural Model (CRAM) and the Canadian Agricultural Greenhouse Gas Monitoring Accounting and Reporting System (CanAG-MARS). CRAM was used to estimate the resource use patterns in the agriculture sector which were then fed into CanAG-MARS to provide estimates of emissions/removals from cropland remaining cropland.

Ongoing- A base funding

AAFC

AAFC Research Centres

AAFC conducts research into reducing greenhouse gas emissions associated with agriculture at primary research centres such as: Semiarid Prairie Agricultural Research Centre (SPARC) in Swift Current, Saskatchewan; Brandon Research and Development Centre (BBDC) in Brandon, Manitoba; and Dairy and Swine Research and Development Centre (DSRDC) in Sherbrooke, Quebec.

Ongoing – A-base funding

AAFC

Canadian Agri-Science Clusters (Clusters)

To encourage key agricultural organizations to create, plan and implement a national program of applied science and technology research and development activities by mobilizing and coordinating a critical mass of scientific and technical capacity in industry, government and academia. Several projects include GHG mitigation efforts.

2008-2013- Growing Forward
2013-2018- Growing Forward 2

Not-for profit organizations agricultural organizations with Contribution Agreements and Collaborative Research Development Agreements with AAFC.

Canadian Agricultural Adaptation Program (CAAP)

To facilitate the agriculture, agri-food and agri-based products sector's ability to seize opportunities, respond to new and emerging issues and pilot solutions to new and ongoing issues to adapt and remain competitive. Several projects include GHG mitigation efforts.

2009-2014 – A-base funding
2014-2019 – A-base funding

Any Canadian legal entity capable of entering into a contract including but not limited to organizations and associations, cooperatives, marketing boards, aboriginal groups for profit companies and individuals.

Appendix D: List of Projects

Province Client Name Project Type Allocated Dollar Amount
Alberta University of Alberta (Chang & Bork) AF $598,400.00
Alberta University of Alberta (Wilson) LS $828,850.00
Alberta Total $1,427,250.00
British Columbia British Columbia Ministry of Agriculture AF $223,120.00
British Columbia University of British Columbia Okanagan (Jones) WU $1,291,761.00
British Columbia Total $1,514,881.00
Manitoba University of Manitoba (Amiro) LS $2,996,451.00
Manitoba Upper Assinboine River Conservation District. LS $160,000.00
Manitoba Total $3,156,451.00
Nova Scotia Nova Scotia Agricultural College (Burton) CS $850,168.00
Nova Scotia Total $850,168.00
Ontario University of Guelph (Thevathasan) AF $999,900.00
Ontario University of Guelph (Wagner-Riddle) LS $2,827,000.00
Ontario South Nation River Conservation Authority (SNC) WU $639,412.00
Ontario Canadian Fertilizer Institute CS $700,000.00
Ontario Total $5,166,312.00
Quebec McGill University (Madramootoo) CS $1,999,710.00
Quebec Eastern Township Forest Research Trust AF $925,311.00
Quebec 41 Institut de recherche et de développement en agroenvironnement (IRDA) (2) CS $349,990.00
Quebec Nature Quebec CS $99,440.00
Quebec Total $3,374,451.00
Saskatchewan University of Saskatchewan (Van Rees) AF $1,555,610.00
Saskatchewan 3.1 - University of Saskatchewan (Helgason) WU $919,948.00
Saskatchewan 3.6 - University of Saskatchewan CS $950,308.00
Saskatchewan Total $3,425,866.00
Canada Total $18,915,379.00

Appendix E: Bibliography

  • Agriculture and Agri-food Canada, (2003). Climate Change and Greenhouse Gas Awareness Study.
  • Agriculture and Agri-food Canada, (2011). GF2 Consultations. What was Heard Report.
  • Agriculture and Agri-Food Canada, (2012). News Release. New Growing Forward Agreement Will Drive Innovation, Market Development and Long-term Growth in Canadian Agriculture.
  • Agriculture and Agri-food Canada, (2013). An Overview of the Canadian Agriculture and Agri-Food System. Accessed on March 13, 2014.
  • Canadian Federation of Agriculture, (2012). Towards a National Food Strategy: A Framework for Securing the Future of Food.
  • Conference Board of Canada, 2011. Insights You Can Count On. Report: May 8, 2011, Greenhouse Gas Mitigation in Canada, Energy, Environment and Transportation Policy.
  • Canola Council, Crop Production Research, Accessed on April 2, 2014 from: http://www.canolacouncil.org/research/crop-production-research/sustainability-projects/, Environment Canada (2012)1. National Inventory Report 1990-2012: Greenhouse Gases, Sources and Sinks in Canada – Executive Summary, Accessed on October 9, 2014 from: http://www.ec.gc.ca/ges-ghg/default.asp?lang=En&n=3808457C-1&offset=3&toc=show, April 2014.
  • Environment Canada (2012) 2. Federal Sustainable Development Strategy: 2012 Progress Report. Accessed on May 14, 2014 from: http://www.ec.gc.ca/dd-sd/default.asp?lang=En&n=AD1B22FD-1
  • Environment Canada (2013). Synthesis Report: Public Consultations on the Draft Federal Sustainable Development Strategy 2013-2016, Sustainable Development Office. Accessed on May 14, 2014 from: http://www.ec.gc.ca/dd-sd/default.asp?lang=En&n=F7780ED7-1
  • Food and Agriculture Organization of the United Nations. 2003. World Agriculture Towards 2015-2030. An FAO Perspective. Corporate Document Repository. Climate Change in Agriculture: Physical and Human Dimensions. Accessed on October 11, 2014 from: http://www.fao.org/docrep/005/y4252e/y4252e15.htm
  • Food and Agriculture Organization of the United Nations. 2012. (Sandra Corsi, Theodor Friedrich, Amir Kassam, Michele Pisante and João de Moraes Sà). Soil Organic Carbon Accumulation and Greenhouse Gas Emission Reductions from Conservation Agriculture: A Literature Review. Integrated Crop Management. Volume 16. Accessed on October 10, 2014 from: http://www.fao.org/fileadmin/user_upload/agp/icm16.pdf
  • Government of Canada (2013) Canada's Action on Climate Change, Accessed on April 3, 2014 from: http://www.climatechange.gc.ca/default.asp?lang=En&n=FAC4B67D-1
  • Government of Manitoba (2012) Growing Forward 2. A Content Summary of Public Consultations in Manitoba. Accessed on April 3, 2014 from: http://digitalcollection.gov.mb.ca/awweb/pdfopener?smd=1&did=21398&md=1
  • Horticulture Council (2014). Projects and Programs, Accessed on April 3, 2014 from: http://www.hortcouncil.ca/projects-and-programs/agri-science-cluster.aspx
  • Intergovernmental Panel on Climate Change (IPPC) (2006). "2006 IPCC Guidelines for National Greenhouse Gas Inventories, Volume 4, Agriculture, Forestry and Other Land Use." Accessed on April 1, 2014 from: http://www.ipcc-nggip.iges.or.jp/public/2006gl/vol4.html
  • Intergovernmental Panel on Climate Change (IPPC) (2014). Climate Change 2014: Impacts, Adaptation and Vulnerability. Part A. Global and Sectoral Aspects. Chapter 7: Food Security and Food Production Systems. Contribution of Working Group II to the Fifth Assessment Report of IPCC. Accessed on December 1, 2014 from http://ipcc.ch/report/ar5/wg2/
  • Intergovernmental Panel on Climate Change (IPPC) (2014). Agriculture, Forestry and Other Land Use (AFOLU), Chapter 11. In: Climate Change 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the IPCC. Accessed on December 1, 2014 from https://www.ipcc.ch/pdf/assessment-report/ar5/wg3/ipcc_wg3_ar5_chapter11.pdf
  • Organic Federation, (2014). Improving Competitiveness and Profitability of Canadian Agriculture with an Organic Science Cluster, Accessed on April 2, 2014 from: http://www.organicfederation.ca/organic-science-cluster-i.
    Pulse Cluster, (2014). Environment Sustainability, Accessed on April 2, 2014 from: http://www.pulsecanada.com/environment/sustainability/low-carbon-footprint
  • Statistics Canada (2012). Census of Agriculture, Table 004-0010. Selected Land Management Practices and Tillage Practices Used to Prepare Land for Seeding, Canada and Provinces.
  • Statistics Canada (2014). Exports of goods on a balance-of-payments basis, by product. Accessed on March 13, 2014 from http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/gblec04-eng.htm
  • Vineland Research and Innovation Centre, Canadian Ornamental Horticulture Research and Innovation Cluster, National Strategy for Ornamental Horticulture Research and Innovation, Accessed on April 2, 2014 from: http://vinelandresearch.com/Default.asp?id=98&l=1
  • World Health Organization, 2014 Climate Change and Health. Accessed October 9, 2014 from: http://www.who.int/mediacentre/factsheets/fs266/en/
  • Additional documents were also accessed from Agriculture and Agri-Food Canada such as: AAFC Departmental Reports on Plans and Priorities, Departmental Performance Reports and other foundational program documents.