Evaluation of Growing Forward 2 Cost-shared Programming Strategic Initiatives

November 15, 2017

Office of Audit and Evaluation

Table of contents

Abbreviations

AAFC

Agriculture and Agri-Food Canada

AIC

Adaptability and Industry Capacity

BMP

Beneficial Management Practice

BRM

Business Risk Management

CMD

Competitiveness and Market Development

FPT

Federal-Provincial/Territorial

FTE

Full-Time Equivalent

GDP

Gross Domestic Product

MBFI

Manitoba Beef and Forage Initiatives

OAE

Office of Audit and Evaluation

Executive summary

This evaluation examines Growing Forward 2 Cost-shared Non-Business Risk Management (Non-BRM) programming: a partnership between the federal government and individual provincial and territorial governments whereby both the federal and PT governments fund initiatives designed and delivered by the PT governments. The report presents a national picture of the provincial/territorial-administered cost-shared programs under Growing Forward 2 during its first three years, as well as Agriculture and Agri-Food Canada's (AAFC) management and oversight (governance) framework for the programming. The evaluation was conducted in accordance with the Treasury Board Policy on Results.

Background and profile

Growing Forward 2 was designed to allow provinces and territories to tailor cost-shared programming to meet their regional needs, while still contributing to the achievement of national objectives. Federal, provincial and territorial (FPT) governments agreed to achieve the objectives by committing to an investment in the three priority areas: Competitiveness and Market Development (CMD); Innovation; and, Adaptability and Industry Capacity (AIC) as outlined in the Growing Forward 2 Multilateral Framework Agreement.

The three priority areas are broken down into six areas of intervention indicating the types of programming typically included in each priority area. Based on the bilateral agreements, PTs aligned their cost-shared programs to one of the three priority areas.

Evaluation methodology

The evaluation supports both the Government of Canada accountability of federal-level program expenditures, and the FPT governments in developing the next FPT agricultural policy framework.

The evaluation was conducted in accordance with the Growing Forward 2 Multilateral Framework Agreement’s commitments, the Treasury Board Policy on Results, the Financial Administration Act, and AAFC’s 2014-15 to 2018-19 Departmental Evaluation Plan. This evaluation is able to report reliably on relevance issues, and issues of design and delivery.

Findings

Growing Forward 2 Cost-shared Programming remains relevant to support the Canadian agriculture and agri-food sector toward increased market share, improved productivity and environmental sustainability, improved resilience, and overall enhancement of agricultural Gross Domestic Product growth.

Other key results from the evaluation include:

• There is a continued need for programming to support the Canadian agriculture and agri-food sector toward increased market share, improved productivity and environmental sustainability, improved resilience, and overall enhancement of agricultural Gross Domestic Product growth
• Cost-shared programming is aligned with federal government priorities and departmental strategic outcomes
• Progress is being made toward the achievement of the expected outcomes of cost-shared programming
• Cost-shared programming operates economically. A consolidated analysis of efficiency was beyond the scope of the evaluation, however instances of efficiency were found
• Growing Forward 2 Cost-shared Programing gives provinces and territories greater flexibility in programming choices; however, provinces and territories perceive that a number of administrative processes are constraints to the timely awarding of funding capital
• Limitations regarding data collection hamper the ability to assess the impact of the Program
• Bilateral communications are generally effective. However, coordination and communication between provinces and territories could be strengthened

Recommendations

Recommendation 1: Programs Branch and Strategic Policy Branch should include in each bilateral agreement a requirement for standardized data against common indicators that will enable AAFC to assess the effectiveness of the Cost-shared Program. 

Recommendation 2: Programs Branch, in collaboration with other Branches, should leverage existing communication tools and mechanisms to enable sharing and coordinated communication among provinces and territories supporting innovations, operational best practices, and other areas of common interest.

1.0 Introduction

The Evaluation of Growing Forward 2 Cost-shared Programming Strategic Initiatives (Non-Business Risk Management) was undertaken to assess the first three years of the five-year Growing Forward 2 program cycle to:

• provide evidence-based findings related to the program; and
• assess progress toward the expected impacts of Growing Forward 2 Cost-shared Programming Strategic Initiatives

The evaluation addressed the core evaluation issues related to relevance and performance outlined in the Directive on the Evaluation Function. For a detailed evaluation methodology, see Annex A.

2.0 Growing Forward 2 Cost-shared Programming Strategic Initiatives (Non-BRM)

2.1 Federal-Provincial/Territorial Agreements and Governance

Agriculture is a shared jurisdiction and, for more than a decade, Federal-Provincial/Territorial (FPT) governments have built strong collaboration in support of the sector to increase its competitiveness, profitability and sustainability. This was done through a succession of policy frameworks, including the Agricultural Policy Framework (2003-08), Growing Forward (2008-13), and Growing Forward 2 (2013-18). The Growing Forward 2 policy framework was designed to respond to industry priorities, including: Competitiveness and Market Development; Innovation; and, Adaptability and Industry Capacity. The April 2013 Agreement for Growing Forward 2 (hereon known as the Agreement) establishes policy direction, defines the scope of designated programming and provides guidance for the development of cost-shared initiatives that encompasses programming  designed and delivered by FPT governments. Programming is divided in four groups:

• provincial and territorial delivered cost-shared strategic initiatives
• business risk management programs
• federally delivered strategic initiatives; and
• federally delivered strategic initiatives that complement cost-shared programming

Growing Forward 2 Cost-shared Programming Strategic Initiatives referred to as “cost-shared programming”, fund initiatives designed and delivered by provincial and territorial governments on a 60:40 cost-shared basis. While provinces and territories may determine the suite of programs that best suit their regional needs, all provinces and territories must commit to meet the objectives set out in the Agreement.

Under the umbrella Agreement, there are 13 individual bilateral agreements which define provincial and territorial programming, along with reporting and financial commitments specific to AAFC and each province/territory. Every bilateral agreement defines performance indicators and targets for each designated program and subprogram. Provinces/territories have until August 31 of each year to submit to AAFC their annual performance reports for the previous year’s designated programming, detailing, in particular, performance against their output indicator targets. While previous bilateral agreements required provinces/territories to submit to AAFC a full evaluation of their activities under cost-shared programming, the Growing Forward 2 bilateral agreements did not include this requirement.

Each of the thirteen bilateral management committees (one for each province or territory) are made up of FPT government representatives responsible for coordinating Growing Forward 2 activities related to the implementation of the bilateral agreements in the provinces and territories.

2.2 Program activities

The Agreement outlines shared objectives and investments in three priority area:

• Competitiveness and Market Development (CMD)
• Innovation; and
• Adaptability and Industry Capacity (AIC).

The three priority areas are broken down into six areas of intervention, as follows:

	Competitiveness and Market Development	Innovation	Adaptability and Industry Capacity
Market and Trade	X
Assurance Systems	X
Environment		X
Research		X
Business Development			X
Infrastructure			X
X = applicable Source: Growing Forward 2 FPT Performance Measurement Strategy for cost-shared programming Strategic Initiatives (non-BRM)

Provinces and territories design and deliver programs which provide funding to support projects benefiting the agricultural, agri-food and agri-product sector. Target recipients for funding, and expected applicants, include:

• agri-businesses, including producers
• agri-processing industry
• industry associations and organizations
• research and innovation institutions and organizations
• not for profit organizations
• aboriginal organizations; and
• academia

Applicants seek funding from the provincial or territorial program of interest, typically through an online application process, outlining the objectives and potential benefits of their proposed project, along with the project’s planned activities and funding requirements. In most cases, applicants must demonstrate access to additional funding sources. For example, in some provinces and territories, applicants seeking support under a CMD program to attend a trade show must demonstrate that they will match cost-shared program dollars with an equal amount of their own money. Individual project funding amounts range from a few thousand dollars to several million dollars. For approved projects, payments are typically made in installments, with the final installment contingent on the submission of a project report that includes data corresponding to relevant provincial and territorial indicators. These data permit the provinces and territories to generate indicator databases used in support of both annual performance reports to AAFC and their internal provincial and territorial reports.

2.3 Program objectives and Logic models

Three logic models (see Annex B) were developed by the Program to depict cost-shared programming and provide a foundation for the evaluation. Not all provinces and territories have programming that contributes to every outcome. In most cases, outcomes reflect the results to be achieved from programming undertaken by the majority of provinces and territories (for example, funding for Environmental Farm Plans). However, some outcomes reflect programming that is common to only a few provinces (for example, funding for infrastructure).

Competitiveness and market development

CMD programming, under the “Market and Trade” and “Assurance Systems” areas of intervention, aims to increase Canadian producer and processor market share. CMD programming focuses on maintaining and creating market-based opportunities through such activities and outputs as market information, trade shows, marketing strategies, and product attributes, including attributes obtained from assurance systems. The abbreviated logic model below provides a summary of the key outputs, as well as the immediate, intermediate and end outcomes associated with the CMD priority area.

Area of Intervention	Outputs	Immediate Outcomes	Intermediate Outcomes	End Outcomes
Market and Trade	value chain development market plans meetings and events advisory and consulting services research and market analysis trade shows and missions	increased producer/processor knowledge and skills about market development new market-ready products increased awareness on the part of the market of Canadian products	increased access to new markets reduced production costs	increased Canadian producer/ processor domestic and international market share
Assurance Systems	equipment and facilities (including surveillance systems) training assessments and consulting services	increased producer/processor knowledge and skills about assurance systems new assurance plans, systems and practices	increased capacity to manage risks	increased Canadian producer/ processor domestic and international market share

Innovation

Innovation programming, under the “Environment” and “Research” areas of intervention, aims to improve environmental and productivity sustainability by focusing on activities that directly enhance or enable innovation, from discovery through to commercialization and adoption. The abbreviated logic model below provides a summary of the key expected outputs and outcomes associated with the Innovation priority area.

Area of Intervention	Outputs	Immediate Outcomes	Intermediate Outcomes	End Outcomes
Environment	research and environment analysis infrastructure and technologies training assessment and consulting services environment plans implementation of BMPs	increased producer/processor knowledge and skills about environment greater incorporation of environmental practices in planning	minimized environmental risks more efficient use of inputs	greater environmental sustainability increased productivity
Research	research prototype development, demonstration, and commercialization training innovation jobs	new products, practices and technologies increased producer/processor knowledge and skills about products, practices and technologies	adoption of new products, practices and technology	greater environmental sustainability increased productivity

Adaptability and industry capacity

AIC programming, under the “Business Development” and “Infrastructure” areas of intervention, aims to improve resilience. AIC programming focuses on activities that build agri-business management capacity, attract new entrants to the sector, and develop agriculture and agri-processing infrastructures. The abbreviated logic model below provides a summary of the key expected outputs and outcomes associated with the AIC priority area.

Area of Intervention	Outputs	Immediate Outcomes	Intermediate Outcomes	End Outcomes
Business Development	research and business analysis safety, risk management equipment training assessment and consulting services leadership and peer-to-peer networking promotional products	increased producer/processor knowledge and skills about management increased awareness on the part target clientele of the Canadian agri-food industry	industry adoption of new business and risk management practices attraction of new entrants and investments to the Canadian agri-food industry	increased industry resilience
Infrastructure	equipment water management and land development projects assessment and consulting services	maintained or increased acreage/water	increased production capacity	increased industry resilience

Linkages between priority areas

The ultimate objective across the three priority areas is the enhancement of agricultural Gross Domestic Product (GDP) growth. The three priority areas are linked and activities under one priority area can have impacts on outcomes in other priority areas. The outcomes depicted in the three logic models contribute together to create growth in the sector.

2.4 Program resources

The total FPT investment in cost-shared programming is just under $2 billion over five years. As of the end of 2015-16, the first three years under GF2, federal contributions totalled $623 million and provincial contributions totalled $505 million.  This represents annual contributions of $207 million federally and $168 million provincially^{Footnote 1}. Forty-six percent (46%) of the overall budget is for Innovation, 39% for CMD, and 15% for AIC.

The Agreement specifies that spending for programming associated with Innovation and CMD is to be at a minimum 25% of each of the program’s total spending^{Footnote 2}. Through their bilateral agreements, FPT dollars and/or in-kind resources are counted toward the respective 60% and 40% shares of total expenditures over the five-year period.

To manage the program, AAFC employs six full-time equivalent employees in headquarters and five full-time equivalents in regional offices.

3.0 Evaluation methodology

The evaluation supports the Government of Canada accountability of federal-level program expenditures, and the FPT governments in developing the next FPT agricultural policy framework, that is the Canadian Agricultural Partnership.

The evaluation was conducted in accordance with the Agreement’s commitments, the Treasury Board of Canada Policy on Results, the Financial Administration Act, and AAFC’s 2014-15 to 2018-19 Departmental Evaluation Plan. This evaluation is able to report on relevance issues, and issues of design and delivery. The case studies provide a source of illustrative examples of the kinds of outputs and outcomes that can result from program activities. For detailed evaluation methodology, see Annex A.

3.1 Evaluation scope

The evaluation is national in scope and examined cost-shared programming over the first three years of the five-year program cycle, from 2013-14 through 2015-16. The focus was on federal government activities, with no intent to evaluate provincial or territorial program delivery. The evaluation addressed the core evaluation issues related to relevance and performance.

4.0  Program relevance

4.1 Continuing need

The 2013 Evaluation of the Cost-shared Non-Business Risk Management (Non-BRM) Contribution Programming under Growing Forward concluded that: “cost-shared Non-BRM programs were developed to support the long-term competiveness of the agriculture and agri-food sector. Cost-shared programming was designed to address sector needs in broad priority areas.” While both the domestic and international markets continue to evolve, the need to support Canada’s agriculture and agri-food industry through broad-based programming essentially remains the same as it was during the time of Growing Forward.

Growing Forward 2 Cost-shared Programming was designed following consultations across the country, with extensive involvement on the part of FPT agriculture ministers. Growing Forward 2 Cost-shared Programming emphasizes slightly different priorities than those of the Growing Forward era; for example, the Growing Forward focus of food safety, bio-security and traceability is given less emphasis under Growing Forward 2 Cost-shared Programming reflecting the maturity of programs in these areas as well as changing priorities identified through the Growing Forward 2 consultation process.

Documents reviewed for the evaluation consistently supported the findings of the consultations, that is they demonstrated a continuing need for programming focused on increasing market share, improving productivity and environmental sustainability, improving resilience, and the overall enhancement of agricultural GDP growth. The Growing Forward 2 framework outlines a vision of Canada’s agricultural and agri-food sector as a world leader in food safety, innovation, and environmentally responsible production, as well as an engine of Canadian economic growth. In 2015, agriculture and agri-food exports were valued at nearly $60 billion.

Interviewees universally supported the perspective that cost-shared programming is needed and is focused on the right areas, and appreciated the increased flexibility for provinces and territories under Growing Forward 2 compared with previous frameworks. Interviewees valued the continued program evolution and the corresponding need to continue to consult widely in the development of the next framework. They noted that emerging areas of focus such as climate change, animal care, consumer awareness, and organic products should play a more prominent role in future.

4.2 Alignment with government and Agriculture and Agri-Food Canada priorities

The 2013 evaluation concluded that Growing Forward’s programming framework aligned with AAFC’s Program Activity Architecture, and cost-shared programming was aligned with AAFC’s strategic outcomes and federal priorities. This continues to be the case under Growing Forward 2, which focuses on increased market share, improved productivity and environmental sustainability, improved resilience, and the overall enhancement of agricultural GDP growth. This aligns with the two strategic outcomes described in the 2015-16 AAFC Report on Plans and Priorities:

Strategic Outcome 1: A competitive and market-oriented agriculture, agri-food and agri-based products sector that proactively manages risk, incorporating Program 1.1 Business Risk Management, and Program 1.2 Market Access, Negotiations, Sector Competitiveness, and Assurance Systems; and,

Strategic Outcome 2: An innovative and sustainable agriculture, agri-food and agri-based products sector, incorporating Program 2.1 Science, Innovation, Adoption and Sustainability, and Program 2.2 Industry Capacity.

These strategic outcomes align with the whole-of-government outcome, “Strong Economic Growth,” as well as with elements in the Minister’s mandate letter. These observations are reinforced in the 2016 federal budget: “The Canadian agriculture and agri-food sector is a vital part of our economy that supports both rural and urban communities across the country. At its foundation are the farmers and ranchers that work hard to feed Canada and the world.”

4.3 Alignment with federal responsibilities

Agriculture is a constitutionally mandated shared responsibility between the federal government and provincial and territorial governments. As such, the cost-shared programming model is aligned with federal responsibilities as outlined in the 2015-16 Departmental Performance Report (DPR) and Agriculture and Agri-Food Act.

5.0  Program performance

5.1 Outputs

In the first three years of a five-year program, $1.13 billion was spent by the FPT governments on cost-shared programming (as of March 31, 2016). This represents 61% of the combined five-year cost-shared programming budget. Federal funding of $623 million and $505 million of provincial and territorial level funding represents 56% and 69% of total program budget. Thus, while federal government spending is a little behind, and provincial and territorial spending is a little ahead as of the three-year mark of the program cycle, these variances are within the annual re-profiling allowance of plus or minus 25%.

The table below shows consolidated national output totals for 2013-14 and 2014-15^{Footnote 3}. For the purpose of this analysis, the 43 indicator clusters were grouped into four categories, as shown below. In the Program’s first two years 36,490 events; 7,265 infrastructure, pilot test, and research projects; and, 19,676 reports and assessments were supported across all provinces and territories by cost-shared programming funds. In all, 475,168 organizations/recipients participated or benefited in some way.

Category	Events / Activities	Infrastructure, Pilot Tests, Research	Beneficiaries	Reports / Assessments
Competitiveness and Market Development	27,181 (82%)	1,548 (84%)	124,199 (125%)	2,501 (182%)
Innovation	4,104 (42%)	4,112 (64%)	67,142 (28%)	16,615 (52%)
Adaptability and Industry Capacity	5,205 (416%)	1,605 (56%)	283,827 (685%)	560 (185%)
Total	36,490 (83%)	7,265 (65%)	475,168 (289%)	19,676 (59%)

Targets were pro-rated for the purpose of this analysis to adjust for the evaluation of activities and outputs mid-way through the program cycle. Hence, as Program areas could experience increases in the activities and outputs, the analysis of this data suggests the likelihood the Program may meet its targets should progress continue as seen to date. As indicator targets were found to vary widely and, in some cases, bore little relation to actual performance, the output data does not paint a clear picture of achievements. Hence, the survey, key informant interviews, and case studies all provide related evidence. The following examples illustrate selected notable progress of expected outputs:

• Competitiveness and Market Development / Assurance Systems: Interviewees described training conducted under the Assurance Systems area of intervention. The Yukon Grain Farm project,^{Footnote 4} in Yukon, led to the construction of a new 40 feet by 80 feet cold storage facility for root vegetables (equipment and facilities). The project built capacity and increased productivity within the Yukon Grain Farm agriculture and agri-food processing operation by 50% to 74%. The project managers believe that completing this project has contributed and will continue to contribute to an increase in productivity and capacity within the Yukon agriculture and agri-food processing industry. (Case 18)
• Innovation / Environment: Eighty-seven percent (87%) of survey respondents involved with environment programming either planned or completed the adoption of "a best [environment] management practice for my operation.” Interviewees described training conducted under the Environment area of intervention as invaluable. Two projects, the Prince Edward Island Potato Farm Soil Conservation Project and the Québec Ferme Porcine Lina Inc. Ouvrages de conservation des sols Project, involved the construction of soil erosion control structures. Receiving training on better environmental practices and learning about the work that is conducted through Growing Forward 2 has been beneficial to their business. Interviewees believed that the knowledge is spreading, and other farmers are now being influenced to opt for better environmental practices. (Case 13/Case 14)
• Innovation / Research: Interviewees described training conducted under the Innovation area of research. Out to Pasture Farm produced a new nutrient management plan, water well, and manure storage bin (prototype development). The Vineland Research and Innovation Centre project produced horticultural research, development, and commercialization aimed at optimizing Appassimento red wine producer productivity (research). The Manitoba Beef and Forage Initiative engaged in feasibility studies, technology testing, prototype development, intellectual property right procurement, commercialization assistance, support for agri-entrepreneurs, training, and research. (Case 8/Case 11/Case 4)
• AIC / Business Development: Interviewees described training conducted under the Business Development area of intervention. By way of example, more than 250 participants per year are trained in Quebec on international agriculture business opportunities. Producers and processors are physically accessing international environments through trade shows. At the industry level there is agreement that Growing Forward 2 helps companies diversify their market, reduces the chain between the producer and the consumer, generates higher income, and reduces risk. There is agreement among provincial and territorial representatives that companies are investing in new products and trying to get those into new markets.
• AIC / Infrastructure: The Inuksuk High School project, in Nunavut, produced a small indoor greenhouse aimed at supporting youth education in food security (equipment). The Small-Scale Foods Program, in the Northwest Territories, produced facilities, tools and training (equipment). While these projects are small in scale, they both successfully demonstrate a way in which the northern region may increase its acreage available to agriculture, increase sector production capacity, and increase the sectors resilience and self-sustainability. (Case 9/Case 10)

According to interviewees, the agriculture and agri-food-related project activity supported by Growing Forward 2 Cost-shared Programming is substantial relative to other activity in the provinces and territories. Large projects and important research institutes (such as the Vineland Centre) would not exist, or would be diminished, in the absence of this funding.

5.2 Outcomes

The survey, key informant interviews, and case studies provided evidence of progress toward the achievement of expected immediate and intermediate outcomes. The following examples illustrate this progress, particularly in the CMD Market Development and AIC Business Development areas of intervention.

• CMD / Market Development: 48% of survey respondents who benefited from marketing and trade programs reported that these programs “made buyers aware of [their] products” (an immediate expected outcome) while 40% of affected respondents reported increasing their number of clients (an intermediate expected outcome). Industry representatives noted Growing Forward 2 Cost-shared Programming has helped companies diversify their markets, reduce the complexity of the supply chain between the producer and the consumer, and generate higher incomes. Interviewees noted two additional Market Development outcomes:
  • The generation of market reports immediately opened new markets for producers and processors; and
  • Producers and processors are able to attend trade shows in person, and in turn are able to access international environments to showcase their products abroad, meet buyers, and receive positive economic spinoffs in terms of sales

    Several case studies demonstrated the achievement of Market Development intermediate outcomes:

  • Les comptonales, Marché public de Compton, built 20 sales kiosks at the Compton Thursday evening summer market. An intermediate outcome came in the form of increased sales and profits for local producers (Case 15)
  • Three processors–Misty Mountain Industries Ltd., Ethical Bean Coffee Co., and NutraSun Foods–attended trade shows. Misty Mountain attended Asia Fruit Logistica, and Ethical Bean attended numerous international shows including Summer Fancy Foods, National Expo West (in Seattle) and National Products Hawaii. Both companies reported increased sales in new markets and, as a result, have expanded their businesses. NutraSun took its organic flour to Engredea in Anaheim, California, and obtained five new customers ordering 1,000 tonnes of product (Case 3/Case 17); and
  • The Filsinger’s Organic Ltd. Automation of Organic Apple Cider and Vinegar Bottling Line project resulted in the purchase and installation of a new automated continuous flow processing system and automated packaging line. The new line is a continuous, fully automated system with filling, capping, safety seal installment, and labels applied while the bottles move down the line. The project had the immediate effect of eliminating the need for four employees which, in turn, improved profitability and enabled personnel to be leveraged to new business tasks such as marketing and planning. (Case 12)
• CMD / Assurance Systems: Interviewees reported wide-scale measures such as food safety and traceability programs and protocols, as well as farm projects such as the installation of heated floors in a hog farm, that led to assurance-related benefits in terms of food safety and bio-security. Two animal handling systems resulted in improvements to efficiency and animal health–the Cow/Calf Indoor Handling System, and P. Quintaine & Son Ltd., new hog sorting equipment. (Case 2/Case 5)
• Innovation / Environment: Environment programs reported that 84% of affected survey respondents “reduce environmental risks or impacts.” 61% of affected respondents stated that environmental programs “improve [my] operation’s productivity.”
• Innovation / Research: The Pure Holsteins Robotic Milking System project supported renovations to a producer’s barn and the purchase of two robotic milkers, the first robotic milkers in the province. The milking machines are automated operating on a 24-hour basis, seven days a week. During milking, an apparatus records information about the cow, its weight and the temperature of milk, and tests the ketone and salt levels of the milk to help maintain cow health. The automated system has created an operation that is animal-friendly, produces high-quality milk, and controls costs. (Case 7)
• AIC / Business Development: As a result of participating in business development programming, 64% of affected survey respondents “tracked production costs and returns more closely.” 54% of affected respondents created a business plan, and 24% created a risk management plan. Industry representatives noted that Business Development programming has helped producers reduce risks, and resulted in additional outcomes including farm succession plans. The “Inspiring Commercial Agriculture Conference” held in the Northwest Territories shared new practices and ideas with a view to attracting new entrants to the sector. Interviewees stated that funded projects have promoted an increase in public awareness about foods and food processing, an important benefit (given increased interest in food sources and origins). The SaskCanola Agriculture Advocacy Film Project, produced a 30 minute documentary film titled License to Farm that “explores the truth behind common misconceptions of agricultural production in Canada, while empowering farmers to stand up and advocate for their social license to farm.” The film has been viewed over 100,000 times on YouTube by viewers from over 160 countries, has had nearly 60,000 visits to its dedicated website, has been screened numerous times for various audiences including politicians and representatives of agriculture and food related professional associations, and has had over two million social media impressions (Instagram, Facebook, etc.). (Case 16)
• AIC / Infrastructure: Interviewees noted Infrastructure projects such as green tanks and dryers have led to increased production capacity. The Irrigation Efficiency Project, at Vucurevich Farms Ltd., supported the purchase of an upgraded low pressure center pivot irrigation system. The new system generated production cost reductions. (Case 1)

Interviewees also noted “crossover” benefits–that is, benefits related to one priority area stemming from program activities in another priority area–such as competitiveness benefits stemming from Innovation research, including Beneficial Management Practices in areas like bio-security and traceability. Other examples include:

• In the North, projects promoting wildlife and plant harvesting help Indigenous people return to work on the land; and
• Public and customer trust increased as a result of investments in the development of animal care, bio-security, and food safety practices

Interviewees observed that results on certain kinds of projects–in particular innovation and research projects–cannot be expected to occur immediately. Rather, these benefits may be realized years following the initial investment.

5.3 Economy and efficiency

Other than a relatively small complement of AAFC staff at headquarters and in regional offices, cost-shared programming is managed and operated by provincial and territorial officials. An 8% cap on provincial and territorial administrative expenditures ensures that a maximal amount of program money finds its way to beneficiaries.

Information was not provided for an efficiency assessment. However, examples provided by provinces and territories demonstrate instances where a high level of efficiency has reportedly been achieved. One province, for example, after investigating the return-on-investment associated with cost-shared programming expenditures, estimated a return of nine dollars for every one dollar spent. Another province has estimated a return to date on Growing Forward 2 investments as six to one, with a projection of 11 to one in the future.

6.0  Program design and delivery

6.1 Program administration

Government officials interviewed at both the federal and provincial/territorial levels stated that the financial administration of cost-shared programming by AAFC is sound and effective. Provincial and territorial officials across the country noted the effectiveness of AAFC support provided at the local level by AAFC regional staff.

6.2 Program design

Provincial and territorial interviewees appreciated the flexibilities built into the Growing Forward 2 Agreement, permitting each province and territory to design its own suite of Growing Forward 2 Cost-shared Programs, geared to the specific provincial and territorial needs and circumstances. Interviewees found that flexibility had increased over the last three policy frameworks. An examination of the 13 bilateral agreements confirms this view; the programs and subprograms of each province and territory, while complying with the general parameters of the Agreement, are unique. For example, under the AIC priority area, Alberta operates “Business Management, Opportunities and Skills Development,” and “Water Management,” while Ontario operates “Business and Leadership Development.”

Interviewees indicated that because most provincial and territorial programs require an investment on the part of the project proponent, proponents assume greater ownership and take more responsibility for their projects. Interviewees noted that the significant investment in the agriculture and agri-food industry by two levels of government represented in cost-shared programming sends a positive message to industry players and stakeholders that their industry and their actions are important to Canadians making their work feel of greater value.

Representatives of the provinces and territories expressed some concern with operational restrictions. In particular, the Agreement and bilateral agreement terms preclude projects that span more than one priority area and projects that span two or more provinces or territories. There are restrictions related to funding cycles, eligibilities for contributions, and proportions of project budgets required of project proponents. Provincial and territorial interviewees noted administrative delays associated with the requirement for Federal Minister approval of capital spending over $100,000; farming capital expenditures are rarely less than $100,000 and perhaps a higher threshold could be considered. Taken together, interviewees stated that they believe these restrictions result in a delay for provinces and territories to approve projects that they view as having merit. These kinds of “administrative irritants”, as interviewees described it, were not uncommon among other benchmarked programs.

6.3 Performance measurement

The Program took steps to address the inconsistent data against its indicators that was noted in the 2013 Evaluation of Growing Forward Cost-shared Programming by engaging with a FPT working group to develop the FPT Performance Measurement Strategy. However, while both output and outcome indicators were developed in the strategy, the bilateral agreements either included variations in wording or omission of some indicators causing a gap in the Growing Forward 2 Cost-shared Programming performance story due to a lack of comprehensive, reliable, empirical data. As stated in the Program’s commissioned report, which consolidated and analysed provincial-territorial performance data for two fiscal years: “It was anticipated that the approach to a national report would be a straight-forward exercise of simply tallying the indicators and targets under each of the 25 standardized indicators. Instead, there were multiple versions of the standardized indicators and different approaches for reporting targets (some reported annual targets; others reported cumulative targets).”^{Footnote 5} The findings from this commissioned report, the recommendations in the 2013 Evaluation of Growing Forward Cost-shared Programming and this evaluation’s analysis of the Program, suggest there continues to be a gap in obtaining relevant data to assess the impacts of the Program.

The decrease in the number of indicators from 600, in a previous performance measurement document, to 25 in the current performance measurement strategy, showed initiative of the Program to reduce the reporting burden to the provinces and territories. However, the Program did not collect data against all 25 indicators, particularly the outcome indicators. Further, each province and territory is required to report on its own unique set of indicators (which may or may not correspond to the indicators of other provinces and territories or to the 25 indicators in the performance measurement strategy), making national level consolidation problematic. As a consequence, findings related to performance in this evaluation are limited to examples and illustrations.

Provinces and territories, for the most part, have effective internal performance measurement systems. Provincial and territorial representatives indicated during interviews for the case studies their willingness to share their data with AAFC, beyond existing AAFC requirements to supply data on the output indicators in their respective bilateral agreements. Further, as the performance measurement strategy was in place since 2014-15 and as the Performance Measurement Working Group meets regularly to discuss performance data, discussions on the steps needed to collect data against the indicators in the performance measurement strategy could have resulted in the existence of robust data for the purpose of this evaluation.

6.4 Federal, provincial and territorial coordination

Interviewees noted adequate communications between AAFC and individual provinces and territories. Yet, despite the cost-shared program having various mechanisms in place that allow for provincial and territorial coordination, (such as, various FPT working groups, the Issues Management Report which is distributed to provinces and territories with issues raised by other provinces and territories, and, the Annual Planning Meetings that provide a forum to share information), interviewees noted less than ideal coordination among provinces and territories. It was generally felt that AAFC was best placed to play a central coordination role. The absence of more effective coordination means that practices developed in one jurisdiction tend not to be shared with others, resulting in inefficiencies that could be avoided.

7.0  Conclusions and recommendations

7.1 Cost-shared programming remains relevant

In crafting the Growing Forward 2 Agreement, FPT governments continued the practices developed in the eras of the Agricultural Policy Framework and Growing Forward with comprehensive and effective consulting of stakeholders and constituents, while aligning Growing Forward 2 programming with government priorities and AAFC objectives. As such, Growing Forward 2 Cost-shared Programming remains relevant to support the Canadian agriculture and agri-food sector toward increased market share, improved productivity and environmental sustainability, improved resilience, and overall enhancement of agricultural GDP growth.

7.2 Examples illustrate progress towards achievement of outputs and outcomes

As of the end of 2015-16, $1.13 billion had been spent by the FPT governments on Growing Forward 2 Cost-shared Programming. This represents 61% of the combined five-year budget. In the Program’s first two years, 36,490 events; 7,265 infrastructure, pilot test, and research projects; and 19,676 reports and assessments were supported by cost-shared programming. In all, 475,168 orgazinations/people participated or benefited in some way.

Specific outputs included technical installations such as soil conservation systems, greenhouses, tools, and storage facilities, as well as mentoring programs, farm succession plans, training workshops, public engagement events, research, feasibility studies, technology testing, innovations, and commercialization assistance. Outcome data derived from case studies stemming from these outputs were particularly prevalent in the CMD Market Development and AIC Business Development areas of intervention and include:

• improved animal health and food safety
• higher product quality, production capacity and productivity
• greater revenues, profits and public awareness
• improved competitiveness and more diversified markets; and
• reduced environmental and business risks

7.3  Limitations regarding data hampered the ability to assess the Program performance.

Limitations in data provided for this evaluation preclude a comprehensive, empirically-based assessment of Program impacts. There was a lack of consistency in the language used when entering indicators into the bilateral agreements, which made it difficult to roll the data up at a national level and report against the Growing Forward 2 Performance Measurement Strategy. The collected data, including data collected in the Growing Forward 2 Client Impact Survey, does not speak directly to the impacts of the Program, resulting in a reliance on case studies and interviews to illustrate performance. There is an interest and willingness on the part of both AAFC and provincial and territorial Ministries to improve performance measurement, and, for the most part, provinces and territories are already actively collecting reliable performance data at both the output and outcome levels. Work needs to be done to standardize definitions to enable a roll-up of data to assess the national impact. Improvements in the inclusion of both output and outcome indicators in the bilateral agreements, as defined in the cost-shared performance measurement strategies, could place AAFC at the forefront of sound and innovative performance measurement practices associated with FPT cost-shared initiatives.

In the next bilateral agreements, AAFC may want to consider strengthening the requirement for provinces and territories to report on outputs and outcomes linked directly to Program objectives, and that would together comprise an agreed-upon national “master list” with corresponding indicators. A complementary national performance measurement system could take advantage of these commonalities with features as standard indicator codes, the assignment of unique project numbers, and the ability to assign projects to provinces and priority areas to facilitate performance measurement and evaluation.

7.3.1 Recommendation 1: Programs Branch and Strategic Policy Branch should include in each bilateral agreement a requirement for standardized data against common indicators that will enable AAFC to assess the effectiveness of the Cost-shared Program. 

7.4 Program Financial Administration is Effective, but there is Room for Improvement in Program Design and Delivery

The financial administration of cost-shared programming by AAFC was found to be sound and effective, as was AAFC support to provincial and territorial officials by AAFC regional staffs. Provincial and territorial representatives were appreciative of the flexibilities built into the Growing Forward 2 Multilateral Framework Agreement permitting provinces and territories to design their own suites of Growing Forward 2 Cost-shared Programming geared to the needs and circumstances of the provinces and territories.

Provincial and territorial representatives are concerned, however, with administrative delays which can result in their inability to fund projects that they view as having merit. Interviewees noted a lack of coordination and communication between provinces and territories which hampers the sharing of best practices.

7.4.1 Recommendation 2: Programs Branch, in collaboration with other Branches, should leverage existing communication tools and mechanisms to enable sharing and coordinated communication among provinces and territories supporting innovations, operational best practices, and other areas of common interest.

8.0  Management Response and Action Plan

Recommendation	Management Response and Action Plan	Target date	Responsible position(s)
Recommendation 1: Programs Branch and Strategic Policy Branch should include in each bilateral agreement a requirement for standardized data against common indicators that will enable AAFC to assess the effectiveness of the Cost-shared Program.	Agreed. The Multilateral Framework Agreement for the Canadian Agricultural Partnership includes a provision that states that all parties agree to develop consistent and comparable performance indicators and that the process for reporting will be set in each provincial/territorial Bilateral Agreement. Programs Branch is working with Strategic Policy Branch and with its provincial/territorial counterparts to develop and implement a Performance Measurement Strategy for the Cost-Shared Strategic Initiatives under the Canadian Agricultural Partnership. The Strategy will include a logic model that links indicators to outcomes and a comprehensive data dictionary for outputs, outcomes and indicators. The outcomes and indicators for which the PTs are responsible to reporting on annually will be included in the Bilateral Agreements. In addition, a provision will be added to PT Bilateral Agreements indicating that data to support progress against the outcomes in the Performance Measurement Strategy will be collected annually through performance measurement reports. These reports will be reviewed annually by AAFC to ensure the quality and integrity of the data.	April 2018 April 2018	Director General,  Service and Program Excellence Directorate
Recommendation 2: Programs Branch should leverage its existing communication tools and mechanisms to enable sharing and to coordinate communication among provinces and territories supporting innovations, operational best practices, and other areas of common interest.	Agreed. The Multilateral Framework Agreement principles underscore that collaboration amongst stakeholders is a key factor to success and approaches for information sharing are developed in a transparent way. Programs Branch will encourage federal co-chairs of federal, provincial, territorial working groups (for example, federal, provincial, territorial Innovation Working Group) to incorporate, in their mandate, a role for strengthening the sharing and coordination of information on areas of common interest amongst provincial/territorial governments.	June 2018	Director General,  Service and Program Excellence Directorate

Annex A: Detailed Evaluation  Methodology

The evaluation addressed the core evaluation issues related to relevance and performance outlined in the Directive on the Evaluation Function (2009), assessing the following:

Relevance

• Assessment of the extent to which cost-shared programming continues to address demonstrable needs of Canadians
• Assessment of the extent to which cost-shared programming aligns with: (i) federal government priorities, and (ii) departmental strategic outcomes; and
• Assessment of the extent to which cost-shared programming aligns with related federal roles and responsibilities

Performance

• Assessment of progress toward expected outcomes; and
• Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes

The evaluation examined design and delivery related to AAFC’s management and oversight of cost-shared programming.

Evaluation Methodology

The evaluation is based on six sources of evidence. Where possible, at least two sources were used to generate findings for each evaluation issue. Sources of evidence were as follows:

• Document Review.Thirty-three documents were reviewed including: foundational documents such as the Calgary Statement, the Memorandum to Cabinet, and the Agreement; program-related documentation including the 13 bilateral agreements; and, contextual documentation such as the 2016 federal budget. The document review primarily supported Relevance issues.

• Key Informant Interviews. Eighty-two interviews were conducted, 47 in the context of case studies (see below), and 35 with key informants.
  • AAFC staff directly involved in Growing Forward 2 Cost-shared Programming (n=3)
  • AAFC senior management including regional directors (n=8)
  • financial and performance measurement provincial and territorial  representatives (n=15); and
  • industry association representatives (n=9)

The interview component provided comprehensive, in-depth, qualitative data supporting the evaluation issues.

• Output Data. The Program commissioned an analysis of provincial and territorial performance data for the first two years of the Growing Forward 2 program cycle. The report provided analysis related to performance issues, especially concerning expected outputs. Annex C contains a summary of outputs consolidated at the national level.

• Producer and Processor Survey. In 2016-17, the Program commissioned a survey of producers and processors affected by cost-shared programming. 3,148 respondents represent 26% of the population. This data addressed performance issues, especially those concerning expected outcomes.

• Case Studies. Eighteen case studies were conducted, each involving a local site visit and interviews with project managers and associated Provincial government program officials. At least one case study was conducted in each province and territory, with two conducted in each of the five provinces with the largest cost-shared programming budgets^{Footnote 6}. Each case study was representative of priority area; budget proportions; province and territory; and/or, evaluation issue.  Annex D contains descriptions of the 18 projects studied.

• Benchmarking. Thirteen FPT cost-shared programs were benchmarked, primarily to examine design and delivery alternatives. Benchmarked programs were similar in scope to AAFC Growing Forward 2 Cost-shared Programming and were found in Infrastructure Canada, Canadian Heritage, Justice Canada, Employment and Social Development Canada, and Public Safety Canada. Annex E contains a summary of the findings from the benchmarking exercise.

Methodological Limitations

Limitation	Mitigation Strategy	Impact on Evaluation
AAFC collects limited outcome data from provinces.	Additional lines of evidence were used. This includes the stakeholder survey (phase 1) and a case study approach of 18 case studies.	Survey questions did not always correspond to evaluation issues; thus, empirical evidence related to outcomes is limited. Case studies provide only illustrative examples of the outputs and outcomes that resulted from cost-shared program activities.
Timing of evaluation limited ability to assess outcomes.	Other relevant sources of data have been integrated into the report.	Observations related to performance issues are limited to indications that an expected outcome may ultimately be achieved.
Empirical evidence related to outputs is limited	Commonality among indicators facilitated the grouping of indicators into clusters, but there is the possibility that seemingly like indicators are not actually the same.	Indicator targets vary widely between bilateral agreements, and in some cases, bore little relation to actual performance of the program.

Annex B: Growing Forward 2 Cost-Shared Programming Logic Models

Source: FPT Performance Measurement Strategy For Cost-shared Programming

Competitiveness and Market Development

• Market Development Programming
  • Generic Activities
    • Funding value chain development
    • Supporting industry growth through strategic sectorial planning
    • Networking and partnership building
    • Developing, delivering and funding training related to export preparedness/ readiness activities
    • Developing, delivering and funding business development training activities
    • Offering advisory and consulting services on export preparedness/ readiness activities for producers and agri-processors
    • Conducting market intelligence
    • Gathering, analysing and disseminating existing market intelligence reports and data
    • Conducting/ facilitating/ supporting market development, business growth, products and industry
    • Branding activities/ products internationally and in the domestic market
  • Standardized Outputs
    • Value chain development
    • Project plans, sectorial analysis, strategic plans, SWOT, gap analysis and benchmarking reports
    • Meetings, consultations, collaboration agreements, networking events, networks
    • Training events for producers and agri-processors; people trained
    • Training sessions; people trained
    • Advisory and consulting services on export development activities for producers and agri-processors
    • Reports, presentation of information and gaps identified, new market research and seminars
    • Trade shows, seminars, table tops, missions, international initiatives, business development and investment attraction activities, translation of materials, legal consultations, promotional materials (including labels) product development, market development strategies, branded product and locations, branding tools
  • Immediate Outcomes
    • The industry is aware of the need to collaborate and improve structure and its building capacity
    • The sector/industry is increasing awareness, knowledge and skills of potential market-based opportunities and challenges, and actions that needs to be taken
    • The market is aware of Canadian agri-related products
    • The sector has products ready to meet market demands
  • Intermediate Outcomes
    • The sector/ industry is acting on opportunities (for example, changing market attributes, managing production costs, accessing new markets)
  • End Outcomes
    • The sector is maintaining/ increasing its market share, both at home and abroad
• Assurance Systems Programming
  • Generic Activities
    • Funding the implementation of assurance systems, facility upgrading and good management practices
    • Developing and/or funding pilot projects, exploring and investigating new approaches around assurance systems
    • Developing and/or implementing surveillance and agri-intelligence for pests, disease, pathogens and food-borne illness
    • Developing and/or delivering and/or funding training related to assurance systems
    • Offering advisory/ assessment services
    • Communicating and/or promoting and/or informing activities related to risks and mitigations
    • Networking
  • Standardized Outputs
    • Payments; manuals, equipment, verification, trained staff within facility/farm, equipment purchased, facility upgrading
    • Pilot projects
    • Surveillance systems and agri-intelligence systems
    • Training events; people trained
    • Assessments, consulting sessions
    • Promotional products, analysis of agri-environment portrait with a PT
    • Networking opportunities
  • Immediate Outcomes
    • Agribusinesses are incorporating assurance systems practices and risks into the planning process
    • The industry/sector is developing or enhancing plans, systems and protocols
    • The sector/ industry is increasing awareness, knowledge and skills around the assurance systems
  • Intermediate Outcomes
    • The sector is increasing capacity to mitigate and minimize risks, and to respond to emergencies
  • End Outcomes
    • The sector is maintaining/ increasing its market share, both at home and abroad

Innovation

• Environment Programming
  • Generic Activities
    • Developing, delivering and funding training on environmental risks, information and mitigations
    • Offering advisory and assessment services to industry producers and processor
    • Communicating/ promting/ informing/ activities/ of environmental risks and mitigations aroung BMPs
    • Developing and disseminating environmental information
    • Funding for environment farm plans
    • Funding for agri-processors to complete environment risk assessment
    • Funding to implement BMPs on farm
    • Providing financial assistance to producers to take high risk land out of production
    • Funding updated infrastructure/technologies related to energy, efficiency, water efficiency, soil protection
  • Standardized Outputs
    • Training sessions, seminars, workshop, social media to present research results
    • Consulting sessions, assessments, audits
    • Promotional products, analysis of agri-environment portrait within a PT
    • Research analysis and reports, soil surveys, agri-meteorology systems; dissemination material (websites, publications, etc.)
    • New and/ or updated environmental farm plans
    • Implemented BMPs
    • Unseeded acreage; acreage converted to another type of production
    • New and/ or updated infrastrure/ technologies implemented
  • Immediate Outcomes
    • The sector/ industry is increasing awareness and knowledge of good environmental practices and risks
    • The sector is incorporating environmental practices and risks into the planning process
  • Intermediate Outcomes
    • The sector is taking actions to minimize environmental risks and using inputs efficiently
  • End Outcomes
    • The agriculture and agri-processing sector is environmentally sustainable
    • The sector/ industry is increasing its productivity
• Innovation Programming
  • Generic Activities
    • Providing assistance to the sector for commercialization
    • Funding adoption/ evaluation/ assessment/ demonstration on activities in the agricultural and agri-food industry
    • Transferring and prioritizing knowledge to the sector which could include entrepreneurs, businesses and/ or governments
    • Funding research activities and positions
    • Funding applied research activities to bring new products/ technologies/ processes to commercialization/ adoption
  • Standardized Outputs
    • Feasibility studies, test markets, prototype development, intellectual property rights for products, commercialization centre assistance, support for agri-entrepreneurs
    • Technologies practices, processes assessed; demonstration of new products, technologies, and processes; Technology, practices processes adopted
    • Training sessions, seminars, workshop, social media to present research results
    • Innovation positions; research project and reports (which could include funding for specific equipment)
    • Research projects; research reports; peer reviewed article, HQP development; field and management trials
  • Immediate Outcomes
    • The sector/ industry has access to new agri-related products, practices and technology
    • The sector/ industry is increasing knowledge on agri-related products, practices, and technology
  • Intermediate Outcomes
    • The sector/ industry is adopting and commercializing agri-related technologies, processes, practices, services
  • End Outcomes
    • The agriculture and agri-processing sector is environmentally sustainable
    • The sector/ industry is increasing its productivity

Adaptability and Industry Capacity

• Business Development Programming 
  • Generic Activities
    • Expanding risk management tools
    • Supporting Implmentation of business recommendations
    • Offering advisory and assessment services to industry producers and processors; funding assessments and assessment tools
    • Developing and offering training and education activities and opportunities within the industry/sector of business management
    • Gathering and disseminating business management information
    • Networking and developing strategic alliance
    • Funding leadership programs and clubs
    • Communicating/ promoting/ informing the general public
  • Standardized Outputs
    • New risk management tools/ wildlife prevention/ farm safety
    • Financial support; equipment; machinery; software
    • Consulting sessions; assessments, business plans; feasibility and cost of production report; succession planning
    • Training events, workshops and people trained
    • Research, surveys; promotional materials
    • Business clubs; peer to peer activities; strategic alliance
    • Leadership programs/ projects and clubs; resources related to leadership programs and clubs
    • Promotional products
  • Immediate Outcomes
    • The sector/ industry is increasing awareness, knowledge and skills around business management and leadership
    • Targeted clientele awareness of the agriculture and agri-food sector is increasing
  • Intermediate Outcomes
    • The sector/ industry is increasing its adoption of business management and strategic risk management practices and tools
    • The sector/ industry is attracting new entrants and new investments
  • End Outcomes
    • The sector is increasing its ability to be resilient and self-sustaining
• Infrastructure Programming
  • Generic Activities
    • Funding for the renewal, expansion and adoption of infrastructure
    • Offering advisory services
  • Standardized Outputs
    • Infrastructure technical projects (for example, on farm water management projects); irrigable acres developed; equipment funded (for example, land clearing equipment)
    • Infrastructure developed
    • Technical advisory services/assessments provided
  • Immediate Outcomes
    • Maintained or increased acreage and water is made available to agriculture and agri-food industry
  • Intermediate Outcomes
    • The sector/ industry is increasing production capacity
  • End Outcomes
    • The sector is increasing its ability to be resilient and self-sustaining

Annex C: Consolidated National Outputs^{Footnote 7}

Outputs Achieved for Competitiveness and Market Development Priority Area

Activities	Outputs	Indicators	Target	Actual	% Achieved
Offering training opportunities	Training events for agri-processors	# of training events by type	56	84	150%
Offering training opportunities	Training events for agri-processors	# of participants by training event	338	864	256%
Funding agri-processing expansion	Expanded/upgraded agri-processing facilities	# of participants by training event	94	717	763%

Activities	Outputs	Indicators	Target	Actual	% Achieved
Developing an/or delivering and/or funding training related to assurance systems	Training events; people trained	# training events by type of training event	10024	21155	211%
Developing an/or delivering and/or funding training related to assurance systems	Training events; people trained	# of participants	25289	53206	210%
Offering advisory/assessment services	Assessments, consulting sessions	# of assessments completed by type of assurance system	946	1814	192%
Funding the Implementation of assurance systems (develop, write, implement program and practices), facility upgrading and good management practices	Payments; manuals, equipment, verification, trained staff within facility/farm, equipment purchased, facility upgrading	# of activities completed by type of assurance system (food safety, biosecurity, etc.) and by type of commodity.	21408	5081	24%
Developing and/or implementing surveillance and agri-intelligence for pests, disease, pathogens and food-borne illness.	Surveillance systems and agri-intelligence systems	# of surveillance systems developed	7	13	186%
Developing and/or implementing surveillance and agri-intelligence for pests, disease, pathogens and food-borne illness.	Surveillance systems and agri-intelligence systems	# of surveillance systems implemented by type of surveillance system	1155	985	85%

Activities	Outputs	Indicators	Target	Actual	% Achieved
Developing, delivering and funding training related to Export preparedness/readiness activities	Training events for producers and agri-processors; people trained	# of training events by type	86	52	60%
Developing, delivering and funding training related to Export preparedness/readiness activities	Training events for producers and agri-processors; people trained	# of participants by training event	21030	41977	200%
Offering advisory and consulting services on export preparedness/readiness activities for producers and agri-processors	Advisory and consulting services on export development activities for producers and agri-processors (available?)	# of clients	52823	29016	55%
Conducting market intelligence (market exploration activities; in-market activities);	Reports, presentation of information and gaps identified, new market research and seminars	# of analysis/reports made available to a target audience by type.	194	501	258%
Conducting/Facilitating/Supporting market development, business growth, product and industry and branding activities/products internationally and in the domestic market.	Trade shows, seminars, table tops, missions, international initiatives, business development and investment attraction activities, but translation of materials, legal consultations, promotional materials (including labels) product development, market development strategies, branded product and locations, branding tools	# of activities funded by type; commercial benefits realized through GF funded market development activities and projects;	1435	887	62%
Funding value-chain development	Value chain development	# of value chain activities and/or initiatives funded	33	6	18%
Supporting industry growth through strategic sectorial planning	Project plans, sectorial analysis, strategic plans, SWOT, gap analysis and benchmarking reports	# of plans/projects funded by type of Industry	236	186	79%

Outputs Achieved for Innovation Priority Area

Activities	Outputs	Indicators	Target	Actual	% Achieved
Developing, delivering and funding training on environmental risks, information and mitigations	Training events, people trained	# of training event by type	210	222	106%
Developing, delivering and funding training on environmental risks, information and mitigations	Training events, people trained	# of participants by training event	1940	1795	93%
Offering advisory and assessment services to industry producers and processors	Consulting sessions, assessments, audits	# of clients by specific focus (that is energy efficiency, soil management etc.)	30	74	247%
Funding for environmental farm plans	New and/or updated environmental farm plans	# of new environmental farm plans/ # of updated farm plans	13226	8633	65%
Funding to implement BMPs on farm	Implemented BMPs	# of BMP implemented and completed by type	18113	7644	42%
Funding for agri-processors to complete environment risk assessment;	agri-processors environment assessment	# of assessments (by type of processing)	380	388	102%
Financial assistance to producers to take high risk land out of production;	Unseeded acreage; acreage converted to another type of production	# of acres under the program	11730	6937	59%
Developing and disseminating environmental information	Research analysis and reports, soil surveys, agri-meteorology systems; dissemination materials (websites, publications etc.),	# of analysis/reports	1	0	0%
Funding updated infrastructure/technologies related to energy efficiency, water efficiency, soil protection	New and/or updated infrastructure/technologies implemented	# of infrastructure/implemented technology projects (could be broken by types)	900	1046	116%

Activities	Outputs	Indicators	Target	Actual	% Achieved
Transferring and prioritizing knowledge to the sector which could include entrepreneurs, businesses and/or governments; Communicating and/or promoting and/or informing and/or disseminating target population of innovation information; Identifying priorities around innovation	Training sessions, seminars, workshop, social media (you tube, videos, Facebook, Twitter, blogs etc.) to present research results	# of training events by type of training event	9501	3882	41%
Funding research activities and positions	Innovation Positions; research project (which could include funding for specific equipment);	# of innovation positions;	2	123	6150%
Funding research activities and positions	Innovation Positions; research project (which could include funding for specific equipment);	# of personnel funded by type	9	12	144%
Funding research activities and positions	Innovation Positions; research project (which could include funding for specific equipment);	# of private sector contributions recorded by type of industry	20	5	25%
Funding research activities and positions	Innovation Positions; research project (which could include funding for specific equipment);	# of HQP positions supported through GF2 funding	135	187	139%
Funding applied research activities to bring new products/technologies/ processes to commercialization/adoption	Research projects; research report; peer reviewed article; HQP development; field and management trials	# of new projects funded by type of industry;	4871	11441	23%
Funding adoption/evaluation/ assessment/demonstration activities in the agricultural and agri-food industry	Technologies, practices, processes etc. assessed; Demonstration of new products, technologies, and processes, Technology, practices, processes adopted	# of new technologies, practices and processes assessed/evaluated/ demonstrated by type of industry;	609	1794	295%
Funding adoption/evaluation/ assessment/demonstration activities in the agricultural and agri-food industry	Technologies, practices, processes etc. assessed; Demonstration of new products, technologies, and processes, Technology, practices, processes adopted	# of new technologies, practices and processes adopted by type of industry	26	123	473%
Providing assistance  to the sector for commercialization	Feasibility studies, test markets, prototype development, intellectual property rights for products, commercialization center assistance, support for agri-entrepreneurs	# of clients receiving assistance for commercialisation  by type of industry	612	1285	210%

Outputs Achieved for Adaptability and Industry Capacity Priority Area

Activities	Outputs	Indicators	Target	Actual	% Achieved
Developing and offering training and education activities and opportunities within the industry/sector of business management	Training events, workshops, people trained	# of training events by type of training event	1149	5102	444%
Developing and offering training and education activities and opportunities within the industry/sector of business management	Training events, workshops, people trained	# of participants by training event	38863	281015	723%
Offering advisory and assessment services to industry producers and processors; Funding assessments and assessment tools	Consulting sessions, assessments, business plans; feasibility and cost of production report; succession planning;	# of clients by type (industry)	2239	1948	87%
Funding leadership programs and clubs	Leadership programs/projects and clubs; Resources related to leadership programs and clubs	# of activities funded/	14	10	72%
Funding leadership programs and clubs	Leadership programs/projects and clubs; Resources related to leadership programs and clubs	# of leadership programs and clubs funded	30	9	30%
Supporting implementation of business recommendations	Financial support; equipment; machinery; software;	# of projects funded by type of industry	2783	888	32%
Gathering and disseminating business management information	Research, Surveys; promotional materials (websites, publications etc.),	# of reports made available to target population	303	560	185%

Annex D: Case Studies

Case 1: Irrigation Efficiency Project Implemented at Vucurevich Farms Ltd., Alberta

• Priority Area: Adaptability and Industry Capacity
• Area of Intervention: Infrastructure
• Value: $15,000

The Irrigation Efficiency program helps producers invest in new or upgraded low pressure center pivot irrigation equipment for their operations.  Accordingly, the contribution recipient purchased a low pressure center pivot system for his farm to irrigate a field that was previously irrigated by a side-wheel system. A total of $15,000 of the eligible costs incurred by the applicant for an upgrade to a new low pressure center pivot system was received. The irrigation equipment upgrades aimed to improve energy efficiency and water usage on the farm. The system uses electricity as its main source of energy. The methods used to determine when to irrigate the field include the conditions of the crop, feel of the soil, reports on daily crop water use and computer simulation models (that is Alberta Irrigation Management Model).

Case 2: Cow/Calf Indoor Handling System, Alberta

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Assurance Systems
• Value: $35,000

The aim of this project is to improve animal care on the farm. The recipient aimed to contract a calf/cow handling system within a vacant building in his farm that included constructing an indoor handling system. The system was designed to ensure the safe flow and movement of cattle and is safe for both the animals and farmers and reduced stress.

Case 3: BC Agri-food and Seafood Export Program, featuring Misty Mountain Industries Ltd. & Ethical Bean Coffee Co.

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Market and Trade
• Value: $6,750 (Misty Mountain), and approx. $30,000 (Ethical Bean)

The BC Agri-food and Seafood Export Program is designed to support BC agri-food sector businesses in accessing and growing market share in domestic and international markets. The Program provides matching funds to support industry participation at international tradeshows, promotional events, incoming and outgoing trade missions and to develop marketing materials for international audiences. Within this program framework, Misty Mountain Industries Ltd. attended the Asia Fruit Logistica hosted by the BC Ministry of Agriculture under a BC-branded booth aimed at expanding sales of its mushroom-based products. The event provided Misty Mountain with the opportunity of exposure to Asian markets and potential expansion of sales in Asia. The goal was to identify new customers/ buyers and also connect with existing customers by showcasing various fresh and dried produce from BC, Canada. Also within this program framework, Ethical Bean Coffee Co. attended numerous international trade shows over the last several years, including Summer Fancy Foods (2016), National Expo West (2016) and National Products Hawaii (2016), with the goal to develop business opportunities with foreign buyers and sellers. The funding provided has produced some of the intended outputs. In turn, these outputs have produced the intended immediate outcome of the Market Development program, such as the companies’ increasing awareness, knowledge and skills of potential market-based opportunities, and an awareness of Canadian agri-related products in foreign markets.

Case 4: Manitoba Beef and Forage Initiatives

• Priority Area: Innovation
• Area of Intervention: Research
• Value: $2,850,800

The Manitoba Beef and Forage Initiatives (MBFI) is a centre of agricultural innovation engaging in science-based research to benefit valuable ecosystems, improve producer profitability and build social awareness around the beef and forage industry. MBFI is funded in part by the Growing Forward 2 Innovation program, in collaboration with Manitoba Beef Producers, Ducks Unlimited Canada and the Manitoba Forage and Grassland Association, with input and support from other industry stakeholders. The project is composed of two research farms and facilities all of which were toured during the site visit.

Case 5: P. Quintaine & Son Ltd., Brandon, Manitoba

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Food Safety-Processing and Distribution
• Value: $50,000

The Quintaine & Sons facility in Brandon is an assembly operation and marketing agent for pork producers. This facility acts as an “assembler” which means they either act directly, or on behalf of, a producer to assemble and grade the livestock and broker the sale of the hogs to Food Manufacturers and Processors. Quintaine & Sons invested approximately $350K on upgrading their hog sorting facility and submitted a claim of $50K to the Bio-security Processor Sub-Program under the Growing Forward 2 Assurance Program. This facility which is part of the food marketing channel was reviewed during the site visit.

Case 6: Investigation of Source of Potato Virus Y Problems and Evaluation of Best Management Practices, New Brunswick

• Priority Area: Innovation
• Area of Intervention: Research
• Value: $67,000

The site visit for this project occurred at Agricultural Certification Services Inc. – a laboratory where the project manager works. The overall objective of this Growing Forward 2 funded project was “to investigate problems of Potato Virus Y at several seed farms in New Brunswick and evaluate various best management practices used on grower’s farms for the management of Potato Virus Y.” Growing Forward 2 funding included a $67,000 contribution to conduct a study to help researchers clearly understand Potato Virus Y spread in New Brunswick. Results from the study were used to make recommendations for Potato Virus Y management.

Case 7: Pure Holsteins – Robotic Milking System, Newfoundland and Labrador

• Priority Area: Innovation
• Area of Intervention: Environment
• Value: $800,000 (total project cost, including required renovations to the barn; $640,000 for two robotic milkers). $345,000 received from Growing Forward 2 funding program.

In Newfoundland and Labrador, the Advancing Agriculture Innovation Program is close to a $12 million initiative. The robotic milking system, installed at Pure Holsteins Ltd. dairy farm in Little Rapids, is an example of a project that was funded in 2014 through Growing Forward 2 in Newfoundland and Labrador. The project represents the first robotic milking system in the province. The milking robots are completely automated and operate on a 24-hour basis, seven days a week. The system allows the dairy farmers to retain optimal production and high-quality milk while still focusing on the well-being and comfort of their cows. A grain feeder on the robot entices the cows into the machine and allows them to eat while being milked. During the milking, the robot records information about the cow, its weight and the temperature of her milk, and tests the ketone levels of the milk and salt counts to help dairy farmers be proactive in keeping their cows healthy. Each cow has an identification collar that identifies everything about her upon entering the robot, including if she is due to milk. The automated system creates an operation that is animal-friendly and works efficiently while controlling costs. Growing Forward 2 funding included a $345,000 contribution toward the purchase of two Lely Austronaut Milking Robots.

Case 8: Out to Pasture Farm, Nova Scotia

• Priority Area: Innovation
• Area of Intervention: Environment
• Value: $19,646

Out to Pasture Farm received funding under the Environmental Management and Innovation program, which provides funding for the Environmental Farm Plan program and Beneficial Management Practices. The EFP program is delivered by the Nova Scotia Federation of Agriculture to increase awareness of environmental issues and help farmers implement environmental stewardship initiatives, The Nova Scotia Department of Agriculture offers cost-shared programing in the area of Beneficial Management Practices. Funding under the program was used to renew the farm’s Nutrient Management Plan, construct a water well, and construct a concrete manure storage area.

Case 9: Inuksuk High School Greenhouse, Nunavut

• Priority Area: Adaptability and Industry Capacity
• Area of Intervention: Infrastructure
• Value: $7,000

Inuksuk High School applied for funding to the Community Greenhouse Development program in order to build a small indoor greenhouse operation in the school. The objective of the project is to increase education related to food security.

Case 10: Small Scale Foods Program, Northwest Territories

• Priority Area: Adaptability and Industry Capacity
• Area of Intervention: Infrastructure
• Value: $12,000

Two community garden sites in Yellowknife, Northwest Territories were visited. There are over 30 community gardens that are active across all of the Northwest Territories. These projects fall under the Small Scale Foods Program which has been implemented in all regions of the Northwest Territories by the Department of Industry, Tourism and Investment. The Small Scale Foods Program provides support to clear and develop land, purchase seeds, and provides basic tools for garden maintenance. Furthermore, the Program provides planning and gardening seminars and workshops on an ongoing basis. The project is a $1.775 million Adaptability and Industry Capacity initiative but for illustrative purposes, the case study involved looking at two roughly $6,000 community garden projects.

Case 11: Vineland Research and Innovation Centre – Appassimento Red Wine, Ontario

• Priority Area: Innovation
• Area of Intervention: Research
• Value: $26.5 million over 5 years.

Vineland Research and Innovation Centre is an independent, not-for-profit organization created in 2007 that has received core funding through Growing Forward 2 falling under the Innovation Priority Area and Research Area of Intervention. Vineland’s core funding allows it to drive growth and impact in the horticulture sector. With its renewed Growing Forward 2 core funding, Vineland funds a variety of research projects including the one examined for this case study, the development and commercialization of Appassimento red wine technology geared at simplifying and optimizing the Appassimento grape drying process and techniques for winemakers in the region, optimizing their productivity.

Case 12: Filsinger’s Organic Ltd. – Automation of Organic Apple Cider and Vinegar Bottling Line, Ontario

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Markets and Trade
• Value: $200,000

Filsinger’s Organic Ltd. is an organic apple orchard in Ayton, Ontario, where they grow many varieties of apples on the 100 acre farm. They harvest the apples and make a variety of apple products, including apple cider, apple cider vinegar, applesauce, and apple butter. Filsinger’s Organic Ltd. purchased and installed a new automated continuous flow processing system and automated packaging line to improve labour productivity and efficiency. This was primarily for new bottling equipment and a boiler. The new line is a continuous system, fully automated with filling, capping with safety seals, and labels applied while the bottles move down the line. The previous manual method required four employees to clean the equipment, fill the pasteurizer, and complete the bottling of the cider/vinegar. These employees can now focus on other business tasks. The total cost of the project was a $200,000 endeavour that received $69,178.55 in Growing Forward 2 cost-shared funding. The project was completed May 31, 2016.

Case 13: Potato Farm – Soil Conservation Erosion Control Structures, Prince Edward Island

• Priority Area: Innovation
• Area of Intervention: Environment
• Value: $28,950

These projects completed by Andrew Lawless (Hilltop Produce) fall under the Supporting On-Farm Sustainable Agriculture Practices program area within the Environment Area of Intervention. They represent some of Prince Edward Islands Agriculture Stewardship projects. The two soil conservation projects were completed in 2016. Growing Forward 2 funding included a $28,950 contribution toward the installation of erosion control structures.

Case 14: Ferme Porcine Lina Inc. – Ouvrages de conservation des sols, Québec

• Priority Area: Innovation
• Area of Intervention: Environment
• Value: $15,071 in 2014-15

Recipient of Growing Forward 2 cost-shared programming funds through the Quebec sub-program Prime-Vert, the work undertaken at the Lina swine farm is part of a project to develop structures and soil conservation, including interception of ditches and gullies. The project had the following mandate:

• Identify erosion problems and perform field surveys
• Recommend work and other actions to be taken to correct erosion problems
• Prepare plans and specifications
• Prepare a list of materials and an estimate of project costs
• Recommend mitigation measures to be applied during construction
• Submit and explain the "technical file" and its appendices to the representative of the farm concerned
• Ensure that the permits required to carry out the project have been issued before the start of the work

Case 15: Les comptonales - Marché public de Compton, Québec

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Diversification et développement des marchés de proximité
• Value: $35,000

Les Comptonales is an organization in the eastern townships of Quebec that exists since 2005. It is made-up of members from the agriculture and processing world, as well as elected members. The organization’s mission is to promote the work and knowledge of producers, processeurs, chefs and artisans of Compton and its surroundings, while ensuring the transmission of the regional eastern townships agri-food heritage. One of the projects they launched and organized was Compton’s public evening market, every Thursday evening, from 16:00 to 18:30, throughout the summer months. For this project, they applied for and received funding under the Proximity program of the Quebec Growing Forward 2 cost-shared programming framework. The base funding was supplemented with other funding sources, to build 20 permanent wooden kiosks and related plumbing and electricity work to develop the Compton public evening market site. The market’s objectives were to offer producers an environment better suited to the outdoor market, as well as improving their sales revenues. Project partners: Compton, Municipalité Régionale de Comté Coaticook, Desjardins Caisse des Verts-Sommets de l’Estrie, Lions International – Club Lions de Compton, CIGN 96.7FM, New Hampshire Charitable Foundation, UPA Pouvoir Nourrir Pouvoir Grandir Coaticook Estrie, Assurance Claude Marcoux Inc., CABICO Armoires sur mesure, Gouvernement du Canada, Gouvernement du Québec, and Cultivons l’avenir.

Case 16: License to Farm: SaskCanola Agriculture Advocacy Film Project Saskatchewan

• Priority Area: Adaptability and Industry Capacity
• Area of Intervention: Business Development
• Value: $49,500 (combined across two related projects)

License to Farm is a 30 minute documentary film [produced between October 2016 and March 2016 by SaskCanola, that is, Saskatchewan Canola Development Commission] that “explores the truth behind common misconceptions of agriculture production in Canada, while empowering farmers to stand up and advocate for their social license to farm.” Growing Forward 2 funding included a $22,000 contribution for production and a $27,500 contribution for marketing and promotion.

Case 17: NutraSun Foods Outgoing Mission to Engredea Trade Show, Saskatchewan

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Trade and Market Development
• Value: $9,896

NutraSun Foods Ltd. attended and exhibited its organic flour products at the Engredea 2016 trade show in Anaheim. The objective of the mission was for NutraSun Foods Ltd. to obtain five new customers from the California area, representing approximately 1,000 tonnes of new product sold. As a result of activities funded through Saskatchewan Trade and Market Development program, NutraSun was able to produce some of the outputs and immediate, intermediate and end outcomes intended by the program. Most importantly, these outputs and outcomes produced by NutraSun did have a positive impact on its business, introducing its products to new markets and developing client relationships and deals.

Case 18: Yukon Grain Farm – Root Vegetable Storage, Yukon

• Priority Area: Competitiveness and Market Development
• Area of Intervention: Assurance Systems
• Value: $140,362

The Yukon Grain Farm in Whitehorse, Yukon received Growing Forward 2 cost-shared funding for building a new 40 feet by 80 feet cold storage with temperature and humidity controls for potatoes, and retrofitting the existing 40 feet by 80 feet cold storage for appropriate root vegetable storage (carrots, beets, cabbage, rutabagas, parsnips). The retrofit included new refrigeration, structural modifications, new humidity control, more storage boxes and a separate section for store-ready (bagged) product. The project was successfully completed and is more than meeting expected outputs and outcomes.

Annex E: Benchmarking summary

Benchmarking Government Cost-Shared Programming

What We Learned

AAFC Is Fully Aligned With Other Government Programs

• Clear, comprehensive, robust multilateral frameworks, allowing flexibility for P/Ts with an adequate federal support to ensure consistency and rigour

Opportunities To Enhance AAFC’s Policy Framework

• Decrease communication gaps relating to program administration or troubleshooting by creating formal opportunities to share best practices
• Keep an open and honest dialogue to improve program design and delivery
• Continue to streamline application procedures and single window client portals

New Approaches for AAFC

• Build adequate capacity to support program from start-up through day-to-day operations, including ongoing performance measurement
• Consider longer program cycles
• Develop best practice training guides, data dictionaries for indicators, co-develop monitoring/reporting tools with the P/T and hold performance indicator development workshops

Agriculture and Agri-food Canada	Infrastructure Canada	Canadian Heritage	Justice Canada	Employment and Social Development Canada	Public Safety Canada
$2 billion	$16.7 billion	$3.6 billion	$2.8 billion	$1.9 billion	$0.7 billion
5 years	10 years	3 and 5 years	5 years	3 and 4 years	5 years
60:40	33:66	50:50	50:50	84:16 and 50:50	50:50
GF2 Cost Shared non-BRM Programming	Provincial Territorial Infrastructure – National and Regional Program, Communities Component, and Small Communities Fund	Official Languages in Education Protocol, Hosting Program, and Sport Support Program	Youth Justice Fund, Aboriginal Court Workers Program, and Legal Aid	Targeted Initiative for Older Workers, and Labour Markets Agreements for Persons with Disabilities	First Nations Policing Program

Governance - Program Features

• Strongest models have steering committees with FPT co-chairs and meet frequently
• FPT working groups are crucial mechanisms to encourage communications, identify best practices and develop program policy changes
• FPT groups share information, work through shared issues and monitor and profile progress

Administration

• Best models have programs with a balance between project funding and administration costs
• Best models feature clear application criteria and instructions, and project proponents register through a single window client portal
• Strict frameworks are difficult to roll out, and PTs see as intrusive and unproductive
• Cumbersome application and approval processes are common and remain an irritant for applicants

Operational Capacity

• Almost all programs reported capacity concerns, at both the federal and P/T levels, in relation to the volume of administrative work generally, and particularly with respect to performance measurement; no program reported having a team dedicated to performance measurement
• In some cases P/Ts contract out program delivery

Performance Measurement

• All interviewees emphasized the importance of PM, and much time is spent with P/T developing PM strategies, but there is a gap between development and implementation
• The most effective performance management models have well developed performance measurement and risk management strategies in place
• Best models identified a list of potential P/T activities and related output indicators in Multilateral Framework Agreements, to the outcomes and results achieved
• Most performance measurement systems are limited to attempts to track transfers and record outputs, often using incompatible systems and indicators across P/T leaving federal partners – and P/T– unable to determine program impacts
• None of the interviewed representatives was satisfied with the state of performance measurement for their programs:
  • data are not available on time
  • few templates in regular use
  • inconsistent interpretation of indicators
  • little or no training provided on data collection, interpretation and reporting
  • lack of federal partner or P/T influence over data collection in the field
• Interviewees felt the federal government should exhibit more leadership in developing common data platforms and driving data collection; ideally, develop and provide in consultation with P/T in the collection of standard indicators such as best practice training guides, performance indicator development workshops, data dictionaries, monitoring data collection and reporting templates

Communication

• 2 out of 12 programs have formal communications strategies despite all stating that communication is critical to program success
• Communication gaps for larger programs lead to missed opportunities to share best practices among working group members and P/Ts especially in programs with strong working relationships between federal and P/T counterparts worked best

Methodology

In the context of the evaluation of AAFC’s Growing Forward 2 (GF2) Cost-Shared program, one area of interest is the FPT cost-shared funding model being utilized by AAFC. A benchmarking exercise was undertaken to determine how this model compares to other departments that have FPT cost-shared funding programs.

Representatives (managers and program officers) of six selected federal departments and agencies representing 13 (F/P/T) cost-shared programs were invited to participate in interviews to discuss their department’s approach and implementation of their programming. Interviews with program managers along with basic program documentation were used to identify key features and best practices.

Limitations

The exercise was limited to general facts respecting a small sample of Canadian cost-shared programs; observations contained in this page should be viewed as an introduction to cost-shared programming, not an exhaustive study. An accompanying report contains additional detail on the benchmarked programs.