Evaluation of the Trade and Market Expansion Program

March 9, 2016

Report:
Office of Audit and Evaluation

List of acronyms

AAFC

Agriculture and Agri-Food Canada

CFIA

Canadian Food Inspection Agency

EU

European Union

FTA

Free Trade Agreement

G8

Group of Eight

G20

Group of Twenty

GF2

Growing Forward 2

MAS

Market Access Secretariat

MISB

Market and Industry Services Branch

NAFTA

North American Free Trade Agreement

OAE

Office of Audit and Evaluation

OECD

Organisation for Economic Co-operation and Development

PAA

Program Alignment Architecture

PMS

Performance Measurement Strategy

PT

Province and Territory

SED

Sector Engagement and Development

TAND

Trade Agreements and Negotiations Directorate

TME

Trade and Market Expansion

TPP

Trans-Pacific Partnership

TRECS

Tracking and Reporting Executive Correspondence System

US

United States

WTO

World Trade Organization

Executive Summary

Introduction

Trade and Market Expansion (TME) is one sub-program within the Market Access, Negotiations, and Sector Competitiveness Program. TME includes the activities of the Market Access Secretariat (MAS) and the Trade Agreements and Negotiations Directorate (TAND), both of which are part of the Market and Industry Services Branch (MISB). The TME program aims to support Canada's producers and processors by helping to improve market access, expand market opportunities and strengthen international trade rules. TME is responsible for trade related policy development and negotiations, research and analysis, and partnership development.

The evaluation of TME was conducted between November 2014 and May 2015. This evaluation, which is the first of the program, complies with the requirements of the Treasury Board Policy on Evaluation (2009). In accordance with this policy, the evaluation addresses the relevance of TME, the achievement of its outcomes, its efficiency and economy, and design and delivery of TME. The evaluation covers the work of TME between fiscal years 2009-10 and 2013-14.

Methodology

The evaluation methodology consisted of a document and administrative review; literature and media coverage review; interviews with representatives of TME and of other areas of Agriculture and Agri-Food Canada (AAFC), and representatives of other federal departments, provinces and industry stakeholders; as well as a case study. Triangulation was used to verify and validate the findings obtained through these methods and to arrive at the overall evaluation finding.

Findings

Relevance

The evaluation found that there is an ongoing need within the sector for effective policy development and negotiations, reliable market research and analysis, and partnership development in support of global market expansion and development. TME addresses these needs. TME's leadership and coordination in marshalling a Canadian response to agriculture, agri-food market opportunities and threats is warranted.

TME directly supports the federal government's trade agenda of increasing market access through its policy development and negotiations work and by supporting sector stakeholders in their market development initiatives. As such, TME's work has been closely aligned with governmental priorities and commitments outlined in the 2014 Economic Action Plan and Throne Speech.

Achievement of expected outcomes

TME achieved its immediate outcomes, including: engagement in advancing Canadian agriculture market access issues and trade priorities; engagement with industry for new and expanded global market opportunities; and, cooperation between federal and provincial governments in the advancement of Canadian agriculture.

TME engaged frequently with the industry and the provinces with respect to market development initiatives and took a leadership role in terms of coordination among key stakeholders, representation of Canada, and background research in support of market access and development.

Through its engagement and cooperation with industry and provinces, TME achieved two of its intermediate outcomes: increased recognition, adoption and enforcement of clear, transparent, science-based trade rules; and, increased market access for Canadian agriculture. In addition, progress was made towards the other two intermediate outcomes: new bilateral and multilateral trade agreements with key trading partners; and, increased Canadian industry entry into new markets and expansion of existing markets. As international conditions change, work will always be required to open and expand markets.

TME made progress toward its expected end outcome which is "Canadian agriculture and agri-food products exporters are successful in seizing market opportunities and in achieving commercial success". Over the five year period from 2009-10 to 2013-14 Canadian agri-food and seafood exports rose by 43 per cent. Greater Canadian export growth occurred in non-OECD countries compared to OECD countries. By 2012-13, AAFC's 2017 targets were met in the case of non-OECD countries and nearly met in the case of OECD countries.

Efficiency and economy

The evaluation evidence suggests that TME was delivered economically and efficiently. The value of Canada's agriculture and agri-food industry is large in relation to TME's annual cost; the volume and quality of TME outputs combined with qualitative evidence from TME contributions suggests a positive financial impact.

Design and delivery

TME's organizational structure is effective. The evaluation found that TME could increase the effectiveness of the integration and cooperation with its delivery partners, in particular Global Affairs Canada^{Footnote 1} and CFIA. The evaluation found weaknesses in TME's performance measurement strategy, which needs to be refined.

1.0 Introduction

1.1 Context of the evaluation

The evaluation of the Trade and Market Expansion (TME) program was conducted by Agriculture and Agri-Food Canada's (AAFC) Office of Audit and Evaluation (OAE) as part of AAFC's Five-year Departmental Evaluation Plan (2014-15 to 2018-19) and complies with the requirements of the Financial Administration Act (1985) and the Treasury Board Policy on Evaluation (2009). The evaluation focused on TME's activities from 2009-10 to 2013-14 and the report consolidates all findings from each source of evidence.

1.2 Structure of the report

The evaluation addresses core evaluation issues related to relevance and performance, as defined in the Treasury Board Directive on Evaluation (2009), as well as additional questions and issues determined by OAE and AAFC senior management as being key for future program development. The evaluation questions, sub-questions and indicators are contained in the evaluation matrix (Annex A: evaluation matrix).

The report contains a profile of TME including a program logic model (Annex B: Trade and market expansion logic model), a description of the evaluation methodology and its limitations, findings organized by evaluation issue, and conclusions.

2.0 Profile of trade and market expansion

2.1 Program activities, outputs and expected outcomes

Through a series of programs, Agriculture and Agri-Food Canada (AAFC) aims to support the economic growth and long-term prosperity of the agriculture and agri-food industry domestically and internationally. The Trade and Market Expansion (TME) program aims to support Canada's producers and processors by helping to improve market access, expand market opportunities and strengthening international trade rules.

2.1.1 Activities

TME is responsible for activities in three main areas, as follows:

• Policy Development and Negotiations.The sub-activities within this activity area are concerned with representing the interests of the Canadian government abroad in agriculture and agri-food. The sub-activities include:
  1. Market Access: building and maintaining bilateral trade relations to advance market access; identifying and leading on market access activities related to countries of interest; leading the coordination and support of incoming and outgoing missions related to market access;
  2. Trade Negotiations: leading the negotiations of the agriculture component of free trade agreements (FTAs); supporting negotiators with analysis on Canadian agri-food market access interests; participating in negotiations, including, in relation to technical barriers to trade and sanitary phytosanitary related issues;
  3. World Trade Organization (WTO): leading the review, consultation and draft input on WTO notifications and trade policy reviews; monitoring the implementation of WTO agreements by WTO Members to assess compliance with their international obligations; negotiating new WTO trade rules for agriculture that will cover domestic support, market access and export competition; and,
  4. Trade Barriers: influencing the development of international science-based standards and decisions in multilateral organizations to improve the predictability and transparency in agri-food trade; engaging with key trading partners (for example, United States (US), China, Japan, Korea, India, and the European Union (EU)) to mitigate unnecessary disruptions to trade of Canadian food, feed and seed exports.
• Research and Analysis.The sub-activities within this activity area are concerned with conducting policy and socio-economic research and analysis. The sub-activities include:
  1. Market analysis requests in support of AAFC: producing and distributing market research and analysis internally to the Market and Industry Services Branch (MISB), senior management and Minister's Office, including all requests through the Tracking and Reporting Executive Correspondence System (TRECS);
  2. Market research and intelligence reports in support of market development activities:producing and distributing market intelligence reports in support of trade show participants as well as other events, such as the Annual Market Access meeting and trade summits; and,
  3. External ad hoc requests responded:producing and distributing market data and analysis externally to AAFC for provinces, other government departments, Trade Commissioners and industry associations.
• Partnership Development. The sub-activities within this activity area aim to develop collaboration and partnership opportunities with international organizations, interest groups, other government departments, provinces and territories (PTs) and the public. The sub-activities include:
  1. Communications and Consultations: coordinating consultation mechanisms to inform governments and stakeholders on market access issues; monitoring and providing analysis and feedback on market access issues;
  2. Market Development Coordination: fostering coordination among provinces, industry, Trade Commissioners, and other Canadian stakeholders on market development activities and encouraging a pan-Canadian approach in key markets, as well as, providing information and analysis;
  3. Multilateral Organizations: managing Canada's engagement with multilateral organizations such as Group of Eight (G8), Group of Twenty (G20), Organization for Economic Cooperation and Development (OECD), Food and Agriculture Organization of the United Nations, Inter-American Institute for Cooperation in Agriculture; and, providing briefings for Canadian representatives to multilateral organizations;
  4. Flagship Trade Shows: designing and managing a Canadian branded and organized presence of Canadian exhibitors in Canada pavilions at selected international flagship trade shows; leading AAFC, other government departments, industry associations and private sector participation in flagship trade shows in key markets; and,
  5. Trade Commissioners: undertaking activities developed in line with TME priorities relating to trade irritants, trade agreements, market access issues, market development, market intelligence, trade leads, missions and flagship trade shows.

2.1.2 Outputs

The outputs generated from the above sub-activities for each activity area include the following:

• Policy development and negotiations
  1. Market Access: strategic advice to senior management on market access issues; reporting on the status of market access negotiations, including analysis and briefing material; contribution to bilateral trade meetings; support to incoming and outgoing missions; and, alignment of Trade Commissioner activities with market access priorities;
  2. Trade Negotiations: strategic advice to senior management on trade policy issues; reporting on the status of trade negotiations including analysis and briefing material; addressing barriers to trade within the context of FTA negotiations, analysis and technical support to the WTO Committees (inclusive not only of agriculture but also of technical barriers to trade and sanitary phytosanitary); contributions to trade meetings; AAFC and industry interests reflected in the negotiating priorities and in the FTAs; communications and consultation processes with governments and stakeholders on Canada's approach to FTA negotiations; and, briefing material prepared for appearances of the Minister of Agriculture and Agri-Food and senior officials to parliamentary committees; and,
  3. Trade Barriers: development of science-based trade rules and policies; evergreen analysis on the development of domestic trade policies of other WTO Members including key trading partners; advancement of Canada's trade policy interests (for example, case against the US country-of-origin labeling policy); and, defence of Canadian programs and policies against potential trade challenges from other countries.
• Research and analysis
  1. Market analysis requests in support of AAFC: production and distribution of market research and analysis internally to Market Access Secretariat (MAS), senior management and Minister's Office, including all TRECS requests;
  2. Market research and intelligence reports in support of market development activities:production and distribution of market intelligence reports in support of flagship trade show participants such as the Annual Market Access meeting and trade summits; and,
  3. External ad hoc requests responded: production and distribution externally to AAFC for market data and analysis, including provinces, other government departments, Trade Commissioners and industry associations.
• Partnership development
  1. Communications and Consultations: targeted communication materials to stakeholders on the progress of trade negotiations and on the content of concluded agreements; and, responses on market access issues provided to Canadian agriculture stakeholders through the TME Single-Window service and Annual Market Access Meetings;
  2. Multilateral Organizations: reports provided to stakeholders on strategies and developments in multilateral organizations; and, briefings given to Canadian representatives to multilateral organizations;
  3. Market Development Coordination: coordination of a framework for development of country strategies and supporting activities among provinces for priority markets through the International Market Engagement Team process; participation by Canadian agriculture stakeholders in flagship trade show venues around the world; on-line strategy to facilitate market success; and, coordination framework provided to facilitate development of country strategies; and,
  4. Trade Commissioners: resolution of trade irritants, resolution of market access issues, new markets developed, market intelligence and trade leads provided, missions and flagship trade shows supported.

2.1.3 Expected immediate outcomes

The immediate outcomes from the combined outputs of the three activity areas include:

1. Engagement in advancing Canadian agriculture market access issues and trade priorities;
2. Engagement with industry for new and expanded global market opportunities; and,
3. Cooperation between federal and provincial governments in the advancement of Canadian agriculture.

2.1.4 Expected intermediate outcomes

The intermediate outcomes from the combined outputs and immediate outcomes of the three activity areas include:

1. Increased recognition, adoption and enforcement of clear, transparent, science-based trade rules;
2. Increased market access for Canadian agriculture;
3. New bilateral and multilateral trade agreements with key trading partners; and,
4. Increased Canadian industry entry into new markets and expansion of existing markets.

2.1.5 Expected end outcome

The end outcome from the combined outputs, immediate and intermediate outcomes of the three activity areas is: Canadian agriculture and agri-food products exporters are successful in seizing market opportunities and in achieving commercial success. The indicators for the end outcome are the expansion of Canada's exports in agriculture and agri-food gained through breaking down trade barriers in Canada and internationally, assisting Canada's agriculture and agri-food sector to identify and capture new global market opportunities, developing strategies to differentiate Canadian products and, in the long term, through deepening the sector's strength in the global marketplace. Annex B: Trade and market expansion logic model contains a logic model summarizing activities, outputs, outcomes, and the linkages between them.

2.2 Target populations and stakeholders

The TME activities are targeted to:

• Canadian producers, food processors and exporters of agriculture and agri-food products, and industry associations;
• Provinces and territories;
• Canadian government departments and agencies: AAFC, Global Affairs Canada^{Footnote 2}, Canadian Food Inspection Agency (CFIA), Department of Finance and Privy Council Office;
• International trading partners (for example, US, China, Japan, Korea, India, and the EU), current FTA negotiating partners (for example, EU, Trans-Pacific Partnership (TPP) members, Japan, and India), WTO Members; and,
• Multilateral organizations including OECD, Food and Agriculture Organization of the United Nations, Inter-American Institute for Cooperation in Agriculture, G20, G8, World Organization for Animal Health.

TME's stakeholders are the national organizations in the public and private sectors promoting and representing Canadian agriculture and agri-food and trade including federal departments and agencies (AAFC, Global Affairs Canada, CFIA), provincial departments of agriculture and trade, industry associations and clients within agriculture and agri-food, trading and marketing organizations, and individual businesses involved in the production, processing and exporting of agricultural and agri-food products. International stakeholders include multilateral organizations such as the Food and Agriculture Organization of the United Nations, G8 and G20.

2.3 Governance structure

TME activities are conducted by two AAFC directorates within MISB:

Market Access Secretariat (MAS): This directorate is headed by a director general and divided into four divisions. The divisions are: the Market Access Coordination Division (led by an executive director); the Asia and Oceania Division; the Americas Division; and, the Europe, Middle East and Africa Division (each led by a director).

Trade Agreements and Negotiations Directorate (TAND): This directorate is headed by a director general and divided into three divisions each led by a director. The divisions are: the Strategic Trade Policy Division; the Trade Negotiations Division; and, the Technical Trade Policy Division.

2.4 Program resources

The expenditures for TME for the fiscal years of 2009-10 to 2013-14, Table 1, shows that the number of full-time equivalents ranged from 157 (2013-14) to 215 (2011-12). For the period from 2009-10 to 2013-14, the expenditures totaled $136.37 million. Below, all expenditures are for salary and operations (Vote 1). In 2014-15, Trade Negotiations and Market Access, Market Growth, Market Information and Export Capacity Building, and Canada Brand were consolidated to form TME in the departmental Program Alignment Architecture (PAA).

Table 1: TME expenditures for 2009-10 to 2013-14

	2009-10	2010-11	2011-12	2012-13	2013-14
Salary	$16,274,655	$17,274,101	$18,317,418	$18,415,158	$17,962,344
Non Pay Operations	$7,910,001	$9,906,815	$13,756,598	$10,238,910	$6,318,504
Total	$24,184,656	$27,180,916	$32,074,016	$28,654,068	$24,280,848
Full-Time Equivalents	187	205	213	194	157

3.0 Methodology

3.1 Sources of evidence

3.1.1 Document and file review

A document and file review was conducted to address evaluation questions pertaining to alignment with federal government priorities and departmental strategic outcomes, program performance, program economy and efficiency, and program design and delivery. The document and file review assisted in the development of a profile of the Trade and Market Expansion (TME) Program. Documents included performance reports, Reports on Plans and Priorities, program profiles, financial information, business plans, policy frameworks, and annual reports. Government of Canada documentation reviewed included Throne Speeches, Budgets and government-wide policy statements.

3.1.2 Literature review

A literature review was conducted to assess the program's relevance, performance, and its efficiency and economy. Reviewed literature and documentation included reports from Statistics Canada, and external peer-reviewed publications, articles and internet sources.

3.1.3 Interviews

Twenty-four interviews were conducted with 16 internal and 11 external stakeholders (Table 2). Interviews were conducted in person or by phone. Selection of the respondents was done by the evaluation team. A mix of government and non-government representatives were selected including representatives from the major subsectors of the agriculture and agri-food sector; meats, grain, vegetables/fruits, seafood, processed foods and import/export.

Table 2: interviews by sub-group

Interview Sub-Group	Number of Interviews	Number of Interviewees
AAFC Program Management and Staff	4	4
Federal Stakeholders in AAFC, Global Affairs Canada, Health Canada, and CFIA	9	12
Industry Representatives	11	11
Total	24	27

3.1.4 Case study

Information from the CDC Triffid Flax Case Study conducted as part of the evaluation of the Sector Engagement and Development (SED) program of Agriculture and Agri-Food Canada (AAFC) (conducted at the same time as this evaluation) was used as market access issues are also present in this case study. The case study concerned a situation in the summer of 2009 when Canadian shipments of flax headed for Europe tested positive for low-level presence of a genetically modified flax (CDC Triffid) approved for food, feed, and environmental release in Canada, but was not evaluated in several export markets, including the European Union (EU). As a result of this detection of CDC Triffid in the EU, imports of Canadian flax were halted. The TME worked with SED, the Canadian Grain Commission, Programs Branch, legal counsel, as well as with other federal government and industry partners to put in place a sampling and testing protocol that allowed exports of Canadian flax to the EU to resume.

3.2 Methodological limitations

A methodological limitation of the evaluation was that a full set of reliable quantitative evidence was not available for all expected outcomes, as well as for the outputs generated by TME. Empirical evidence related to outcomes was limited to:

1. anecdotal evidence in relation to, for example, particular incidents in which TME was involved and
2. macro-level economic indicators which are difficult to reliably attribute to TME program activities.

The lack of outcome and output data limited the evaluation's capacity to conduct comprehensive cost-benefit and costs-per-outputs analyses, which would have been useful to assess TME efficiency and economy.

In order to mitigate this limitation, interviews with a representative sample of stakeholders were used as a primary source of evidence. TME's expected outcomes are in large part qualitative and observable; thus a systematic program of interviews was able to provide a reasonable base of evidence from which to address outcome questions. This evidence base was bolstered by several complementary sources of evidence – the literature and document review and the case study – enabling the evaluation to corroborate and illustrate findings.

4.0 Evaluation findings

4.1 Relevance

4.1.1 Need for Trade and Market Expansion

The evaluation found that there is an ongoing need within the sector for reliable research and analysis, for policy development and negotiations, and for partnership development in support of industry access to new and existing markets, and a continued role for Trade and Market Expansion (TME) in providing these services.

Export markets are critical to the agriculture and agri-food sector, and to the Canadian economy. Canadian agriculture is a modern, technologically advanced, export-oriented sector that provides more than two million jobs nation-wide and over 8 per cent of Canada's Gross Domestic Product^{Footnote 3}. Producers and processors export 42 per cent of their primary agricultural products and 23 per cent of their processed food and beverages^{Footnote 4}. In 2013-14, farmers earned more money from the global marketplace than ever before, with exports reaching over $50 billion. To support the agriculture and agri-food sector to continue exporting and to seize new market opportunities, there is ongoing need for market information and export capacity building.^{Footnote 5}

This is especially true in the developing world where higher economic growth represents new and expanding opportunities. Industry has repeatedly noted the importance of continuing and enhancing the Government's activities to improve market access and development. For example, in March 2014, the Report of the Standing Committee on Agriculture and Agri-Food recommended that: "the Government of Canada continue to pursue additional comprehensive trade agreements to open new markets and provide opportunities for growth for Canadian agricultural and food exporters."^{Footnote 6}

Canada is a strong supporter of trade liberalization primarily due to the importance of international trade to the Canadian economy. As a country with a low population but vast arable lands, Canada is heavily reliant on export markets to drive growth in the agriculture and agri-food sector. Canada has, therefore, signed a number of multilateral, regional and bilateral agreements, and continues to do so in order to protect its interests by helping to establish a rules-based international trade regime.

Evidence suggests that gaps would exist in addressing sector needs in the absence of TME. The private sector does not have any mechanism for establishing the capacity to systematically generate market intelligence and information or develop markets on a national basis. Market intelligence informs analysis, strategies and decision-making to improve market access and market development efforts. TME's market intelligence is based on participation in missions abroad, bilateral meetings with trading partners, and access to primary market information (for example, via trade commissioners). TME's market intelligence reports produced for industry serve to build the capacity of industry to export. The reports cover topics such as retail market size, consumer trends, distribution channels, competing companies and countries, brands, products and innovations.^{Footnote 7}

Information costs can impede trade by making it more challenging for small and medium sized enterprises to get established in export markets. According to Farm Credit Canada (Copeland, 2007):

"Information costs can impede trade in a number of ways. These include costs of identifying new markets, developing distribution channels, finding suitable and reliable suppliers and dealing with local regulations. They can also be related to learning how to adapt a product to local market conditions, learning the right marketing strategy for the foreign market, issues of asymmetric information about quality of both one's own product and those utilized in the foreign market, and many others. To date federal government support for export promotion and market development activities has been important due to the lack of private sector investment in the information needed to access foreign markets."

Market intelligence and export capacity building is particularly important when markets change and the evaluation found that markets are changing. According to Statistics Canada data, in 2013 Canada's agri-food and seafood exports crossed the borders into 192 markets. While the US is still Canada's largest export market, the 2013-14 Farm Credit Canada Report documents that the percentage of exports going to the US has dropped substantially over the past ten years, from 60.3 per cent in 2004 to 51.8 per cent in 2013. This happened as overall exports increased, an indication of the growth in Canadian exports to markets like China, Japan, the EU, India, Indonesia, Mexico, South Korea, Taiwan, and Russia which together accounted for 47 per cent of Canada's agri-food and seafood exports in 2013. Documented evidence shows that over the last eight years Canada's export growth to BRIC countries (Brazil, Russia, India and China) and other emerging markets have outpaced growth to Organisation for Economic Co-operation and Development (OECD) counties. The ongoing liberalization of world trade, in combination with the increasing strength of emerging economies, offers further potential for growth in the exports of Canadian agriculture and agri-food products.^{Footnote 8}

Canada's success in negotiating various bilateral and multilateral Free Trade Agreements (FTAs) has driven this evolution. Government action, through TME, including efforts to prioritize, maintain and expand markets and to pursue FTAs with key trading partners helps industry build commercial success.

Market challenges and opportunities are also changing and organizations need to adapt to these changes to remain competitive. Increasingly, non-tariff barriers to trade are creating market challenges (for example, addressing variations of maximum residue limits across markets and risks associated with low-level presence of genetically modified crops). Efforts to reduce uncertainty and increase transparency through the development and promotion of science-based standards, guidelines and regulations, are needed. These complex issues require a delegation of collaborative expertise to intervene and expedite their resolution, a capacity provided by TME.

4.1.2 Alignment of TME with government and departmental priorities

The evaluation examined the extent to which TME activities align with the priorities of the Department and the government as a whole. The evaluation found that the services and products that TME provides are aligned with government and departmental priorities.

Alignment with Government-Wide Priorities TME's objectives and activities, which focus on increasing market access, are aligned with federal government priorities. The 2014 Economic Action Plan and Speech from the Throne emphasized the importance of market access to ensure success in the global economy. The 2014 Economic Action Plan noted that: "We need to ensure that Canadian businesses and investors have the market access they need to succeed in the global economy." It also detailed a commitment to: "…intensify Canada's pursuit of new and deeper trading relationships, particularly with large, dynamic and fast-growing economies."^{Footnote 9} In particular, the 2014 Economic Action Plan made reference to bilateral free trade deals with Korea, Japan and India, and negotiations on the TPP, a market with nearly 800 million people and a combined gross domestic product of $27.8 trillion.^{Footnote 10}

At the sector level, Government priorities with regard to agriculture are reflected in the Growing Forward 2 (GF2) policy framework for the agriculture, agri-food and agri-products sector. The five-year agreement includes investments in strategic initiatives for innovation, competitiveness and market development. The intent is to achieve a profitable, sustainable, competitive and innovative agriculture, agri-food and agri-products industry that is market-responsive, and that anticipates and adapts to changing circumstances and is a major contributor to the well-being of Canadians.^{Footnote 11} TME's activities to support the sector in accessing new and existing markets are consistent with the competitiveness and market development focus of the GF2 policy framework.

Alignment with Departmental Priorities within AAFC's Program Alignment Architecture (PAA), the TME sub-program 1.2.1 falls under Program 1.2: Market Access, Negotiations, Sector Competitiveness, and Assurance Systems. This program is captured in the 2014-15 PAA under Strategic Outcome 1: A competitive and market-oriented agriculture, agri-food and agri-based products sector that proactively manages risk. This Strategic Outcome focuses on Canada's capacity to produce, process and distribute safe, healthy, high-quality and viable agriculture, agri-food and agri-based products and expanding the sector's domestic and global markets.^{Footnote 12} TME activities are focused on expanding the success of Canadian sector enterprises in global markets.

4.1.3 Alignment of TME with the federal government's role

The evaluation found that TME activities are aligned with the federal government's roles in agriculture and in export promotion.

Role in agriculture AAFC's roles and responsibilities are mandated under the Department of Agriculture and Agri-Food Act (1985). As stated in the Act:

The powers, duties and functions of the Minister extend to and include...matters...relating to:

1. agriculture;
2. products derived from agriculture; and,
3. research related to agriculture and products derives from agriculture including the operation of experimental farm stations.

These roles and responsibilities are embedded in Agriculture and Agri-Food Canada's (AAFC) mandate of providing information, research and technology, and policies and programs to achieve an environmentally sustainable agriculture, agri-food and agri-based products sector, and establishing a competitive and innovative products sector that proactively manages risk. AAFC is also responsible to help ensure the agriculture, agri-food and agri-based products industries can compete in domestic and international markets, deriving economic returns to the sector and the Canadian economy as a whole.^{Footnote 13}

TME's objectives are aligned with this mandate as the TME program aims to help the agriculture sector to maintain and expand access export markets, enhance domestic and international competitiveness and provide the tools necessary to seize immediate market opportunities and build commercial success.

Federal agricultural policy is intended to serve national economic and political goals as well as the interests of those directly involved in and affected by Canadian agriculture - primarily the provinces, territories and industry producers. Therefore, it is incumbent on the federal government to engage these key stakeholder groups on an ongoing basis to identify priorities and objectives.

The market development coordination activities of TME provide the environment and framework to foster coordination and collaboration among industry and provinces in the creation of market development strategies and the implementation of joint activities. TME's knowledge transfer includes the creation of practical and pertinent information products as well as the promotion and communication of the latter to Canadian industry stakeholders online via multiple platforms.

Role in Export Promotion The federal government has traditionally played a role in encouraging exports. This has generally included services designed to assist with exports including:

• the provision of market information and intelligence to assist Canadian commercial interests abroad;
• advocacy and market access interventions when practices and regulations constrain Canadian companies' ability to do business abroad;
• support to missions and flagship trade shows; and,
• trade negotiations to improve access to markets and to facilitate a trading environment based on rules agreed to by all participants.

Many foreign governments have a history of supporting similar programs and activities to those of TME. For example, in Australia, Austrade features multiple programs intended to support exports, including in the areas of food and agriculture. Austrade supports export promotion by providing advice to producers and companies that are interested in exporting their products. In Europe, the European Commission provides financial support for campaigns to promote farm products and to inform consumers about how they were produced.^{Footnote 14}

In 2010, the United States launched the National Export Initiative, a whole-of-government plan to double exports in five years, which, in addition to priority trade agreements, has a heavy emphasis on advocacy, business development services and export financing. Emerging markets have also embraced these approaches. As China's companies emerge on the world stage, they are receiving significant political and financial support from Beijing. Composed of activities that lie outside the realm of trade policy—which typically refers to the negotiation and implementation of trade agreements—trade promotion and economic diplomacy are an increasingly important part of states' international economic strategies. These activities include training in conducting business in certain countries, the provision of market information or vetted local contacts, financing or insurance for exports and outward investment, logistical support for and leadership of trade delegations, technical assistance to improve the regulatory environment, advocacy to host governments, and promotion of the country brand.^{Footnote 15}

In summary, TME is relevant. To continue to significantly contribute to the Canadian economy, Canada's agriculture and agri-food sector must, in the face of unrelenting worldwide competition, take advantage of the ongoing growth in international market opportunities. Tangible, related federal government support, collaboration and partnerships with industry are needed and valued by the sector and are aligned with the federal government's and AAFC's mandate and priorities. Leadership and coordination in marshalling a Canadian response to foreign agriculture and agri-food market opportunities, and threats is warranted. Industry respondents agreed that there was an ongoing need for engagement by TME in regard to market access and development issues as well as soliciting input on industry priorities.

4.2 Performance – effectiveness

According to the Treasury Board's 2009 Policy on Evaluation, evaluating performance involves assessing effectiveness, as well as efficiency and economy. The subsections below discuss the effectiveness of TME, the extent to which TME is achieving its expected outcomes.

The TME Performance Measurement Strategy (PMS) lists the expected outputs and outcomes of TME, but the majority of the indicators do not provide targets. The evaluation assesses whether TME met its PMS targets in the cases where targets existed and data was available.

4.2.1 Generation of expected outputs

The evaluation matrix did not include questions related to program outputs. Output data were not systematically collected by TME. Nevertheless, evidence obtained in addressing questions related to expected outcomes revealed a substantial level of output production including:

• production of market research reports in support of internal requests, provinces, trade commissioners and industry;
• production of market intelligence reports to inform decision making within the AAFC as well as building capacity within industry to export and identify market opportunities;
• the generation of some 1,000 trade leads per year for industry stakeholders;
• maintenance of working relations with some 2,000 Agriculture and Food Trade Commissioners Service clients;
• support and leadership for Canadian participation in eight major international flagship trade shows every 20 month cycle (including supporting research reports);
• leadership in supporting Canada's role in the review of various technical issues by, for example, the World Wine Trade Group, the Codex Alimentarius, The the World Organization for Animal Health, and World Trade Organization (WTO) committees on Technical Barriers to Trade;
• participation in numerous other international fora (for example, the North American Free Trade Agreement (NAFTA) Technical Working Group, the Food and Agriculture Organization of the United Nations Biotechnology Forum);
• stakeholder coordination and a pan-Canadian approach to market development activities; and,
• participation in the resolution of critical trade disputes.

According to the program logic model these outputs stemming from TME activities were expected to result in the achievement of specific immediate, intermediate, and end outcomes. The remainder of this section answers evaluation questions addressing the validity of the model and, specifically, the extent to which PMS targets and expected outcomes were achieved.

4.2.2 Achievement of expected immediate outcomes

The program logic model describes three immediate outcomes. For the purpose of the evaluation, the first of these, "engagement in advancing Canadian agriculture market access issues and trade priorities," was folded into the second and third expected immediate outcomes relating, respectively, to engagement with industry and engagement with the provinces. Thus the evaluation addressed the extent to which TME engaged with industry in advancing Canadian agricultural and agri-food exports and with the provinces in advancing Canadian agricultural and agri-food exports.

4.2.2.1 Engagement with industry

Looking first at Canadian agricultural and agri-food industry, the evaluation found that the expected immediate outcome related to engagement was achieved. This was done through two primary avenues. First, TME is responsible for the Agriculture and Food Trade Commissioner Service which is comprised of 33 positions in 20 locations around the world. Together with a larger complement of Trade Commissioners working under Global Affairs Canada, the Agriculture and Food Trade Commissioner Service forms part of a worldwide network of professionals in international business development with preferred access to local business knowledge and contacts, aiming to advance Canada's interests abroad. The Agriculture and Food Trade Commissioners deliver services to help Canadian clients – primarily representing Canada's agricultural and agri-food industry – make better business decisions in order to achieve their market development and export goals abroad.

In terms of commercial achievements, the Agriculture and Food Trade Commissioner Service consistently generated over 1,000 trade leads for industry stakeholders each year of the evaluation period, including some 1,500 in 2013-14.^{Footnote 16} The TME PMS did not include any target for the number of trade leads generated. External stakeholders see the Agriculture and Food Trade Commissioner Service as a valuable service. Interviewees noted that the Agriculture and Food Trade Commissioner Service provides context and key intelligence for industry decision making, provides preferred access to business knowledge and client contacts, generates ongoing trade leads for industry, and, supports and leads trade missions and flagship trade show participation. As one industry respondent stated: "[The Trade Commissioner Service has] been essential – I can't underscore how important they are to us. They've addressed critical market entry problems that could jeopardize an entire [industry], have addressed them quickly and in good faith, and also provided great referrals that have resulted in new business. Their localized knowledge, business contacts, etc. have been very important to us."

Other partnership activities mentioned by industry interviewees were communication during trade negotiations, responses to market access issue through the Market Access Secretariat (MAS) single window service, the Annual Market Access meetings, and market development activities around developing country strategies.

All industry respondents rated the engagement before and during trade negotiations as good. One respondent noted: "On low-level presence of genetically modified flax (CDC Triffid), they've been really helpful to making sure industry can put into place effective solutions; both on the policy side and through trade negotiations. Consultations here have been critical to our success commercially." A large majority of respondents also stated that having MAS as a focal point of inquiry for the industry made sense and facilitated engagement.

Finally, all respondents agreed that country specific export strategies made sense. Those respondents who had experience with International Market Engagement Team meetings saw value in them. However, industry participation was described as spotty and half of those interviewed had not heard of the International Market Engagement Teams. One interviewee stated, "I would like to see expanded opportunities to participate at International Market Engagement Teams. I spoke at one meeting and it was great. Very few industry people know about International Market Engagement Teams and that should be changed."

The second avenue through which TME engages industry is through the provision of targeted research and analysis. Industry interviewees were appreciative of the market research and intelligence reports that were developed in support of Canadian industry food show participants for the eight food shows supported by AAFC. Industry representatives stated that these reports were valuable for follow-up leads and in formulating commercial strategies. TME also produces and distributes market intelligence reports and maintains an on-line priority setting tool. These activities were highlighted as having value for industry on a selective basis. Industry respondents representing large industries felt there was some duplication of TME market intelligence reports by producer associations. Larger associations have the capacity to generate market research reports on their own. However, respondents representing smaller to medium sized associations stated they rely on AAFC for crucial data support. A suggestion from this segment of industry respondents – from both large and small industries – was for AAFC to solicit more industry involvement to inform the focus and content of the market intelligence reports. A suggestion was to add more demographic data as well as general information on the political and economic climates within target markets.

Almost all industry respondents stated a desire to be involved in setting priorities. A majority were also in agreement that the current on-line priority setting tool designed by TME to allow industry to input information, is in need of improvement. The objective behind the tool is good, but industry respondents claimed it is not easy to work with and takes too much of their time to complete.

4.2.2.2 Engagement with the provinces

Turning to the provinces, the evaluation found that the expected immediate outcome related to engagement was achieved. TME engages with the provinces in two ways. The first of these involves direct partnership building. TME organizes a 20-month cycle Canadian participation, including provincial representation, in eight flagship trade shows: Food & Hotel Asia (FHA), Seafood Expo Global (SEG), Salon international de l'alimentation (SIAL) Paris, Allgemeine Nahrungs- und Genussmittel-Ausstellung (ANUGA), Food & Hospitality China (FHC), China Fisheries and Seafood Expo (CFSE), Gulfood, and Foodex-Japan. As the target set in the PMS is 6 to 7 flagship trade shows, the evaluation concludes that TME met its target. Industry respondents were overwhelmingly favorable in their comments around the value and cooperation between levels of government in bringing these events to fruition.

The second way TME engages with the provinces is in the context of policy development and negotiations. AAFC staff spoke of the shared jurisdiction of agriculture between the federal and provincial governments and the efforts undertaken to engage with the provinces. Staff and federal stakeholders identified the following examples of federal market development outreach initiatives with provinces and territories:

• Annual Conference of Federal-Provincial-Territorial Ministers of Agriculture;
• Federal – Provincial Agricultural Trade Policy Committee;
• Federal – Provincial Market Development Council;
• International Market Engagement Team Meetings; and,
• Cost-shared GF2 agreements with provinces.

Some of the value-added activities that take place in these cooperative forums include: analysis in support of multilateral and bilateral negotiations, assisting in development of WTO negotiation strategies; collaboration with MAS; coordination of advocacy efforts against trade irritants, (for example, WTO country-of-origin labelling Case, WTO Korea case on beef, low-level presence of genetically modified flax (CDC Triffid), and black leg fungus found in canola shipments to China); monitoring changes in trade policy and legislation globally; and, FTA strategic planning.

Industry respondents also stated that the federal government was delivering on its obligation to show leadership with federal-provincial outreach in particular in market access issues. However, noting a general trend toward increasing efforts by some provinces to promote their provincial brand abroad, some industry interviewees cautioned that vigilance is required in maintaining a good balance between provincial specific market development activities and pan-Canadian efforts. A specific concern was that this may undermine efforts to promote a pan-Canadian identity. As one industry interviewee noted: "It is important to have a Canadian approach to market development. After all when you are in China, the provinces really don't have the profile. If the [federal government reduces] market development, the provinces will go it alone. We have already seen this at flagship trade shows."

In summary, TME achieved its expected immediate outcomes of engaging with, and supporting industry, and developing and maintaining partnerships with the provinces and territories, with respect to market development initiatives. TME took a leadership role in coordinating key stakeholders representing Canada, and conducting critical background research in support of market access and development. It is important that TME continues to work closely with industry to focus and prioritize market development efforts and support a pan-Canadian identity internationally.

4.2.3 Achievement of expected intermediate outcomes

4.2.3.1 Improved market access for Canadian agriculture and agri-food products

The evaluation found that the expected intermediate outcome related to improved market access for Canadian agriculture and agri-food products was achieved. This is a combination of two of the intermediate outcomes depicted in the program logic model: "increased market access for Canadian agriculture" and "increased Canadian industry entry into markets and expansion of existing markets."

Achievement of this expected outcome was done through two avenues. The first of these involved policy development and negotiations.

All internal stakeholders interviewed stated that TME activities had improved market access for Canadian agriculture. On the FTA side, TME's efforts to open new markets and expand others have reportedly contributed to the economic prosperity of the sector. The value of Canadian agriculture and agri-food exports covered under in-force FTAs increased by a total of 32 per cent ($6.4 billion) between 2009 and 2012. Year over year growth was consistently positive, ranging between one per cent (2009 to 2010) and 15 per cent (2010 to 2011). This growth pattern was roughly at pace with the growth in total Canadian agriculture and agri-food exports (a total 30 per cent increase between 2009 and 2012) of which FTA-covered exports represented more than half the dollar value (55 to 57 per cent) annually across the time period.^{Footnote 17} The target set in the PMS for December 31, 2017 is 75 per cent.

Industry respondents recognized that their situation was helped by TME's efforts in trade negotiations and in dealing with trade export irritants. For example, TME's efforts have benefited the canola industry. As noted by an industry representative: "Canada exports 90 per cent of its canola. We work alot with TME on trade negotiations and trade policy. We worked closely with them on the TPP, Korea, and Europe agreements. We got rid of tariffs to Korea through the agreement and expect to double exports as a result. TME staffs were competent and engaged with us, and these agreements will have significant benefits for the canola industry."

In terms of progress made in offensive and defensive trade irritants and disputes, the 2013 AAFC Departmental Performance Report stated that TME's performance measurement strategy targeted the percentage of trade irritants where progress is made at 75 per cent by March 31, 2013. Progress was made or resolution was achieved in 83 per cent of Canadian agriculture and agri-food market access issues in 2012-13.

AAFC respondents also noted TME's efforts to resolve market access issues as a key contributor to the overall success of Canadian agricultural exports. Fifty-one market access issues were successfully resolved by MAS in 2012-13, and 33 were resolved in 2013-14. The Market Access Support System database documents these in detail^{Footnote 18}, but no target was set in the PMS for the number of market issues resolved. One example of the successful resolution of a market access issue stemmed from an announcement by the Government of Vietnam that it intended to change its stance on maximum residue limits of ractopamine, a substance used by Canadian producers in the raising of livestock that can remain at trace levels in export products. Vietnam's decision could have had a negative impact on Canadian sales to Vietnam. TME officials negotiated with Vietnam over a 13-month period resulting in Vietnam reversing its decision and accepting the international standard to which Canadian producers already adhered, thus assuring no interruption in Canadian exports to Vietnam.

An example of long-term, multi-disciplinary TME efforts to regain a significant market for Canadian producers is the case of the country-of-origin labelling requirements in the US.  When implemented in 2008, these requirements caused significant price and quantity decreases for Canadian exports of cattle and hogs. TME officials were involved in the WTO dispute settlement process initiated by Canada against this measure. The WTO ruled in Canada's favour four times between 2011 and 2015, finding that the US country of origin labelling measure discriminates against Canadian cattle and hogs. Canada requested authorization to retaliate against US exports, as the US has failed to remove the discriminatory elements of country of origin labelling legislation. TME officials have engaged in extensive advocacy work in the US in an effort to regain this important market for Canadian livestock producers. On December 7, 2015, the WTO ruled that Canada can impose retaliatory surtaxes on US exports to Canada as a result of the economic harm caused by the US country-of-origin labelling policy, once final WTO authorization is obtained. The US cannot appeal the ruling. On December 21, 2015, Canada obtained WTO authorization to impose retaliatory surtaxes on US exports to Canada. The US will no longer enforce the country-of-origin labelling requirements for beef and pork products because country-of-origin labelling was repealed by the US. This is the final technical step in the WTO dispute settlement process.

The second way that TME supported the achievement of improved market access for Canadian agriculture and agri-food products was through engagement with partners. TME was involved in ongoing consultations with industry both through formal vehicles and informally, especially during emerging crises in market access.

The CDC Triffid Flax case is an example of a successful coordinated multi-disciplinary approach to resolving a market access issue. The situation began in the summer of 2009 with the detection of low-level presence of genetically modified flax (CDC Triffid) in Canadian shipments to the EU. The sampling and testing protocol that was developed in response is still in place today and the file remains active for the Government of Canada.

The case was complex, multifaceted and touched every producer, processor and exporter of flax in Canada. It encompassed trade and market access issues, science, logistics, diplomacy, law, AAFC program support and featured a great deal of coordination and communication across federal departments, between industry and government, as well as internationally (that is, with officials from the EU, Japan and to a lesser extent Brazil). In addition to TME, the key stakeholders included:

• The flax industry (for example, The Flax Council of Canada, which represented Canadian flaxseed producers, processors, exporters, and a few large individual companies);
• The Canadian Grain Commission (which is the federal authority responsible for establishing and maintaining Canada's grain quality standards and certifying export shipments);
• CFIA (under Canada's Seeds Act, CFIA is responsible for administering Canada's seed regulatory framework);
• Other groups within AAFC (for example, SED, Programs Branch, Legal Counsel, and senior management); and,
• Global Affairs Canada representatives at diplomatic posts (notably Canada's agricultural trade representative in Brussels).

The stakes for Canada's flax industry were high. According to interviewees, the crisis could have devastated the industry to the point of near total loss. As one industry representative noted: "Once people stop growing flax, are you ever going to get them back?" The combined efforts of industry, AAFC and other federal agency partners led to a strategy to remove the genetically modified material from the system and regain key export markets.

A key part of the TME organization involved in engagement with partners is the Trade Commissioner Service. The Agriculture and Food Trade Commissioner Service helped improve market access for Canadian producers by both identifying and facilitating commercial opportunities for industry as well as supporting the trade and negotiation team. Even if no target was set in the PMS for the number of Trade Commissioner Service clients, the evaluation found that the number of Agriculture and Food Trade Commissioner Service clients rose from 1,681 in 2009 to 2,084 in 2012, an increase of 24 per cent. One key interviewee representing a large association noted: "We get lots of cooperation from the Trade Commissioner Service. The Agriculture and Food Trade Commissioner Service can be strong advocates for beef exports. We have good relationships with the Department. Trade associations need to be running themselves as businesses and the Agriculture and Food Trade Commissioner Service can save money. They help us manage our reputation."

The Agriculture and Food Trade Commissioner Service provides strategic communication that can also help Canada's negotiating team. This type of internal partnership is valuable. Federal stakeholders described the following collaborative efforts by the Agriculture and Food Trade Commissioner Service that help negotiators be better advocates for Canadian industry:

• market specific intelligence including the political climate;
• refine policy positions of negotiators;
• provide intelligence on the domestic market of our trading partners; and,
• provide key intelligence on trading partners with respect to possible trading of market access issues that may lead to a successful trade agreement.

4.2.3.2 Advancement of Canadian interests bilaterally and multilaterally

The evaluation found that progress was made toward the achievement of the expected intermediate outcome related to advancing Canadian interests bilaterally and multilaterally through trade agreements. As international conditions change, this objective will always be in play, and there will always be more to achieve.

During the evaluation period (that is, 2009-10 to 2013-14), AAFC, in concert with CFIA and Global Affairs Canada, brought six FTAs into force. Additionally, the Canada-EU Comprehensive Economic and Trade Agreement negotiations concluded in August 2014 and the Canada-Korea FTA was signed in September 2014. Canada also moved ahead with TPP. Canada joined the negotiations in October 2012 and an agreement was reached in October 2015. Canadian participation in the Trans-Pacific Partnership (TPP) has the potential to significantly improve Canada's access to Asian markets, encouraging innovation and stimulating the sector's productivity. TME was the Department's primary actor in these efforts.

As a result of these accomplishments over half of Canada's agricultural exports are now covered by FTAs. Despite these successes, another third of world markets remains open for negotiated trade agreements, according to interviewees. Negotiating bilateral and multilateral FTAs is a complex process which can take many years to bring to fruition. For example, Canada and Korea began discussing the establishment of a FTA in 2005 and held 14 rounds of bargaining sessions before finalizing the agreement. The Comprehensive Economic and Trade Agreement negotiations lasted 5 years.

Canadian negotiators have succeeded in securing agreements that are in the best interests of the Canadian agricultural sector while the Trade Negotiations and Market Access teams ensured a strategic approach to international trade disputes. Defining Canada's interests emerges from both an internal and external consultation process. The Government of Canada defines priority countries in an effort to establish markets with the largest overall return on trade investments. AAFC then consults with industry to define sector strengths and weaknesses.

AAFC has taken an active role in various international fora to reduce the number of technical trade issues in order to ensure market access for Canadian agriculture and agri-food products. TME's research and analysis activities have supported AAFC in becoming a contributor to the development of policy positions regarding several technical issues being reviewed by international organizations and committees such as the World Wine Trade Group, Codex Alimentarius, the International Plant Protection Convention and the World Organization for Animal Health, as well as the WTO Committees on Technical Barriers to Trade and Sanitary and Phytosanitary Measures.

There are many examples where TME has actively sought to increase predictability for importers and exporters, and minimize the potential for trade disruptions due to non-tariff (technical) barriers to trade. Actions have included working with industry to address issues such as variations in maximum residue limits across markets and risks associated with low-level presence of genetically modified flax (CDC Triffid). TME has worked to ensure the effective and consistent implementation of WTO Agreements, including through dispute settlement mechanisms, to maintain export opportunities for Canadian agriculture (for example, as in the case with the US country-of-origin labeling legislation^{Footnote 19} In July 2011, Canada secured an extension to the transitional trade measures granted by the Chinese in 2010 related to blackleg fungus. This provided continued access for exporters of Canadian canola seed crop. In 2011, Canada exported $1.6 billion in canola products to China.^{Footnote 20}

4.2.3.3 Increased recognition of science-based rules

The evaluation found that progress was made toward the achievement of the expected intermediate outcome related to increased international recognition, adoption and enforcement of science-based trade rules. As with the previously discussed outcome, as international conditions change, this objective will always be in play, and there will always be more to achieve.

TME has contributed to progress towards resolutions of trade irritants such as those relating to variations in pesticide maximum residue limits across markets, as well as those related to risks associated with low-level presence of genetically modified crops. Interviewees cited Canada's work in developing its own low level presence policy and framework as a model for international engagement. The objective of the low level presence policy and framework was to provide transparency and predictability for importers and exporters and minimize disruptions to trade while protecting the health and safety of people, animals and the environment. It is also intended to facilitate an efficient risk-based approach to manage an expected increase in occurrences of low level presence of genetically modified crops in international trade while promoting compliance with Canadian regulatory requirements.^{Footnote 21} Canada was cited as "instrumental in establishing world standards through the Global low level presence Initiative and through the Biotech Forum of the Food and Agriculture Organization of the United Nations." TME officials led in these efforts.

TME contributed towards the resolution of trade irritants involving maximum residue limits for pesticides. For example, through the NAFTA Technical Working Group, the US, Mexico, and Canada work with stakeholders to ensure national pesticide standards are met. One of the goals of the NAFTA Technical Working Group is developing a coordinated pesticide regulatory framework among NAFTA partners. The US, Canada and Mexico, and many other countries set regulatory enforcement levels for pesticide residues on food commodities. These limits help ensure that pesticides are used in accordance with the registered label.^{Footnote 22} TME's efforts contributed to the NAFTA Technical Working Group Food Residues Trade Irritant Resolution.

AAFC and other federal stakeholders commented on the potential large dollar value involved in resolving market access issues. A successful resolution of a market access issue can be the difference between commercial success and business failure for producers. Citing the CDC Triffid flax case as an example, one AAFC interviewee noted, "we have pushed the EU towards more science-based, clear rules for trade. TME advocates to the European Commission while others will make the regulatory recommendations." Interview respondents were in agreement that trade concerns are increasingly technical, science-based, and related to regulatory issues that are beyond the scope of industry to manage on their own.

In summary, TME achieved or made progress toward its expected intermediate outcomes. TME played a significant role in opening and expanding markets, the resolution of critical trade irritants and disputes, and in the generation of more internationally-accepted, science-based trade rules.

4.2.4 Achievement of expected end outcomes

4.2.4.1 Expansion of global market opportunities for Canadian agriculture and agri-food products and success on the part of exporters in seizing market opportunities

The evaluation found that progress has been made toward the expected end outcome related to expansion of global market opportunities and Canadian industry taking advantage of that expansion. Evidence for this finding comes from an examination of export trends.

Table 3 shows that over the five year period from 2009-10 to 2013-14 Canadian agri-food and seafood^{Footnote 23} exports to both OECD countries and non-OECD countries combined rose by 43 per cent^{Footnote 24} Over the four year period^{Footnote 25} from 2009-10 to 2012-13 Canadian agri-food and seafood exports to OECD countries increased each year. There was a 25 per cent increase, to $35.7 billion, in exports to OECD countries in 2012-13 compared with 2009-10.

The 2012-13 figure for OECD countries is close to AAFC's stated target for 2017: $36.1 billion. Over this same period Canadian agri-food and seafood exports to non-OECD countries increased each year to $14.7 billion, 36 per cent in 2012-13 compared with 2009-10. AAFC's stated 2017 target for exports to non-OECD countries was $14.7 billion. The evidence is not sufficient for the evaluation to comment on the extent to which these accomplishments can be attributed to TME.

Table 3: Canadian agri-food and seafood exports to OECD and non-OECD markets, in Canadian dollars ($)^{Footnote 25}

Fiscal Year	2009-10	2010-11	2011-12	2012-13	2013-14
N/A: not available Source: Statistics Canada, CATSNET Analytics
34 OECD Countries	28,564,287,236	32,723,609,357	33,579,949,869	35,727,939,861	N/A
Non-OECD Countries	10,786,984,827	11,674,155,152	14,143,550,874	14,673,159,556	N/A
Total	39,351,272,063	44,397,764,509	47,723,500,743	50,401,099,417	N/A

In summary, progress was made toward the TME expected end outcome. Over the five year period from 2009-10 to 2013-14 Canadian agri-food and seafood exports rose by 43 per cent. Greater growth occurred in non-OECD countries compared to OECD countries. By 2012-13, AAFC's 2017 targets were met in the case of non-OECD countries and nearly met in the case of OECD countries.

4.3 Performance – efficiency and economy

4.3.1 TME economy

According to the 2009 Treasury Board Policy on Evaluation, economy is said to have been achieved when a program's activities and expected outputs have been delivered for the lowest possible cost, taking into consideration the program's context as well as requirements respecting output quality. A comprehensive, empirical assessment of TME's economy would require a detailed study of costs-per-outputs associated with comparison programs (for example, in other countries) and/or a business process mapping exercise. Both of these methodological approaches were beyond the scope of the evaluation. The evaluation is thus limited to a descriptive approach, reinforced by interview data.

Over the evaluation period, TME's annual expenditures were $24.1 million in 2009-10, rose to $32 million in 2011-12, and dropped to $24.2 million in 2013-14. The expenditures covered primarily TME staff salaries; TME had 187 Full-Time Equivalents in 2009-10, 213 in 2011-12, and 157 in 2013-14. Outputs generated for these expenditures included market research and intelligence reports. TME managed the Agriculture and Food Trade Commissioner Service, provided support and leadership to Canada's participation to flagship trade shows and to the review of various technical issues. TME was heavily involved in the generation of trade leads, the resolution of critical trade disputes and the coordination of pan-Canadian approaches to market development activities. However, costs-per-outputs are not possible to calculate.

Anecdotal evidence from interviewees suggests that the high level of experience and expertise on the part of TME officials helps enable TME to respond to regular and ad hoc requirements quickly and effectively. This capacity is bolstered by the relationships TME officials maintain with partners in Canada as well as with counterparts overseas allowing quick access to key stakeholders. These observations suggest the potential for an economical operation.

4.3.2 TME efficiency

According to the 2009 Treasury Board Policy on Evaluation, program efficiency is a measure of the achievement of expected outcomes in relation to resources expended. For programs like TME, whose suggested benefits can mostly be described in monetary terms (for example, value of exports), cost-benefit analysis is the most appropriate vehicle for the assessment of program efficiency. A comprehensive cost benefit analysis was beyond the scope of the evaluation; the necessary range of outcome data was not available and, even where data were available, an assessment of the relative contribution of TME to specific outcomes was not possible. The evaluation is thus limited to a descriptive approach drawing on macro-level outcome data plus anecdotal evidence, reinforced by interview data. The available evidence suggests efficiency, although definitive conclusions in this regard cannot be drawn.

Macro-level Outcomes As highlighted in Table 4, below, Canadian agri-food and seafood exports grew by some 43 per cent between 2009-10 and 2013-14, from $39 billion to $56 billion. TME expenditures over the same period were generally stable with some yearly fluctuations. From 2009-10 to 2013-14, TME expenditures totaled just over $136 million while total sector exports over the same period were in excess of $238 billion. Over the five year period covered by this evaluation, the exports to expenditures ratio corresponding to TME was 1,748:1. In other words, TME's expenditures represented 0.057 per cent of total sector exports.

Table 4: Canadian agri-food and seafood exports and TME expenditures, 2009-10 to 2013-14^{Footnote 26}

Fiscal Year	Sector Exports	Exports Growth	TME Expenditures	Exports to Expenditures Ratio
2009-10	$39,351,272,063	N/A	$24,184,656	1,627:1
2010-11	$44,397,764,509	12.8 per cent	$27,180,916	1,633:1
2011-12	$47,723,430,102	7.5 per cent	$32,074,016	1,488:1
2012-13	$50,401,191,043	5.6 per cent	$28,654,068	1,759:1
2013-14	$56,451,463,404	12.0 per cent	$24,280,848	2,325:1
Total	$238,325,121,121	43.4 per cent	$136,374,504	1,748:1

Given the size and complexity of the agriculture and agri-food sector, and given the influence of a wide range of extraneous variables on the sector, attributing export sales increases or decreases directly to the TME program is difficult. If TME activities and outputs led to improvements in the revenue generating capacity of the sector in excess of 0.057 per cent, or if the program contributed to maintaining or mitigating the loss of market shares as a result of external events by the same amount, TME would have to be considered efficient.

Interviewees who commented on the question of efficiency believed this to be the case. AAFC staff interviewees felt that the TME program provided significant value compared to program costs. Export opportunities were cited as the biggest need addressed by TME. Respondents from all interviewee sub-groups stated that Canada's multilateral and bilateral negotiations have yielded demonstrably positive results for many Canadian industries. Respondents generally felt that the relatively small costs of operating TME were outweighed by the large dollar amounts generated by the sector.

Representatives of industry interviewed for the evaluation were also consistent in attributing some of their commercial success to TME. Most respondents had difficulty assigning a dollar amount and the value attributed to TME varied among producers. Representative quotes include:

• "On the market access side, it's huge. [Canada's] canola industry exports somewhere in the range of $2 billion per year between our market access to China, the US, and the EU. TME has been instrumental in opening these markets and keeping them open for us. Also, for us to be able to use biotechnology in canola leads to about $800 to $900 million per year in exports. TME helped us to get our biotechnology canola accepted in foreign markets. TME does government-to-government work that councils like ours couldn't do."
• "The Canada Beef return on investment is about 6:1. I would say TME activities are about 50 per cent of that value creation."
• "We can attribute 10 per cent of export sales to Canada Pork International activities. This amounts to approximately $300 million and clearly some of this can be attributed to TME activities. This finding came from our own Canadian Pork International evaluations."

In summary, although definitive empirical evidence was not available, it appears that TME is economical and efficient. Anecdotal evidence suggests that TME operates economically, and there was no evidence found to the contrary. The value of Canada's agriculture and agri-food industry is very large in relation to TME's annual cost; TME would need to have relatively little incremental impact to pay for itself. In fact, the volume and quality of TME outputs combined with qualitative evidence pointing to the positive effects stemming from what are seen as important TME contributions, suggests a positive financial impact.

Design and delivery

The evaluation question associated with design and delivery asks whether or not the TME program was delivered as intended. The evaluation assesses design and delivery with a particular focus on TME's integration and cooperation with delivery partners. The evaluation also examined TME's robustness and sustainability, management structure and processes, and performance monitoring and reporting.

4.4.1 Management structures and processes

The evaluation found TME to be adequately structured. TME is managed by MAS and TAND within MISB. A number of TME initiatives funded through GF2, including AAFC Trade Commissioners, CFIA's assistance in building relationships with regulatory counterparts, and market access and market development services, require accountability to GF2 as a component of TME's governance. Global Affairs Canada provides administrative support to AAFC Trade Commissioners through funds transferred from AAFC under an interdepartmental Memorandum of Understanding on Operations and Support at Missions. Global Affairs Canada's role in this context, as mandated by Treasury Board, is to manage support services used by partner departments such as AAFC at Canada's diplomatic and consular missions. As TME manages the Agriculture and Food Trade Commissioner Service, this linkage with Global Affairs Canada represents another component of TME's governance. In addition, CFIA supports market access activities through funds transferred from AAFC under a Memorandum of Understanding. CFIA's main role in this regard is to demonstrate to other countries the effectiveness of Canada's plant pests, animal health, and food safety controls in order to help TME officials to negotiate the export certificates and equivalency agreements that countries require for trade in agricultural products.^{Footnote 27}

When asked about the governance of the TME program, key interviewees' answers were consistent; the management and governance of TME is perceived to be sufficient. The accountabilities and decision making processes were clear and well understood. The majority of internal AAFC respondents suggested the TME program is well designed to meet its stated objectives, with a straightforward mandate and well-articulated objectives.

TME and SED produce market data, intelligence and analyses reflecting different perspectives and audiences. TME's reports aim to support trade and market expansion internationally while SED's reports aim to increase knowledge, competitiveness and adaptability within the sector. Evidence suggests coordination between both programs currently exists that avoids duplication of efforts. To maintain the effectiveness and efficiency of TME, continued coordination with SED is necessary to avoid duplication of future efforts.

4.4.2 Integration and cooperation with delivery partners

The evaluation found TME to be making progress toward effective integration and cooperation with its delivery partners.

The Government of Canada's approach to increasing market access for agriculture and agri-food export captures the collective efforts of the federal Market Access Secretariat Team members: AAFC, CFIA, and Global Affairs Canada. MAS is designed to improve the level of coordination among key stakeholders and build more effective consultation mechanisms and partnership agreements with federal partners, industry, provinces and territories.^{Footnote 28}

When asked to state areas for improvement, a majority of federal government interviewees noted challenges in relation to the integration and cooperation with CFIA and Global Affairs Canada, highlighting the potential for different opinions and territoriality over issues, and possible missed opportunities to partner. The alignment of CFIA and AAFC mandates are not evident: CFIA has a regulatory and safety mandate while AAFC has commercial priorities. While CFIA is the technical expert driving the enforcement of clear science-based trade rules, AAFC aims to improve market access and expand market opportunities. Those interviewed suggested that the linkages between both departments are not fully understood. Some interviewees felt there was a lack of clear understanding and some inherent challenges due to the crossover responsibilities between departments. This lack of alignment was heightened with a 2013 change in oversight duties; CFIA was formerly under the Minister of Agriculture, and it is now under the Minister of Health.^{Footnote 29}

Global Affairs Canada has a much broader trade mandate than that of AAFC which opens up possibilities of challenges with policy cohesion. As TME focuses solely on agriculture and agri-food, Global Affairs Canada's mandate is to manage all of Canada's diplomatic and consular relations, to encourage the country's international trade and to lead Canada's international development and humanitarian assistance.

Almost all government interviewees predicted that in the future, Canada's response to trade challenges will need to be evermore robust and be built collaboratively by key partners. The differing mandates of CFIA and Global Affairs Canada in relation to that of AAFC highlight the need for TME to evolve and further define the support and cooperation it currently has with its key federal partners.

4.4.3 TME robustness and sustainability

Almost all internal interviewees underscored that the ability to achieve program objectives was negatively affected by resource limitations. Specifically, they felt that TME lacked resources in the science and technical area and that TME required more capacity to effectively partner with other agencies to resolve market access issues. A federal government respondent stated: "TME [is] overtaxed in terms of technical analysis capacity. This is a sore point within our department. Industry is feeling it and they require more timely responses to their concerns. [Global Affairs Canada] is rolling forward negotiating more and more trade agreements but the resources to support these agreements, that is, within TME, are insufficient. Almost every trade issue has a technical and regulatory component to it so we need to address the capacity gaps so we can all better deliver on our objectives."

4.4.4 Performance monitoring and reporting

A performance measurement strategy for the TME Program was produced in 2014. TME also monitors data related to outcomes such as volume and value of exports. The program's ability to establish an empirical link between its activities and outcomes is limited. There were also specific data gaps associated with a number of key objectives and their associated indicators. For example, market issue resolution data was not captured before 2011-12 and only the 2012-13 and 2013-14 data were useful for the evaluation. "Progress made on market issues" is noted as an indicator for immediate and intermediate outcomes in the TME PMS, but no specific data sources were found in support of this indicator. Neither were any relevant data sources found in support of the indicator "increase in value of Canadian agriculture and agri-food exports through market access interventions leading to resolution of trade irritants."

Several internal TME units have a performance measurement framework in place and collect related data. While interviewees did refer to using such information for accountability reporting, almost all had difficulty identifying ways in which it was actually used for decision making. Some performance indicators for 2013-14 have been revised to accommodate the contributions that MAS' information management tool will bring to measuring progress on AAFC's market access goals. Improvements in the articulation of performance measures for TME should create better synergies between outputs and outcomes. Enhancing market access interventions indicators that lead to resolution of trade irritants would help the determination of the causal relationship between TME's work and the value of Canada's agricultural and agri-food product exports. These suggested changes would contribute to an ability to perform an assessment of program efficiency in the future.

In summary, TME is effectively structured and has generally good working relations with its federal partners. Given the increasingly multidisciplinary nature and complexities of trade, AAFC's partnerships with, in particular, Global Affairs Canada and CFIA, must remain robust and effective; to this end there is room for TME to work toward achieving a clear understanding of its role when working with Global Affairs Canada and CFIA. While progress has been made in the area of performance monitoring, there may be room for a more rigorous approach to identifying, collecting, and using empirical performance data. In this regard, the existing TME performance measurement strategy could be refined.

5.0 Evaluation conclusions

This section of the report presents conclusions based on the findings outlined in the previous sections. The information is structured along the main evaluation issues.

5.1 Relevance

Trade and Market Expansion (TME) is relevant and well aligned with government and departmental priorities and roles. To continue to contribute to the Canadian economy, Canada's agriculture and agri-food sector must, in the face of unrelenting worldwide competition, take advantage of the ongoing growth in international market opportunities. TME provides tangible support to the sector, and engages with industry and provinces and territories (PTs) to expand market access. TME's leadership and coordination in marshalling a Canadian response to agriculture and agri-food market export opportunities and threats is warranted.

5.2 Performance – effectiveness

TME has met all three immediate outcomes, including:

• Engagement in advancing Canadian agriculture market access issues and trade priorities;
• Engagement with industry for new and expanded global market opportunities; and,
• Cooperation between federal and provincial governments in the advancement of Canadian agriculture.

TME engaged effectively with, and supported, industry. TME developed and maintained partnerships with the provinces and territories with respect to market development initiatives. TME took a leadership role in terms of coordination among key stakeholders that represent Canada's agriculture sector, and background research in support of market access and development.

By engaging with industry and provinces, TME increased recognition, adoption and enforcement of clear, transparent, science-based trade rules, and increased market access for Canadian agriculture. As a result of TME's involvement in the sector, the following intermediate outcomes have been met:

• Increased recognition, adoption and enforcement of clear, transparent, science-based trade rules; and
• Increased market access for Canadian agriculture.

In addition, progress was made towards the following two intermediate outcomes:

• New bilateral and multilateral trade agreements with key trading partners; and,
• Increased Canadian industry entry into new markets and expansion of existing markets.

As international conditions change, these last two outcomes will always be in play and there will always be more to achieve.

Despite the challenges to attribute macro-level performance directly to TME activities, the evaluation found that TME contributed to its end outcome which is "Canadian agriculture and agri-food products exporters are successful in seizing market opportunities and in achieving commercial success". Over the five year period from 2009-10 to 2013-14 Canadian agri-food and seafood exports rose by 43 per cent. Greater growth of Canadian exports occurred in non-Organisation for Economic Co-operation and Development (OECD) countries compared to OECD countries.

5.3 Performance – efficiency and economy

Although definitive empirical evidence was not available, anecdotal evidence suggests that TME operates economically, and there was no evidence found to the contrary. The value of Canada's agriculture and agri-food industry is large in relation to TME's annual cost; the volume and quality of TME outputs combined with qualitative evidence pointing to the positive effects stemming from TME contributions suggests a positive financial impact, that is, efficiency.

5.4 Design and delivery

TME is effectively structured and has generally good working relations with its federal partners, including Sector Engagement and Development (SED). Given the increasingly multidisciplinary nature and complexities of trade, AAFC's partnerships with, in particular, Global Affairs Canada and Canadian Food Inspection Agency (CFIA), must remain robust and effective; to this end there is room for TME to work toward achieving a clear understanding of its role when working with Global Affairs Canada and CFIA. While progress has been made in the area of performance monitoring, there may be room for a more rigorous approach to identifying, collecting, and using empirical performance data. In this regard, the existing TME performance measurement strategy could be refined.

6.0 Issues and recommendations

The evaluation includes the following issues and recommendations:

Issue 1

The Government of Canada's approach to increasing market access for agriculture and agri-food export captures the collective efforts of the federal Market Access Secretariat Team members: Agriculture and Agri-Food Canada (AAFC), Canadian Food Inspection Agency (CFIA), and Global Affairs Canada. The evaluation noted challenges in relation to TME's integration and cooperation with CFIA and Global Affairs Canada.

Recommendation

Given the high level of coordination needed between AAFC, CFIA and Global Affairs Canada, AAFC's partnerships must remain robust and effective; to this end there is room for TME to work in support of greater role and goal clarity. The Market and Industry Services Branch, in consultation with its delivery partners, needs to find measures to increase the effectiveness of its integration and cooperation with its delivery partners.

Management response and action plan

Agreed: As part of the refreshed International Strategy, the Market and Industry Services Branch will undertake a review of coordination and collaboration mechanisms in place with Global Affairs Canada and CFIA with a view to assessing its ongoing relevance and effectiveness by the end of the next fiscal year (that is 2016-17).

The Market and Industry Services Branch will also undertake a review of all terms of reference for governance committees/bodies in place with Global Affairs Canada and CFIA in support of the Trade and Market Expansion (TME) with a view to maximizing the effectiveness of the existing governance structure by the end of the next fiscal year (that is 2016-17). Appropriate changes will be made when necessary.

Target date for completion

March 31, 2017.

Responsible positions

• Assistant Deputy Minister, Market and Industry Services Branch;
• Director General, Trade Agreements and Negotiations Directorate, Market and Industry Services Branch; and,
• Director General, Market Access Secretariat, Market and Industry Services Branch.

Issue 2

A performance measurement strategy (PMS) for the TME Program was produced in 2014. TME monitors data related to outcomes such as volume and value of exports. The program's ability to establish an empirical link between its activities and outcomes is limited. There were specific data gaps associated with a number of key objectives and their associated indicators.

Recommendation

Although progress has been made, the evaluation found weaknesses in performance monitoring and reporting. It is recommended that the Market and Industry Services Branch refines TME's current performance measurement strategy to ensure that meaningful performance measures are developed. These measures include indicators for program activities, outputs and outcomes so that future, more robust assessments of program effectiveness, efficiency and economy can be undertaken.

Management response and action plan

Agreed: The Market and Industry Services Branch will undertake a review of the TME's PMS with a view to refining its logic model and performance measures to ensure that performance is effectively monitored and reported. Updated PMS will be approved by MISB senior management by the end of the fiscal year (2015-16).

Target date for completion

March 31, 2016.

Responsible positions

• Assistant Deputy Minister, Market and Industry Services Branch;
• Director General, Trade Agreements and Negotiations Directorate, Market and Industry Services Branch; and,
• Director General, Market Access Secretariat, Market and Industry Services Branch.

Annex A: Evaluation matrix

Relevance

Continued need for the initiative

Evaluation question: Within the Canadian agriculture, agri-food and agri-based products sector, what are the needs addressed by the Trade and Market Expansion (TME) program? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which Office of Audit and Evaluation (OAE) will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities. TME: Trade and Market Expansion
Demonstrated or expressed need for TME on the part of industry representatives and other targeted population segments (C)[c]	Yes	Yes	Yes	Yes	Yes
Presence/absence of other programs that complement or duplicate TME	Yes	No	No	Yes	No
Gaps would exist in addressing societal / agriculture sector need in absence of the program	Yes	No	No	Yes	Yes

Evaluation question: Is there a continued need for the TME program? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. TME: Trade and Market Expansion
Views on the extent to which the TME program is still necessary	No	Yes	No	Yes	Yes

Evaluation question: Have the needs changed, and if so, has the program evolved accordingly? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. TME: Trade and Market Expansion
Demonstrated or expressed changes in need	Yes	Yes	Yes	Yes	Yes
Views on the extent to which TME has evolved to meet changes in need	No	No	No	Yes	Yes

Evaluation question: Within the sector, which population segments are targeted by the program, and is this the appropriate target group? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities.
Identification of population segments targeted (C)[c]	Yes	No	No	Yes	No
Views on the extent to which this is the appropriate target group	No	No	No	Yes	Yes

Alignment with government priorities

Evaluation questions: How is the TME program aligned with federal government and Agriculture and Agri-Food Canada (AAFC) priorities and objectives? Have priorities/objectives changed and if so, has the program evolved accordingly? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised.
Program's objectives correspond to recent/current federal government priorities/objectives as defined in the federal budget, the speech from the throne, departmental strategic outcomes, and/or other relevant documents	Yes	No	No	No	No
Views on the alignment of program objectives to recent/current federal government and departmental priorities	No	No	No	Yes	No

Alignment with federal roles and responsibilities

Evaluation question: What is the nature of the federal government's role and mandate to deliver the TME program, and to what extent do the objectives of the TME program align with this role/mandate? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. TME: Trade and Market Expansion
TME objectives aligned with federal government role and responsibilities	Yes	No	No	No	No
Views on the appropriateness of federal involvement	No	No	No	Yes	Yes

Evaluation questions: What would be the impact on the sector if the TME program was not in place? Could some TME activities be shared with, or delivered by, industry? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. TME: Trade and Market Expansion
Views on whether some TME activities could be shared with, or delivered by, industry	No	Yes	No	Yes	Yes

Performance: Achievement of expected outcomes

Achievement of immediate outcomes

Evaluation question: To what extent has industry been engaged to identify new and to expand global market opportunities? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. N's in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities.
Views on extent to which industry has been engaged in exploring new and expanded global market opportunities (A)[c]	No	No	No	Yes	Yes
Number of trade leads uncovered by Trade Commissioners	Yes	No	No	No	No

Evaluation question: To what extent has there been cooperation between the federal and provincial levels in the advancement of Canadian agriculture? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities. PT: Province and Territory
Number of flagship trade shows with federal and provincial participation	Yes	No	No	No	No
Views on level of federal-provincial cooperation in support of market development by industry (A)[c]	No	No	No	Yes	Yes
Number of federal market development outreach initiatives with PTs	Yes	No	No	No	No

Achievement of intermediate outcomes

Evaluation questions: To what extent has market access improved for Canadian agriculture and agri-food? To what extent has Canadian industry entry into new markets and expansion of existing markets increased? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities.
Number of market access issues resolved (A)[c]	Yes	No	No	Yes	Yes
Number and duration of market access issues currently being worked on (context of complexity)	Yes	No	No	Yes	No
Views on progress on market access issues in negotiations (A)[c]	No	No	No	Yes	Yes
Increase in value of Canadian agriculture and agri-food exports over the course of the period covered by the evaluation (A)[c]	Yes	No	No	Yes	Yes
Increase in the number of trade commissioner clients	Yes	No	No	No	No
Number of Trade Commissioner clients identifying a commercial transaction as a result of Trade Commissioner services	Yes	No	No	No	No
Level of adoption of Canada Brand in differentiating and promoting Canadian agricultural and agri-food products (C)[c]	Yes	No	No	No	Yes
Percentage (%) of Canadian agriculture and agri-food exports covered by free trade agreements (FTAs)	Yes	No	No	No	No

Evaluation questions: To what extent have Canadian interests been advanced and defended in bilateral and multilateral negotiations and fora? To what extent have new bilateral and multilateral trade agreements been concluded with key trading partners? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities. FTA: Free Trade Agreement
Views/documented evidence on extent to which Canadian interests have been advanced and defended (C, A)[c]	Yes	Yes	Yes	Yes	Yes
Extent to which negotiated trade agreements reflect Canada’s negotiation position (C, A)[c]	Yes	No	No	Yes	Yes
Number of new trade agreements concluded with key trading partners over the course of the period covered by the evaluation (A)[c]	Yes	No	No	Yes	Yes
Percentage (%) of Canadian agriculture and agri-food exports by value covered by FTAs	Yes	No	No	No	No

Evaluation question: To what extent has there been increased recognition, adoption and enforcement of clear, transparent, science-based trade rules? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. [c] A "C" or an "A" identifies the indicator as one for which the corresponding interview question will include a focus on confirmation and/or attribution respectively. Confirmation questions will be asked of interviewees when OAE has another source of data – usually documentary or administrative data – and OAE wishes to confirm the validity of the data. Attribution questions will elicit opinions on the link between an outcome and program activities.
Increase in value of Canadian agriculture and agri-food exports through market access interventions leading to resolution of trade irritants (C, A)[c]	Yes	No	No	Yes	Yes
Progress made (advances/resolutions) in offensive and defensive trade challenges (for example, low level presence, international trade disputes, maximum residue limits, C, A)[c]	Yes	No	No	Yes	Yes

Achievement of ultimate outcomes

Evaluation question: To what extent have global market opportunities expanded for Canadian agriculture and agri-food products? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. OECD: Organisation for Economic Co-operation and Development
Value ($) of exports to OECD countries	Yes	No	No	No	No
Value ($) of exports to non-OECD countries	Yes	No	No	No	No

Efficiency and economy

Evaluation questions: What were the costs of activities/outputs undertaken by the TME program, including Full-Time Equivalent costs, plus all other expenditures? Does evidence exist that program resources were acquired at the lowest cost consistent with the required quality, quantity, appropriateness and timeliness? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised.
Analysis of actual program costs ($) over the course of the period covered by the evaluation	Yes	No	No	No	No
Views on whether the cost of producing program outputs is as low as possible/whether there are alternative, more efficient, ways of delivering them.	No	Yes	Yes	Yes	Yes

Evaluation question: What was the value of outcomes achieved – immediate, intermediate, and end – and how does that compare to program costs? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised.
Views of program management & staff, and other federal stakeholders on extent to which program intended outcomes have been achieved at the least possible program cost	No	No	No	Yes	No
Views of program management & staff, and other federal stakeholders on how the efficiency of program could be improved	No	No	No	Yes	No
Evidence of/views on whether there are alternative program models that would achieve the same expected outcomes at a lower-cost	No	Yes	No	Yes	Yes

Design and delivery

Evaluation questions: Are the appropriate management structure and processes in place to support effective delivery of the TME program? What factors have facilitated or impeded the effective delivery of the TME program? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised.
Appropriateness of the governance structure to support delivery of program	Yes	No	No	Yes	No
Delivery challenges as identified by program management & staff, and other federal stakeholders	No	No	No	Yes	No

Evaluation question: Are the TME program activities sufficiently integrated and coordinated to support the achievement of outcomes? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. TME: Trade and Market Expansion
Level of coordination/integration of TME activities	Yes	No	No	Yes	Yes
Identified mechanisms/structures that facilitate information exchange internally and externally, and improvements that could strengthen collaboration	Yes	No	No	Yes	Yes

Evaluation question: Are the TME program activities sufficiently integrated and coordinated to support the achievement of outcomes? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised. TME: Trade and Market Expansion
Level of coordination/integration of TME activities	Yes	No	No	Yes	Yes
Identified mechanisms/structures that facilitate information exchange internally and externally, and improvements that could strengthen collaboration	Yes	No	No	Yes	Yes

Evaluation question: Is the TME program facing any resource challenges in terms of available funding, organizational structure/staffing, or internal support (training, technological, research) that are affecting its ability to achieve its expected outcomes? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes: in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised.
Sufficiency of resources (human, financial, technical, competencies, training) to meet objectives	Yes	No	No	Yes	No

Evaluation question: Does the TME program have in place appropriate methods/systems for monitoring performance and reporting on outcomes? Refer to the following table.

Indicator	Method/Data Source - Document & Financial Data Review	Method/Data Source - Literature & Media Review[a] - Lit	Method/Data Source - Literature & Media Review[a] - Media	Method/Data Source - Key Indicator Interviews[b] - Internal	Method/Data Source - Key Indicator Interviews[b] - External
[a] "Yes" in these columns mark the indicators for which OAE will search literature and media sources for corresponding evidence. Evidence will not necessarily be found for each indicator. "No" in these columns mark the indicators for which OAE will not search literature and media sources for corresponding evidence. [b] "Yes" in these columns mark the indicators for which interview questions will be devised. Not all questions will be asked of all interviewees; it will depend on the relevance of the question to the interviewee. "No" in these columns mark the indicators for which interview questions will not be devised.
Extent to which appropriate mechanisms are in place to monitor performance and report on outcomes	Yes	No	No	Yes	No

Annex B: Trade and market expansion logic model

Annex B presents the logic model of the Trade and Market Expansion (TME) program. The contents of the logic model are as follows:

TME Linkages are:

• Growing Forward 2 (GF2) Stream A (market access)
• A Base
• GF2 Stream B (market development)

TME Activities are:

• Policy Development
• Research and Analysis
• Partnership Development

The Sub Activities are:

• Represent the interests of the Canadian government abroad
• Conduct policy and socioeconomic research and analysis
• Develop collaboration and partnership opportunities with international organizations, interest groups, other government departments, Provinces and Territories (PTs) and the public

The Outputs of TME are:

• 1. Outputs
  1. Strategic advice on market access and trade policy issues
  2. Advocacy towards science-based trade rules and policies
  3. Contribution to trade related meetings
• 2. Outputs
  1. Market analysis requests in support of Agriculture and Agri-Food Canada (AAFC)
  2. Market research and intelligence reports in support of market development activities
  3. External ad hoc requests responded
• 3. Outputs
  1. Targeted communications and consultation processes
  2. Reports on multilateral strategies and developments
  3. International Market Expansion Teams (IMET) meetings to coordinate formulation of joint market development strategies, including the use of Canada Brand
  4. Trade commissioner services

The Immediate Outcomes of TME are:

• 4. Engagement in advancing Canadian agriculture market access issues and trade priorities
• 5. Engagement with industry for new and expanded global market opportunities
• 6. Federal provincial cooperation in the advancement of Canadian agriculture

The Intermediate Outcomes are:

• 7. Increased recognition, adoption and enforcement of clear, transparent, science-based trade rules
• 8. Increased market access for Canadian agriculture
• 9. New bilateral and multilateral trade agreements with key trading partners
• 10. Increased Canadian industry entry into new markets and expansion of existing markets

The Ultimate Outcome is:

• 11. Canadian agriculture and agri-food products exporters are successful in seizing market opportunities and in achieving commercial success

The Beneficiaries are:

• Canadian agriculture and agri-food industry and associations
• OCDs, provinces, territories, interest groups, and international organizations