Audit of the AgriMarketing program – market development stream

December 3, 2015

Office of Audit and Evaluation

Executive summary

The AgriMarketing program - Market Development stream (AMP MD) is aimed at building and promoting Canada's ability to expand domestic and export markets, undertake promotional activities to help position and differentiate Canadian products and producers, and ensure industry's ability to meet market requirements.

The annual budget allocation for AMP MD is $21.23MFootnote 1. Projects are cost-shared between Agriculture and Agri-Food Canada (AAFC) and recipients, with AAFC's maximum contribution generally not exceeding 50% of the total eligible costs. AMP MD is managed within the Programs Branch, Competitiveness Division.

The audit of AMP MD was included in the 2015-2018 Risk-Based Audit Plan. The audit focused on the management control framework related activities, addressing governance, risk management and controls, associated with AMP MD.

The audit determined that effective controls were in place to support the management and delivery of AMP MD.

The audit recommendations, addressed to the Assistant Deputy Minister (ADM), Programs Branch, are as follows:

  • Review and update processes in place to obtain and relay key information to the Director General Review Committee for decision making on project funding;
  • Review and update processes in place to obtain technical reviews to ensure that a sufficient level of information is gathered from the organizational areas;
  • Review and update the internal guidance documents for the AgriMarketing program - Market Development stream;
  • Review and update process in place to collect, track and report on AgriMarketing program – Market Development stream's performance against service standards.

1.0 Introduction

1.1 Background

  • 1.1.1 On April 1 2013, federal, provincial and territorial governments marked the official launch of the Growing Forward 2 agricultural policy framework in Canada. Growing Forward 2 is a five-year, $3 billion framework that aligns federal, provincial and territorial policies and programming to support the agriculture sector and food industry. This framework includes the following three federally delivered programs – AgriInnovation, AgriMarketing and AgriCompetitiveness.
  • 1.1.2 The AgriMarketing Program (AMP) is a five-year (covering fiscal years 2013-14 to 2017-18), $341 millionFootnote 2 program comprising a combination of government initiatives and contribution funding for industry-led projects. It supports the agriculture industry by creating and maintaining access to markets and taking advantage of market opportunities. Within AMP, there are four streams: Market Development (MD); Assurance Systems; Breaking Down Trade Barriers; and, Building Market Success. Two of these streams, MD and Assurance Systems, provide contribution funding that support industry-led projects.
  • 1.1.3 AMP MD is aimed at building and promoting Canada's ability to expand domestic and export markets, undertake promotional activities to help position and differentiate Canadian products and producers, and ensure industry's ability to meet market requirements. Agriculture and Agri-food Canada (AAFC) contribution funding supports national industry organizations in implementing long-term international strategies (herein referred to as the generic component), as well as small and medium sized enterprises in implementing international market export plans, which include promotional and market development activities in traditional and new markets.
  • 1.1.4 The annual budget allocation for AMP MD is $21.23M. Projects are cost-shared between AAFC and recipients, with AAFC's maximum contribution generally not exceeding 50% of total eligible costs. AAFC funding levels may be adjusted upwards where activities are aimed at expanding into new markets and downward where activities are aimed at maintaining established markets.
  • 1.1.5 For the generic component, contribution agreement terms ranged from one to five years, with the maximum annual AAFC contribution generally not exceeding $2.5M. Generic contribution agreements signed resulted in a total commitment of $19.7MFootnote 3 for the 2014-15 fiscal year.
  • 1.1.6 For the small-medium sized enterprises component, contribution agreements were for a single year, with the maximum annual AAFC contribution not exceeding $50,000. Small-medium sized enterprise contribution agreements resulted in total commitments of $416,000Footnote 3 for the 2014-15 fiscal year.
  • 1.1.7 AMP MD is managed by the Programs Branch, Competitiveness Division. AMP MD has recently conducted a lean process review for the small-medium sized enterprise component of the program to reduce the application processing times and to improve performance against service standards. The resulting process improvements were phased in commencing in March 2015.

1.2 Audit objective

  • 1.2.1 The objective of the audit was to provide assurance on the adequacy and effectiveness of the controls in place to support the management and delivery of AMP MD.

1.3 Audit scope

  • 1.3.1 The audit examined the management control framework addressing governance, risk management and controls, associated with AMP MD.
  • 1.3.2 The audit examination covered 2013-14 and 2014-15 fiscal years, with an emphasis on the most recent year.
  • 1.3.3 The Assurance Systems, Breaking Down Trade Barriers, and Building Market Access AMP program streams were excluded as part of this engagement.

1.4 Audit approach

  • 1.4.1 The audit approach and methodology was risk-based and consistent with the International Standards for the Professional Practice of Internal Auditing and the Internal Auditing Standards for the Government of Canada, as required under the Treasury Board Policy on Internal Audit.
  • 1.4.2 The conduct phase of the audit took place from July to September 2015.
  • 1.4.3 Internal Audit performed the following audit steps:
    • Document Review: Internal Audit assessed key program documentation to obtain an understanding of the program operations, including key documents used by program staff and recipients of program funding, and examples of reports prepared for management.
    • File Review: Internal Audit selected a sample of project application assessments and claims payments for review. Sample selections were made on transactions from 2013-14 and 2014-15 fiscal years, with an emphasis on the most recent year in order to reflect the most recent management practices followed by AMP MD management and staff.
    • Interviews: Internal Audit conducted interviews with members of AMP MD program management and staff, and with Corporate Management Branch – Financial Controls, Operations staff, who provide support in terms of processing claim payments.

1.5 Conclusion

  • 1.5.1 The audit determined that effective controls were in place to support the management and delivery of AMP MD.
  • 1.5.2 Recommendations were made for improvements in the following areas:
    • Information within Project Recommendation Forms provided to the Director General Review Committee;
    • Project application technical review process;
    • AMP MD internal guidance documents; and,
    • Reporting on AMP MD performance against service standards.

1.6 Statement of conformation

  • 1.6.1 In the professional opinion of the Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence was gathered to support the accuracy of the conclusion provided and contained in this report. The conclusion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The conclusion is applicable only to the entity examined.
  • 1.6.2 This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

2.0 Detailed observations, recommendations and management responses

  • 2.0.1 This section presents the key observations, based on the evidence and analysis associated with the audit, and provides recommendations for improvement.
  • 2.0.2 Management responses are included and provide:
    • An action plan to address each recommendation;
    • A lead responsible for implementation of the action plan; and,
    • A target date for completion of the implementation of the action plan.

2.1 Governance structures

  • 2.1.1 There are Departmental-level governance structures, as well as a program-level governance committee for the oversight of AgriMarketing program – market development stream (AMP MD).
  • 2.1.2 The Department has the following oversight bodies in place, which oversee departmental grants and contribution programs, including AMP MD: Departmental Management Committee; Director General Management Committee; Policy and Program Management Committee; and, Director General Policy and Program Management Committee. The Departmental Management Committee and Director General Management Committee provide an oversight role associated with management and operational issues, and the Policy and Programs Management Committee and Director General Policy and Program Management Committee provide an oversight role related to policy and program design. The audit team reviewed these governance committees' meeting agendas, records of decision from June 2014 to May 2015, as well as their Terms of Reference, and determined that these oversight bodies were provided with AMP MD information and reports.
  • 2.1.3 At the program level, the Director General Review Committee plays a key role in reviewing proposed projects and their funding allocations, prior to Ministerial approval. The Director General Review Committee members were provided with Project Recommendation Forms, which summarized the views of the program area and provided key information through technical reviews by up to seven other organizational areas. The audit found that in 6 of the 21 (29%) applications sampled (out of 65 generic files), the Project Recommendation Forms would have benefited from additional project risk and market access information listed within the technical reviews. The audit also found that recipient organizations' past performance was not included in the Project Recommendation Forms for 13 of 21 generic files (62%). As a result, the information being relayed to committee members via the Project Recommendation Forms may not provide complete information to determine the potential value of proposed projects.
  • 2.1.4 Recommendation 1: The Assistant Deputy Minister (ADM), Programs Branch should review and update the processes in place to obtain and relay key information to the Director General Review Committee for decision making on project funding.

    Management Response and Action Plan:

    Agreed. The Market Development stream began a lean process improvement initiative in October 2015. One of the key objectives of this initiative will be to review and update the Program's collection, presentation, and content of information products going to Director General Review Committee for decision making on project funding; including the development and implementation of a new briefing material format for Director General Review Committee. The new briefing format for the Director General Review Committee will provide better, more organized information and stronger analytics supporting the recommendations.

    Target Date for Completion: January 2016

    Lead(s) Responsible: ADM, Programs Branch and Director General, Business Development and Competitiveness Directorate

2.2 Risk management

  • 2.2.1 Risk management practices for AMP MD take place at both the program and recipient / project levels.
  • 2.2.2 The program utilized the Program Risk Management Assessment tool, as developed by the Service and Program Excellence Directorate, to identify key program risks. The audit team reviewed the risk documentation completed by AMP MD and found that this tool took into consideration key aspects of a program lifecycle and was updated by AMP MD program staff on an annual basis.
  • 2.2.3 In addition, the Department has a Recipient / Project Risk Management Framework in place that outlines an approach to identify recipient / project risks and mitigating strategies, which helps to determine advance frequency, holdback limitations, project reporting frequency and claim document sampling percentages. The audit team sampled 21 AMP MD application files (out of 65 generic files) and found that all applications had a recipient / project risk assessment that was completed in accordance with the Recipient/Project Risk Management Framework.

2.3 Management controls

  • 2.3.1 The AMP MD has management controls in place throughout the recipient / project lifecycle, including during the assessment of funding applications, contribution agreement approvals, project monitoring, as well as claims submission and payment processes.
  • 2.3.2 The audit reviewed a sample of 24 successful funding application assessments (28% of 87 generic and small-medium sized enterprises contribution agreements signed between April 1, 2013 and March 31, 2015) and found that:
    • 2.3.2.1 Funding application assessments were assessed in accordance with Service and Program Excellence Directorate's Assess Due Diligence Guide. The Guide identifies factors to be assessed when reviewing project funding applications.
    • 2.3.2.2 Conflict of Interest requirements were respected in accordance with the Service and Program Excellence Directorate's Conflict of Interest Guidelines for Grant and Contribution Programs.
    • 2.3.2.3 Project funding recommendations and contribution agreements were reviewed and approved through the governance hierarchy to the delegated signing authority.
  • 2.3.3 As part of the application process, the AMP MD solicits input from up to seven other organizational areas in the form of a technical review questionnaire. Responses to the questionnaire help support the assessment listed within the Project Recommendation Forms and provides meaningful information to the Director General Review Committee. In a sample of 21 successful funding applications (out of 65 generic files), the audit team found that there was a large variability in the response rate between each organizational area for the completion of technical reviews. Response rates ranged from 5% to 90%. The audit team also noted that AMP MD has no formalized agreement in place with other organizational areas regarding the completion of the technical review questionnaires. The varying levels of input within technical reviews may not provide program staff and the Director General Review Committee with complete information to assess proposed projects.
  • 2.3.4 Recommendation 2: The ADM, Programs Branch should review and update the processes in place to obtain technical reviews, to ensure that a sufficient level of information is gathered from other organizational areas.

    Management Response and Action Plan:

    Agreed. The Market Development Stream began a lean process improvement initiative in October 2015. One of the key objectives of this lean initiative will be to review and update the process in place for technical reviews; including the development and implementation of an updated technical assessment form (with dedicated questions to specific areas of expertise in other organizational areas), process improvements (e.g.; a technical assessment discussion with technical reviewers on projects before they go to Director General Review Committee) and related guidelines.

    Target Date for Completion: January 2016

    Lead(s) Responsible: ADM, Programs Branch and Director General, Business Development and Competitiveness Directorate

  • 2.3.5 The audit also reviewed a sample of 24 claim payment files (57% of the total of 42 claim files between April 1, 2014 to July 27, 2015) and found that:
    • 2.3.5.1 Claims were reviewed and approved by the Program Manager / Officer and Manager, Program Operations and Financial Claims, and were certified pursuant to Section 34 of the Financial Administration Act by the delegated authority, prior to Section 33 account verification and payment approval by Corporate Management Branch – Financial Controls, Operations.
    • 2.3.5.2 Claims verification processes were followed, which included ensuring that claims documentation was complete, and that financial updates were received in accordance with contribution agreements.
    • 2.3.5.3 Advances, holdbacks and sample selection percentage rates were in accordance with the Recipient / Project Risk Assessment.
    • 2.3.5.4 Amendments were signed by the delegated authority and were supported by documented justifications (five such amendments were identified within the sample). In addition, the audit found that the application review and funding decision due diligence process was appropriately followed for the amendments that involved an increase in total project funding and an increase in the agreement period (three amendments of the five).
  • 2.3.6 The audit examined guidance documents developed for AMP MD and found some were current, for example: AMP MD Applicant Guide and Recipient Guide, and the Roles and Responsibilities for Claim Positions; while other guidance documents were not updated for Growing Forward 2, for example: AgriMarketing Program – Program Officer Procedures Manual (2012), or not compiled, for example; roles and responsibilities of AMP MD program staff. The audit team also observed some inconsistent practices within the file sampling reviews, including: approaches to sampling, and monitoring of project activities and results. Interviews with staff indicated the need for additional guidance on key activities. As a result, there is a potential risk of inconsistent practices in the management of recipient files.
  • 2.3.7 Recommendation 3: The ADM, Programs Branch should review and update the internal guidance documents for the AgriMarketing program – Market Development stream.

    Management Response and Action Plan:

    Agreed. The Market Development stream will update the "AgriMarketing Program – Program Officer Procedures Manual 2012", which will provide program officer with detailed information/procedures on: the application process, approval process, contribution agreement process, project monitoring/management, project reporting and shutdown.

    The AgriMarketing Market Development stream will work with colleagues from other areas in Program Branch (AgriInnovation Program) to complete the "Guidance Document for Claims Processing -- Questions and Answers Inventory". This document will complement the work undertaken on standardization and harmonization with other program areas.

    Target Date for Completion: March 2016

    Lead(s) Responsible: ADM, Programs Branch and Director General, Business Development and Competitiveness Directorate

  • 2.3.8 The AMP MD compiles information on performance against service standards and develops monthly, as well as quarterly reports for management review and decision making. The audit tested the accuracy and completeness of the main tracking spreadsheet used by program staff to document key milestone dates for application and claim processing. The audit team sampled 24 out of 87 generic and small-medium sized enterprises application files from April 1, 2013 to March 31, 2015, and 16 out of 42 claims files from April 1, 2014 to July 27, 2015. The audit team found that the program area recorded all key application milestone dates accurately, whereas the claim spreadsheet was incomplete, with 8 of 48 claim milestone dates (17%) not recorded. The audit also examined performance against service standards reports and found that additional qualitative information / analysis (e.g.: how and why variances occurred), would have been beneficial for management review and decision-making. The incomplete gathering of data by program staff may result in inaccurate reporting of performance against service standards, while the absence of value-added information / analysis may contribute to less informed decision-making by management.
  • 2.3.9 Recommendation 4: The ADM, Programs Branch should review and update the processes in place to collect, track, and report on AgriMarketing – Market Development stream's performance against service standards.

    Response and Action Plan:

    Agreed. The Market Development stream will work with other areas of Program Branch to harmonize and standardize service standards collection and reporting, with a view to provide more timely feedback to management on the status of applications and claims.

    The Market Development stream is developing a SharePoint system that will electronically capture and report on key service standard milestones, as projects move through the technical review and approval process. The intention is to pilot this initiative for broader application across multiple programs.

    The Market Development stream has put in place a new claims tracking system (spreadsheet) and created a generic "claims inbox", with a dedicated position to coordinate the screening and tracking of claims.

    Target Date for Completion: May 2016

    Lead(s) Responsible: ADM, Programs Branch and Director General, Business Development and Competitiveness Directorate

Annex A: Audit criteria

  1. Adequate program implementation is in place.
  2. Effective program monitoring and reporting is in place.
  3. Appropriate recipient project funding application assessment, award and agreement development processes are in place.
  4. Effective recipient project funding administration, monitoring and reporting processes are in place.

Annex B: Acronyms

AAFC
Agriculture and Agri-Food Canada
ADM
Assistant Deputy Minister
AMP
AgriMarketing program
MD
Market development stream
Date modified: