March 27, 2015
Office of Audit and Evaluation
Under the Growing Forward (GF) multilateral agreement, Agriculture and Agri-Food Canada (AAFC) committed to implementing several federally funded programs, including the Canadian Agri-Science Clusters ("Clusters") and Developing Innovative Agricultural Products (DIAP) programs.
The Clusters program was designed to be an industry-led concentration of academic, government and industry science and technical expertise in a particular sector. The DIAP program was intended to provide Canadian individuals, agri-entrepreneurs, firms and organizations, who may have had limited scientific, technical and marketing intelligence resources, greater access to government, university and other resources required to support successful transformation of innovative ideas into viable business ventures.
The Clusters and DIAP programs provided funding for third-party research projects through Vote 10 transfer payments via contribution agreements with recipients. The programs also allowed third-parties to work with AAFC through collaborative activities funded by non-pay operating funds (Vote 1). Collaborative Research and Development Agreements (CRDAs) were signed with third-parties to define the scope of research project work and the use of AAFC agricultural research resources.
During GF, the Clusters and DIAP contribution programs were managed within the former Research Branch. In 2012-2013, the contribution programs were transferred to Programs Branch (PB). At present, PB manages the contribution agreement funding (Vote 10) and the Science and Technology Branch (STB) manages the processes related to the CRDAs.
An audit of GF federally funded programming was included in the Office of Audit and Evaluation’s 2013-2016 Risk-Based Audit Plan. The Clusters and DIAP programs were selected for the audit based on the materiality of their Vote 10 contribution funding components; the fact that the number of recipients for these programs was relatively higher than the other federally funded programs; and due to the complexity of the programs.
For the areas reviewed, the audit concluded that effective controls were generally in place to support the administration of the Clusters and DIAP contribution agreements and the management of the CRDAs during GF.
Recommendations were made to improve compliance monitoring, guidance to funding recipients, and CRDA Steering Committee governance.
1.1 Audit Context
- 1.1.1 Under the GF multilateral agreement, which commenced in Fiscal Year (FY) 2007-2008 and ended in FY 2012-2013, AAFC committed to implementing several federally funded programs designed to support the achievement of GF objectives.
- 1.1.2 The Office of Audit and Evaluation (OAE) identified federally funded programming as an area for audit in FY 2013-2014 as part of its 2013-2016 Risk-Based Audit Plan. Following a scoping assessment, the Clusters and DIAP programs were selected for the audit based on the materiality of their Vote 10 contribution funding components; the fact that the number of recipients for these programs was relatively higher than the other federally funded programs; and due to the complexity of the programs.
- 1.1.3 During GF, the Clusters and DIAP contribution programs were managed within the former Research Branch. In 2012-2013, the programs were transferred to PB. At present, PB manages the contribution agreement funding (Vote 10) and the STB manages the processes related to the CRDAs. The contribution programs and the related CRDAs are described further in the following sections.
1.2 Canadian Agri-Science Clusters Program
- 1.2.1 An Agri-Science Cluster was designed to be an industry-led concentration of academic, government and industry science and technical expertise in a particular sector. The purpose of the Canadian Agri-Science Cluster program was to encourage key agricultural organizations to mobilize and coordinate a critical mass of scientific and technical capacity in industry, government and academia to create, plan and implement a national program of applied science and technology research and development activities in support of strategies for enhanced profitability and competitiveness.
- 1.2.2 Under GF, 10 Clusters agreements were signed: $43,526,029 for vote 10, $24,461,357 for vote 1 and $7,967,405 of industry contributions for a program total of $75,954,791.Footnote 1
1.3 Developing Innovative Agricultural Products Program
- 1.3.1 The DIAP program was designed to provide Canadian individuals, agri-entrepreneurs, firms and organizations, who may have had limited scientific, technical and marketing intelligence resources, greater access to government, university and other resources required to support successful transformation of innovative ideas into viable business ventures. The purpose of the program was to provide funding to accelerate the pre-commercialization of new agri-practices, products and processes by:
- Supporting collaboration amongst stakeholders creating new, or engaged in existing, agricultural value-chains;
- Increasing access of these value chains to applied science and technological resources; and
- Increasing access of individual agri-business and entrepreneurs to support applied science, technology development, piloting and demonstration services.
- 1.3.2 Under GF, 41 DIAP agreements were signed: $20,663,540 for vote 10, $20,168,619 for vote 1 and $7,759,030 of industry contributions for a program total of $48,591,189Footnote 1.
1.4 Collaborative Research and Development Agreements
- 1.4.1 The Canadian Agri-Science Clusters and DIAP programs were designed to provide funding for third-party research projects through transfer payments (Vote 10 contributions). The programs also allowed third-parties to work with AAFC through collaborative activities funded by non-pay operating funds (Vote 1) CRDAs, which were signed with third-parties to define the scope of research project work and the use of AAFC agricultural research resources.
1.5 Audit Objective
- 1.5.1 The objective of this audit was to provide assurance that the management control framework in place to support the delivery of the selected Growing Forward-federally funded programs was adequate and effective.
1.6 Audit Scope
- 1.6.1 The scope of the audit included the following elements of Clusters and DIAP programming:
- Contribution Agreement Administration: Monitoring recipient compliance with agreements, including recipient audit; claims payment processes; and agreement close-out.
- Financial Monitoring: Financial monitoring of the contribution agreement and CRDA funding.
- Values and Ethics and Conflict-of Interest: Training and sharing of information within PB and STB.
- CRDA Governance: Governance processes related to financial administration.
- 1.6.2 Audit work was focussed within PB and STB.
- 1.6.3 The audit fieldwork phase included a sample review of files for the period of April 1, 2011 to March 31, 2013. This timeframe was chosen to examine the most recent management practices which may assist with the management of Growing Forward 2 (GF2) programming.
- 1.6.4 The audit did not include the following elements of the management control framework:
- Application, Assessment and Award of Contribution Funding and Research Agreements: Program staff was in the process of completing the application, assessment and award of funding for GF2 projects while this audit was in the fieldwork phase. Due to the fact that this audit was focussed on GF, the audit team recommends that these processes be considered for a future audit that would assess GF2 processes with an objective of leading to improvements for Growing Forward 3.
- Program Close Out: At the time that audit fieldwork concluded in June 2014, there were still outstanding final claims and the two programs had not yet concluded; however agreement close-out for individual sampled projects was included in the audit scope.
- 1.6.5 The audit took into account that few key staff who managed the programs during GF were still involved in innovation programming in the current time period. There were significant changes to the programs during the audit scope period, including the move of the contribution programs to PB, senior management changes, and staff departures. AAFC’s current Chief Audit Executive was the Director General, Innovation Programs during GF, until March 2012.
1.7 Audit Approach
- 1.7.1 During the audit planning phase, the audit team completed a risk assessment to identify inherent risks associated with the Clusters and DIAP programs, and to assist in defining the objective and scope of the audit. As part of this process, and in order to obtain a basic understanding of the Clusters and DIAP program processes, interviews were conducted with various PB and STB staff. The audit team also reviewed relevant policies and directives, the Clusters and DIAP program processes, sample program reports, and sample claim and payment files.
- 1.7.2 The risks identified by the audit team were assessed based on their expected impact and their likelihood of occurrence. As a result of the risk assessment, the areas identified as being of greatest risk were used to establish the audit objective, scope and criteria. Audit criteria were selected from the AAFC’s Grants and Contributions Audit Criteria, as well as being developed based on specific control objectives of the programs. Annex A of this report provides a list of the selected audit criteria.
- 1.7.3 The audit approach and methodology was risk-based and consistent with the International Standards for the Professional Practice of Internal Auditing and the Internal Auditing Standards for the Government of Canada, as required under the TBS’ Policy on Internal Audit. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that the audit objective is achieved. The audit was conducted in accordance with an audit program that defined audit tasks to assess each audit criterion.
- 1.7.4 Audit evidence was gathered through various methods including interviews, observations, walkthroughs, documentation review and analysis.
- 1.7.5 The selection of DIAP and Cluster projects for sampling was based on the materiality of their Vote 10 and Vote 1 funding components. Due to the generally large file sizes, extensive sampling was not possible during the audit conduct phase time period; however, the selected sample sizes were sufficient for the audit team to understand the management review processes associated with the files. The project files examined were also under the responsibility of different project officers, so that any differences in management processes could be highlighted.
- 1.7.6 The audit file review included 3 Cluster and 5 DIAP project files to examine financial monitoring processes; Financial Administration Act (FAA) Section 34 certifications; claims review processes and CRDA agreement governance. In addition, 3 completed Clusters projects and 2 completed DIAP projects were examined for agreement close-out processes.
1.8 Audit Conclusion
- 1.8.1 For the areas reviewed, effective controls were generally in place to support the administration of the Clusters and DIAP contribution agreements and the management of CRDAs during GF.
- 1.8.2 Recommendations were made to improve compliance monitoring, guidance to funding recipients, and CRDA Steering Committee governance.
1.9 Statement of Conformance
- 1.9.1 In the professional opinion of the Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusion provided and contained in this report. The conclusion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The conclusion is applicable only to the entities examined.
- 1.9.2 This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.
2.0 Detailed Observations, Recommendations and Management Responses
- 2.0.1 This section presents the key observations, based on the evidence and analysis associated with the audit, and provides recommendations for improvement.
- 2.0.2 Management responses are included and provide:
- An action plan to address each recommendation;
- A lead responsible for implementation of the action plan; and
- A target date for completion of the implementation of the action plan.
2.1 Contribution Agreement Administration
- 2.1.1 Recipient Financial Monitoring: Clusters and DIAP contribution program management developed guidelines for recipient financial and performance reporting that were communicated to recipients. These guidelines included the required templates to be completed by the recipients for financial and performance reporting and claims submission.
- 2.1.2 In addition, an internal Payment Process Checklist was developed that identified each required verification step that program officers were to undertake and initial as completed prior to claims payment. A number of financial monitoring verification items were included in this process checklist.
- 2.1.3 Based on the sample of Clusters and DIAP project files reviewed, all expected financial reporting was obtained and the Payment Process Checklist was initialed accordingly by the program officers.
- 2.1.4 Internal audit observed that evidence of the financial monitoring performed by program officers could be strengthened by cross-referencing completed verification checklists to the actual monitoring work performed (for example, source documents and calculations).
- 2.1.5 FAA Section 34 Certification: Internal audit interviewed program management and reviewed the programs’ claims verification process documentation. The process included the following steps:
- Preparation of the claim by the program officer, including ensuring the completeness of the claim, completing the Payment Process Checklist, and maintaining correspondence related to the claim;
- Review of the claim package by the program manager;
- Detailed financial review of the claim package by the program financial officer and finance manager; and
- FAA Section 34 certification signed by a person with appropriate delegated financial authority.
- 2.1.6 For the sample of Clusters and DIAP claims reviewed, Internal Audit noted that Payment Process Checklists were completed and initialed, and the Section 34 certifications were appropriately signed and dated for each claim.
- 2.1.7 Internal Audit observed that several claims included a high volume of invoices due to the program requirement to verify 100 percent of travel and hospitality expenses claimed. Program management indicated that this requirement was removed due to the administrative delays it created. Under GF2, travel and hospitality claims will be reviewed on a sample basis, with the percentage of sampling required to be based on the results of the annual recipient risk assessments.
- 2.1.8 Internal Audit also noted that the large annual claims submitted by recipients, in particular for Clusters projects, created administrative challenges to process these claims in a timely manner. However, program management has revised the requirements stated in the contribution agreements for GF2 to encourage quarterly claims submission.
- 2.1.9 Monitoring Agreement Compliance: AAFC’s standard annual recipient risk assessment template was used by Clusters and DIAP to assess recipient risk and identify the monitoring requirements for the following year.
- 2.1.10 Based on a review of a sample of project files for Clusters and DIAP, Internal Audit observed that recipient risk assessments were prepared as required and approved. Internal Audit also observed that program officers were in regular contact with recipients and maintained evidence of this informal monitoring, including email correspondence and notes to file.
- 2.1.11 For the contribution programming, the Cluster Recipient Guide explained the reporting requirements for Recipients under the contribution agreements. Cluster recipients were required to submit annual and final performance reports to AAFC and these reports were reviewed by AAFC staff with scientific expertise.
- 2.1.12 Finding 1 – Under the contribution agreements, DIAP recipients were also required to submit annual performance reports to AAFC. Internal Audit observed that no process was established to have these annual reports reviewed by AAFC staff with a scientific background and, therefore, no annual feedback was available to feed into the recipient monitoring and risk assessment processes. Final reports, however, were reviewed by STB staff with scientific expertise as part of the agreement close-out process.
- 2.1.13 Recommendation 1 – The Assistant Deputy Minister (ADM) PB, supported by the ADM STB, should ensure that annual performance reports for DIAP are reviewed by PB staff with appropriate expertise and that this information is used in recipient monitoring and risk assessments.
- Management Response and Action Plan: Agreed. For GF2, a process has been implemented where Annual Performance Reports are reviewed jointly by the Program Design and Performance (PDP) Team of Innovation Programs Directorate from PB and the Cross-Sectoral Strategic Direction (CSSD) of STB. Each group focuses on different elements of recipient reports according to their respective expertise. PDP verifies reporting against performance targets. CSSD undertakes an analysis of the reporting against science activities, consulting with STB scientists involved in the project on an as needed basis, to ensure science-related activities are in line with the Contribution Agreement and /or CRDA. CSSD also makes note of any delays, changes or accelerations for any activities and makes recommendations to PB. This assessment is shared with program officers for recipient performance monitoring and future reporting improvements. A comment sheet to be shared with recipients is prepared jointly by the two teams for future reporting improvements.
- Lead Responsible: Director General (DG), Innovation Programs Directorate, PB
- Target Date for Completion: December 2014
- 2.1.14 Recipient Audit: Annual risk assessments were prepared for each program recipient. The results were used to identify which recipients to audit and were incorporated into the departmental recipient audit plan. The Recipient Audit Unit in the Service Programs and Excellence Directorate (SPED) coordinated the departmental plan and arranged for external auditors to perform the audit work.
- 2.1.15 Finding 2 - The internal audit team reviewed the Clusters and DIAP recipient audit reports finalized during GF and observed that for Clusters, there were significant scope limitations and audit adjustments included in the reports. The recipient auditors were unable to conclude on the eligibility of expenditures related to research completed by universities because, at this time, the recipients were not required to obtain and maintain the expenditure documentation for their university research expenditures.
- 2.1.16 Cluster recipients have service contracts with Universities for research that supports Cluster project objectives. These services are claimed by the Cluster recipient as eligible expenditures under the contribution agreement with AAFC. As such, the Recipient should maintain documentation on site to support expenses for service contracts.
- 2.1.17 Recommendation 2 – The ADM PB should ensure that guidance provided to recipients clearly defines expectations for recipient record-keeping in future programs.
- Management Response and Action Plan: Agreed. For GF2, a standard Recipient Guide has been developed where the requirements for record-keeping are clearly described, including the requirements to maintain source documentation for research expenditures undertaken under service contracts, such as contracts with universities, and the necessary evidence supporting ongoing recipient due diligence in managing service contracts.
- Face-to-face training sessions were held with each of the Cluster recipients to discuss the documentation requirements and expected supporting documents to be provided for the payment of claims under the AgriInnovation Program (AIP). Some project recipients were also trained in person where possible; if not, teleconference training took place with all of the recipients as agreements were being negotiated to ensure that requirements for supporting documents would be clear.
- An annex to the Recipient Guide, which is specific to the AIP, is updated regularly and provided to recipients as they sign new agreements. Questions and Answers (Qs and As) are also currently being developed to provide clarifications on expectations.
- Lead Responsible: DG, Innovation Programs Directorate, PB
- Target Date for Completion: December 2014
- 2.1.18 Agreement Close-Out: Program management developed agreement close-out templates. Based on Internal Audit’s review of a sample of closed-out agreement files, they contained the required checklists and approval signatures of the program manager.
- 2.1.19 Internal Audit observed that enhanced organization of closed-out files could improve the audit trail and overall timeliness of the close-out process. This would also better demonstrate the nature of the work performed by program officials. For example, the use of standard documentation file orders; linkages to the records and document management system; explanations for blank items on checklists; and removal of duplicate copies would strengthen the records management practices and ready the files for eventual archiving.
- 2.1.20 Grants and Contribution Training for Program Officers: SPED developed a mandatory Grants and Contribution training program for departmental program officers. Internal Audit verified learning records and determined that all GF program officers received the required training modules.
- 2.1.21 Other Observations: During the review of the contribution agreement project files, Internal Audit observed that the files contained sensitive information. Program management should evaluate if program templates and files require Protected B designation.
2.2 Financial Monitoring
- 2.2.1 Financial Monitoring: Internal Audit observed that periodic financial reporting was prepared for contribution agreement and CRDA funding.
- 2.2.2 Financial reporting on contribution agreement funding was prepared bi-weekly and included information at the project and program levels. Program management also noted that under GF2, financial tracking will be implemented as part of the existing project tracking database. This database will be used until the Department-wide Grants and Contributions Database System (GCDS) is operational.
- 2.2.3 For each CRDA, STB prepared an annual Budget Allocation Document. This document contained budget allocations for AAFC Vote 1 funding and collaborator-provided funding (Net Voted Revenue) allocated to each Fund Centre, Project Activity, Researcher, and Location. The document also summarized budget allocations for each year of the agreement and required sign-off by the responsible Research Manager. For the sample of CRDAs examined, Internal Audit observed that Budget Allocation Documents were prepared and approved as required. Financial reporting and tracking for sampled CRDA files was also available in the Departmental financial system, SAP-SATURN. The Vote 1 non-pay operating funding was allocated in the system at the agreement and scientist level via Work-Breakdown Structure Element (WBSE) and Internal Order (IO) numbers. The audit noted that salary costs associated with the research performed at AAFC research facilities were managed at the Branch level through the budget planning process.
2.3 Values and Ethics and Conflict-of-Interest
- 2.3.1 The area of values and ethics and conflict-of-interest (COI) was raised as an area of higher risk for Clusters and DIAP due to the close working relationship of AAFC staff with Industry partners.
- 2.3.2 To mitigate this risk, AAFC provides employees with a combination of training and senior management affirmations about the importance of values and ethics and COI, including post-employment measures.
- 2.3.3 Internal Audit verified that values and ethics training was provided to Cluster and DIAP program officers during GF. In addition, the intention during GF2 is to require each program officer to sign a COI acknowledgement form that must be maintained on the project file. This acknowledgement form has been developed by SPED and will be mandatory for all AAFC grants and contributions programs.
- 2.3.4 Within STB, values and ethics and COI were affirmed by senior management through emails and town hall meetings. In addition, Headquarters management indicated that manager-led training has taken place in the research centres. During GF2, management intends that each STB project assessor will be required to sign a COI acknowledgment form.
2.4 Collaborative Research and Development Agreements Steering Committee Governance
- 2.4.1 Governance of CRDA Steering Committees: Agreement governance is an important element of the control framework for collaborative projects and, to that end, Steering Committees are required to fulfill their mandate:
- To oversee the implementation of the agreement and the smooth fulfillment of Canada's and the collaborator's commitments under the agreement;
- To review the required reports, budgets, and resources allocated to different work activities and sub-projects;
- To consider proposed changes to the agreement or work activities or budgets; and
- To consider the potential for the exploitation of Foreground Intellectual Property.
- 2.4.2 For the sample of CRDAs identified, the audit team verified that each agreement stipulated that a Steering Committee be established and that it was to meet not less frequently than once per year.
- 2.4.3 Finding 3 - The audit team confirmed that Steering Committees were established for the sample of 8 agreements reviewed (5 DIAP collaborative agreements and 3 Cluster collaborative agreements). Based on the file review of the 8 CRDAs, the audit team noted that:
- For two agreements, minutes or records of decision were available for all years of the agreement.
- For three agreements, minutes or records of decision were not available for each year.
- For three agreements, minutes or records of decision were not available for each year, but other documentation provided indicated that meetings were held in each year (for example, meeting tracking sheets and preparation documents).
- 2.4.4 Recommendation 3 – The ADM STB should clarify the requirements for Steering Committee meetings and records, in order to ensure that adequate collaborative agreement project governance is exercised and evidenced in official records.
- Management Response and Action Plan: Agreed. Under GF2, STB provides guidance in the establishment of the initial meetings of the Steering Committee for all Projects and Clusters, and provides ongoing support to these Committees to facilitate the transfer of information (for example, work plans, budgets, non-pay operating (NPO) costs, salaries, equipment).
- STB has developed sample agenda and minutes templates which are used for current and future Steering Committee meetings to ensure best practices for the retention of official records. Meeting minutes are shared and reviewed by Committee members and saved on internal STB shared folders. This process helps to create a streamlined system to ensure that the terms of the collaborative agreements are met and records are available.
- A monitoring process will be established to ensure that Steering Committees meet at least once in every calendar year, as required under the terms of the CRDAs. It will be up to the discretion of the members to determine if more frequent meetings are required.
- Lead Responsible: DG, Cross-Sectoral Strategic Direction, STB
- Target Date for Completion: December 2014
Annex A: Audit Criteria
- Recipient Financial Monitoring:
- Recipient financial reporting was monitored and this information was used to support recipient risk assessments.
- FAA Section 34 Certification:
- Payments were processed in accordance with FAA Section 34 requirements.
- Monitoring Agreement Compliance:
- Recipient compliance to Program Terms and Conditions and funding agreements was monitored, and feedback was provided to recipients regarding opportunities for improvement.
- Recipient Audit:
- Recipient audits were conducted based on an established recipient risk framework to ensure that recipient activities were in compliance with agreement terms and conditions.
- Agreement Close-Out:
- Agreement closeout procedures were conducted in a timely manner to ensure that agreement responsibilities of both parties were met.
- Employees were provided with training related to their grants and contributions (G&C) responsibilities.
- Financial Monitoring:
- Financial performance was monitored for both the contributions programs and collaborative research and development activities.
- Values and Ethics:
- Employees involved in contribution program delivery and CRDAs formally acknowledged their duty to comply with Public Service values and ethics and received related training.
- Conflict of Interest (COI):
- The Department established and implemented a COI process for staff involved in the management and execution of CRDAs.
- Governance of CRDAs:
- The Department established a monitoring control framework to ensure consistency and adequacy in administrative processes across CRDAs.
Annex B: List of Acronyms
- Agriculture and Agri-Food Canada;
- Assistant Deputy Minister;
- AgriInnovation Program;
- Collaborative Research and Development Agreements;
- Cross-Sectoral Strategic Direction;
- Director General;
- Developing Innovative Agricultural Products;
- Fiscal Year;
- grants and contributions;
- Grants and Contributions Database System;
- Growing Forward;
- Internal Order;
- Non-pay Operating;
- Office of Audit and Evaluation;
- Programs Branch;
- Program Design and Performance;
- Qs and As
- Questions and Answers;
- Service Programs and Excellence Directorate;
- Science and Technology Branch;
- Work-Breakdown Structure Element