Joint audit of SAP internal controls

2015-2016
Agriculture and Agri-Food Canada - Office of Audit and Evaluation, December 4, 2015
Canadian Food Inspection Agency - Audit and Evaluation Branch, January 13, 2016
Natural Resources Canada - Audit Branch, December 17, 2015

 

Executive summary

SAP is the Enterprise Resource Planning system standard for the Government of Canada used for the Integrated Financial and Materiel System (IFMS). As the administrator of the IFMS, (Agriculture and Agri-Food Canada) AAFC is responsible for providing SAP application management services, as well as managing the SAP database infrastructure for various cluster partners.

The Chief Audit Executives of AAFC and two of its cluster partners, the Canadian Food Inspection Agency (CFIA) and Natural Resources Canada (NRCan), agreed to have a joint audit conducted to provide assurance to their respective organizations regarding the effectiveness of the SAP Enterprise Resource Planning application in providing accurate, reliable, accessible and timely financial information.

Objective

  • The audit objective was to provide assurance that:
    • Governance structures are in place and are operating effectively to support sound and equitable investment planning and the allocation of resources in the maintenance and continued evolution of the SAP Enterprise Resource Planning hosted by AAFC.
       
    • Information technology (IT) controls are in place and operating effectively to support the production of accurate, reliable, accessible and timely financial information.

Summary of observations

  • In order to address the audit objective, the audit focused on the overall governance structure guiding the operation and maintenance of SAP, IT controls (including IT General Controls and automated application controls), business continuity planning and disaster recovery planning, as well as governance between the Partnership and third-party service providers, such as Shared Services Canada (SSC). In summary:
    • Partnership governance: The audit determined that the Partnership governance structure was implemented as designed in the governing Service Level Agreement, and was functioning effectively at the operational level; however the audit recommends improvements to strengthen the tactical and strategic layers of the governance structure for which meeting attendance and frequency were not in compliance with the Service Level Agreement requirements.
       
    • IT controls: Key controls in the areas of IT change management, IT operations and application controls were determined to be effective in design and operation throughout the audit period. The audit recommends improvements in the areas of User Acceptance Testing (UAT) and user access management processes.
       
    • Business continuity planning and disaster recovery planning: The audit identified that AAFC's and CFIA's Business Continuity Plans require updates and improvements so that the organizations are prepared to continue the operation of critical functions and services, as defined in their Business Impact Analysis, in the event of a business disruption or disaster.
       
    • Service provider governance: The audit recommends that governance mechanisms between the Partnership and SSC are improved so that SAP and underlying IT infrastructure can be effectively recovered in a timely manner in the event of a disaster, and that any concerns with the timing or potential impact of IT infrastructure changes will be communicated and escalated appropriately.

Conclusion

The audit determined that governance structures were in place to support sound and equitable investment planning and the allocation of resources in the maintenance and continued evolution of the SAP Enterprise Resource Planning; however key governance groups did not meet at the frequency defined in the Partnership Service Level Agreement and attendance was not consistently at the appropriate level. Overall, the audit determined that the AAFC Centre of Excellence consistently executed key IT controls under its responsibility effectively and in a timely manner. Opportunities for improvement were identified to address weaknesses in some IT controls owned by Client Partners or jointly owned by the Partnership, as well as in the areas of business continuity and disaster recovery. Finally, the audit determined that governance structures for service providers could be strengthened.

1.0 Introduction

1.1 Background

  • 1.1.1 In the context of financial and materiel management systems, a cluster group is a collective of departments working together to promote and implement business-driven and standardized solutions for interoperable financial and materiel management. The Treasury Board (TB) Policy on the Stewardship of Financial Management Systems recognizes that cluster groups play an important role in ensuring standardization of financial management by acting as a centre of expertise and central coordinating body for the development and management of more common departmental financial management systems across government.
  • 1.1.2 As the leader of an SAP cluster, AAFC is responsible for providing SAP application management services, as well as managing the SAP database infrastructure for various cluster partners.
  • 1.1.3 The Chief Audit Executives of AAFC and two of its cluster partners, the CFIA and NRCan, agreed to have a joint audit conducted to provide assurance to their respective organizations regarding the effectiveness of the SAP application in providing accurate, reliable accessible and timely financial information.
  • 1.1.4 Given the reliance of the cluster partners on AAFC internal controls, conducting a joint audit of the SAP internal controls will provide all parties with an efficient means of obtaining assurance regarding whether implemented controls support providing financial information in a manner that preserves the integrity of financial transactions entered in the SAP application.
  • 1.1.5 AAFC, CFIA and NRCan (herein referred to as "Client Partners", altogether "the Partnership") have established agreements that articulate the governance mechanisms and roles and responsibilities of the various stakeholders, including the Centre of Excellence, which is the AAFC support team responsible for coordinating and providing the SAP solution and services to the Partnership. The agreements between Client Partners included a right-to-audit clause, which the partner organizations have opted to not exercise, instead coordinating to have a common, joint audit of the SAP internal controls, and supporting governance structures, including Service Level Agreements.
  • 1.1.6 A list of acronyms used in the report can be found at Annex A.

1.2 Audit objective

  • 1.2.1 An internal audit of SAP internal controls was included in the Three-Year Risk-Based Audit Plans for each of the partner departments (AAFC, CFIA and NRCan) for fiscal year 2015-2016. The audit objective was to provide assurance that:
    • Governance structures were in place and operated effectively to support sound and equitable investment planning and the allocation of resources in the maintenance and continued evolution of the SAP Enterprise Resource Planning hosted by AAFC.
       
    • IT controls were in place and operated effectively to support the production of accurate, reliable, accessible and timely financial information.
  • 1.2.2 Please note that "IT controls" included both IT General Controls and key SAP application controls that support financial processes.

1.3 Audit scope

  • 1.3.1 The scope of the audit included controls pertaining to the SAP application and its supporting IT environment that were in place between April 1, 2014 and June 30, 2015 under the responsibility of the following Client Partners: AAFC, CFIA and NRCan. The audit did not extend to controls pertaining to any other AAFC-led cluster services, such as PeopleSoft or SAP Business Warehouse.
  • 1.3.2 SSC is the owner of the IT infrastructure supporting the SAP solution used by the Partnership. The testing of controls under SSC's responsibility was excluded from the scope of this audit, but governance structures and oversight pertaining to SSC areas of responsibility were included.

1.4 Audit approach

  • 1.4.1 The audit approach and methodology was risk-based and consistent with the Internal Auditing Standards for the Government of Canada, as required under the TB Policy on Internal Audit. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that the audit objective is achieved. The audit was conducted in accordance with the audit program, which defined audit tasks to assess each audit criterion. Audit evidence was gathered through various methods including interviews, observations, walkthroughs, documentation review and analysis.
  • 1.4.2 We reviewed previous audit and assessment work performed by other auditors and assessors pertaining to the SAP control environment to determine if a reliance strategy could be adopted. We considered various factors, including the:
    • Audit period covered
    • Nature and extent of work performed
    • Objectivity and competence of other auditors
  • 1.4.3 The audit period covered by previous audit work was not sufficient to adopt a reliance strategy, and therefore we performed independent testing to complete the audit program.
  • 1.4.4 The Control Objectives for Information and Related Technology (COBIT) is an industry standard for control requirements aligned with IT and business risks. COBIT has been leveraged during the risk assessment and in the creation of the audit criteria by aligning our planned work to selected elements of COBIT.
  • 1.4.5 The TB Policy on the Stewardship of Financial Management Systems and related instruments, along with key elements of the Service Level Agreement between the Client Partners were also used as sources of criteria and sub-criteria.
  • 1.4.6 The audit criteria are organized in the following three lines of enquiry, which together allowed the audit to meet its objective:
    • Partnership governance: The Partnership governance structure enables the Partnership to jointly discuss, establish and communicate strategic priorities and objectives to guide investment and resource planning and prioritization in alignment with business requirements and government-wide direction.
       
    • IT controls: IT controls are in place at the Centre of Excellence and Client Partners to support the production of accurate, reliable, accessible and timely financial information.
       
    • Service provider governance: Governance structures are in place to manage risks related to IT General Control services provided by third parties.
  • 1.4.7 Audit criteria and sub-criteria are presented in Annex B.
  • 1.4.8 The conduct phase of the audit began in June 2015 and was completed by September 2015.

1.5 Conclusion

  • 1.5.1 The audit determined that governance structures were in place to support sound and equitable investment planning and the allocation of resources in the maintenance and continued evolution of the SAP Enterprise Resource Planning; however key governance groups did not meet at the frequency defined in the Partnership Service Level Agreement and attendance was not consistently at the appropriate level. Overall, the audit determined that the AAFC Centre of Excellence consistently executed key IT controls under its responsibility effectively and in a timely manner. Opportunities for improvement were identified to address weaknesses in some IT controls owned by Client Partners or jointly owned by the Partnership, as well as in the areas of business continuity and disaster recovery. Finally, the audit determined that governance structures for service providers could be strengthened.

1.6 Statement of conformance

  • 1.6.1 In the professional judgment of the Chief Audit Executives, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusion provided and contained in this report. The conclusion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The conclusion is applicable only to the entity examined.
  • 1.6.2 This audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

2.0 Detailed observations, recommendations and management responses

2.1 Partnership governance

  • 2.1.1 The Partnership governance structure was created to enable joint discussion, and the establishment and communication of strategic priorities and objectives (refer to Figure 1). The governance model created to establish, deploy and monitor the strategic, tactical and operational layers of the SAP Partnership includes the following structures:
    • The SAP partnership steering committee (SPSC): Assistant Deputy Minister standing committee that provides leadership and sets the strategic direction of the Partnership, and is the final approval body for change, management of escalated issues and risk mitigation. This committee is composed of representatives from each of the Client Partners.
       
    • The SAP partnership management committee (SPMC): Director General led standing committee focused on the functionality, operations and management of the SAP service for the Partnership. This committee is composed of representatives from each of the Client Partners.
       
    • Centre of excellence: AAFC-based organization that is responsible to deliver SAP system solutions that adhere to the strategic objectives and priorities directed by the SPSC and SPMC. The Centre of Excellence provides system development and support services for the SAP Solution.
       
    • Change control committee (CCC): AAFC-led group that assists in the assessment, prioritization and scheduling of changes.
       
    • Client advisor: An individual designated by each Client Partner to act as the key contact person and liaison to the Partnership.
       
    • Client partner forums: Groups that support the governance by working closely with the Centre of Excellence to provide operational subject matter expertise to the Partnership by helping to plan and prioritize business requirements, identify operational issues and act as a conduit for information sharing and knowledge transfer.

Figure 1 - Partnership governance structure

Description of this image follows.
Description of Figure 1

The above chart outlines the following:

SAP partnership steering committee (SPSC) is Strategic
SAP partnership management committee is Tactical
Centre of excellence, Change control committee, Client advisor, and Partner forums are Operational.

Finding

  • 2.1.2 The audit determined that the Partnership governance structure was implemented as designed in the governing Service Level Agreement for the SAP solution (refer to Figure 1); however areas for improvement were identified. The SPSC and the SPMC do not meet at frequencies that are in accordance with those specified in the Service Level Agreement. The SPSC met only once during the 15-month audit period (1 out of an expected 3+) and SPMC met twice during the same period (2 out of an expected 5). As a result, there is an increased risk that issues hindering the achievement of strategic priorities and objectives are not identified and resolved in a timely manner. The Service Level Agreement that specifies the expected frequency of governance meetings was developed at a time when the SAP Partnership was new to all parties. As the Partners now have many years of experience and familiarity in working with the SAP solution and each other, it is possible that the frequency in the Service Level Agreement no longer reflects the frequency that is required or expected by all partners. Internal Audit also observed that attendance was not consistently at the appropriate level, as delegates attended the meetings on behalf of the invitees.

    Finding 1: A governance structure with senior level participation for strategic planning, decision-making and problem resolution is adequately designed and established, however it is not operating as defined in the Partnership Service Level Agreement. The SAP Partnership Steering Committee and the SAP Partnership Management Committee do not meet at frequencies that are in accordance with the Service Level Agreement, and attendance was not consistently at the designated level (that is, delegates attended the meetings).

  • 2.1.3 Recommendation

    AAFC 1 - AAFC, in collaboration with the other Partners, should strengthen the governance processes by jointly re-assessing the Service Level Agreement requirements to determine if the required frequency of governance meetings is appropriate based on the needs of the Partnership, and update the Service Level Agreement if necessary. AAFC should designate a lead or develop a process to schedule governance committee meetings at the agreed upon frequency and with appropriate representation.

    AAFC 1
    Management response: Agree

    Action plan:

    Frequency of the governance meeting will be re-assessed jointly during the upcoming Director General/Assistant Deputy Minister governance meetings in order to address the needs of the partnership. The Service Level Agreement will be updated accordingly. A lead will be designated to schedule governance committee meetings at the agreed upon frequency and with appropriate representation.

    Target date for completion: January 31, 2016

    Lead(s) responsible: Chief Information Officer

2.2 Information technology controls

  • 2.2.1 IT controls are broadly defined as activities performed by persons or IT systems, to ensure that the IT applications continually operate as designed to support the achievement of business objectives and the integrity of data. IT controls are assessed within two main categories: IT General Controls and application controls.

Information technology general controls

  • 2.2.2 IT General Controls represent the broad foundation of the IT control environment, and are primarily manual in their operation. IT General Controls traditionally fall within three categories: IT change management, user access management and IT operations. Results for each category are described in detail below.

Information technology change management

  • 2.2.3 IT change management controls are key in supporting the continued operations and evolution of IT applications, and contribute to ensuring IT applications and supporting infrastructure function as intended in a consistent manner. The Centre of Excellence is responsible for performing system development and support services, including the prioritizing, planning, designing, developing, testing and implementation of service requests.
  • 2.2.4 To track changes and control the entire change management process, the AAFC Centre of Excellence uses a custom module within SAP called the Change Management System. The audit observed that all changes examined followed the change management process as defined in the Service Level Agreement and the Change Management Service Agreement, and the following key control points are in place and operating effectively:
    • Change initiation: Changes are initiated by appropriate personnel based on business or technical operational requirements.
       
    • Functional testing: Changes are subject to functional testing by members of the Centre of Excellence.
       
    • Change approval: Changes are approved by appropriate personnel before being moved into production.
       
    • Change monitoring: The use of super user accounts used to transport changes to production, and other privileged activities related to change management, are effectively monitored by the Centre of Excellence.
       
    • Segregation of incompatible duties: There is an appropriate segregation of incompatible duties among those involved in key responsibilities: approving, developing and moving changes to production.

    Strength: All changes that were examined followed the change management process in a controlled manner, and changes were planned to minimize impact on operations.

Finding

  • 2.2.5 Within the software development lifecycle, User Acceptance Testing (also referred to as end user testing) is an activity in which software is tested by the intended audience or business users prior to being moved into production. User Acceptance Testing is key within the IT change management suite of controls, because it is critical that changes have the intended effect within the context of the impacted business processes. Within the Partnership environment, User Acceptance Testing is of increased importance because there may be nuanced differences to each Client Partners' business processes that the Centre of Excellence is not aware of; and therefore there is a higher risk that changes may have an unintended effect if not tested by Client Partners prior to being moved to production.
  • 2.2.6 The audit found that Client Partners are not consistently involved in User Acceptance Testing of changes to SAP, and User Acceptance Testing requirements have not been formalized as part of the change management process. There is a risk that changes do not meet clients' needs or do not function as intended by all Client Partners. In some cases, the lack of client involvement has resulted in inefficiencies, because additional change requests had to be completed to meet clients' needs.

    Finding 2: Client Partners are not consistently involved in User Acceptance Testing of changes to SAP.

  • 2.2.7 Recommendation

    AAFC 2 - AAFC, in collaboration with the Client Partners, should augment the change management process to include consideration for User Acceptance Testing based on risk before changes are made to the SAP application and its supporting IT environment.

    AAFC 2
    Management response: Agree

    Action plan:

    In collaboration with the Client Partners, the Change Management process will be improved to include consideration of User Acceptance Testing going forward

    Target date for completion: April 1, 2016

    Lead(s) responsible: Chief Information Officer

User access management

  • 2.2.8 User access management controls govern how access to applications is restricted, provisioned, modified and removed, to help ensure that only authorized individuals have access to data and related IT applications, and have the ability to perform only specifically authorized functions. Strong logical access control environments include a balance of both preventative and detective controls. Controls pertaining to user access approvals, user access revocation, password requirements and security configuration settings are considered key preventative controls, while detective controls generally consist of periodic review of access rights and monitoring of user activity. The audit assessed controls in the following key user access management areas:
    • Passwords
    • Security configuration
    • Access to privileged IT functions
    • User access monitoring
    • User access provisioning
    • User access removal
    • User access periodic review and certification
Strengths:
  • Password controls are appropriate and aligned with Communication Security Establishment Canada's standards.
     
  • Key SAP configuration and security settings are appropriate.
     
  • Privileged IT functions at the SAP application layer, including access to sensitive transactions, default SAP profiles, and SAP authorization objects are limited to appropriate individuals.
     
  • Controls to monitor sensitive and higher-risk users' access rights and activities are effective in design and operation over the audit period.
     
  • The AAFC Centre of Excellence in collaboration with Client Partners, have developed an effectively designed approval process for new access, which includes definitions of roles and identification of approvers; and the access provisioning process is consistently executed by the Centre of Excellence.
     
  • The Centre of Excellence consistently removes access in a timely manner upon receiving requests from Client Partners.

Findings

  • 2.2.9 The audit also identified opportunities to improve the user access management control environment. While the Centre of Excellence consistently removes access in a timely manner upon receiving requests from Client Partners, it was observed that the Centre of Excellence is not consistently notified by Client Partners in a timely manner upon the departure of users. As a result, SAP access rights for users leaving the organizations are not consistently removed in a timely manner. This is due to each Client Partners' employee departure process not being well-integrated with SAP access removal processes. There is a risk that unauthorized users could access the SAP application through departed employees' user accounts.

    Finding 3: The Centre of Excellence is not consistently notified of the departure of SAP users in a timely manner, therefore SAP access rights for users leaving the organizations are not consistently removed in a timely manner.

  • 2.2.10 To promote the ongoing appropriateness of user access rights required to process financial transactions or that can potentially impact the IT environment components, the audit expected that there would be a periodic review of users' access rights. It was expected that reviews were performed by appropriate IT or business management personnel, at a reasonable frequency and following a risk-based approach so that the level of effort in performing the review was commensurate with the associated risks (for example moderate and higher-risk users should be reviewed more frequently than lower risk users, who may only require an annual review). The audit identified that each Client Partner's periodic user access review is only performed on an annual basis, which is not a sufficient frequency to mitigate risks related to deficiencies in the access removal process (refer to paragraph 2.2.9). User accounts belonging to employees who have left the organizations could remain active for up to a year, and accessed by unauthorized users (that is, if account is used frequently enough so that automatic locking due to inactivity after 30 days is not applied).

    Finding 4: Each Client Partner's periodic user access review is only performed on an annual basis, which is not a sufficient frequency to mitigate risks related to deficiencies in the access removal process.

  • 2.2.11 Recommendation

    AAFC 3/CFIA 1/NRCan 1 - Each client partner should re-assess its:

    • Departure process to ensure that the Centre of Excellence is notified in a timely manner of all departures of employees that have SAP access rights.
       
    • Periodic user access review process, and consider adopting a more frequent (for example on a quarterly basis) and risk-based approach.

    AAFC 3
    Management response: AAFC 3: Agree

    Action Plan:

    AAFC 3 (first bullet): AAFC will re-assess its departure process to ensure that the SAP Centre of Excellence is notified in a timely manner of all departures. AAFC will re-assess its user access review process and consider adopting a more frequent, risk-based approach.

    Target date for completion: March 1, 2016

    AAFC 3:

    Lead(s) responsible: Assistant Deputy Minister, Corporate Management Branch

    Action Plan:

    AAFC 3 (second bullet): SAP Centre of Excellence will communicate and engage with its partners in order to put in place a standard user access review process.

    Target date for completion: March 31, 2016

    AAFC 3:

    Lead(s) responsible: Chief Information Officer

    CFIA 1
    Management response: Agree

    Action Plan:

    The Accounting Operations group does strive to notify the Centre of Excellence of a departing employee with SAP access rights. The issue is more with employees who leave a position for another within the agency. The proposed periodic review (below) will help to address those situations.

    While the Agency does conduct an intensive SAP user review, it is only done annually. The Accounting Operations division will conduct either a quarterly or semi-annual review aimed more at the users rather than their roles to ensure that (1) the people are still in the position and that (2) they still requires the SAP access held. Targeted areas (for example Accounts Payable) may also be used when pursuing a risk-based approach.

    Target date for completion: March 31, 2016

    Lead(s) responsible: Director, Accounting Operations

    NRCan 1
    Management response: Agree. It should be noted; however, that the audit revealed a minimal number of instances where departed employees still had potential access to SAP.

    Action Plan:

    NRCan has recently implemented a new departure process (March 2015), which is integrated with SAP access rights. With respect to adopting a more frequent access review process, NRCan will introduce a semi-annual review for higher risk access profiles, such as corporate reporting and section 33 officers.

    In addition, automated reports will compare SAP access information to other systems such as the Salary Forecasting Module and Directory of People and Services (DPS). This additional compensatory control will serve to further mitigate related risk. Also, please note that compensatory controls currently exist, whereby user accounts are inactivated if users do not access SAP for a period of 90 days.

    Target date for completion: March 31, 2016

    Lead(s) responsible: Director Finance and Procurement Services, Finance and Procurement Branch

Information technology operations

  • 2.2.12 The objective of IT operations controls is to provide a reliable processing environment and mitigate risks related to the loss of IT applications and data, and the incomplete processing of information. The key control areas assessed as part of the audit were data backups and scheduled processing (that is, batch jobs). The Centre of Excellence is responsible for executing key controls related to IT operations.
  • 2.2.13 The audit determined that the suite of backup controls and redundancy built into the architecture reasonably mitigates the risk related to data backups. Additionally, controls to identify and resolve deviations from scheduled processing (for example batch job failures) in a timely manner are effective in design and operation over the audit period, and access to make changes to the job schedules is restricted to appropriate Centre of Excellence individuals.

    Strength: IT operations controls related to data backup and scheduled processing are managed in a controlled manner, and monitoring of automated processing is performed.

Application controls

  • 2.2.14 Application controls are automated in nature and are embedded within business processes (for example SAP will not allow an unbalanced transaction to be posted).
  • 2.2.15 In some cases, the descriptions of automated application controls pertaining to business processes of AAFC, CFIA and NRCan do not accurately align with the SAP system's functionality. While minor opportunities for improvement were identified pertaining to the accuracy of documented control descriptions, the audit determined that the automated application controls are designed and operating effectively. The list of controls for which there are opportunities to improve the accuracy of the control descriptions, has been provided to management.

2.3 Business continuity planning and disaster recovery planning

  • 2.3.1 Every organization is at risk from potential business disruptions caused by various incidents and disasters, which could include:
    • Natural disasters such as tornadoes, floods, blizzards, earthquakes and fire
    • Accidents
    • Sabotage
    • Power and energy disruptions
    • Communications, transportation, safety and service sector failure
    • Environmental disasters such as pollution and hazardous materials spills
    • Cyber-attacks and hacker activity.
  • 2.3.2 A Business Continuity Plan enables Client Partners to respond to business disruptions in order to continue the operation of critical functions and services.
  • 2.3.3 Disaster recovery planning is focused on recovering IT assets from disasters or disruptions. A Disaster Recovery Plan is a critical component enabling the Partnership to respond to incidents and disruptions in order to continue the operation of the SAP solution and related IT infrastructure, as well as maintain the availability of information at a level acceptable to the Client Partners. A Disaster Recovery Plan describes procedures and arrangements in place to recover critical IT assets within a predetermined time frame.

Findings

  • 2.3.4 As per the SAP Partnership Service Level Agreement, each Client Partner is responsible for performing their own Business Impact Analysis and corresponding Business Continuity Plan (if warranted by the Business Impact Analysis results) for services supported by SAP. The audit assessed each Client Partner's Business Impact Analysis and Business Continuity Plan and determined that the latest complete and approved AAFC Business Continuity Plan was from March 2012, although a Business Continuity Plan refresh is currently in progress. There is a risk that AAFC is not well positioned to continue business operations in a timely manner in the event of a business disruption.
  • 2.3.5 For CFIA, the Business Continuity Plan documentation provided as part of this audit did not provide for scenarios where the SAP application would be impacted. A maximum allowable downtime of 1 to 3 days was identified by the business units in the Business Impact Analysis, and business continuity may not be achievable in the absence of a Business Continuity Plan, potentially hindering CFIA in continuing its critical operations in the event of a business disruption.
  • 2.3.6 NRCan's Business Impact Analysis did not identify any critical functions or services that rely on SAP; therefore, the audit did not expect to find evidence of a Business Continuity Plan and a Disaster Recovery Plan related to SAP.

    Finding 5: Business continuity documentation requires updates and improvements so that AAFC and CFIA are prepared to continue the operation of critical functions and services in a timely manner in the event of a business disruption.

  • 2.3.7 As per the Partnership Service Level Agreement, each member of the Partnership has individual responsibilities regarding disaster recovery planning. The Service Level Agreement between the Client Partners states that "the cost model for the SAP solution does not include disaster recovery planning or business continuity planning", however "should a Client Partner be interested in creating a Disaster Recovery Plan with the assistance of AAFC, this would be addressed separately". The audit expected that each member of the Partnership had arrangements in place to ensure business and IT requirements will be met in the event of a disaster; however disaster recovery planning arrangements and associated costs have not been formally agreed upon by the Partnership. Therefore, the AAFC Disaster Recovery Plan was not developed in collaboration with NRCan and CFIA. While NRCan and CFIA data is replicated and backed up by the same methods as AAFC's data, there are no provisions to provide Client Partners' personnel workspaces and connectivity at the hot site in the event of a business disruption or disaster. Based on the NRCan Business Impact Analysis, this would not impact the ability of NRCan to deliver critical functions or services. CFIA's business requirements for SAP, as defined in its Business Impact Analysis, may not be met in the event of a disaster. However, management has stated that it believes that SAP is not a critical service and therefore, the business and IT requirements in its Business Impact Analysis are inaccurate.

    Finding 6: For the SAP application, CFIA does not have arrangements in place to enable business and IT requirements, as defined in its Business Impact Analysis, to be met in the event of a disaster.

  • 2.3.8 Recommendation

    AAFC 4 - AAFC should complete the in-progress Business Continuity Plan refresh to update and approve its Business Continuity Plan to ensure departmental readiness.

    AAFC 4
    Management response: Agree

    Action Plan:

    The SAP Business Continuity Plan is completed and approved at the Director General level and will be submitted for final approval.

    Target date for completion: December 1, 2015

    Lead(s) responsible: Chief Information Officer

    CFIA 2 – CFIA should reassess the criticality of the SAP application. If services supported by SAP are subsequently determined to no longer be critical, then the Business Impact Analysis should be updated accordingly. If SAP is confirmed as being critical, then the following improvements to the Business Continuity Plan documentation and recovery arrangements should be made to enable the continuity of critical functions and services:

    • CFIA should update the Business Continuity Plan documentation to include scenarios in which SAP and IT infrastructure is impacted or unavailable. The requirements to continue critical services in the event that the SAP and IT infrastructure are unavailable should be documented in the Business Continuity Plan.
       
    • CFIA should engage AAFC to implement arrangements so that its business and IT requirements for SAP will be met in the event of a disaster. The arrangements should be supported by formal documentation describing the agreement, cost model and resources that will be made available to CFIA in the event of a disaster.

    CFIA 2
    Management response: Agree

    Action Plan:

    The Agency admits that the Maximum Allowable Downtime as indicated on the Business Impact Analysis of 1 to 3 days (level 2) was inadequate. It will be changed to a Maximum Allowable Downtime of 8 to 21 days (level 4) during the next review.

    With the correction of the Maximum Allowable Downtime, the services move from critical to non-critical and therefore further updates to the Business Continuity Plan are not warranted.

    Target date for completion: July 31, 2016

    Lead(s) responsible: Director, Accounting Operations

2.4 Service provider governance

  • 2.4.1 Partnering with third party service providers can provide significant benefits, such as reduced cost and increased efficiency, however the use of such providers also introduces additional risks that need to be managed accordingly. The most critical service provider supporting the Partnership's SAP environment is SSC, as they are responsible for hosting and maintaining the IT infrastructure. SSC was created on August 4, 2011, and IT infrastructure-related services that were formerly performed by AAFC were subsequently transitioned to SSC.
  • 2.4.2 The audit assessed service provider governance in the following key areas: disaster recovery planning and IT infrastructure change management.

Disaster recovery planning

Finding
  • 2.4.3 It is critical that there is a high-level of communication and coordination between the Partnership and SSC, due to SSC's integral role in disaster recovery. The audit identified that AAFC and SSC do not regularly meet to discuss strategic or operational requirements in support of disaster recovery for the SAP environment.
  • 2.4.4 The audit determined that the AAFC Disaster Recovery Plan was most recently updated in March 2012 and did not reflect the transfer of data centre responsibilities to SSC. SSC subsequently changed the location of the SAP system's backup data centre from a Winnipeg facility to an Ottawa facility; however the SAP Disaster Recovery Plan has not been updated to reflect the current environment. Additionally, due to the relocation of the backup data centre, the primary and backup data centres supporting the SAP application are now within close proximity (~5km), which is not aligned with best practices. Primary and backup data centres should have sufficient separation and differences between geography, climate, seismology, and power grids. There is a risk that AAFC is not well positioned to recover IT assets in a timely manner in the event of a disaster. Overall, there is a risk that contingencies to support disaster recovery are not sufficient to recover from certain types of disasters in a timely manner.

    Finding 7: The Disaster Recovery Plan does not reflect the current environment or clearly delineate roles and responsibilities between the Partnership and SSC. The primary and backup data centres are in close proximity, increasing the likelihood that a particular disaster or disruption could affect both data centres simultaneously, thereby hindering the timely recovery of the SAP environment.

Information technology infrastructure change management

Finding
  • 2.4.5 The audit identified that effective communication channels were not in place for AAFC to be advised in a consistent and timely manner of IT changes at the infrastructure level that may cause unavailability of the SAP application, which prevents advance notice and disclosure to Client Partners. Since the creation of SSC, the extent of communication with, and involvement of, SSC personnel in the AAFC change management process has decreased, and SSC representatives no longer attend the change control meetings at AAFC. Although meetings between AAFC and SSC occur at the Senior Management level, there are no change management meetings attended by both SSC and AAFC representatives.

    Finding 8: Effective governance structures are not in place in support of consistent and timely communication of changes at the infrastructure level that may cause unavailability of the SAP application to enable disclosure to all Partners and minimize impacts on operations.

  • 2.4.6 Recommendation

    AAFC 5 - AAFC should collaborate with SSC to formalize Service Level Agreements or other business arrangements so that:

    • The Disaster Recovery Plan is continually updated to reflect the current environment. The Disaster Recovery Plan documentation should clearly delineate roles and responsibilities between the Partnership and SSC in the event of a disaster.
       
    • The location of the backup data centre and hot site is re-assessed and remediated as necessary.
       
    • Planned infrastructure changes with potential impact on SAP availability are consistently discussed with Client Partners in a timely manner.

    AAFC 5
    Management response: Agree

    Action Plan:

    AAFC will collaborate with SSC to ensure SAP service expectations and needs are fully understood and work with SSC to:

    • Formalize the periodic review and adjustment to the Disaster Recovery Plan to reflect the current SAP environment including the delineation of roles and responsibilities for AAFC and SSC in the event of a disaster.
    • Request that the location of the Backup Data Centre (Hot site) for SAP be re-assessed and remediated as necessary.
    • Define the approach for SSC to have timely discussions with AAFC on proposed infrastructure changes with an anticipated impact on SAP.

    Target date for completion: April 1, 2016

    Lead(s) responsible: Chief Information Officer

Annex A: acronyms

AAFC
Agriculture and Agri-Food Canada
CCC
AAFC Change Control Committee
CFIA
Canadian Food Inspection Agency
COBIT
Control Objectives for Information and Related Technology
IMFS
Integrated Financial and Material Systems
IT
Information technology
SPMC
SAP Partnership Management Committee
SPSC
SAP Partnership Steering Committee
NRCan
Natural Resources Canada
SSC
Shared Services Canada
TB
Treasury Board

Annex B: audit criteria

Line of enquiry 1.

The Partnership governance structure enables the Partnership to jointly discuss, establish and communicate strategic priorities and objectives to guide investment and resource planning and prioritization in alignment with business requirements and government-wide direction.

  • 1.1. A governance structure with senior level participation for decision-making and problem resolution is established to ensure that the investments and resourcing of the SAP solution support the Client Partners in an effective and equitable manner.

    Source: Service Level Agreement for SAP Solution between AAFC, NRCan and CFIA

    • 1.1.1. The SAP Partnership Steering Committee reviews and approves the resourcing of the SAP solution and ensures planned investments are aligned with Client Partners' strategic objectives. The Committee meets regularly and attendance at the meetings is appropriate.
    • 1.1.2. The SAP Partnership Management Committee reviews and approves initiatives from Client Partners, including major functionality changes to the SAP solution, and determines plans and priorities through negotiation. The Committee meets regularly and attendance at the meetings is appropriate.
    • 1.1.3. The AAFC Centre of Excellence engages Client Partner functional business operations areas and service delivery organizations in a timely manner when developing solution and application plans to ensure Client Partners' business requirements are met and that initiatives from each Client Partner can be leveraged to the extent possible at the Partnership level.
    • 1.1.4. Client Partners are represented by appropriate individuals at each Client Partner forum and the forums occur periodically to discuss upcoming development to the SAP solution and allow Client Partners to provide feedback in a timely manner.
  • 1.2. Prudent investments are made in the SAP Integrated Financial Management System (IFMS) that meet operational requirements and are aligned with the strategic direction for Financial Management Systems across government, in compliance with TB directions.

    Source: TB Policy on the stewardship of Financial Management Systems

    • 1.2.1. Partnership representatives have been assigned to the Government of Canada (GC) IFMS cluster group's committees and they actively participate to ensure that the Partnership's business needs are considered in that forum while adhering to Office of the Comptroller General of Canada (OCG) direction for Financial Management Systems.
    • 1.2.2. New information or operational requirements that affect the IFMS at a government-wide level are discussed with Client Partners, and collaboration between the Partnership, the OCG and the GC IFMS cluster group occurs to develop an appropriate systems change management process and an implementation plan.
    • 1.2.3. Collaboration between the Partnership and the OCG is in place to align, to the greatest extent possible, the Partnership's IFMS with the government-wide direction.

Line of enquiry 2.

Information Technology (IT) controls are in place at the AAFC Centre of Excellence and Client Partners to support the production of accurate, reliable, accessible and timely financial information.

  • 2.1. Effective controls are in place for adding, updating and deleting user access rights to financial data and programs and access to financial data and programs is appropriately restricted.

    Source: COBIT5 - DSS05.04 Manage user identity and logical access

    • 2.1.1. SAP configuration settings, including password requirements, are appropriate and aligned with industry standards.
    • 2.1.2. Access to privileged IT functions, including powerful accounts, is limited to appropriate individuals.
    • 2.1.3. User access is authorized, appropriately established, monitored, periodically reviewed and removed in a timely manner when no longer required.
    • 2.1.4. Segregation of incompatible duties exists within the logical access management process.
  • 2.2. Application-level changes are made in a controlled manner, including standard changes and emergency maintenance. This includes change standards and procedures, impact assessment, prioritization and authorization, emergency changes, tracking, reporting, closure and documentation.

    Source: COBIT5 - BAI06 Manage Changes

    • 2.2.1. Changes are initiated by the appropriate personnel based on business or technical operational requirements.
    • 2.2.2. Testing of changes is performed by the appropriate personnel based on the nature of the change and before the change is moved into production. Client Partners are involved when appropriate.
    • 2.2.3. Changes are approved by the appropriate personnel before being moved into production.
    • 2.2.4. Planned changes to the production environment and maintenance activities are discussed at the Change Control Committee and scheduled to minimize the impact on operations.
    • 2.2.5. Segregation of incompatible duties exists within the change management process.
  • 2.3. IT operations are managed in a controlled manner and monitoring of the automated processing is performed.

    Source: COBIT5 - DSS01.01 Perform operational procedures

    • 2.3.1. Backups of SAP financial data are performed regularly and tested to ensure they can be restored. Backup failures are monitored and remediated.
    • 2.3.2. Deviations from scheduled processing are identified and resolved in a timely manner, and access to makes changes to the job schedule is restricted to appropriate individuals. Changes to the job schedule are approved.
  • 2.4. A plan has been established and is maintained to enable the business and IT to respond to incidents and disruptions in order to continue operation of the SAP solution and maintain availability of information at a level acceptable to the Client Partners.

    Source: COBIT5 - DSS04 Manage continuity

    • 2.4.1. Each Client Partner has performed a business impact analysis and a Threat and Risk Assessment of the SAP solution, including the establishment of target recovery time objectives.
    • 2.4.2. A Disaster Recovery Plan has been established in collaboration with Client Partners and is aligned with the business impact analyses.
    • 2.4.3. A Business Continuity Plan has been established by each Client Partner when deemed necessary based on the business impact analysis.
  • 2.5. Key application controls in support to financial business processes, as identified by Client Partners' Chief Financial Officer, are functioning as intended.

    Source: TB Policy on Internal Control

    • 2.5.1. Application controls have been designed and configured adequately.
    • 2.5.2. Access to modify key parameters and configurable items is restricted to authorized personnel.

Line of enquiry 3.

Governance structures are in place to manage risks related to services provided by third parties.

  • 3.1. Changes or maintenance impacting the infrastructure supporting the Partnership's SAP solution are discussed in advance with the Partnership's representatives to mitigate unintended impacts or interruptions to operations.

    Source: COBIT5 - APO10.04 Manage supplier risk

    • 3.1.1. A mechanism is in place for third parties to liaise with the Partnership's representative(s) and discuss upcoming infrastructure changes.
    • 3.1.2. Upcoming changes at the infrastructure level that may result in downtime are discussed among Client Partners and any issues are raised with third parties.
  • 3.2. Mechanisms are in place to ensure that services provided by third parties can be restored at a level and within a timeframe acceptable to the Client Partners in the event of a disaster, or mitigating measures are in place.

    Source: COBIT5 - APO10.04 Manage supplier risk

    • 3.2.1. Requirements in support to business continuity, including the availability of complete backup data within the recovery time objective, are formally discussed with third parties.
    • 3.2.2. Roles and responsibilities in the event of a disaster have been clearly delineated between the Partnership and third parties.
    • 3.2.3. An integrated Disaster Recovery Plan has been developed and communicated to the Partnership, and is aligned with the Partnership's objectives, or mitigating measures are in place if such plan is not in place.