Audit of Grants and Contributions Digital Platform: Project Planning

Departmental Audit Committee
June 27, 2019

Executive summary

The Programs Branch (PB) and the Information Systems Branch (ISB) are developing a new Grants and Contributions Digital Platform (GCDP) for transfer payment programs (Voted) that will be integrated with other Agriculture and Agri-Food Canada (AAFC) systems, to streamline program administration; provide for flexible and integrated program delivery, management and reporting; and improve client experience. The system is expected to require less intervention by technical resources, provide improved turn-around time, and decrease operational costs to onboarding new programs.

Development of the GCDP is expected to span a three-year time period, starting in June 2018 and ending in March 2021 with three main phases:

  • Project planning, including prototyping, solution requirement and design, infrastructure and security requirements identification;
  • Build and delivery of Release 1;
  • Integration and enhancements, plus delivery of Release 2.

Given the size and complexity of this information technology project, its potential risks and impacts, the Office of Audit and Evaluation will conduct three real-time audits, with one for each of the three phases. This approach is intended to enable providing advice and recommendations on issues that could hamper implementation and delivery, so that management actions can be taken to mitigate these as they arise.

The Audit of Grants and Contributions Digital Platform: Project Planning was included in the 2018-19 AAFC Integrated Audit and Evaluation Plan. The objective of the audit was to determine whether project management processes are in place and function as intended to support the effective delivery of the planning phase of the GCDP project.

The audit concluded that adequate project management processes were in place and functioning as intended to support the planning phase of the Grants and Contributions Digital Platform project. In particular, the following good management practices were noted:

  • Effective governance structures for oversight and decision-making have been established;
  • Well-documented project plans and coordination efforts guided key project management activities, including roles and responsibilities, project scope, project costs, quality, and vendor management; and
  • Requirements have been defined to demonstrate that the solution design supports the achievement of program objectives.

The audit identified opportunities to improve project management processes, including:

  • Revisiting project timelines for subsequent phases and adjust as required;
  • Ensuring resource plans match project tasks with resources required;
  • Ensuring that organizational change management plans are prepared; and
  • Establishing a working agreement for agile development team members, as well as specific project metrics to track development progress in the execution phase.

Management has agreed with the findings in the audit report and has provided action plans to address the recommendations.

1.0 Introduction

1.1 Background

  • 1.1.1 The Programs Branch (PB) manages consolidated program delivery on behalf of Agriculture and Agri Food Canada (AAFC) for Canada's agriculture, agri-food and agri-based products sector. Current programs are primarily a result of the Canadian Agricultural Partnership, the five-year policy framework that came into effect on April 1, 2018.
  • 1.1.2 The PB is responsible for administering and managing the systems and processes that support program delivery. In recent years, PB has been working to improve management of Grants and Contributions programs (Voted), including: standardization of program materials, harmonization of service standards for application approvals and claims processing, as well as implementation of an online application submission process.
  • 1.1.3 The current Grants and Contributions (Voted) systems do not provide a fully integrated business delivery solution. To address current expectations and align with AAFC’s service vision of providing easy to use, efficient, timely, and responsive service, PB and the Information Systems Branch (ISB) are developing a new Grants and Contributions Digital (GCDP) platform based on Microsoft Dynamics that will be integrated with other AAFC systems. It is anticipated that this platform will require less intervention by technical resources, provide improved turn-around time and decrease operational costs when on-boarding programs.
  • 1.1.4 Development of the Grants and Contributions Digital Platform (GCDP) is expected to span a three-year time period with the following key planned activities and deliverables:
    • June 2018 – April 2019: Project planning, including prototyping, solution requirement and design, infrastructure and security requirements.
    • April 2019 – April 2020: Build and delivery of Release 1.
    • April 2020 – March 2021: Integration and enhancements, plus delivery of Release 2.
  • 1.1.5 In the first project planning phase, governance structures were established and project management practices were put in place. Project organizational readiness and capacity were being reviewed and determined. In terms of prototyping, business leads were working directly with Microsoft Dynamics Customer Relationship Management experts to develop mock-ups of different processes to support discussion with subject matter experts. Demonstrations of prototypes were also held with business/program officers to refine requirements, with a goal of validating that Microsoft Dynamics can meet or surpass program requirements, and obtain endorsement to proceed.
  • 1.1.6 Given the size and complexity of this information technology project and its potential impacts, the Office of Audit and Evaluation (OAE) will conduct three internal audits, including one for each of the three phases of the system’s planned development. This approach is intended to provide advice and recommendations on issues that could hamper implementation and delivery, so that management actions can be taken to mitigate these as they arise.
  • 1.1.7 As per the Office of Audit and Evaluation's 2018-19 Integrated Audit and Evaluation Plan, this audit engagement was planned to be undertaken in the second half of fiscal 2018-19, with completion by the first quarter of fiscal 2019-2020.

1.2 Audit objective

  • 1.2.1 The objective of the audit was to determine whether project management processes are in place and function as intended to support the effective delivery of the planning phase of the GCDP project.

1.3 Audit scope

  • 1.3.1 The audit focused on GCDP project activities relating to project planning, including the establishment of governance and project management practices, development of project organizational readiness and capacity, as well as prototyping, solution requirement and design, infrastructure and security requirements, which occurred from June 2018 to April 2019.
  • 1.3.2 The following areas were scoped out of this audit, as they were to be implemented in later phases of the project:
    • Interfaces with other corporate systems (SAP, email, and GCDocs in the future);
    • Integration with external portal; and
    • Accounting treatment of GCDP project (which costs are capitalized or expensed).

1.4 Audit approach

  • 1.4.1 The audit approach and methodology were risk-based and consistent with the International Standards for the Professional Practice of Internal Auditing, and the Treasury Board Secretariat Policy on Internal Audit and Directive on Internal Audit. These standards require that the audit be planned and performed so as to conclude against the objective. The audit was conducted in accordance with an audit program that defined audit criteria and audit tasks to be performed in the assessment of each line of enquiry.
  • 1.4.2 Evidence was gathered through various methods including interviews, review and analysis of documentation.

1.5 Conclusion

  • 1.5.1 Internal audit concluded that adequate project management processes were in place and functioning as intended to support the planning phase of the GCDP project. In particular, the following good project management practices were noted:
    • Effective governance structures for oversight and decision-making have been established;
    • Well-documented project plans and coordination efforts guided key project management activities, including defined roles and responsibilities, project scope, project costs, quality, and vendor management; and
    • Requirements have been defined to demonstrate that the solution design supports the achievement of program objectives.
  • 1.5.2 The audit identified opportunities to improve certain project management practices. These included:
    • Revisiting project timelines for subsequent phases and adjusting as required;
    • Ensuring resource plans match project tasks with resources required;
    • Ensuring that organizational change management plans are prepared; and
    • Establishing a working agreement for development team members, as well as specific project metrics to track development progress in the execution phase.

1.6 Statement of conformance

  • 1.6.1 In the professional judgment of the Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusion provided and contained in this report. The conclusion is based on a comparison of the conditions, as they existed at the time of the audit, against pre-established audit criteria that were agreed on with management. The conclusion is applicable only to the entity examined.
  • 1.6.2 This audit conforms to the Institute of Internal Auditors' International Professional Practices Framework, as supported by the results of the internal audit quality assurance and improvement program.

2.0 Observations, recommendations, and management responses

This section presents the key observations and recommendations for improvement. Management action plans were provided and include:

  • An action plan to address each recommendation;
  • A lead responsible for implementation of the action plan; and
  • A target date for completion of the implementation of the action plan.

The results of the audit are organized into four themes:

  • Project Governance
  • Project Management
  • Organizational Readiness and Capacity
  • Grants and Contributions Digital Platform (GCDP) Prototyping

2.1 Project governance

  • 2.1.1 Internal Audit expected to find that structures and mechanisms for project oversight were defined and in place to support and monitor the execution of the project. Project governance would entail providing support, guidance, oversight, direction, and a risk-based approach to the management of the Agriculture and Agri-Food Canada (AAFC) GCDP development project.
  • 2.1.2 A governance structure was established for GCDP with two steering committees for the project: the Assistant Deputy Minister (ADM) Steering Committee and the Director General Steering Committee. Both steering committees were made up of representatives from Information Systems Branch and Programs Branch.
  • 2.1.3 Internal Audit examined the Terms of Reference and meeting materials of these two GCDP steering committees, and found that they utilized effective governance practices, including formalized accountabilities for project oversight, direction, decision making, and project health monitoring.
  • 2.1.4 Internal Audit noted that project reporting to the GCDP governance committees was effective:
    • 2.1.4.1 Records of decision from governing bodies indicate that project health and progress were regularly discussed at each meeting.
    • 2.1.4.2 Three Director General GCDP Steering Committee meetings took place since project launch and email briefings have been used to communicate updates between meetings. Directors General are also members of the ADM GCDP Steering Committee meetings.
    • 2.1.4.3 Four ADM GCDP Steering Committee meetings took place since project launch. The meeting agenda and records of decision indicate that the ADM GCDP Steering Committee monitored the project health, project objectives and progress, and approved key decisions.
  • 2.1.5 The GCDP project was also supported by AAFC’s Investment Planning Committee, Information Systems Branch’s (ISB) Project Review Committee, Architecture Review Committee, Change Control Committee, and Corporate Management Branch’s Procurement Review Board, in accordance with the departmental Information Management/Information Technology Portfolio Management Framework for AAFC Investments. Additionally, GCDP provided monthly reporting of project costs, schedule, scope, key risks, and issues to the Investment Planning Committee, of which the Chief Financial Officer and Chief Information Officer are co-chairs, in a manner which aligns with Treasury Board Secretariat guidance for reporting executive project dashboards.
  • 2.1.6 Internal Audit determined that an effective governance structure and appropriate mechanisms have been established to provide oversight and decision making for the GCDP project. 

2.2 Project management

2.2.1 Project planning

  • 2.2.1.1 Internal Audit expected that the project plans would be well-documented to adequately address the processes for managing the project throughout its lifecycle, including timelines for integration points and dependencies.
  • 2.2.1.2 Internal Audit examined the Project Management Plan, Project Charter, and Business Case, and found that plans were approved and well-documented to guide key project management activities throughout the project lifecycle, including project scope, timelines, cost, changes, procurement, communications, integration points, and dependencies.
  • 2.2.1.3 The GCDP project’s prototyping phase extended approximately two months beyond its original schedule to further define requirements that were originally planned to be performed in later phases of the project. Even though the defining of these requirements was advanced to the planning phase, the work performed was within the overall project scope.
  • 2.2.1.4 While completing some steps earlier than planned will provide some efficiencies for the execution phases of the GCDP project, it is unclear whether the next phase of the project will be completed by April 2021 as planned. Revisions to the GCDP project timelines may be required.

Recommendation 1
The Assistant Deputy Minister, Programs Branch should task the GCDP Project to formally revisit and adjust the project execution and release timelines as required.

Management response and action plan:
Agree.
As the project progresses from planning to executing, and priorities for development in 2019-2020 are confirmed, the project plan and timelines will be updated. Regular reporting against approved deliverables and timelines at both the ADM GCDP Steering Committee and to the investment Planning Committee will be undertaken. The project plan and timelines will be updated by July 30, 2019.

Lead responsible:
ADM, Programs Branch
Senior Director, GCDP Project, Programs Branch

Target date for completion: July 30, 2019

2.2.2 Project roles and responsibilities

  • 2.2.2.1 Internal Audit expected that roles and responsibilities would be clearly defined, assigned across the project, well understood and accepted.
  • 2.2.2.2 The Project Charter and Project Management Plan clearly defined and assigned roles and responsibilities for the project team and business stakeholders. Interviews with key project team members from Program Branch’s (PB) Service and Program Excellence Directorate, ISB, and PB’s Programs confirmed that roles were well understood and accepted.

2.2.3 Project scope/Change control

  • 2.2.3.1 Internal Audit expected that the project scope would be clear and documented, and that any scope changes would be well-managed, approved, and documented.
  • 2.2.3.2 The Project Charter and Project Management Plan clearly define the project scope and the planned process for managing scope changes. This process includes use of a standard form and requires discussion in project status meetings. The process also requires that new and in-progress changes be tabled for decision/action/discussion at ADM GCDP Steering Committee meetings.
  • 2.2.3.3 There were no scope changes at the time of the audit, so Internal Audit was unable to assess whether the change process was effectively implemented according to the planned process.
  • 2.2.3.4 The audit team analyzed project scope and the rationale for prototype completion delays experienced and determined that the activities affecting the delay were attributable to in-scope activities planned for a later date, as noted in paragraphs 2.2.1.3 and 2.2.1.4.

2.2.4 Project cost management

  • 2.2.4.1 Internal Audit expected that project costs (forecast and actuals) would be tracked, accurate, up-to-date, and reported.
  • 2.2.4.2 Internal Audit noted that a formal cost management approach has been defined, including a structured approach to measuring and reporting project costs. Project costs have been accurately tracked and regularly reported using the AAFC Information Management/Information Technology Investment Portfolio Dashboard to the Investment Planning Committee and through presentations to the ADM GCDP Steering Committee.

2.2.5 Project quality standards and management

  • 2.2.5.1 Internal Audit expected that quality management planning and reporting processes would be in place over the project deliverables, and product development activities.
  • 2.2.5.2 Internal Audit noted that a Quality Management Plan and related quality control processes have been documented to help ensure the overall success of the GCDP project.

2.2.6 Project resource management

  • 2.2.6.1 Internal Audit expected that current and required/end-state skills and competencies would be identified, structured resource management processes were in place, and that training plans would be provided to project staff to augment skillsets where necessary.
  • 2.2.6.2 The core GCDP project team is comprised of PB’s Service and Program Excellence Directorate, ISB, and PB’s Programs resources, and is augmented using professional services as required. Resource availability (including subject matter experts) has been identified as a key success factor for the GCDP project.
  • 2.2.6.3 Internal Audit noted that management has prepared a listing of resources and associated costing by year, including identification of subject matter experts and Microsoft Dynamics technology professionals, however the current human resource plan does not match required resources to tasks.
  • 2.2.6.4 In the absence of a defined resource plan that matches required human resources and tasks needed to meet delivery deadlines, risk exists that tasks may not be completed within scheduled delivery dates.

Recommendation 2
The Assistant Deputy Minister, Programs Branch should prepare a human resource management plan that matches required resources and tasks needed to meet delivery deadlines.

Management response and action plan:
Agree.
As the project moves from planning to the execution phase, more detailed sprint planning and linking resources to specific tasks to be completed will be developed. A resource management plan will be developed by July 30, 2019.

Leads responsible:
ADM, Programs Branch
Senior Director, GCDP Project, Programs Branch

Target date for completion: July 30, 2019

2.2.7 Project contracting

  • 2.2.7.1 Internal Audit expected that vendor management practices for GCDP would be established to deliver on the requirements of the resource management plan and project activities.
  • 2.2.7.2 Internal Audit found that effective vendor management practices for GCDP were established. Procurement for the GCDP project leverages existing contracting vehicles in place at AAFC. The Project Management Plan includes a procurement strategy for sourcing GCDP project resources. The strategy has been approved by the ADM GCDP Steering Committee.

2.3 Project organizational readiness and capacity

2.3.1 Organizational change plan

  • 2.3.1.1 Internal Audit expected to find that an organizational change plan would be in place that identifies stakeholders and includes change impact assessments. A change plan should be used throughout the project to minimize the impact that the GCDP has on stakeholders, help to support GCDP adoption, and its successful implementation.
  • 2.3.1.2 Internal Audit noted that the need for organizational change management has been recognized in key project documents and that two organizational change management resources were deployed on the project in March 2019 and April 2019 and are currently developing the organizational change management strategy and plan.
  • 2.3.1.3 While the project has resources in place to develop the organizational change management plan, limited organizational change management activities have occurred at the time of the audit. For example, a formal change impact assessment has not been documented and a process for monitoring stakeholder commitment and buy-in at the prototyping phase has not been defined.
  • 2.3.1.4 If organizational change management plans are not in place, there is a potential risk that Programs Branch users resist or do not buy-in to the GCDP solution once implemented. As a result, the GCDP solution may not realize its expected benefits in terms of results, outcomes, return on investment, adoption, usage, cost, and risk mitigation.

Recommendation 3
The Assistant Deputy Minister, Programs Branch should task the GCDP Project to ensure that organizational change management plans are prepared to successfully guide stakeholders through the transition to the GCDP and help ensure the project meets its intended outcomes.

Management response and action plan:
Agree.
Interviews with Programs Branch officers, managers, Directors, Directors General, and the Assistant Deputy Minister have been undertaken to guide development of an organizational change management strategy and associated plans. The organizational change management strategy will be developed by August 31, 2019.

Leads responsible:
ADM, Programs Branch
Senior Director, GCDP Project, Programs Branch

Target date for completion: August 31, 2019

2.3.2 Development Principles and Policies

  • 2.3.2.1 Internal Audit expected that clear principles and polices would have been developed to define the project approach.
  • 2.3.2.2 Internal Audit found that the project team has adopted an agile approach to help foster collaboration with the business and effectively manage risks throughout the project life cycle. An agile development team structure is in place and effectively designed to plan and deliver on development efforts. The GCDP project team is also using a centralized tool to manage the project deliverables, and to track and report on their agile development activities.
  • 2.3.2.3 Internal Audit noted that development efforts to date are limited to the prototype build. Since significant development has not yet occurred, Internal Audit was unable to assess effectiveness of development practices at the time of audit conduct.
  • 2.3.2.4 Internal Audit observed that the GCDP project development approach for the execution phase provides limited details on the working agreement for the project team and how the project's progress in execution will be measured and reported. For example, plans and metrics for measuring agile results and processes for reporting to the project sponsors, such as velocity, cycle time (days/per story) or quality metrics (that is defects) to provide measures on progress and to help forecast how long the project will take have not been defined.
  • 2.3.2.5 There is a potential risk that velocity and release progress is not measured which could result in poor estimates, cost overruns, delays or rework. In turn, this could result in inaccurate or incomplete reporting on project risks and progress reporting to oversight bodies.

Recommendation 4
As the GCDP project moves to the execution phase, the Assistant Deputy Minister, Programs Branch should task the GCDP Project to establish a working agreement for agile development team members, and define specific project metrics to provide visibility into release/project progress for the project oversight committees, including velocity, team capacity, schedule tracking, and product quality.

Management response and action plan:
Agree.
As the project moves from planning to the execution phase, more detailed sprint planning, inclusive of defined project metrics, will be established to allow for transparent schedule tracking and product quality monitoring. The detailed sprint plans will be available starting July 30, 2019.

Leads responsible:
ADM, Programs Branch
Senior Director, GCDP Project, Programs Branch

Target date for completion: July 30, 2019

2.4 Grants and Contributions Digital Platform prototyping

2.4.1 Solution requirement and design

  • 2.4.1.1 Internal Audit expected that solution requirements would have been identified, analyzed, reviewed by business users, and formally approved by project stakeholders.
  • 2.4.1.2 Interviews with Programs Branch staff indicated collaboration and involvement of business stakeholders from the Programs Branch and technical teams in the requirements gathering process through formal and informal methods: workshops, weekly meetings, and ad-hoc feedback sessions. In addition, prototype demonstrations were held to refine requirements and data models.
  • 2.4.1.3 Internal Audit determined that business and technical requirements have been defined during the planning phase with the intent of demonstrating that the solution design supports the achievement of program objectives. By incorporating stakeholder feedback in an iterative manner, the prototype is planned to deliver expected functional requirements for the following common business processes for (Voted) Grants and Contributions: Apply, Assess, Award, Administer, Audit, and Acquit.

2.4.2 Infrastructure and security

  • 2.4.2.1 Internal Audit expected that infrastructure and security requirements would have been analyzed and a plan established to ensure adherence to requirements from AAFC, and Government of Canada policies.
  • 2.4.2.2 The GCDP technology environments are being provided by Public Services and Procurement Canada (PSPC) through Shared Services Canada. Public Services and Procurement Canada is responsible for providing the GCcase software as a service to AAFC through a Memorandum of Understanding and service agreement. This arrangement leverages Public Services and Procurement Canada’s responsibilities for hosting the GCcase environment and the management of the Microsoft Dynamics Customer Relationship Management application.
  • 2.4.2.3 In terms of its supporting infrastructure, the GCDP project has identified and maintained compliance with the following relevant policies and guidelines:
    • 2.4.2.3.1 Treasury Board Secretariat's Policy on Project Management, Policy on Government Security, Directive on Privacy Impact Assessment;
    • 2.4.2.3.2 AAFC’s information security policies, information management, architecture, and gating framework.
  • 2.4.2.4 Internal Audit reviewed the conceptual solution architecture and process flows and determined that the business process, functional and technical requirements to execute and deliver the new Grants and Contributions solution have been adequately defined.

Annex A: Audit criteria

  1. Line of enquiry 1: Project governance
    Structures and mechanisms for project oversight are defined and in place to support and monitor the execution of the project.
  2. Line of enquiry 2: Project management
    Disciplines, tools, and coordination activities are in place to initiate, plan, execute, control, and complete the prototype phase.
  3. Line of enquiry 3: Organizational readiness and capacity
    Plans are in place to define and measure the organization’s requirements for readiness and capacity for change.
  4. Line of enquiry 4: Grants and Contributions Digital Platforms (GCDP) prototyping
    Requirements have been identified, analyzed, reviewed, and approved to confirm the solution design can meet or surpass program objectives.

Annex B: Abbreviations

AAFC
Agriculture and Agri-Food Canada
ADM
Assistant Deputy Minister
GCDP
Grants and Contributions Digital Platform
ISB
Information Systems Branch
PB
Programs Branch