Audit Report - SVRS Approved - Elections in Canada(The business case for ITR had been approved by EXCOM in February 2008.) Thus, SVRS had the benefits but also the burden of being

Internal Audit

Audit of the Special Voting Rules System (SVRS) Redevelopment Project

March 2010

Review of IT Business Continuity Plan 3

Table of Contents

EXECUTIVE SUMMARY .........................................................................................................................................5

1. INTRODUCTION..............................................................................................................................................8

1.1 BACKGROUND..................................................................................................................................................8 1.2 OBJECTIVES .....................................................................................................................................................9 1.3 SCOPE ..............................................................................................................................................................9 1.4 METHODOLOGY ...............................................................................................................................................9

2. STATEMENT OF ASSURANCE...................................................................................................................11

3. OVERALL ASSESSMENT.............................................................................................................................12

3.1 AUDIT CONCLUSION ......................................................................................................................................12 3.2 SCORE CARD..................................................................................................................................................13

4. OBSERVATIONS AND RECOMMENDATIONS.......................................................................................16

4.1 PROJECT GOVERNANCE..................................................................................................................................16 4.2 BUSINESS NEEDS............................................................................................................................................21 4.3 PROJECT MANAGEMENT ................................................................................................................................24 4.4 TECHNOLOGY ................................................................................................................................................31

5. CONCLUSION.................................................................................................................................................34

APPENDIX A – AUDIT METHODOLOGY..........................................................................................................35

APPENDIX B – LIST OF ACRONYMS, ABBREVIATIONS AND TERMS.....................................................38

Executive Summary 5

Executive Summary

Context The Special Voting Rules System (SVRS) redevelopment is a four-year project that was initiated in 2006, at an estimated cost of $7 million, and sponsored by the Electoral Events sector. Its mission statement is: “To facilitate electors in exercising their right to vote whether at home or abroad, meeting the needs of the electorate in an efficient, innovative and collaborative manner, while maintaining confidence in the electoral process.” The SVRS project was included in the 2009–2010 Risk-Based Audit Plan, which was reviewed by the Executive Committee (EXCOM) and the Audit Committee, then approved by the Chief Electoral Officer. It was considered important to include this project in the plan because it will replace the existing Special Voting Rules (SVR) system, which is based on obsolete technology, and modernize some SVR business processes. It should be noted that Elections Canada approved the SVRS project going ahead, knowing that it would encounter some challenges because it was proceeding in advance of the IT Renewal (ITR) initiative. (The business case for ITR had been approved by EXCOM in February 2008.) Thus, SVRS had the benefits but also the burden of being a “pathfinder” Information Technology (IT) project. Audit Objectives The objectives of this system under development (SUD) audit are to provide a timely, independent assessment and assurance to senior management that:

effective governance practices are in place

project management practices are sound

financial controls are adequate and support the decision-making and monitoring processes

business requirements are clearly defined and aligned with the business strategy

key IT controls required are included in the SVRS application Audit Scope The scope of the audit included reviewing and assessing the status of the SVRS project to date, including the costs, schedules and performance compared with the baseline, as well as the current risk and issues affecting the delivery of SVRS and its intended benefits. The audit was conducted from June to October 2009. Audit Conclusion The SVRS project team has implemented many elements of adequate project management processes, techniques and controls for managing the project, and these are generally in line with its importance and size. The team includes a strong involvement from the business side and has

6 Audit of the Special Voting Rules System (SVRS) Redevelopment Project

taken appropriate steps to meet business needs. The key areas that the team needs to focus on to complete the project include enhancing governance of IT projects at the agency level, managing the implementation and organizational changes, and resolving technological issues such as hardware requirements and the development environment. The governance structure should also be improved for the delivery of future IT projects of similar size. Alignment with other initiatives at Elections Canada, such as ITR and e-registration, has been a challenge, but this is to be expected for such a pathfinder project. The organization has decided to support both the ITR technology and the one used by SVRS. To complete this project, it will be important to enhance governance at the agency level, complete the technological requirements and develop an implementation plan. The key conclusions and recommendations for each audit objective are presented below. Effective governance practices are in place The governance of the project, including the senior management control framework, management of scope and change, and investment management, needs moderate improvement. To strengthen the governance of an IT project like SVRS, the agency should:

clarify the roles and responsibilities of IT, the business leader and the various committees

ensure that all IT project approvals are properly documented

implement standard, regular project status for IT projects

ensure better visibility of cross-project linkages and dependencies Project management practices are sound Project management and controls are largely in place and working. To further enhance project management, the agency should:

establish the operational and maintenance cost of the new system

develop a “lessons learned” report for future projects because this is a pathfinder project

develop an implementation plan to release the new system into production

prepare a plan to acquire and implement production infrastructure to ensure that the requirements can be met by the implementation date

Financial controls are adequate and support the decision-making and monitoring processes Financial controls exist at the project level, but at the agency level, they need to be improved for IT projects. There is a lack of financial reporting available at the agency level for monitoring and supporting the decision-making process. To strengthen financial controls, the agency should:

develop standard financial reports for IT projects

review Elections Canada procurement and contracting practices for future projects

Executive Summary 7

Business requirements are clearly defined and aligned with the business strategy Business-requirements management and solution-design processes are largely in place and adequate, thereby aligning the business requirements of SVRS with the business strategy. To further define and meet business requirements, the agency should:

review and complete the appropriate requirement relating to Privacy Impact Assessment (PIA)

ensure that detailed planning for the impact of changes on internal and external stakeholders is carried out

Key IT controls required are included in the SVRS application Generally, key IT controls are being included in the SVRS application. However, to improve the development process, the agency should:

establish a process for releasing IT projects and identifying points of authority

develop a quality assurance (QA) process for IT projects

complete the development environment framework

determine the licensing cost of the development environment framework


1. Introduction

1.1 Background

The mandate of Elections Canada is to be prepared at all times to “conduct a federal general election, by-election or referendum, administer the political financing provisions of the Canada Elections Act, monitor compliance and enforce electoral legislation.” One of Elections Canada’s key responsibilities is to ensure that electors have access to the electoral system. For some electors, voting by Special Voting Rules (SVR) is the only way to cast a ballot. The provisions for SVR are set out in Part 11 of the Canada Elections Act and apply to the following five categories of electors:

1. Canadian electors in their electoral districts who cannot or do not wish to go to an ordinary or advance poll to vote (local electors)

2. Canadian electors temporarily away from their electoral districts, whether in Canada or abroad (national electors)

3. Canadian citizens temporarily residing outside Canada (international electors) 4. Incarcerated electors 5. Canadian Forces electors (including civilians outside the country)

The Special Voting Rules System (SVRS) redevelopment is a four-year project that was initiated in 2006, at an estimated cost of $7 million, and sponsored by the Electoral Events sector. Its mission statement is: “To facilitate electors in exercising their right to vote whether at home or abroad, meeting the needs of the electorate in an efficient, innovative and collaborative manner, while maintaining confidence in the electoral process.” During an election period, a national centre for SVR processing is established. This centre is responsible for identifying SVR voters, processing SVR requests, mailing out national and international SVR ballot kits, receiving and processing completed SVR kits and counting, distributing and reporting returned SVR ballots. To improve the delivery of electoral events, and in response to stakeholders’ concerns and expectations, the SVRS is being redeveloped to enable Elections Canada to manage SVR electors’ application forms more efficiently and effectively, maintain the register of electors residing temporarily outside Canada and maintain stakeholder information. A number of factors have changed or delayed the initial SVRS baseline: a general election, implementing the Information Technology Renewal (ITR) infrastructure and resolving many interface and linkages issues. Nevertheless, a major portion of the SVRS functionality is expected to be ready by July 2010. The SVRS project was included in the 2009–2010 Risk-Based Audit Plan, which was reviewed by the Executive Committee (EXCOM) and the Audit Committee, then approved by the Chief Electoral Officer. It was considered important to include this project in the plan because it will replace the existing SVR system, which is based on obsolete technology, and modernize some SVR business processes.

1. Introduction 9

It should be noted that Elections Canada approved the SVRS project going ahead knowing that it would encounter some challenges because it was proceeding in advance of ITR. (The business case for ITR had been approved by EXCOM in February 2008.) Thus, SVRS had the benefits but also the burden of being a “pathfinder” Information Technology (IT) project.

1.2 Objectives

The objectives of this system under development (SUD) audit are to provide a timely, independent assessment and assurance to senior management that:

effective governance practices are in place

project management practices are sound

financial controls are adequate and support the decision-making and monitoring processes

business requirements are clearly defined and aligned with the business strategy

key IT controls required are included in the SVRS application

1.3 Scope

The scope of the audit included reviewing and assessing the status of the SVRS project to date, including the costs, schedules and performance compared with the baseline, as well as the current risk and issues affecting the delivery of SVRS and its intended benefits. The audit was conducted from June to October 2009.

1.4 Methodology

The methodology used to audit the SVRS project is based largely on the Control Objectives for Information and related Technology (COBIT) created by the IT Governance Institute. COBIT is a framework and supporting tool set that provide IT managers, auditors and IT users with a set of generally accepted measures, indicators, processes and best practices to assist them in maximizing the benefits of using IT and developing appropriate IT governance and control in a company. COBIT allows organizations to bridge the gap between control requirements, technical issues and business risks. The COBIT model, in turn, draws heavily on other standard IT approaches, including the IT Infrastructure Library (ITIL), a set of industry best practices. ITIL has become the de facto standard for IT service delivery and is being implemented across the federal government. The Software Engineering Institute (part of Carnegie-Mellon University) has published a series of critical risk measurements for both technology projects and technology-management organizations. The Capability Maturity Model (CMM) outlines the necessary or strongly desirable elements of Information Management (IM) and IT management required for achieving higher levels of acknowledged maturity. Using a combination of these frameworks, the proposed SUD risk methodology deals with the normal set of accountabilities involved in acquiring software and hardware, and managing technology projects. The following four domains are involved:

governance risk


business risk

project risk

technology risk For each domain, audit criteria are represented by one or more indicators. Indicators were chosen to represent key areas of interest to management based on the preliminary survey’s initial assessment of risk. See Appendix A for a more detailed description of the four risk domains. Our audit approach relied on reviewing existing documentation and gaining an understanding of the processes by conducting interviews with key Elections Canada staff and stakeholders. The audit team conducted 27 individual interviews and reviewed all relevant documentation.

2. Statement of Assurance 11

2. Statement of Assurance

In my opinion as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management. The opinion applies only to the entity examined, as described in the Audit Scope section of this report.


3. Overall Assessment

3.1 Audit Conclusion

The SVRS project is essential for implementing business processes and automated application functions at Elections Canada. The existing SVR process uses manual activities and is supported by an application that is based on obsolete technology. The situation is complicated by having multiple stakeholders from varying levels of government: internal users, three federal departments and 13 provinces and territories. Thus, the impact of a new SVRS on SVR business processes and stakeholders will be significant. As the project moves forward, management must ensure that key risks are addressed. This section presents the overall conclusion for each objective of the audit.

Effective governance practices are in place

The governance of the project, including the senior management control framework, management of scope and change, and investment management, needs moderate improvement at the agency level. Key elements will require attention to strengthen governance, including defining roles and responsibilities, documenting decisions, writing a project status report and creating better project linkages.

Project management practices are sound

Project management and controls are largely in place and working. However, key areas can be improved, including confirmation of operational and maintenance costs, the implementation plan and production infrastructure requirements. These key elements must be addressed in order to mitigate the risk to the success of the project.

Financial controls are adequate and support the decision-making and monitoring processes

Financial controls generally exist at the project level, but at the agency level, they need to be improved for IT projects. There is a lack of financial reporting available at the agency level for monitoring and supporting the decision-making process. To strengthen financial controls, the agency should develop standard financial reports for IT projects and review its procurement and contracting practices for future projects.

Business requirements are clearly defined and aligned with the business strategy

Business-requirements management and solution-design processes are largely in place and adequate. These processes ensure that the business requirements of the project are aligned with the business strategy. This has been observed during the audit as a strong element of the project. To further enhance this objective, the agency should review and complete the appropriate requirement relating to Privacy Impact Assessment (PIA) and carry out detailed planning for the impact of changes on internal and external stakeholders.

Key IT controls required are included in the SVRS application

Generally, key IT controls are being included in the SVRS application development. However, improvements are required in IT project assurance methods, the release process, quality assurance (QA), the development environment framework and its licensing cost.

3. Overall Assessment 13

3.2 Score Card

The score card below shows the current status of the SVRS project for each criterion at the time of the audit. The rating and explanation indicate the current implementation and level of improvement required, keeping in mind that SVRS is still a system under development.

Criteria Rating Explanation

Governance

Senior Management Control Framework

Committee structure Y Current structure is not effective: roles and responsibilities are unclear decisions are poorly documented at times reporting is not specific or consistent decision process has been variable

Stakeholder involvement G No project governance structure is documented. Situation is complicated, with 3 federal government departments and 13 provinces and territories. Management will continue to further engage stakeholders in the project.

Management of interrelated projects Y Relationship with other initiatives at Elections Canada needs attention. Late or non-delivery of ITR products and services affects progress.

Management of Scope/Change

Scope management process Y No adequate traceability of decisions; frequent changes in scope and technical direction.

Investment Management

Investment decisions/management financial reporting

Y IT project status report and financial reporting standards not established.

Business case management/benefits achievement

B Benefits included in business case and approval presentation. Business case must be updated to confirm operational and maintenance costs of new system.

Business Requirements

Business requirements management B A process has been used to identify and prioritize business requirements. Management must review and complete appropriate requirements for PIA.

Business solutions design G Final approved design documents are adequate, complete and appropriate.

Management of organizational development

B Planning for impact of changes on stakeholders has started. With implementation date pushed back, project team has more time to plan.

Project Management

Project Organization and Structure

Project management framework G Project team has implemented its own project management framework.



Standard project documentation B Despite not having a project charter, most roles and responsibilities seem to be understood. Several areas need strengthening: IT boundaries, management reporting, processes and procedures.

SVRS project organization, staffing and supervision

B Project organization structure is designed to include business, IT and contract personnel. Limited number of IT staff at Elections Canada affects engagement by the Information Technology Directorate (ITD).

Development Process

Adequate software development and acquisition standard

G Development and acquisition standard is adequate.

Business solution design approval G Approved design documents communicated with appropriate stakeholders.

Project developed in accordance with design specification and standards

G Project developed in accordance with design specifications and documented in approved standards.

Testing plan

G Project is detailed in design documentation. Test plan has been developed for each module.

Project assurance methods

O Some assurance tasks to support accreditation of new system identified. No current procedures describe what assurance tasks must be completed before migrating new system to production. Process and authority to approve releases not yet agreed to. Delay in implementation date mitigates some risk, but this is a critical area.

Implementation and fallback plan O Will use existing application. Later implementation date allows time to address this issue.

Project Control Processes

Approved integrated project plan G Has been developed and approved.

Project change control process Y Numerous changes to scope have occurred, with no consistent approach.

Quality assurance

B CGI Group Inc. (CGI) has a formal QA process. Elections Canada QA process tied to implementation of Information Technology Service Management (ITSM) project, planned for next year.

Project risk management G Process is in place. Risk is discussed in weekly application construction reports.

Problem management G Process has been established. Problems resolved quickly as both CGI contractor and Elections Canada personnel located at Coventry warehouse.

3. Overall Assessment 15


Procedures and standards for acquiring IT-related solutions

O Initial estimated project costs close to currently established contract costs. Contract options treated as contract amendments because contract value has changed. Possible perception of contract splitting. Elections Canada does not handle multi-year projects well in corporate plans. Current Treasury Board Secretariat (TBS) standard of full project costs plus first year operating costs not being met.

Technology

Infrastructure Management

Security, privacy, identification, authentication

Y ITR will no longer deliver the single sign-on (SSO). A report is available on level of implementation of recommendations in first Threat and Risk Assessment (TRA); PIA requirements should be reviewed.

Architecture and standards G SVRS pre-dates and is independent of ITR. SVRS is using different standards and agency has decided that it will support both standards.

Development/test environment O Development environment framework (DEF) not available when required (when application construction started). An alternative was installed by the project. DEF now partially furnished but still not complete; this has delayed the project, and some coding necessary for interfaces still cannot be started.

Technology Transition

Transition strategy – structure, requirements

O Technology infrastructure plan not complete or up-to-date. Final hardware requirements for production not yet set.

Transition planning Y Not yet started. July 2010 production date allows sufficient time to complete planning, acquisition and implementation.

G B Y O R Gr

Satisfactory Needs Minor

Improvement

Needs Moderate

Improvement

Needs Significant

Improvement

Unsatisfactory Unknown Cannot Be Measured


4. Observations and Recommendations

The details of each observation, conclusion and recommendation resulting from our audit procedures are outlined below.

4.1 Project Governance

The governance of the project is of concern. The structure and responsibilities of the committees providing oversight to the project are unclear and ineffective. Management reporting requires improvement, as does the management of linkages and dependencies among different IT projects. There is no adequate traceability of decisions taken regarding changes in scope and technical direction.

4.1.1 Senior Management Control Framework

Committee Structure

Two committees presently oversee the SVRS project: the IM/IT Committee and EXCOM. There was also at one time an SVRS Steering Committee, which addressed operational and administrative issues. However, that committee ceased to operate when the project direction was confirmed and the scope of the project substantially reduced to exclude integration with other Elections Canada systems.

The SVRS Steering Committee met during the first two years of the project but has not met since June 2008. Only one set of SVRS Steering Committee minutes exists.

The IM/IT Committee was set up in November 2007 at the request of the Chief Electoral Officer to form part of Elections Canada’s new governance structure, and its terms of reference were approved by EXCOM. There is now an initiative to review its membership and mandate.

EXCOM meets weekly at round tables to exchange information, meets monthly to make decisions and also occasionally has special meetings focusing on one topic. EXCOM is sporadically informed or consulted regarding SVRS. In fact, there were a few mentions of SVRS in 2009 at the weekly round tables, but no mentions of SVRS in any minutes of the monthly meetings. In addition, there were two special EXCOM sessions to discuss SVR in the spring of 2007 and 2008 for which no minutes are available. It was also noted during the audit that the roles and responsibilities of IT and the business leader were not clear or consistent during the project. Clearly defined roles and responsibilities would increase the effectiveness of governance of similar IT projects. Recommendation 1. It is recommended that senior management clarify and strengthen the governance of IT

projects, such as the SVRS initiative and other future similar projects. For example, it needs to clarify the roles and responsibilities of IT, the business leader and the various committees.

4. Observations and Recommendations 17

Management Response Responsible Position: Chief Information Officer We agree with the recommendation. The IM/IT Committee is undergoing a reorganization and has developed a work plan for the remainder of 2009–2010. The work plan includes specific items dealing with IM/IT governance and the role of the committees in the governance structure. This should provide a clear delineation of roles, responsibilities and relationships. This work will be completed by March 31, 2010. Project Approval Documentation

Partly as a result of the confusion regarding roles and responsibilities within the committee structure, management decisions concerning SVRS are not consistently documented. For example, although interviewees mentioned that at least two important decisions relating to SVRS were taken at EXCOM, no minutes of EXCOM discussions about SVRS are available. There is no established process for streaming decisions to one of the three oversight committees. For example, when it was decided recently to move the project implementation date to July 2010, the IM/IT Committee was not available to approve this decision in time. A different approval process had to be used, thereby diverging from previous practice. Recommendation 2. It is recommended that Elections Canada senior management ensure that all IT project

approvals are properly documented. Management Response Responsible Position: Chief Information Officer We agree with the recommendation. The IT sector, through the Project Management Office (PMO) and the extended management team, will ensure that all decisions relating to IT projects are properly documented. Copies of documented decisions will be maintained by the responsible IT manager and a copy sent to the PMO to be kept with the project file. The Chief Information Officer (CIO) will request that all project decisions taken by any committee be documented and that a copy of the document be sent to the PMO, to be kept with the project file. The SVRS Steering Committee’s roles and responsibilities will be updated to include approvals and decisions. The Committee will also be responsible for documenting them.

Management Information

Project and financial reporting standards are not established at the agency level. Management status reporting also needs improvement. Oversight committees do not receive specific or consistent reports and are not always available to provide decisions when needed.


There is no established standard for management reporting on project progress or financials. A review of committees’ agendas and minutes shows no consistent or regularly scheduled reports. Elections Canada has yet to define what it means by “satisfactory status reports,” which it expects to receive for IT projects. The manager of the Client Portfolio Management Office (CPMO) is developing a standard report format, but it is not yet completed. The first cost-versus-forecast report was produced only in July 2009. Recommendation 3. It is recommended that Elections Canada senior management ensure that standard regular

project status be implemented for IT projects. Management Response Responsible Position: Chief Information Officer We agree with this recommendation. As noted in our response to Recommendation 2, the work relating to standard regular project status for IT projects is currently being carried out by the IT PMO with the support of the IM/IT Committee, and it should be completed by March 31, 2010. This work will ensure that standard reporting dashboards are in place for all projects with an IT component. Financial reports are developed with Finance, as indicated in the response to Recommendation 5. Stakeholder Engagement

Although there is no project charter, the SVRS project card contains most of the elements of a project charter except for a project management structure, which is shown in the SVRS Steering Committee Terms of Reference. As a result, there is no documented project governance structure. The situation regarding stakeholder involvement complicates SVRS governance. Internal Elections Canada users are involved, as are other federal government departments – for example, the Correctional Service of Canada (CSC), the Department of National Defence (DND) and the Department of Foreign Affairs and International Trade (DFAIT) – and 13 provinces and territories. All of these users have their own requirements. Stakeholders were involved in the Business Requirements Definition (BRD) stage and are comfortable that their requirements are being addressed. They believe that the new business processes and system will reduce their workload and improve the quality of the data. They will also be involved in the testing. Some stakeholders interviewed were concerned about when SVRS would be implemented because there have been numerous delays. Management will continue to engage external stakeholders in project activities such as testing and keep them informed about project status. Interrelated Projects

Because project governance documentation is lacking, roles and responsibilities in project governance are undefined. The relationship between business and IT participation in the project is also undefined, and this has led to some instances of poor coordination.


Cross-project linkages are not well established. Tracking cross-project linkages is the responsibility of the CPMO manager, and this person included SVRS in a new overall cross-project plan only in the fall of 2009.

There have been several instances of poor communication regarding project dependencies – specifically relating to providing funds for production workstations and the impact of the selection of development tools. These are discussed further under Technology, below. The business areas are getting better at identifying cross-project impacts and becoming more aware of the potential of new technology. For example, Web-enabled, centrally held data are now seen as a benefit to Elections Canada. Recommendation 4. It is recommended that senior management ensure better visibility of cross-project linkages

and dependencies among IT projects. Management Response Responsible Position: Chief Information Officer The identification and tracking of project linkages and dependencies is now being accomplished through the annual planning cycle and the project reviews done by the IM/IT Committee. The annual planning cycle is used by the IT sector as the first opportunity to identify project linkages and dependencies. This is done through a review of the project proposals by the IT Front Office in the PMO, the review of the IT Sector Operating Plan by the IT Extended Management Team, the project proposal review by EXCOM and the Business/IT Sector Operating Plan proposals. Each of these activities is a step in the annual planning process that will identify any project linkages and dependencies. The IM/IT Committee has a role to review and identify possible project linkages and dependencies, and it will continue to carry out this role.

4.1.2 Management of Scope/Change

The impact of interrelated projects at Elections Canada makes managing change and scope more complicated. SVRS depends on services and capabilities from other projects such as ITR. It also depends on having certain architectural standards available. (This is discussed further under Technology, below.)


There is inadequate record keeping of all decisions affecting the scope of SVRS taken at different forums. The audit team has not seen any documentation of scope discussions at any of the oversight committee meetings. For example, important decisions regarding data architecture and the selection of development tool standards for SVRS were apparently made at EXCOM meetings, but no minutes exist. Traceability of decisions regarding scope changes and technical directions is inadequate. As indicated in Recommendation 2, decisions relating to IT projects should be documented.

4.1.3 Investment Management

Investment Decisions/Management Financial Reporting

There is no formal identification of which oversight committee should consider financial information. The practice is that project budgets go to EXCOM, not to the IM/IT Committee. Project financial tracking is carried out, but it does not include the time of full-time IT staff. Yet Elections Canada expects to be able to compare costs of projects. Financial reporting is carried out mostly at the project level, without guidance from Finance. There has been a lack of visibility of costs versus forecasts at the agency level to support decision making and monitoring of the investment. This may also mean that SVRS project costs cannot be accurately compared with other projects. The full financial report, covering multiple years and showing costs versus forecast, was not provided to the Finance sector until July 2009, approximately 2.5 years after the start of the project.

Contingency funds were cut from all projects this fiscal year. Thus, there is a risk that the project will encounter problems, such as those enumerated in the risk registers (Excel spreadsheets), which cannot be resolved. In the past, management has usually cash-managed for contingencies, expecting to receive unspent funds from other large projects. There is a lack of reporting on overall financial impacts. Uneven reporting methods mean that cross-project comparisons may be inaccurate. The lack of contingency funds is an unknown risk. Recommendation 5. It is recommended that Corporate Services work with the SVRS project manager to develop

standard financial reports for IT projects. Management Response Responsible Position: Chief Financial Officer We agree with this recommendation. A newly created Financial Management Advisors unit has been set up, and its mandate is to work with operational sectors to assist them in capturing and reporting financial information in a consistent manner. A standardized report has been created for IT projects and will be distributed in time for next fiscal year’s reporting period. The finalized version should be available March 31, 2010.


Business Case Management/Benefits Achievement

A very detailed business case was produced in April 2007. The main focus was on the need to replace the existing obsolete and unsupportable applications. In many cases, costs were based on assumptions from preliminary operating and voter data collected in 2006 and 2007 and on early assumptions about the cost of application development. The business case anticipates and recommends an update because it states: “It is expected that ulterior updates to the model and the Business Case will include updating the underlying model sheets.”

The business case cost overview shows that projected operating and support costs (excluding system development costs) are actually $3 million higher over the next three elections with the new system than without it. However, this is based on support cost estimates for ongoing application maintenance and production support that have not been confirmed by ITD. It is important to revisit the cost-benefit analysis and finalize it.

Many non-financial benefits are mentioned in the business case and in the project card. The project will reduce the number of repositories that store elector information, reduce the number of SVR applications requiring follow-up, reduce the time required to retrieve elector information and increase the volume of applications that can be handled. Recommendation 6. It is recommended that Elections Canada senior management establish the operational and

maintenance cost of the new system. Management Response Responsible Position: Director, Alternative Voting Methods We agree with this recommendation. The operational and maintenance costs of the new system will be established by March 31, 2010.

4.2 Business Needs

Throughout the SVRS project life cycle, a process has been followed to identify and prioritize the SVR business requirements. In general, the business rules and processes are clear and stable. The processes used to build the solution appear to be robust and have been followed. However, it was difficult to assess whether the business organization will be able to accommodate the changes because implementation planning has just started.

4.2.1 Business Requirements Management

The SVRS business rules are based on the provisions for SVR set out in Part 11 of the Canada Elections Act. These rules apply to five categories of electors, and for each category, a set of business rules and related processes is required. The business rules must meet the requirements of Elections Canada’s internal stakeholders as well as external stakeholders from DND, CSC, DFAIT and the 13 provinces and territories. This significant level of complexity means that sound management of the business requirements is essential. The audit team has found that the project has been well managed, addressing the


business needs. The business leader has been strongly involved in all phases of the IT project. The SVRS project has been following an iterative process to identify and prioritize business requirements. It has included all users and stakeholders at various stages in the process, and sessions regarding tools such as Joint Application Design were held with stakeholders. The business requirements are being managed by using a traceability matrix and related processes. Requirements traceability documents the life of a requirement throughout the life cycle of a project. It enables a project to trace back to the origin of each requirement and every change made to the requirement. A high-level preliminary PIA was conducted. A PIA provides a framework for ensuring that privacy is considered when programs or services are being designed. A questionnaire was completed to determine whether a full PIA was required; it was decided that it was not. Subsequently, Elections Canada reviewed its policy on PIAs, and it would be important to review whether a PIA is needed for the new SVRS.

Recommendation 7. It is recommended that SVRS project management review and complete the appropriate

requirements relating to PIA. Management Response Responsible Position: Director, Alternative Voting Methods Elections Canada agrees that a PIA should be conducted for SVRS. The calendar for the completion of this PIA will be determined at the conclusion of the 2010–11 business planning cycle, following an assessment of the organizational priorities and capacity to conduct this PIA.

4.2.2 Business Solution Design

The SVRS project has been following a system development methodology. It used a phased iterative process throughout the project life cycle, one that translated the business requirements into the proposed solution. A traceability matrix has been used to track the business requirements for each phase. The BRD defined the business requirements at a high level. A technical proof of concept was conducted to determine the feasibility of the solution. The Functional Design Document (FDD) was built on the baseline provided by the BRD by specifying the scope of functionality required to satisfy the requirements. The specifications contained in the FDD were used to design the new system (using the Application Design Definition, or ADD). The audit has included reviewing the final, approved BRD, FDD and ADD. They appear to be adequate in terms of degree of completeness, appropriateness and business satisfaction.

4.2.3 Management of Organization Development

The SVRS application being developed will have a significant impact on current business processes for stakeholders – in other words, how they do their business. Processes that are now primarily manual will be carried out electronically. Here are some examples.


Canadian Forces staff will be able to complete and submit their Statement of Ordinary Residence on the Internet. Information will be shared electronically between DND and Elections Canada.

CSC and the provinces and territories can opt to move from a paper process to a standalone application for registering incarcerated voters. They will be given the SVRS application and supporting materials on a Universal Serial Bus (USB) storage device.

Elections Canada users will have automated tools and procedures for conducting their work. Instead of receiving hard copy applications by fax or mail at Elections Canada, a new partner, Public Works and Government Services Canada (PWGSC)/Matane, will receive the applications. PWGSC/Matane will use an imaging process to scan the applications into an electronic format, then send them electronically to Elections Canada.

The project team has been conducting sessions with the various stakeholders to plan the implementations. A memorandum of understanding is in place for Phase 1 of the outsourcing project, which will send scanned images of application forms to Elections Canada for printing and data capture. Negotiations are underway with PWGSC/Matane to develop an agreement/contract that defines the services to be provided and the service levels required to move to Phase 2 of the outsourcing project (Phase 1 services plus data capture and transfer to Elections Canada). Recently, planning began to ensure that stakeholders have the ability and capacity to deal with the overall change. Planning for the impact of the changes on the business process and the organization is key for successful implementation and critical for efficient and effective use of the new application. If the implementation date had not recently been pushed back, the audit would have identified this area as high risk. As it is, the project team now has more time to plan and carry out the necessary activities for managing change. This will mitigate some of the risk, but it still remains an area of concern. Recommendation 8. It is recommended that SVRS project management ensure that detailed planning is carried

out to measure the impact of change on internal and external stakeholders. Management Response Responsible Position: Director, Alternative Voting Methods We agree with this recommendation. A detailed system rollout strategy will be developed in the first quarter of the 2010–11 fiscal year for each system component; it also considered and involved relevant internal and external stakeholders. The strategy should be completed by June 30, 2010.


4.3 Project Management

4.3.1 Project Organization and Structure

Project Management Framework

Elections Canada does not have standards in place for the project management framework, although it is in the process of developing them. SVRS pre-dates these activities, and as a result, it has implemented its own project management framework. Normally, a project charter would describe the project management framework (e.g. roles, responsibilities, accountabilities, processes and procedures for key project functions such as risk management, QA and change management). For this project, there is no one document that describes the project management framework. The business case, management presentations and SVRS project score card all describe elements of the management framework. The project management framework is not critical at this stage and does not present a serious risk to the success of the project. Standard Project Documentation

As indicated in the previous section, a project charter is usually the main document to describe the project, the roles of all involved and the desired outcomes. There is no SVRS project charter, but roles and responsibilities seem to be understood nevertheless. A charter normally describes key project management functions and related processes, such as QA and risk management, among others. SVRS does not have one source for this, but some of it is outlined in general terms in presentations and other documentation. No one has been specifically assigned the role of QA, and there is no formal QA process. A risk management function exists, but it is not being actively pursued. Management reporting for the SVRS project is not clearly defined, and full reporting on both business and IT to senior management is unclear. Elections Canada does not have any standards in place for what a project should report and to what level of management. The availability of project standards and toolkits should be improved. Management should ensure that a “lessons learned” study is completed to benefit future projects. SVRS Project Organization, Staffing and Supervision

The project organizational structure is designed to include business, IT and contract personnel. The project manager of SVRS is responsible for managing business project staff, who include analysts and testers. They have expertise and knowledge of SVR and Elections Canada business as subject matter experts. The IT project manager coordinates between the project team and IT staff who are not members of the project. The IT manager spends about 20 per cent of his time on the project. Recently, another IT staff person was assigned to assist the IT manager, and he will spend 80 per cent of


his time on the project on site. Elections Canada is a small organization, so it only has a small number of IT personnel available to participate in the project. Contract staff are headed up by a project manager from CGI, who supervises CGI personnel by way of meetings and day-to-day interaction. During the various phases of the project, CGI has provided personnel with different skill sets – programmers, architects, data modellers, security analysts, etc. Some CGI personnel have a fundamental knowledge of Elections Canada subject matter. Not all personnel are needed for each phase. There are weekly meetings with project managers from SVR, Elections Canada IT and CGI. This provides a mechanism to manage and supervise all parties. Recommendation 9. It is recommended that SVRS project management develop a “lessons learned” report for

future projects.

Management Response Responsible Position: Director, Alternative Voting Methods We agree with this recommendation. As part of the post-implementation review, a “lessons learned” report will be produced and filed with the PMO as part of the project documentation. This review should be completed by March 31, 2011.

4.3.2 Development Process

Software Development and Acquisition

The SVRS project follows the standard project phases of BRD, detailed design, construction, testing and implementation in the production environment. The project team has made some adaptations by creating a more iterative process for areas like user requirements identification. The project team has indicated that they are getting a better product as a result. The standard for software development and acquisition being used by SVRS is adequate and low risk, especially at this stage in the development life cycle.

Business solution designs have been communicated and articulated between systems development (CGI) and stakeholders during all phases of the project. All design documents have been reviewed by key stakeholders and approved by Elections Canada.

Internal stakeholders were involved in defining the business requirements. They received demos and participated in the pre-user acceptance testing (UAT). Meetings, presentations and e-mails have facilitated communication with external stakeholders. Their level of involvement has varied because the level of change and complexity of their business processes is different.

The SVRS project is being developed following the design specifications as well as CGI development and documentation standards. Software components are seen as configurable items and baselined. These and related processes, such as configuration management, are based on CGI being granted certification by the International Standards Organization (ISO) for industry standards.


The project is currently at the stage of developing test plans and processes. A test plan has been developed and is documented in the ADD. The approach to testing is iterative.

CGI is responsible for system testing. It developed test scripts that are being used to test each module of the application as it is constructed. Results and defects are recorded in Mantis (the automated tool used by CGI) and monitored through to resolution.

Elections Canada is responsible for UAT. It has expanded on the test scripts developed by CGI, and its testing will verify the business processes. Arrangements have been made and agreements reached with the stakeholders to be involved in testing (internal users, DND, CSC, DFAIT and the provinces and territories). The latter will receive a standalone application on a USB storage device. The new processes that will be carried out by PWGSC/Matane are also being tested.

In March and April 2009, a pre-UAT was conducted. It was primarily a review of documentation and preliminary screen shots. SVRS users were asked to comment and verify whether their requirements were being met. Other demos have been used to verify functionality. Users have provided feedback and suggestions, but so far, no major issues have been identified. Test results will be reported and monitored to ensure that defects are corrected and workarounds identified. The overall test plan must be signed off. Project Assurance Methods

New applications must normally go through an accreditation and certification process before being moved into the production environment. This ensures that they adhere to certain organizational standards, such as security standards. Externally defined requirements and assurance tasks are identified and completed as part of the release process. Industry standards stipulate moving applications through the various environments (development, test and live) as well as user involvement. The audit identified some assurance tasks that support the accreditation and certification of the new system. An IT Security Certification and Accreditation Guide existed at the time of the audit, although still in draft form. SVRS was the first application to go through the certification and accreditation process, which was finalized in December 2009. It is important to finalize the guide and formalize the certification and accreditation process for IT projects. The overall project assurance methods still need to be developed. There are currently no in-house procedures or processes that describe what assurance tasks must be completed before a new system moves into production. It is not yet clear what activities and tasks will be part of the release process. Pushing back the implementation date mitigates some risk, but this is a critical area that has not yet been clarified. Recommendation 10. It is recommended that a process for releasing IT projects is established and authority points

identified.


Management Response Responsible Position: Chief Information Officer We agree with this recommendation. Two separate release processes are being developed. The process for releasing software into the DEF will be completed by September 2010. It will include the release processes for the development, test, model production and production environments in DEF. The ITSM initiative will develop the release processes required for legacy systems. The first set of processes will be completed by March 2011. The QA function in the Systems Development and Client Service Organization will work with the ITSM project to ensure that the QA, test and release processes for both DEF and legacy environments are identified, documented and implemented by March 2011. Implementation and Fallback Plan

Projects must have an implementation plan that includes fallback or back-out. The SVRS project contingency plan is to use the existing application. The process of approving releases, and the authority to do so, has not been agreed to yet. However, now that the most recent contract amendment has been approved, implementation planning has begun. With the implementation date being pushed back, there is additional time to address the issue, but it is a risk nevertheless. Recommendation 11. It is recommended that SVRS project management develop an implementation plan to release

the new system for production use. Management Response Responsible Position: Director, Alternative Voting Methods Agreed. An implementation plan will be developed by March 31, 2010.

4.3.3 Project Control Processes

Approved Integrated Project Plan

Project planning includes using schedules to plan, and subsequently report on, the progress of a project. Careful planning at the outset, as well as during a project, can help avoid costly mistakes. It guides project execution, control and monitoring. It also increases the probability that the project will accomplish its goals on schedule and within budget. The SVRS project is required to participate in the annual planning process at Elections Canada. The project has developed an integrated project master plan, which brings together each individual plan to develop an overall view of the project. More detailed plans for sub-projects, such as UAT and PWGSC/Matane outsourcing, are tracked using Microsoft Project.


As part of its weekly status report, CGI provides an update on its part of the project plan. It is specific to the deliverables indicated in the contract and related amendments. Project Change Control Process

The purpose of change management is to control and approve changes to the scope (functionality), schedule (time frame) and/or costs of a project and system. SVRS has experienced three major changes.

The project had to be put on hold during the general election.

The scope had to be reduced. As a result of centralizing various corporate databases and changing the technical infrastructure, it is not possible to meet the requirements for uploading information at this time.

The project needed to be released into production and needed ongoing maintenance support from CGI.

The project used change request forms to explain the reason for a change and its impacts. It is unclear who or what committee has the authority to approve changes – whether it is the SVRS Steering Committee, the IT Committee or EXCOM. Changes have followed different approval processes, and all have resulted in amendments to the CGI contract. The contract authority is IT, and these amendments received that authorization. This issue is related to the lack of clarity regarding project governance and approval processes. The recommendations in governance address this observation. Quality Assurance

The SVRS project does not have a formal QA function. However, this does not mean that Elections Canada does not review CGI deliverables for completeness, accuracy or quality. When CGI submits deliverables, they are reviewed by various members of the project team, some users and some IT personnel. Comments on the reviews are usually given by e-mail. They vary in level of detail from general to very specific. Comments are consolidated in Excel spreadsheets, then sent to CGI to be addressed and incorporated into the subsequent version of the deliverables. The audit verified that this process was followed for the BRD and ADD deliverables by examining the spreadsheets and related e-mails. There were comments from the business, IT and other stakeholders. The version control sheet for all design documents indicated that the version, for example, was revised to include comments from Elections Canada. Recommendation 12. It is recommended that a QA process be developed for IT projects.


Management Response Responsible Position: Chief Information Officer We agree with this recommendation. A QA manager was hired by Systems Development and Client Services in February 2010, and he has begun the work of establishing a documented QA function in the IT sector. This work will need to be carried out in conjunction with the ITSM and DEF projects and supported by the agency-wide deployment of the updated System Development Life Cycle and project management disciplines. The QA framework will need to be established concurrently with testing activities that support the rollout of a number of agency systems. We expect to have basic standards, governance and processes in place by the end of the 2010–11 fiscal year, with a fuller functional presence the following year. Project Risk Management

Risk management is a process that identifies, analyzes, controls, mitigates and communicates risk. It is an important management tool because it identifies areas that could negatively impact the ability to deliver a project on time, within budget and to requirements. The SVRS project has a risk management process in place that identifies risks and records them in a risk register. For each unique risk identified, a standard set of data is collected. Probability and impact are rated on a scale starting at very low and ranging through low, moderate and high to very high. Risks are colour-coded to indicate low, medium and high, and thresholds have been established to assign the levels. The CGI contract defines the roles and responsibilities of CGI with respect to risk management. Part of the firm’s contractual obligations is the requirement to identify risks from its perspective and report them to Elections Canada. Risks are reported in CGI’s regular status reports. The most current risk register was last updated in January 2009. Risk management is not a regular agenda item at IM/IT Committee meetings. It was not evident from minutes or agendas of the various levels of management meetings whether risks or risk management was discussed. However, risk management is discussed at internal project meetings and included in the weekly application construction report. Problem Management

A problem management process has been established. The SVRS project is using Mantis to identify, assess, document and track problems until they are resolved. Items identified during each stage of the project life cycle are logged into the system database; they can include problems, issues, decisions (for example, to use a workaround) and any defects discovered during testing. The database is monitored at least once a week and more frequently during key activities, such as testing. The automated tool has the capability to generate status reports, a feature used regularly by the project team.


Project management establishes a priority for resolving each item and assigns it to the appropriate person (such as a developer). Items remain “open” until Elections Canada and CGI agree that the problem has been resolved and sign off on it. With CGI personnel working at the same location as the Elections Canada project team, problems and issues can be resolved quickly. Procedures and Standards for Acquiring IT-Related Solutions

Procuring services was a competitive process, with firms using the Government of Canada on-line tendering service (MERX) and responding to the Request for Proposal (RFP). A formal evaluation process was conducted, and a contract was awarded to CGI for the BRD phase of the project. Contract arrangements for hardware are the responsibility of IT. There have been issues in acquiring equipment and implementing the DEF as well as the testing and production environments, including several delays and missed milestones. The IT project manager is the technical authority for the contract, and he approves the invoices. He is not involved day to day with CGI personnel or deliverables; this is done by the business project manager. Every week, the two have a status meeting, and before approving any invoices, they analyze and discuss the deliverables and invoices. The RFP and the CGI contract included a clause stating that Elections Canada might continue with the same firm for additional phases of the redevelopment by using the existing procurement vehicle. Completing each phase produced deliverables that CGI used to develop a proposal for the next phase, and on the basis of each proposal, an amendment was issued to CGI for the next phase. Of the 15 amendments approved, five have necessitated increases in dollar amounts. The contract was set up to allow for options during the life of the agreement that aligned with management decisions to continue with system development. Although an estimate was generated early in the project as part of the business case, the initial contract was not awarded for that estimate. More normal contracting arrangements have an overall contract limit for the whole project and then use task authorizations to manage the deliverables. In this case, exercising options meant changing the contract amount in five cases – a condition that financial managers usually consider an amendment. The use of amendments could give the perception of contract splitting. In addition, redefining the size of a multi-year project over the life of that project does not permit Elections Canada to handle such projects well in corporate plans. The current TBS standard of full project cost estimates plus first year operating costs is not being met. Recommendation 13. It is recommended that Elections Canada review its procurement and contracting practices

for future projects.


Management Response Responsible Position: Chief Financial Officer We agree with this recommendation. To enable the organization to develop consistent procurement and contracting practices, a procurement framework and plan are currently being developed. This will include, among other topics, roles and responsibilities, processes for a competitive solicitation process, non-competitive procurement and security considerations. Identified staff will receive training on when and how to use this framework. It is anticipated that this framework will be ready for distribution to the various sectors throughout the organization by June 2010.

4.4 Technology

4.4.1 Infrastructure Management

Security, Privacy, Identification and Authentication

To allow the new SVRS system to recognize Elections Canada users without unduly taxing its resources, an SSO solution was proposed. Originally, the SSO solution was to be developed by the ITR project. However, when the project team realized that an SSO would not be available in time, the Architecture and Standards Board decided to go with a project solution. The decision about an SSO is currently not a priority; it may only be made toward the end of 2010. SVRS may have to adapt to a new corporate standard in the future, with unknown development costs. Defining the level of security needed for a system requires an analysis of threats and risks. The first TRA of SVRS was completed in 2007 and the second in mid-December 2009. A report is available that describes to what extent the recommendations from the first TRA have been implemented. About half of the recommendations have been completed. Architecture and Standards

One of the objectives of the ITR project was to determine a standard development toolset for Elections Canada. However, SVRS started ahead of ITR, so a decision had to be made about the toolset within the context of SVRS instead. Estimates were made about the cost of reworking the SVRS system to align it with ITR standards, assuming that ITR would result in a single standard for toolsets. This has not happened, so Elections Canada will continue to support both the Microsoft .NET stack and an Oracle/Java stack, making rework unnecessary.

Development/Test Environment

The DEF was not available when application construction began, so the project team installed an alternative. The DEF has now been partially furnished by ITD, but it is not complete and is being observed for stability. This has delayed the project, and some necessary development of interfaces still cannot be started. The delay in setting up the DEF delayed the SVRS project, and in September 2009, the project implementation timeline was revised and the production date set at July 1, 2010. The delay


worked to the advantage of the project in some ways because it allowed testing to be done more thoroughly over a longer period of time. Licensing costs for the DEF are still unknown. Licensing is an issue because of the number of environments being created in the DEF and because it is a complex task to calculate the number of licences needed if the virtualized desktop concept is used. Recommendation 14. It is recommended that the CIO ensure that the DEF be completed as soon as possible. Management Response Responsible Position: Chief Information Officer We agree with this recommendation. The work on the initial version of the DEF will be completed by March 31, 2010. This will complete the implementation of the development, test and model production components of the DEF. In the next fiscal year, a production environment will be added and capacity increased in all DEF components to allow new projects to enter the DEF. This work will be completed by March 31, 2011. 15. It is recommended that ITD and SVRS project staff determine the DEF licensing costs as

soon as possible. Management Response Responsible Position: Chief Information Officer We agree with this recommendation. A DEF licensing working group has been created to review the licensing issues related to the DEF and to propose solutions. The working group has resolved many of the current licensing issues; however, future issues should be identified during the annual planning process so that appropriate resources can be set aside to address the licensing requirements of the DEF. The licensing costs for SVRS were resolved in February 2010.

4.4.2 Technology Transition

Strategy for Acquiring and Implementing the Technology Infrastructure

The original Technology Architecture Design for SVRS was drawn up in August 2007, and it has not been updated formally or in a comprehensive way. For example:

ITD has not received any updated information regarding storage or production server requirements. The process for communicating infrastructure requirements to IT is through meetings held between the business project leader and the IT project leader, but there is no overall comprehensive documentation of these requirements. There is also confusion about whether the specifications have been provided and whether the equipment has been ordered.

Infrastructure capacity may be an issue because of the potential for high volumes of applications received at critical times and during short time frames, but no technical capacity study has been carried out since 2006.


Development and testing of technical specifications for the production workstations are not complete. Telecommunications requirements are not fully understood. There are no measures in place to provide business continuity planning for SVRS.

In addition, no decision has been made about whether SVRS will be built and implemented at Elections Canada or at PWGSC. SVRS is still running out of the server rooms at Elections Canada. Whether PWGSC will provide a second hot site data centre is still to be decided. The Chief Electoral Officer has said that redundancy will be established after the primary data centre has moved. This is a medium risk, although the risk is lower if the primary data centre is moved to PWGSC. An audit of the business continuity plan was performed between August and November 2009. Transition Planning

The production date for SVRS has recently been extended to July 2010; this should allow more time to complete planning, acquisition and implementation. Conversion of old data is low risk. Implementation plans, such as migration strategy, fallback and contingency, have only recently been started. A detailed conversion plan has been created for the data. A straightforward conversion plan is in place for the old SVRS application data, and there is no requirement to keep the old FoxPro applications. Recommendation 16. It is recommended that senior management ensure that SVRS project management prepare a

plan for acquiring and implementing production infrastructure, specifically to determine whether all the necessary infrastructure can be implemented before the SVRS production deadline.

Management Response Responsible Position: Chief Information Officer We agree with this recommendation. Management will ensure that staff prepare a plan for acquiring and implementing production infrastructure, specifically to determine whether all the necessary infrastructure can be implemented before the July 2010 deadline. This plan will be in place before March 31, 2010.


5. Conclusion

SVRS will modernize SVR business processes at Elections Canada, and it has had strong buy-in from the business side. We hope that this audit will provide an objective and useful audit of the system under development. Some recommendations, such as those in the Project Governance section, will be important for future similar IT projects at Elections Canada. Other recommendations are more specific to completing the SVRS project successfully. This project was started before the ITR initiative, so it was a pathfinder project. Nonetheless, the recommendations included in this report should support the completion of this project and enhance the management of future similar IT projects. The audit team would like to thank all those in the SVRS directorate and IT who collaborated during this audit.

Appendix A – Audit Methodology 35

Appendix A – Audit Methodology

This appendix describes the methodology used in carrying out the audit of the SVRS redevelopment project. The table below outlines the classes of risk involved and what areas in each class were examined. Class of Risk Areas Examined and Explanation

Project Risk This class of risk pertains to the presence of a well-defined structure of roles, responsibilities and authorities within which the project operates and within which all major decisions about the scope and objectives of the project, including changes to same, are made.

Senior Management Control Framework: The relationship of the project to strategic plans; the assignment of responsibility, owner/sponsor, project structure, committee structure and linkages to related projects; the roles of key organizations and people; the flow of management information; and communications within the organization and with clients. Change/Scope Management: The ability of the project to adapt to changing internal and external conditions. Strategic issues include project scope management; risk management; and relationships to other key projects, initiatives and/or events. Investment Management/Benefits Achievement: The initial business case and the process used to measure project benefits achieved by the organization through the project.

Business Risk This class of risk pertains to the clarity and stability of the business rules and processes from which a system’s requirements will be derived, to the integrity and robustness of the design that will be prepared to address those requirements, and to the capacity of the organization to organize itself for and to manage the changes that the introduction of a new system implies.

Business Requirements: The specification of business requirements related to the processes under consideration. Strategic issues include the breadth of business change represented by the new requirements; the availability of expert users to contribute to the definition; and the level of complexity of the business rules being defined.


Class of Risk Areas Examined and Explanation

Business Solution Design: The process in place to translate the business requirements into the solution; the relationship of investment in cost and time to functionality delivered; the internal control framework; and the provision for security (i.e. confidentiality, integrity and availability). Management of Organizational Development: The impact of the project on the major business processes of the sponsoring organization and the ability of the organization to deal with the overall change.

Project Risk This class of risk pertains to the internal organization and management of the project and to its monitoring, reporting, control and communications functions. This class of risk also considers the tools, techniques, methods and procedures needed to do the actual work of the project: to understand the requirements that have to be addressed, and from that understanding, to design, develop, implement and make operational a relevant, reliable, usable system.

Project Organization and Structure: The roles and responsibilities of each major organizational component of the project structure; the records of decisions made; and the type and quality of project management information made available on a regular basis. Development/Acquisition Process: The existence of a formal process definition, with milestone deliverables; solution design integration and cohesiveness; construction risk minimization; appropriate test planning and management; certification and accreditation; and transition management. Project Control Processes: The planning and scheduling methodology used, critical path management and related resource levels; budgets, financial reporting and variance analysis; project change management and problem and issue identification and resolution; QA strategy; communications vehicles; contract management and amendment control.

Appendix A – Audit Methodology 37

Class of Risk Areas Examined and Explanation

Technology Risk This class of risk pertains to the degree of inherent risk in the technology platforms chosen to support the system. Newer and less widely proven platforms have substantially higher risk than mature and widely used platforms. Not only is there a greater probability of flaws in a newer platform, but the know-how to deal with them is rare. This class also pertains to the transition of the application to the infrastructure within which it will operate. A newly developed and implemented infrastructure poses more risk than a structured mature one.

Infrastructure: The degree of project conformity to the organization’s technical standards and methods, and to its technology environment; and the impact the project will have on this infrastructure. Technology Transition: The readiness of the organization to deal with the new technology; overall technology configuration management; and the ability of the organization to offer support (both short- and long-term).


Appendix B – List of Acronyms, Abbreviations and Terms

Acronym Full Form/Explanation

ADD Application Design Definition

BRD Business Requirements Definition

CGI CGI Group Inc. (the system development contractor delivering SVRS)

CIO Chief Information Officer

CMM Capability Maturity Model (a measure of software development maturity)

COBIT Control Objectives for Information and related Technology

CPMO Client Portfolio Management Office

CSC Correctional Service of Canada

DEF Development Environment Framework

DFAIT Department of Foreign Affairs and International Trade

DND Department of National Defence

EXCOM Executive Committee

FDD Functional Design Document

FoxPro An older database management package used at Elections Canada

IM Information Management

ISO International Standards Organization (source of international standards for IT management and system development)

IT Information Technology

ITD Information Technology Directorate

ITIL IT Infrastructure Library

ITR IT Renewal

ITSM Information Technology Service Management

Java System development software

Mantis An automated software development and project management tool used by CGI

MERX Automated tendering service used by the Government of Canada

PIA Privacy Impact Assessment

PMO Project Management Office

PWGSC Public Works and Government Services Canada

QA Quality Assurance

RFP Request for Proposal

SSO Single sign-on

SUD System under development

Appendix B – List of Acronyms. Abbreviations and Terms 39

Acronym Full Form/Explanation

SVR Special Voting Rules

SVRS Special Voting Rules System

TBS Treasury Board Secretariat

TRA Threat and Risk Assessment

UAT User Acceptance Testing

USB Universal Serial Bus

Audit Report - SVRS Approved - Elections in Canada(The business case for ITR had been approved by EXCOM in February 2008.) Thus, SVRS had the benefits but also the burden of being

Documents