ACC626 Continuous Auditing C Zeibart

UNIVERSITY OF WATERLOO

Masters of Accounting

CONTINUOUS AUDITING FRAMEWORKS AND IMPLEMENTATION

ACC 626

Professor Malik Datardina

University of Waterloo

Waterloo, Ontario

Prepared by:

Caroline SC Ziebart

June 24, 2012

i

Table of Contents

1.0 Introduction ................................................................................................................................. 1

2.0 Continuous Auditing Frameworks and Models ........................................................................... 1

2.1 Difference in Audit Approach ......................................................................................... 1

2.2 Benchmark Continuous Auditing Framework ................................................................. 2

2.3 Risk Indicator Continuous Assurance Framework ......................................................... 3

2.4 Independence Issues ..................................................................................................... 3

3.0 Integration of IT Systems ............................................................................................................ 4

3.1 IT Architecture ................................................................................................................ 4

3.2 Integrating the CA System into the Current Environment .............................................. 5

4.0 Practical Implementation of Continuous Auditing ....................................................................... 6

4.1 Siemens ......................................................................................................................... 6

4.2 AT&T .............................................................................................................................. 7

4.3 HSP ................................................................................................................................ 8

4.4 Third Party Solutions ...................................................................................................... 8

4.5 The Market for Continuous Auditing .............................................................................. 9

5.0 Future Initiatives and Conclusion .............................................................................................. 10

Appendix I Traditional Audit Approach .............................................................................................. 12

Appendix II Continuous Auditing Integrated Audit Approach ............................................................ 13

Appendix III Chan and Vasarhelyi 7-Step Continuous Auditing Paradigm ....................................... 14

Appendix IV Risk Indicator Continuous Assurance (RICA) in an Audit .......................................... 15

Appendix V IIPCA Model ................................................................................................................. 16

Appendix VI Third Party Solutions to Continuous Auditing ............................................................... 17

Appendix VII Continuous Auditing Implementation Stages ............................................................... 18

Works Cited ...................................................................................................................................... 20

Annotated Bibliography ..................................................................................................................... 21

1

1.0 Introduction

The terms continuous auditing and continuous monitoring are often misused. Continuous auditing (CA)

is a tool used by auditors, and defined by the ISACA Standards Board (ISACA) as a methodology used

by auditors, typically assisted by technology, to perform audit procedures and issue assurance on a

continuous basis (e.g., weekly, monthly). On the other hand, continuous monitoring is a management

tool, and defined by ISACA as a process put in place by management, usually automated, to determine

on a recurring and repetitive basis (e.g., weekly, monthly) if activities are in compliance with policies and

procedures implemented by management1 This is more of a quality assurance function.

This paper focuses on continuous auditing frameworks and implementation of a CA system.

2.0 Continuous Auditing Frameworks and Models

Continuous auditing (CA) systems can be used by businesses in several different ways, such as in

governance, risk management and compliance (GRC) departments and used to assist external auditors.

These systems are expected to be increasing prevalent in the future because they offer the ability to more

closely monitor and more frequently audit the higher risk areas of the business. The following section will

explore two different frameworks; the first has been informally called the benchmark continuous auditing

framework for the purposes of this paper, and the second is the risk indicator continuous assurance

approach. Then, IT architecture alternatives are discussed which leads to a discussion of integrating a CA

system with the business current IT environment.

2.1 Difference in Audit Approach

Regardless of the framework being used, reliance on a continuous auditing system fundamentally

changes the audit approach. This will be further discussed throughout this paper, but it is worthwhile to

examine a few key differences upfront. In a traditional audit, risk assessment is done on an annual basis

(see Appendix I). This risk assessment is used to determine the audit approach, which drives the nature,

timing and extent of fieldwork performed. The results of the prior years audit, including the nature and

quantity of errors or exceptions found, largely determine the assessment of risks for the current years

audit. This changes when a continuous auditing system is implemented (see Appendix II). Since the audit

process is now continuous, risk assessment must also be an ongoing process. This is beneficial because

key risks are continuously monitored and proactive approaches can be taken to mitigate these risks when

necessary. Because risks could change at any given time, and will change significantly during the life of

the company, audit approaches need to be continuously updated to reflect new or changing risks.

Periodic fieldwork is still performed during a continuous audit, especially in an external audit, since

substantive procedures are required even when relying heavily on controls (CAS 330 The auditors

1 (Jill Joseph Daigle, 2008)

2

responses to assessed risk). However, the nature of fieldwork becomes more analytical in nature rather

than performing tests of details. Refer to the next section for more detail.

2.2 Benchmark Continuous Auditing Framework2

Continuous auditing introduces an entirely new audit framework. Rather than periodic audits being

performed on past transactions, continuous auditing takes a proactive approach and tests the data and/or

systems more regularly. There are four key steps that need to be addressed when developing a

continuous auditing system:

1. Automate current audit procedures - Continuous auditing does not need to be applied to the

company as a whole. Rather, key business processes are identified and considered for CA. A key

consideration is the access and availability of input data used in the identified business process.

2. Benchmarks are developed Benchmarks need to be developed to determine what is considered

a pass and a fail of an audit procedure. These are developed based on current internal

controls in place, and developed using historical data, estimation techniques, association with

other input data, or clustering techniques. The benchmarks developed should be tested against

historical data. It is important to run these tests on a separate server, and not to use live data.

Note that benchmarks do not have to be based on key risk indicators in this approach, but can be

based on the process as a whole and largely leveraged from the current controls in place. As in

any audit, risks still need to be identified since audit testing is performed with these key risks in

mind, but the specific tests themselves are not necessarily mapped to specific risks. This is in

contrast to the key risk indicator approach discussed below.

3. Exception report clearing Data analytics are performed in the CA system by running the

benchmark against the actual transactional data. A fail against the benchmark is considered an

internal control violation and an exception. For each test run, an exception report is produced,

which is the output of the CA system. A drawback of CA systems is the risk of false alarms, since

there could be several exception reports generated by the system, many of which may not be true

errors or exceptions. Significant manual labour may be required to clear the exception reports.

4. Audit reporting If there are no exceptions found, then a clean audit opinion can be given. Any

exceptions found need to be manually addressed by the auditor and clearing of exceptions

requires significant judgment.

In order to help managers understand CA systems, a 7-step paradigm was developed by Chan and

Vasarhelyi (see Appendix III). A key difference is that the audit model becomes proactive under CA,

rather than the traditional reactive model. This is because rather than auditing historical transactions

many months after they take place, continuous auditing theoretically provides real-time assurance. In

practice, this may take place minutes, hours or days after the transaction. However, the misstatements

2 This reference applies to this entire section. (David Y Chan, 2011)

3

are detected before they make it to the financial statements, making a CA system proactive. The role of

the auditor also changes drastically. Internal auditors are required to primarily exercise judgment in

clearing exception reports, whereas external auditors time will be spent on certifying the continuous

auditing system (see Section 2.4 Independence Issues). Audit procedures in a CA system tend to be

analytical and control-based, as opposed to the test of details frequently done in traditional audits. The

entire population can be tested on a continual basis in a CA system. Therefore, audit reporting is done

more frequently, and if there are no exception reports or significant areas requiring judgment, audit

reporting can theoretically become continuous as well.

2.3 Risk Indicator Continuous Assurance Framework3

An alternative to the approach described above is to think of continuous auditing in terms of risk

indicators. Risk Indicator Continuous Assurance (RICA) uses risk indicators to measure the control

effectiveness within a set of activities or operations. This is used in identifying audit risks, which is part of

the planning stage of the audit process (see Appendix IV). First, the risk indicators (RI) need to be

identified. This is the risk that is mitigated by the control (for example, user access to systems). Then, the

risk indicator metric must be computed. The RI metric quantifies the effectiveness of the control (for

example, [number of obsolete user accounts]/[total number of accounts per system]). Thresholds must be

developed in order to assess the magnitude of the RI metric and evaluate the overall control effectiveness

(for example, >3% of obsolete user accounts means that the control is ineffective).

In order for the risk indicator approach to be effective, auditors must have direct access to internal

systems. Coming up with appropriate RIs and RI metrics is the most difficult task, and it is important that

these metrics are comprehensive enough so material omissions are not made, yet simple enough to

interpret. It is also important for the thresholds and RI metrics to be reviewed and updated regularly. This

means that risk assessment in the audit changes from being done during the planning phase to being an

ongoing process.

2.4 Independence Issues

Several scholarly journals cite continuous auditing (CA) independence issues as a major hurdle to

overcome. Since continuous monitoring is a management tool (refer to section 1.0 Introduction for the

definition), impairment of independence is not an issue. The ISACA Standards Board (ISACA) provides

a six-step approach to assist companies in solving CA independent issues:

1. Understand why the issue is an independence issue based on the specific facts and

circumstances. Audit procedures are frequently developed by external auditors, but the business

is heavily involved in implementing a continuous audit system, meaning the audit procedures

could become transparent to the company. Also, external auditors are heavily involved in

3 This reference applies to this entire section. (Dale Johnstone, 2009)

4

developing a CA system which puts the auditors at risk of acting in an advisory role. There are

also certain instances where external auditors develop code for the CA system, but management

wants to use that code for the CM system.

2. The ethical issues are then identified.

3. Stakeholders are identified.

4. Identify ethical principles to adhere to. These can come from Generally Accepted Auditing

Standards (GAAS), ISACA, IIA, Institute of Management Accountants (IMA), American

Institute of Certified Public Accountants (AICPA) or IESBA.

5. Come up with possible solutions and consequences.

6. Use judgment to determine the best possible alternative.

The exact solution will vary greatly depending on the companys specific CA system. However, signing a

policy clearly stating the role of external auditors, internal auditors and management in development of

the system is a good first step. Agreements such as these help solve issues surrounding the ownership of

code.4

Since continuous auditing systems are developed by both the external auditor and internal management,

there is considerable uncertainty with regards to legal liability if the systems fail to detect material

misstatements.5 This is an issue that has not yet been resolved and therefore requires further research

and guidance from regulatory bodies.

3.0 Integration of IT systems

When adopting a continuous auditing system, it is vitally important to consider both the architecture of the

new CA system and how the system will be integrated into the current IT environment.

3.1 IT Architecture6

There are two different IT architecture alternatives for continuous auditing systems. The first is Embedded

Audit Models (EAM), which are built into the system to provide assurance over certain types of activities.

Usually, external software is purchased and then it is customized with continuous auditing procedures

such as matching certain key documents or comparing transactions against an audit threshold. EAM

ghosting is similar to EAM, but rather than running the continuous auditing functions on the live business

systems, it is run on a copy of the systems. This mitigates the risk that live data will be affected by the

audit process. However, the downside to this approach is that it requires significantly more computing

power and storage, since copies of the system must be made and retained.

4 (Jill Joseph Daigle, 2008)

5 (John R Khun Jr, 2010)

6 This reference applies to this entire section. (Dale Johnstone, 2009)

5

Monitoring Control Layer (MCL) systems are those which the continuous auditing system is external to

the business processing system. This contrasts with EAM systems in which the CA system is embedded

into the business software. Data is transmitted to the CA system at pre-specified internals, and audit

procedures are run with that data.

There are several limitations to these approaches, however. A lot of processing power is demanded by

CA systems, which could be costly to acquire and maintain. With regards to EAM, purchased software

must be customized and EAM must be built into the enterprise resource planning (ERP) systems.

Therefore, EAM must be tested at the ERP system level as well, which could be problematic for large

companies operating many ERP systems.

3.2 Integrating the CA System into the Current Environment7

So far, this paper has discussed continuous auditing systems in isolation, but it is important to realize that

this is only the first step in developing a full CA system. After the CA model is built, it must be integrated

with the current business model. Then, it is crucial that practitioners receive the proper training so they

know how to use the CA system to its full potential.

Traditionally, the internal audit function supported the management, operations and information systems

processes within a company. In continuous auditing systems, these business process systems must be

integrated with the electronic audit evidence. This brings up the issue of who controls the data, since it is

imperative that auditors remain independent and the business cannot control, change or tamper with the

electronic audit evidence in any way.

Researchers developed a full power continuous auditing system called intra/inter process continuous

auditing (IIPCA) to help integrate the CA system into the current IT environment (see Appendix V). This

is the first model to jointly consider CM, internal audit and electronic audit evidence. It is critical that CA

models include internal control testing and testing transactions. Any business rules or policies must also

be followed. This testing should be performed before the CA system is integrated with other existing

systems.

The inter-process auditing function ensures that data goes through each required process in the business

and is not lost along the way. At each process level, the data is ticketed with the phase of business

process and date. Inter-process auditing can be used in operations management as a way to measure

efficiency, since the time between data input and outflow is tracked. The intra-process auditing function

ensures that all tasks within a specific process are completed. Again, the data is ticketed with the activity

performed on or with the data and time. The electronic audit evidence function defines all processes

7 This reference applies to this entire section. (Munir Majdalawieh, 2012)

6

surrounding data analysis, retention and disposition. Not only do the tickets on the data need to be

stored, but any alarms triggered and exception reports need to be kept as well. Since there are effectively

two levels of audit (process level and transactional level), there is the potential for huge amounts of data

to be generated. Businesses need to ensure the appropriate databases are in place to store this data.

4.0 Practical Implementation of Continuous Auditing

Several companies are making the shift towards continuous auditing and/or continuous monitoring

systems. In practice, companies implementing a current auditing system should examine what has been

done at other companies, especially those in a similar industry, to gain an understanding of what

benchmarks are typically used. The challenges faced by other companies during the implementation

process also serve as a key learning tool for future adoption of continuous auditing systems.

4.1 Siemens

A continuous monitoring of business process controls (CMBPC) system was implemented at Siemens.

This is a critical pilot implementation that has been used as a guideline for implementing continuous

auditing systems in other companies.

The internal audit group documents audit action sheets (AAS), which were created to assess any

configurable process controls that could be automated.8 Types of controls are: a) verify by testing a

specific control for existence, correctness and functioning of the control, b) verify by ensuring a prohibited

behaviour cannot happen, c) verify any automatic control settings in the ERP system. Siemens used the

monitoring and control layer approach as opposed to EAM.9 Refer to section 3.1 of this paper for a

description of MCL and EAM. Since the continuous auditing system is external to the rest of the systems

used in the company, there is less intra-department co-ordination required. Another reason why MCL was

chosen over EAM is that the physical and logical access separation from the rest of the entity means that

the auditing system will be less susceptible to manipulation by employees.10

First, it was decided what AASs should be automated. Both automation of the control and assessment of

the effectiveness of the control elements were considered. The challenge was not only making the audit

procedures machine readable but also machine understandable. This required the internal audit team

to re-engineer some audit processes.11

The degree of effectiveness of each control was assessed by the

system based on a 0-4 rating.

8 (Michael Alles G. B., 2006)

9 (Michael Alles A. K., 2008)

10 (Michael Alles G. B., 2006)

11 (Michael Alles A. K., 2008)

7

Each control has several control elements related to it. For example, password controls contains control

elements relating to password length, expiry date, log in attempts. An overall rating based is assigned to

the control based on the aggregate rating of related control elements. This can either be automated (ex

using a weighted average of the control elements) or assessed by the auditors judgment. Some kind of a

control exception hierarchy is also required. It is inefficient and unrealistic to change the entire audit

process, since large companies like Siemens have legacy systems and understaffed audit departments.

Some re-engineering is unavoidable due to the need to separate out formalizable and automatable

controls.12

The continuous auditing system collected data and ran tests every 10 seconds. However, the pilot

implementation at Siemens used a simple MS Access database, which worked for the pilot testing done,

but could not handle the large volume of data being continuously monitored and retained as support in an

entity-wide application of the system. Throughout the implementation process, it was found that the

volume of data retained can be greatly reduced by only retaining the data if the system found control

exceptions. It is critical to have a good database that retains these exceptions.13

An interface where the

auditor can see the 0-4 score achieved by the control and related controls is also required. It was also

particularly difficult to implement compensating controls in the system, and ensuring that alarm floods did

not overwhelm the human auditor. In hindsight, a parallel alarm classification hierarchy should have been

implemented to assess the materiality of the control exceptions. The Siemens team learned that a clear

change management plan needed to be developed before such large changes are made to the auditing

and IT environment.14

4.2 AT&T Bell Laboratories15

In the late 1980s, AT&T became one of the first companies to adopt a continuous auditing-type system

was AT&T. A Continuous Process Audit Methodology (CPAM) was developed to implement Continuous

Process Audit Systems (CPAS). This was a challenge since at the time because corporations generally

used a main database system with other databases connected to it. This means that auditors had to audit

both the system and the reconciliation between the systems, which is not the most efficient way to audit

since these types of procedures do not address certain key issues such as the timeliness of addressing

errors (since audits are only done once a year). Data is only as reliable as the system that generated it so

real-time assurance over system controls was determined to be useful.

CPAS uses two different ways to obtain continuous assurance: 1) data flowing through the system is

continuously tested based on auditor-defined rules, and 2) data is tested indirectly by looking at specific

12

(Michael Alles G. B., 2006) 13

(Michael Alles G. B., 2006) 14

(Michael Alles A. K., 2008) 15

This reference applies to this entire section. (Miklos A Vasarhelyi, 1991)

8

occurrences of errors or individual results. Different types of data can also feed into the system. Data can

be pulled from either the standard application reports, the raw data that feeds to these application reports,

or direct monitoring data. Direct monitoring data is the output data from a monitoring system. Any errors

or exceptions found trigger an alarm, similar to that at Siemens as described in section 4.1. CPAS uses a

hierarchy of alarms to determine the priority of exception report clearing. This makes the audit process far

more efficient since critical errors are examined first.

The team thought it was important for audit work should be done in a separate and independent location,

which is why a MCL system was used (refer to section 3.1 for description). The auditors work is broken

down into two phases. The first is the start-up phase, where the auditor works with the business to gain

an understanding of the control environment, which helps the auditor develop an audit plan and

appropriate procedures. The auditor developed a series of flow charts that were communicated to

computer engineers, IT staff, and management. After this is completed, the auditor can actually use the

system to perform audit work.

The system developed at AT&T primarily used a series of metrics, analysis and alarms. First, metrics are

developed to measure the expected outcome. This is similar to the benchmark continuous auditing

framework described in section 2.2 of this paper. Analysis is broken down into three sub-categories:

functional/natural flow, logical/key data interaction and empirical/observational. The algebraic structure of

the code that tests the control against metric is determined. However, the metric for a specific control

could vary in different situations or across time, so these contingencies that determine the numeric value

of the output also need to be factored into the model. Lastly, certain industry- or company-wide rules of

thumb can be used as a benchmark as well. Whenever the metric or benchmark is not met, an alarm is

generated.

The following hierarchy of alarms was developed at AT&T to prioritize alarm error clearance:

- Type 1: minor issue dealing with functionality of auditing system

- Type 2: low-level alarm dealing with minor operational issues

- Type 3: high-level alarm dealing with issues that must be investigated by the auditor (ex suspense file

becomes too large)

- Type 4: serious crisis

4.3 HSP

The project started by improving the supply chain. HSP had primarily legacy systems, which meant that a

different approach had to be taken. At HSP, the auditor had unrestricted access to raw data, so

benchmarks were determined to test the validity of this data. First, data validity tests were applied to

individual transactions. Then, continuity equations (CE), which used probability models to calculate the

9

expected value and variance of a business process metric, were used to provide additional assurance

over the data. This required coordination between mathematicians, IT personnel, and auditors. The CEs

addressed the risk that the data was not examined in aggregate, but the degree of aggregation needs to

be determined by the accountant. 16

4.4 Third-Party Solutions to Continuous Auditing17

So far, the continuous auditing systems that have been discussed in this paper were built in-house. While

there are some clear advantages to building a system in-house, such as the ability to tailor the system to

meet the companys auditing, risk management and individual internal reporting needs, many companies

do not have the time, expertise or financial capability to build their own CA system. Over the past few

years, third-party solutions to continuous auditing have been developed.

Refer to Appendix VI for a comparison of various third-party continuous auditing solutions. These third-

party solutions are all based on the MCL architecture (refer to appendix 3.1 for a discussion of MCL

systems). This is because it is easier to sell an add-on continuous auditing program rather as opposed to

an EAM system integrated with the ERP environment.

4.5 The Market for Continuous Auditing

As previously mentioned, there are many benefits to continuous auditing. The ability to provide real time

assurance increases the effectiveness and efficiency of the audit since material misstatements are dealt

with on a proactive rather than reactive basis.18

However, a 2010 KPMG survey of 112 respondents in a

variety of industries found that half of respondents are either unconvinced on the benefits of a CA or CM

system or are failing to move forward with implementing such a system, and only 20% of respondents

either currently have a CA/CM system in place or are planning on implementing one. The majority of

companies surveyed use a mix of manual and automated checks.

There are still significant challenges to overcome before the market fully accepts and adopt continuous

auditing. A 2011 PWC survey found that continuous auditing systems are being implemented because

globalization is increasing the complexity and breadth of risks. Automation of the assurance process

allows the company to achieve efficiencies and reducing the cost of compliance, while at the same time

increasing the efficiency of the assurance process by managing compliance more effectively.19

Stakeholders perceive continuous auditing systems as a way to realign the audit process to focus on

anomalies. It is also seen as a more cost effective way to automate issue analysis, reporting, and

documentation.

16

(Michael Alles A. K., 2008) 17

This reference applies to this entire section. (John R Khun Jr, 2010) 18

(David Y Chan, 2011) 19

(The Path Forward for Data Analysis and Continuous Auditing, 2011)

10

The greatest perceived benefit is that greater assurance can be obtained by a continuous auditing

system, which was stated as a benefit by 61% of respondents, but nearly half of respondents believe their

organization is effectively controlled with manual checks.20

This may explain why 61% of respondents

believe that implementation of such a system will not reduce costs in their organization. This report has

provided evidence that continuous auditing provides a higher quality of assurance, but nearly 40% of

respondents are unaware of this. Therefore, there needs to be greater education and awareness

surrounding the numerous benefits of continuous auditing systems. Another reason why CA systems

have not been widely adopted is the concern over potential independence issues, as previously

discussed in section 2.4 of this report.

There is currently demand for continuous auditing systems, but current demand is not very high

compared to potential demand. As more companies adopt CA systems, market momentum will be

created, thereby enticing even more companies to adopt for fear of being left behind.

More recently, a small-scale study was conducted to determine the acceptance of continuous auditing

systems by internal auditors and the degree of adoption. Companies found that CA systems greatly

assisted with SOX compliance requirements.21

Therefore, the companies most likely to start adopting a

continuous auditing system are large public companies that need to comply with SOX requirements.

Management acceptance of CA systems is mixed and the most common drawback cited is the perceived

cost of the system. The degree of adoption varied per industry, with most companies being in the

emerging stage, meaning the continuous auditing system is in the process of being implemented but is

not yet used on a company-wide scale (see Appendix VII). This means there is significant opportunity for

growth in the CA market, even in companies that have already begun the adoption process, since full

implementation of CA systems is not prevalent.22

5.0 Future Initiatives and Conclusion

There are many areas for research in the field of continuous auditing. With regards to the IT architecture,

there has not been much research surrounding EAM, especially with regards to independence and public

perception of the auditors role. There has also not been much research done regarding both the external

and internal auditors legal liability of a material misstatement is found in the financial statements. This

paper discussed several frameworks and methods on how to use continuous auditing systems to obtain

assurance, but there needs to be more work done around how to integrate continuous auditing systems

with management decision making. 23

20

(Continuous auditing and monitoring: Are promised benets now being realised?, 2010) 21

(Miklos A. Vasarhelyi, 2011) 22

(Continuous auditing and monitoring: Are promised benets now being realised?, 2010) 23

(John R Khun Jr, 2010)

11

In terms of the actual continuous auditing process, this paper gave a few examples of exception reporting

and alarm hierarchies (for example, those used at Siemens and AT&T). However, there is still more

research to be done around how to best implement alarms and how to structure alarm hierarchies. For

example, it has yet to be determined the degree to which artificial intelligence can assist in continuous

auditing.24

Lastly, there has not been much work done surrounding the types of organizations that implement

continuous auditing systems. This report suggested that SOX compliance may play a role in which

companies adopt a CA system, but there are also cheaper third-party alternatives that can be adopted by

smaller entities. It has not been determined whether implementation of such a system is driven by the

personalities of management or by the nature of the organization.25

There is significant promise in the area of continuous auditing. Several companies have successfully

implemented such a system, and surveys suggest that potential demand for continuous auditing is quite

high. However, more research needs to be done in the field and managers need to be better educated on

the benefits of CA systems before continuous auditing becomes widespread.

24

(John R Khun Jr, 2010) 25

(John R Khun Jr, 2010)

12

Appendix I Traditional Audit Approach26

Purpose: The purpose of this appendix is to describe the traditional audit process as a basis for

comparison against the continuous auditing process.

26

(The Path Forward for Data Analysis and Continuous Auditing, 2011)

13

Appendix II Continuous Auditing Integrated Audit Approach27

Purpose: The purpose of this appendix is to describe the auditing framework when a continuous auditing

system is in place.

KRI: Key Risk Indicators

27

(The Path Forward for Data Analysis and Continuous Auditing, 2011)

14

Appendix III Chan and Vasarhelyi 7-Step Continuous Auditing Paradigm28

Purpose: This 7-step continuous auditing paradigm shows the 7 key differences between traditional

auditing and continuous auditing. It was developed to be used by researchers and practitioners as a basis

for the current understanding of continuous auditing. Practitioners can use this as a first resort when

trying to implement a continuous auditing system. Researchers can use the paradigm to further the

current understanding of continuous auditing.

28

(David Y Chan, 2011)

15

Appendix IV Risk Indicator Continuous Assurance (RICA) in an Audit29

Purpose: The purpose of this diagram is to show how RICA is integrated into a typical audit.

29

(Dale Johnstone, 2009)

16

Appendix V IIPCA Model30

Purpose: This is a model developed to integrate a continuous auditing model into the existing enterprise

business processes.

Electronic audit evidence functions:

- IM: inventory management

- CO: confirmation by authenticated documents

- ED: external documentation

- RP: re-performance

- ID: internal documentation

- AP: analytical procedures

- IC: inquiries

Ticket number: at each phase of the business cycle, the data is stamped with the time. The tickets also

indicate whether there was a flag or alarm triggered in the data.

30

(Munir Majdalawieh, 2012)

17

Appendix VI Third Party Solutions to Continuous Auditing31

Purpose: The purpose of this appendix is to compare the functionality of various third-party continuous

auditing solutions available.

31

(John R Khun Jr, 2010)

18

Appendix VII Continuous Auditing Implementation Stages32

Purpose: The purpose of this appendix is to show the degree of adoption among 9 companies that have

started adopting CA systems.

Level of adoption:

- Traditional: traditional audit methods are used (see Appendix I) but investments in research and

development have been made

- Emerging: early adoption of a CA system but IT audit still works independently

- Maturing: growth stage with coordination between intra-company departments, and beginning to rely

on benchmarks/key performance indicators

- Full continuous: complete CA system including data warehouse, benchmarking history, error history,

and complete integration with risk management department

Adoption metric:

- Objectives: the degree to which the objective of internal audit is to provide continuous assurance

- Approach: the degree to which continuous audit alarms have been implemented. A maturing

continuous auditing system is one in which the alarms are effectively utilized.

- IT/Data access: the degree to which data access is automated and integrated into the CA system. A

traditional approach relies on manual data extraction.

- Audit automation: the degree to which audit processes and alarms are automated

32

(Miklos A. Vasarhelyi, 2011)

19

- Audit and management sharing: the degree to which management has facilitated the implementation

of a CA system and data sharing among departments

- Management of audit function: the degree to which systems have been implemented to manage the

CA system

- Analytical methods: the usage of analytical methods in a CA system at the transactional level

20

Works Cited

Continuous auditing and monitoring: Are promised benets now being realised? (2010). Retrieved June 24, 2012, from KPMG Advisory: http://www.kpmg.com/BE/en/IssuesAndInsights/ArticlesPublications/Documents/Continuous%20Auditing%20and%20Monitoring.pdf

Dale Johnstone, E. C. (2009). Achieving Continuous IT Auditing: RICA. ISACA , 5.

David Y Chan, M. A. (2011). Innovation and Practice of Continuous Auditing. International Journal of Accounting Information Systems , 9.

Jill Joseph Daigle, R. J. (2008). Auditor Ethics for Continuous Auditing and Continuous Monitoring. ISACA , 4.

John R Khun Jr, S. G. (2010). Continuous Auditing in ERP System Environments: The Current State and Future Directions. Journal of Information Systems , 23.

Michael Alles, A. K. (2008). Putting Continuous Auditing Theory into Practice: Lessons from Two Pilot Implementations. Journal of Information Systems , 21.

Michael Alles, G. B. (2006). Continuous monitoring of business process controls: A pilot implementation of a continuous auditing system at Siemens. International Journal of Accounting Information Systems , 25.

Miklos A Vasarhelyi, F. B. (1991). The Continuous Audit of Online Systems. Auditing: A Journal of Practice & Theory , 17.

Miklos A. Vasarhelyi, M. A. (2011, August). The Acceptance and Adoption of Continuous Auditing by Internal Auditors: A Micro Analysis . Retrieved June 24, 2012, from JEBCL: http://jebcl.com/symposium/wp-content/uploads/2011/08/Continuous-Auditing-Implementation-Study.pdf

Munir Majdalawieh, S. S. (2012). Intra/inter process continuous auditing (IIPCA), integrating CA within an enterprise system environment. Business Process Journal Management , 24.

The Path Forward for Data Analysis and Continuous Auditing. (2011, May). Retrieved June 24, 2012, from ISACA: http://www.isaca-kc.org/Chapter%20Meetings/20110512%20Continuous%20Auditing.pdf

21

Annotated Bibliography

Author Title of Article Periodical/ website

Vol. / No. / Edition

Year published

Pages Date accessed

Location, data base, website, link

Added in the final paper?

Michael Alles, Gerard Brennan, Alexander Kogan, Miklos A. Vasarhelyi

Continuous monitoring of business process controls: A pilot implementation of a continuous auditing system at Siemens

International Journal of Accounting Information Systems

July 2006 2006 25 May 16, 2012

ABI Inform Yes

Annotation

The authors of the paper implemented a continuous monitoring of business process controls (CMBPC) system at Siemens. The paper addresses several issues with regards to the implementation of such a continuous auditing system. Audit action sheets (AAS) are documents that were created to assess any configurable process controls that could be automated. First, it was decided what AASs should be automated. Both automation of the control and assessment of the effectiveness of the control elements were considered. The degree of effectiveness of each control was assessed by the system based on a 0-4 rating. There were some problems with the data base not being able to handle the volume of data being continuously monitored and retained. Since the data is being continuously monitored (every 10 seconds), the volume of data retained can be greatly reduced by only retaining the data if the system found control exceptions. However, a good database is still required to retain these exceptions. An interface where the auditor can see the 0-4 score achieved by the control and related controls is also required. An overall rating based on the aggregate of related control elements (ex all password controls, including those relating to password length, expiry date, log in attempts) then use their judgment in assigning an overall rating to that control (ex passwords). This can either be automated (ex using a weighted average of the control elements) or assessed by the auditors judgment. Some kind of a control exception hierarchy is also required. It is inefficient and unrealistic to change the entire audit process, since large companies like Siemens have legacy systems and understaffed audit departments. Some re-engineering is unavoidable due to the need to separate out formalizable and automatable controls.

David Y. Chan, Miklos A. Vasarhelyi

Innovation and practice of continuous auditing

International Journal of Accounting Information Systems

December 2011 2011 9 February 29, 2012

ABI Inform Yes

Annotation

22

Traditional auditing is outdated in the fast-paced real-time world we currently live in. The following continuous auditing paradigm can be used by researchers and practitioners in developing and furthering the current understanding continuous auditing:

1. Continuous audits rather than periodic audits, implemented in the following 4 stages 2. Audit model becomes proactive, meaning misstatements are detected before they make it to the financial statements,

rather than reactive 3. Several audit procedures are automated 4. Internal auditors primarily test those controls requiring judgment and handle exception reports, whereas external

auditors will need to certify the continuous auditing system 5. Nature of audit procedures: Types of controls are continuous data assurance and monitoring, as opposed to the

analytical procedures and test of details that are frequently done in traditional audits Timing of audit procedures: Testing occurs on a continual basis Extent of audit procedures: Entire population is considered in testing

6. Much of the testing is performed by the system, such as analytics and modeling 7. Audit reporting also becomes more frequent, and if there are no exception reports or significant areas requiring

judgment, audit reporting can theoretically become continuous as well CA systems can be implemented using the following 4 steps:

1. Automation of audit procedures 2. Data modeling and benchmarks developed 3. Data analytics and exception report clearing 4. Audit reporting

This paper suggests that continuous auditing will replace traditional auditing.

Michael Alles, Alexander Kogan, Miklos A. Vasarhelyi

Putting Continuous Auditing Theory into Practice: Lessons from Two Pilot Implementations

Journal of Information Systems

Vol 22, No 2 - Fall 2008

2008 21 February 29, 2012

ABI Inform Yes

Annotation

This paper discusses the implementation of continuous auditing (CA) systems at Siemens and HSP.

Siemens: First the internal audit group needs to determine what control procedures are being followed. Types of controls are: a) verify by testing a specific control for existence, correctness and functioning of the control, b) verify by ensuring a prohibited behaviour cannot happen, c) verify any automatic control settings in the ERP system. Then, the team determined what controls can be automated. Siemens used the monitoring and control approach as opposed to EAM. The challenge was making e-audit output not only machine readable but also machine understandable so reengineering of audit processes took place. The team

23

had particularly difficulty implementing compensating controls in the system, and ensuring that alarm floods did not overwhelm the human auditor. In hindsight, the team should have implemented a parallel alarm classification hierarchy which assesses the materiality of the control exceptions. Also, the team learned that a clear change management plan needed to be developed before such large changes are made to the auditing and IT environment.

HSP: The project started by improving the supply chain. HSP had primarily legacy systems, which meant that a different approach had to be taken. At HSP, the auditor had unrestricted access to raw data, so benchmarks were determined to test the validity of this data. First, data validity tests were applied to individual transactions. Then, continuity equations (CE), which used probability models to calculate the expected value and variance of a business process metric, were used to provide additional assurance over the data. The CEs addressed the risk that the data was not examined in aggregate, but the degree of aggregation needs to be determined by the accountant.

John R. Kuhn, Jr., Steve G. Sutton

Continuous Auditing in ERP System Environments: The Current State and Future Directions

Journal of Information Systems

Vol 24, No 1 Spring 2010

2010 23 February 29, 2012

ABI Inform Yes

Annotation

This paper looks at the IT architecture alternatives for continuous auditing systems. Continuous auditing systems are key in many governance, risk management and compliance (GRC) departments and used to assist external auditors. Two types of systems are: 1) Embedded Audit Modules (EAM): built into the system to provide continuous assurance over certain types of activities, but is typically added onto the purchased accounting system (ex SAP) afterwards and customized for that particular business. This includes modules that match certain key documents (ex purchase order to invoice) or compare transactions against some key audit threshold. Reports over the functionality of these controls are also available on a real-time basis.

Alternatives to EAM:

EAM ghosting: run the EAM functions on a copy of the EAM system on a separate and external system, so as to minimize the risk of live data being affected by the EAM system

Can also have separate (copies of the EAM) systems for the development, quality assurance, and production of the EAM systems, which further reduces risk of live data being changed by the IT department

2) Monitoring Control Layer (MCL): The continuous auditing (CA) system is external to the processing systems. The CA system receives data at pre-specified intervals and runs audit procedures with that data.

Limitations:

24

Processing requirements to implement the system, could be costly

Need to customize purchased software

EAM must be built into the ERP system, and therefore needs to be tested, designed, etc at this level as well; many larger companies operate several ERP systems (ex Siemens has over 20)

Could have large volume of error reports, risk of false alarms

If external auditor designs the system, possible independence issues

Uncertainty regarding legal liability of systems fail to detect material misstatements These limitations provide a basis for future research.

James E. Hunton, Jacob M. Rose

21st Century Auditing: Advancing Decision Support Systems to Achieve Continuous Auditing

Accounting Horizons

Vol 24, No 2 - 2010

2010 17 March 1, 2012

ABI Inform No

Annotation

Decision support systems are defined as any system intended to help improve the information available for decision making purposes, and are used in both external audits and internally. Previous studies have shown that DSS systems are not fully or properly utilized. However, this paper realizes that DSS systems can be useful in implementing continuous auditing systems. Data mining, text mining, and other data analysis techniques (and in some cases, artificial intelligence) can be leveraged in continuous auditing (CA) systems. Therefore, it is the researchers opinion that DSS systems are a necessary precursor to CA systems. Dynamic auditing systems that adapt to new and unusual information should also be considered in CA systems. The authors stress that auditors need to be appropriately trained in these systems. This paper also suggests that continuous auditing/monitoring is inevitable due to increasing pressure on auditors to assess economic, fraud and other financial statement risk factors (ex valuation of complex derivatives).

Sridhar Ramamoorti, Michael P. Cangemi, William M. Sinnett,

The Benefits of Continuous Monitoring

Financial Executives Research Foundation

August 2011 2011 98 May 2, 2012

ABI Inform

Annotation

This is a piece of research done in order to help executives practically implement a continuous monitoring or continuous

25

auditing system. The paper examined 11 companies that had already implemented such a system, and learned the following:

- CM can be a way to achieve better performance across the entire company not just through a return on the investment, but indirectly through operational effectiveness and risk reduction

- A continuous monitoring program needs a CM champion, which is someone who will be held responsible for the program and allocating the internal resources

- Internal auditors play a critical role in moving towards a CM system - Using externally developed software can help in cost management - Companies that initially launch a CM program in one division are rapidly working to expand it to all of their businesses - Companies also want to learn how CM programs are implemented elsewhere with the hope of improving their own CM

system

Munir Majdalawieh, Sofiane Sahraoui, Reza Barkhi

Intra/inter process continuous auditing (IIPCA), integrating CA within an enterprise system environment

Business Process Management

Journal

Vol 18, No 2 2012 24 May 17, 2012

ABI Inform Yes

Annotation

This paper outlines how a company can develop a full continuous auditing (CA) system. There are 3 objectives to developing the CA system: 1) building the CA model, 2) integrate CA model with business model, 3) ensure practitioners know how to use the CA system to its full potential. Several approaches have already been introduced, and they fall into one of the two categories: using CA as a quality assurance too, or automate the auditing process from a monitoring perspective. Critical success factors in CA models include internal control testing, testing transactions, ensuring business rules are followed, and three main requirements in systems are: continuous control monitoring, continuous data assurance and continuous risk monitoring assessment. The paper proposes the following IICPA model:

26

Electronic audit evidence functions:

- IM: inventory management - CO: confirmation by authenticated documents - ED: external documentation - RP: re-performance - ID: internal documentation - AP: analytical procedures - IC: inquiries

Ticket number: at each phase of the business cycle, the data is stamped with the time. The tickets also indicate whether there was a flag or alarm triggered in the data.

Miklos A. Vasarhelyi, Fern

The Continuous Audit of Online

Auditing: A Journal of

Vol 10, No 1 1991 17 May 17, ABI Inform Yes

27

B. Halper Systems Practice & Theory

2012

Annotation

This is a critical piece of literature in the area of continuous auditing since it was the first paper to truly explore the idea of a Continuous Process Audit Methodology (CPAM) to implement Continuous Process Audit Systems (CPAS). At the time, corporations generally used a main database system with other databases connected to it. Therefore, auditors have to audit both the system and the reconciliation between the systems. However, these types of procedures do not address certain key issues such as the timeliness of addressing errors (since audits are only done once a year), and data is only as reliable as the system that generated it so real-time assurance over system controls is useful. There are two different ways to obtain continuous assurance: 1) data flowing through the system is continuously tested based on auditor-defined rules, and 2) data is tested indirectly by looking at specific occurrences. The CPAS can pull the data from either the standard application reports, the raw data that feeds to these application reports, or direct monitoring data. Audit work should be done in a separate, independent location. The paper also describes a hierarchy of alarms that can be used, and the types of controls that can be implemented.

Dale Johnstone and Ellis Chung Yee Wong, CISA, CFE, CISSP

Achieving Continuous IT Auditing: RICA

ISACA 2009, Volume 6 2009 5 May 25, 2012

ISACA: http://www.isaca.org/Journal/Past-Issues/2009/Volume-6/Pages/Achieving-Continuous-IT-Auditing-RICA1.aspx

Annotation

There are several challenges to continuous IT auditing that the article discusses. Achieving auditor independence is a challenge because the auditor is involved in both developing and auditing the CA system. CA systems are also effectively controls on controls, so it is critical that the controls get sufficient depth of assurance. Sufficient and appropriate evidence should also be obtained, which support the auditors scope and objective.

This report suggests an alternative CA approach, called risk indicator continuous assurance (RICA), which focuses on the identification of risk indicators (RI) which are metrics that measure control effectiveness. There are 3 steps to a RICA approach:

1. Identify the RIs 2. Compute the RI metric that will be used (ex a ratio of errors/total population) 3. Develop thresholds to evaluate the RIs

The paper also outlines the three critical success factors to this approach: auditors have direct access to internal systems,

28

critical points and risk indicators are reviewed and updated regularly, risk indicators are appropriately defined based on the population of inputs and are simple enough to interpret.

Jill Joseph Daigle, CISA, CIA, CISSP, Ronald J. Daigle, Ph.D., CPA, and James C. Lampe, Ph.D., CPA

Auditor Ethics for

Continuous

Auditing and

Continuous

Monitoring

ISACA 2008, Volume 3 2008 4 May 25, 2012

ISACA: http://www.isaca.org/Journal/Past-Issues/2008/Volume-3/Pages/Auditor-Ethics-for-Continuous-Auditing-and-Continuous-Monitoring1.aspx

Yes

Annotation

It is important to recognize that continuous auditing (CA) is not the same as continuous monitoring (CM). In both CA and CM, auditor independence is a critical issue which was especially heightened after SOX 404 was implemented. CA is defined by the ISACA Standards Board as a methodology used by auditors, typically assisted by technology, to perform audit procedures and issue assurance on a continuous basis (e.g., weekly, monthly) and CM is defined as a process put in place by management, usually automated, to determine on a recurring and repetitive basis (e.g., weekly, monthly) if activities are in compliance with policies and procedures implemented by management. Therefore CA is an auditing tool used by auditors, and CM is a management tool. Independence impairment affects CM. The paper suggests the following 6-step approach to solve independence problems:

1. Understand why the issue is an independence issue with the specific facts and circumstances (ex consider integration of CA and CM systems and sharing of data and code within the company)

2. Identify ethical issues 3. Identify stakeholders 4. Identify ethical principles to adhere to 5. Identify possible solutions and consequences 6. Determine the best alternative by using judgment

The exact solution will vary greatly by the companys specific CA and CM system.