-
LINKAGES BETWEEN AUDITORS RISK ASSESSMENTS IN A RISK-BASED AUDIT
Natalia Kotchetova Assistant Professor
School of Accountancy University of Waterloo [email protected]
Thomas M. Kozloski Assistant Professor
School of Business & Economics Wilfrid Laurier University
William F. Messier, Jr. Deloitte & Touche LLP Professor
School of Accountancy Georgia State University
and Department of Accounting, Auditing and Law
Norwegian School of Economics and Business Administration [email protected]
August 2006
The authors would like to thank participants at the 2004 New England Behavioral Accounting Research Symposium, the 2005 American and Canadian Accounting Associations Annual Meetings, the EARNet Symposium 2005, and workshops at Georgia State University and the University of Waterloo. Special thanks to Stan Biggs, Larry Brown, Christine Earley, Craig Emby, Lynn Hannan, Sue McCracken, Drew Newman, Ed ODonnell, Luc Quadackers, Ira Solomon, Scott Vandervelde, and Alan Webb for their helpful comments.
-
LINKAGES BETWEEN AUDITORS RISK ASSESSMENTS IN A RISK-BASED AUDIT
Abstract This paper examines auditors assessment of strategic business risks and their linkage of those risks to the risk of misstatement in the financial statements. More specifically, we examine the linkage between (1) auditors identification of business risks and their assessments of the risk of material misstatement at the entity level; (2) their performance of business process analysis and related process-level risk assessments; and (3) their assessments of the risk of material misstatement at account level. These linkages are required during a risk-based audit, and we test six hypotheses related to them. Our results show that auditors document a greater number of specific business risks threatening the client entity when residual strategic risk is high; also, the more business risks are identified by the auditors, the more financial statement implications are identified; and the higher the assessment of client business risk by auditors the higher the assessment of the entity level risk of material misstatement. At the level of a business process, we find that performing a formal business process analysis affects the auditors business process risk assessments. Specifically, the business process risk assessments of auditors who perform a business process analysis do not differ across core and supporting processes, while the auditors who do not perform a business process analysis assess risk for the core process higher than for the supporting process. With respect to the auditors assessments of the risk of material misstatement at the financial statement account level, these assessment are directly related to the auditors assessments of the risk of material misstatement at the entity level. When auditors assess the risk of material misstatement in the accounts and transactions affected by the business process, their judgments are influenced by their process-level business risk assessments and not by their assessments of the entity-level client business risk or by residual strategic risk. Cumulatively, these results indicate that auditors follow the sequential belief updating structure of the audit risk assessment workflow as prescribed by recently promulgated audit risk standards (AICPA 2006; IAASB 2004a). Key words: Business risks, Risk-based audits, Business Processes Availability of Data: The data are available from the first author upon request.
-
LINKAGES BETWEEN AUDITORS RISK ASSESSMENTS IN A RISK-BASED AUDIT
I. INTRODUCTION
This paper examines the linkages that underlie the risk-based audit approach outlined in
the recently issued risk assessment auditing standards and adopted by the major public
accounting firms.1 More specifically, we examine the linkages between the auditors
identification of business risks and their assessments of the risk of material misstatement at the
entity level, their performance of business process analysis and related process-level risk
assessments, and their final assessments of the risk of material misstatement at the account level.
We believe this is the first study to examine the linkages of auditors risk assessments throughout
the planning process of a risk-based audit. Because previous auditing standards and related firm
methodologies have been criticized for not being sufficiently rigorous and explicit regarding
these linkages (e.g., Bell et al. 1997; Jeppessen 1998; Weil 2004), the findings of this study have
important implications for regulators, academics, the accounting profession, and public
accounting firms.
Since the late 1990s, public accounting firms have been implementing audit approaches
generically referred to as risk-based auditing.2 In November 2005, the Auditing Standards
Board (ASB) passed a suite of eight Statements on Auditing Standards (SASs) that dramatically
modified the guidance on the conduct of a financial statement audit (AICPA 2006). The
issuance of these standards represents the most significant change to auditing standards and audit
practice since the passage of the expectation gap standards in 1988 (AICPA 1993). There is an
1 Bell et al. (2005) provide a detailed and comprehensive overview of the changes made to KPMGs audit methodology as a result of the changes in auditing standards. 2 Risk-based auditing is described more fully in the next section. This approach to a financial statement audit is also referred to as a strategic systems audit (Bell et al. 1997; Bell et al. 2002) or a business risk audit (Lemon et al. 2000).
-
emerging body of research investigating various aspects of risk-based auditing (Ballou et al.
2004; Choy and King 2005; Kotchetova and Messier 2006; ODonnell and Schultz 2005;
Salterio, Knechel, Kotchetova 2006), but these studies have not examined the links between the
series of risk assessments made by the auditor at various decision levels (entity, business process,
and account) throughout a risk-based audit. The study of these linkages is important because if
auditors cannot properly relate the identification of business risks to an assessment of what can
go wrong at the account level then audit quality will be affected.
In this paper, we test a number of hypotheses related to the identification of business risks
and their linkage to related risk assessments at the entity, business process, and account levels.
In general terms, we examine the following questions: How does the level of residual strategic
risk3 (RSR) affect the number of business risks identified by the auditors? How does the number
of business risks identified affect the auditors assessment of risk of material misstatement
(RMM) and related financial statement implications? How does the performance of a business
process analysis4 affect the auditors assessments of RMM? Do auditors risk assessments vary
depending upon whether the business process is a core (e.g., product delivery) or supporting
(e.g., human resources) process? Do auditors entity level assessments of RSR and client
business risk (CBR) relate to their account level RMM assessments?
Our results show a number of important findings related to the linkages between auditors
risk assessments in a risk-based audit. First, the auditors documented a greater number of
specific business risks when residual strategic risk was high. Second, the greater number of
business risks identified by the auditors, the greater the number of financial statement
3 Residual strategic risk is the risk that remains after the entity has implemented control activities to monitor and control the effects of the identified business risks. 4 A business process analysis is a formal assessment of a business process, including documentation of its key objectives, key risks, major classes of transaction/accounts affected by these risks, and compensating controls. In our study, participants either performed or did not perform a formal business process analysis.
2
-
implications identified. Third, the business process level risk assessments of auditors who
performed a business process analysis are not associated with RSR, but instead are associated
with their entity level client business risk (CBR) assessments; and this result holds across both
types of business processes (core and supporting). Fourth, the auditors assessments of RMM at
the financial statement account level are directly related to the auditors assessments of the risk
of material misstatement at the entity level. Finally, when auditors assess RMM at the account
level, their judgments are influenced by their process-level business risk assessments and not by
their assessments of the entity-level CBR, or by RSR. Taken together, our results show that
auditors are properly linking the identification of business risks with an assessment of what can
go wrong in the related financial statement accounts as prescribed by recent audit risk standards
(AICPA 2006; IAASB 2004a).
This paper proceeds as follows. In Section II, we provide background and an overview
of risk-based auditing. Section III presents our theory and hypotheses. Section IV discusses the
methodology. Section V presents our results, and Section VI summarizes our conclusions and
implications.
II. RISK-BASED AUDITING
Background
As a result of numerous frauds in the 1990s, the Securities and Exchange Commission
(SEC) asked the Public Oversight Board (POB) to evaluate the way audits were being conducted.
In response, the POB appointed the Panel on Audit Effectiveness, which issued its report in
August 2000 (POB 2000). The report included a number of recommendations to the ASB; the
most important being a need to increase the rigor and specificity of auditing standards.
Concurrently, a Joint Working Group (JWG) composed of standard setters and academics from
3
-
Canada, the United Kingdom, and the United States was formed to research the recent
developments in the audit methodologies of the largest accounting firms. In May 2000, the JWG
published the results of its research (Lemon et al. 2000). Both the Panel and the JWG concluded
that auditing standards needed to be enhanced in order to improve audit effectiveness.
The ASB and International Auditing and Assurance Standards Board (IAASB) undertook
a joint project to respond to the Panels and JWGs recommendations. The two bodies formed
the Joint Risk Assessments Task Force in October 2001 to develop a common set of auditing
standards. However, with the passage of the Sarbanes-Oxley Act in 2002, the Public Company
Accounting Oversight Board (PCAOB) assumed the role of issuing auditing standards for
registered companies in the U.S. in May 2003, and the ASB suspended work on its set of risk
assessment standards. In October 2003, the IAASB completed the international phase of the risk
assessment project and issued three International Standards on Auditing (ISAs).5 After the ASB
was reorganized in early 2004, its Risk Assessments Task Force revised the original exposure
drafts of the risk assessment standards. In November 2005, the ASB approved eight SASs, three
of which map closely to the international standards: SAS No. 106, Audit Evidence, SAS No. 109,
Understanding the Entity and Its Environment and Assessing the Risks of Material Misstatement,
and SAS No. 110, The Auditor's Procedures in Response to Assessed Risks and Evaluating the
Audit Evidence Obtained (AICPA 2006).
Standard setters believe that the introduction of this risk assessment process as the central
framework for auditing will improve the quality and effectiveness of audits, and result in a
substantial change in audit practice (Bell et al. 2005). The ASB has stated that the standards will
enhance auditors application of the audit risk model in practice by requiring (1) a more in-depth
5 The standards were ISA 315, Understanding the Entity and Its Environment and Assessing the Risks of Material Misstatement, ISA 330, The Auditor's Procedures in Response to Assessed Risks, and ISA 500, Audit Evidence (IAASB 2005).
4
-
understanding of the entity and its environment, including its internal control, to identify the
risks of material misstatement in the financial statements and what the entity is doing to mitigate
them, (2) a more rigorous assessment of the risks of material misstatement of the financial
statements based on that understanding, and (3) an improved linkage between the assessed risks
and the nature, timing, and extent of audit procedures performed in response to those risks
(AICPA 2006, p. iii). While the current study provides some limited evidence on the first two
points, its main focus is on the linkage of the auditors business risk assessments to the risk of
material misstatement at the entity and account levels.
An Overview of the Risk Assessment Process
As outlined in the new auditing standards (AICPA 2006), the risk assessment process6
includes the following steps. The first step requires the auditor to obtain an understanding of the
entity and its environment, including the components of internal control, by performing risk
assessment procedures (AICPA 2006, SAS No. 109).7 Based on this understanding of the entity
and its environment, the auditor identifies business risks that may result in material
misstatements in the financial statements. Such business risks result from significant conditions,
events, circumstances, actions, or inactions that could adversely affect the entitys ability to
achieve its objectives and execute its strategies, or through the setting of inappropriate objectives
and strategies (AICPA 2006, SAS No. 109 29).
The second step requires the auditor to use the information from the risk assessment
procedures to assess RMM at the financial statement and assertion levels. Part of this RMM
6 For a detailed discussion of the auditors risk assessment process, see Messier et al. (2006, Chapter 3). 7 Risk assessment procedures include inquiries of management and others within the entity, analytical procedures, and observation and inspection for the purpose of obtaining an understanding of the entity and its environment and to assess the risks of material misstatement at the financial statement and assertion levels (AICPA 2006, SAS No. 109 6). Bell et al. (2005) take a broader view and suggest that all audit procedures (i.e., tests of controls and substantive procedures) are risk assessment procedures because of the dynamic nature of the audit process.
5
-
assessment is an evaluation of how the entity responds to those business risks and a requirement
to obtain evidence that those responses have been implemented. In evaluating the entitys
internal control components, one of the main focus points is an evaluation of the entitys risk
assessment process - the process used by the entitys management to identify and respond to
risks that may prevent the achievement of the entitys objectives. If the entitys response to the
identified risk is adequate, RMM can be reduced. However, if the auditor identifies risks that the
entitys risk assessment process has failed to identify or control (RSR), the auditor should
consider why the process failed and increase the RMM.8
The third step requires the auditor to determine what audit work needs to be completed to
address the RMM identified at the financial statement level, and then to design and perform audit
procedures whose nature, timing, and extent are linked to the assessed RMM at the relevant
assertion level (AICPA 2006, SAS No. 110).9 In other words, the auditor must link the RMM
at the financial statement level to what can go wrong in the related financial statement
account(s). The final step involves evaluating the evidence obtained from the risk assessment
procedures, tests of controls, and substantive testing (AICPA, 2006, SAS No. 106). The output
from this last step is the auditors opinion on the financial statements.
Documentation of audit work has also been an important issue for standard setters. Both
the ASB and PCAOB have recently issued standards (AICPA 2005; PCAOB 2004), which
significantly increase requirements for auditors to document their work. In addition, the new risk
assessment standards (AICPA 2006, SAS Nos. 109 and 110) have very specific requirements for
8 In the SASs, these risks are referred to as significant risks which are defined as identified risks that, in the auditors judgment, require special audit consideration (AICPA 2006, SAS No. 109 110). 9 While the SASs use the term assertion level, the assertions being tested relate to classes of transaction, financial statement accounts, or disclosures. For example, the auditor may have determined through the understanding of the entity and its environment that the entitys products are facing stiff competition. As a result, the auditor should be concerned with the valuation assertion (AICPA 2006b). However, in this example, the RMM at the assertion level resides in the inventory account. Throughout the paper, we use the term RMM at the account level.
6
-
documenting the risk assessment process that were not present in prior auditing standards. In
particular, the auditor should document the following:
! Key elements of the understanding of the entity and its environment, including each of the components of internal control; the sources of information from which the understanding was obtained; and the risk assessment procedures used.
! The assessment of the risks of material misstatement both at the financial statement level and at the relevant assertion level, and the basis for the assessment.
! The risks identified and related controls evaluated. ! The overall responses to address the assessed risks of misstatement at the financial
statement level. ! The nature, timing, and extent of the further audit procedures. ! The linkage of those procedures with the assessed risks at the relevant assertion level. ! The results of the audit procedures (AICPA 2006, SAS Nos. 109 and 110).
The documentation requirements should result in auditors conducting formal business process
analyses that may have a positive effect on the auditors risk assessments.10
Figure 1 presents how we operationalized the audit risk assessment process and how it
relates to our experimental materials (described later). In summary, the entity-level strategic
analysis is linked to specific business processes which in turn are linked to the related financial
statement accounts.
[Insert Figure 1]
III. THEORY AND HYPOTHESES DEVELOPMENT
We develop our hypotheses by following the risk assessment process discussed in the
prior section and outlined in Figure 1.11
Strategic Analysis
Auditing standards require that the auditor assess the risk of material misstatement at the
financial statement (entity) level (RMM-E) by understanding the entity and its environment,
10 It is also possible that documentation requirements can have negative effects when risk assessments are performed in a free-flowing narrative form; such negative effects do not occur, however, when risk assessments are numeric, as in our study (see Piercey 2006). 11 Acronyms used throughout this section are described in Figure 1.
7
-
including internal control (AICPA 2006, SAS No. 109 102). In meeting this requirement, the
auditor uses the information gathered by performing risk assessment procedures to identify
business risks that may affect the entity. A major component of internal control that the auditor
should examine is the entity's risk assessment process. This process involves managements
identification, analysis, and management of strategic risks relevant to the preparation of financial
statements. The auditor must address any significant strategic risks not controlled or mitigated
by the entitys risk assessment process. The new auditing standards (AICPA 2006, SAS No. 109)
presume that an auditor should anchor on RSR, and that this should lead to the identification of
more business risks and result in a higher assessment of entity-level client business risk (CBR-
E). This leads to the following hypothesis:
H1: The number of business risks (NBR-E) identified by the auditors and the auditors assessment of client business risk at the entity level (CBR-E) are greater when residual strategic risk (RSR) is high than when RSR is low.
Auditing standards also require that, based on identifed business risks, the auditor
determine what could go wrong at the related financial statement level. This leads to the
following hypothesis:
H2: The greater the NBR-E identified and the higher the auditors assessments of CBR-E, the higher the level of risk of material misstatement at the entity level (RMM-E) and the greater the number of financial statement implications (NFSI) identified.
Performing a Formal Business Process Analysis12
As we have previously described, the risk assessment process requires the auditor to
evaluate evidence about the entitys business risks and then subsequently use that evidence to
make risk assessments at the account level. In order to assess the effects of business risks at the
account level, the auditor examines the entitys controls over business processes that monitor the
12 Our treatment condition that requires performing a formal business process analysis includes documentation of the results of such analysis.
8
-
initiation, authorization, processing, and recording of transactions that affect the financial
statement accounts. We hypothesize that performing a formal business process analysis will have
a positive effect on the auditors risk assessments and their linkage to potential misstatements in
the related financial statement accounts because such a formal analysis requires the auditor to
specifically consider all the relevant issues/risks related to the process.
Research in psychology (Hastie et al. 1984; Hastie and Park 1986) suggests that judgment
accuracy tends to be higher if individuals utilize an on-line strategy versus a memory-based
strategy. An individual using an on-line strategy tends to build a memory structure during
encoding, whereas an individual using a memory-based strategy forms judgments based on
recall from memory (Haberstroh and Betsch 2002). As a result, the preprocessing of information
(or evidence) should aid in an individuals utilization of an on-line judgment strategy (Hastie and
Park 1986; Haberstroh and Betsch 2002; Hammond et al. 1987). Research in social cognition
and marketing supports the supposition that preprocessing of information aids an on-line
strategy (Feldman and Lynch 1988; Friedman et al. 1985; Hastie and Park 1986; Kardes 1986;
Lynch et al. 1988; Rao and Monroe 1988). For example, numerous studies in marketing have
demonstrated that initial product judgments or prior product evaluations affect subsequent
choices and judgments about product qualities (e.g., Kardes 1986; Lynch et al. 1988; Rao and
Monroe 1988). More specifically, this research shows that consumers initial evaluations of
products influenced their subsequent overall product quality judgments; whereas factual,
descriptive information influenced discrete (performance, dependability) memory-based
judgments.
In an auditing context, individuals who perform a formal business process analysis are
more likely to use an on-line processing strategy for making risk assessments while those who do
9
-
not perform a business process analysis will likely rely on a memory-based strategy. This should
occur because a formal business process analysis results in the auditor preprocessing evidence
that will be used to make risk assessments whereas an auditor who does not perform the business
process analysis will only have memory-based evidence available to make the risk assessments.
Thus, auditors performing such an analysis should form their business process risk assessments
based on a combination of case information and RSR; and information encoded during the
formal analysis of the business process. Auditors who do not explicitly perform a formal
business process analysis will base their business process risk assessments on a memory-based
structure resulting from only the case information and RSR.13
While the auditors who perform a formal business process analysis will have the same
case information and RSR in memory as the auditor who do not perform a business process
analysis, we expect that the auditors who perform a business process analysis to primarily rely on
the information developed from the analysis of the business process to make their business
process risk assessment. This leads to the following hypotheses:
H3: When auditors perform a formal business process analysis of the clients business processes (core and supporting) they use the business process level information that is developed during the analysis and their own assessments of entity-level client business risk (CBR-E) to make process-level business risk assessments (BPR-C and BPR-S). Auditors who do not perform a formal business process analysis of the clients business processes will rely on the RSR that was communicated to them as a part of a case information to make their process-level business risk assessments.
13 Inferences from research on judgment accuracy using on-line versus memory-based strategies (e.g., Haberstroh and Betsch 2002) would suggest that auditors who explicitly perform a formal analysis of a business process will generate more accurate judgments of process-level risks than those who directly proceed from evidence inputs about RSR to process descriptions and associated risk judgments. However, we do not have benchmarks for business process risks judgments for our experimental case; hence, we are limited as to what we can conclude with respect to accuracy.
10
-
Types of Business Processes
The fourth hypothesis examines whether auditors business process assessments differ
when analyzing a core (or critical) process versus a supporting business process, and if this result
varies depending on whether the auditors performed a formal business process analysis.
Auditors who explicitly perform a business analysis of both processes are more likely to become
aware of process-level specific risks existing in both the core and supporting processes.
Conversely, auditors who do not perform a formal business process analysis are likely to focus
on entity level information inputs that relate to the entitys core business processes. Unless the
entity level analysis identifies specific risks that have direct implications for the supporting
business process, auditors who do not perform the business process analysis are likely to focus
on the core process. This likely occurs because their entity level analysis will focus on the more
significant business risks that may result in material misstatements in the core business process
(Ballou et al. 2004; ODonnell and Schultz 2005). Thus, the auditors assessments of client
business process level risk should be higher for the core business process versus the supporting
business process. This discussion predicts an interaction between BPA and ToP and leads to the
following hypotheses:
H4: The process level business risk assessments (BPR-C and BPR-S) of auditors who perform a formal analysis of the entitys business processes will not differ across the core and supporting processes. Auditors who do not perform a formal analysis of the entitys business processes will have higher process level business risk assessments for the core process (BPR-C) than the supporting process (BPR-S).
Linking the Risk of Material Misstatement at the Financial Statement Account Level to the
Entity Level Assessment
The time period and intervening judgment tasks between the entity level analysis (i.e., the
initial evaluation of evidence about RSR and the judgment of the CBR-E) and final business
11
-
process level judgments should reduce their association. If the most recent judgments in the risk-
based approach are available in memory, we expect that the auditors assessments of the risk of
material misstatement for the accounts affected by the business process (RMM-C and RMM-S)
to be primarily based on their assessments of business process-level risks (BPR-C and BPR-S)
rather than RSR at the entity-level. However, we expect that neither type of the process nor
performance of a business process analysis should directly affect the auditors assessments of the
risk of material misstatement in the process-specific accounts. Thus, we hypothesize:
H5: The auditors assessments of the risk of material misstatement at the financial statement account level (RMM-C and RMM-S) are directly related to the auditors assessments of the risk of material misstatement at the entity level (RMM-E).
H6: When auditors assess the risk of material misstatement at the financial statement
account level (RMM-C and RMM-S), their judgments are influenced by the process-level assessments of business risks (BPR-C and BPR-S), and not by their assessments of the entity-level client business risk (CBR-E) or by residual strategic risk (RSR).
IV. METHODOLOGY
Research Design
The experimental design is a 2 (Residual Strategic Risk) x 2 (Business Process Analysis)
x 2 (Type of Process) mixed factorial. The first two factors are manipulated between-subjects,
while the third factor is manipulated within-subjects.14 We also use two measured variables to
examine our hypotheses: the auditors entity-level assessments of the client business risk (CBR-
E) and risk of material misstatement at the entity level (RMM-E).
The Residual Strategic Risk (RSR) factor was manipulated at two levels: high versus
low. In the high risk condition, the participants read the results of the clients strategic analysis
that ended with the following statement: The outgoing senior auditor was concerned that
14 We also manipulate order of the two business processes thereby introducing a third between-subjects factor in our design. When included in our analyses, order was not a statistically significant factor (p-value=.845 or greater).
12
-
National Foods strategy did not adequately address the threats and opportunities in the
companys external environment. The participants were also presented with key financial ratios
that were unfavorable. The ratios indicated a decline in sales and gross profit accompanied by an
increase in interest expense and operating income (6.2%).15 In the low risk condition, the
participants read the results of the strategic analysis that ended with the following statement:
The outgoing senior auditor concluded that National Foods strategy adequately addressed the
threats and opportunities in the companys external environment. The participants also viewed
key financial ratios that were favorable - an increase in sales, gross profit, operating income, and
a small increase in net income (2.6%).
The Business Process Analysis (BPA) factor was manipulated at two levels: an
explicit request to perform and document the analyses of the entitys business processes versus
no explicit request.16 In the explicit request condition, the participants were asked to
document the key objectives of the process, its key activities and associated risks, significant
classes of transactions, and assess two risks: (1) the risk that the overall objectives of the process
are not being achieved, and (2) the risk of material misstatement for the classes of transactions
and account balances affected by the process. In the condition where the analysis of the entitys
business process was not explicitly requested, the auditors were asked to respond to the two risk
assessments only (see section Case Materials below).
The Type of Process (ToP) factor also had two levels: a core (or critical) business
process and a supporting business process. The core business process was operationalized as the
15 We included a larger increase in operating income in the high risk condition than the low risk condition to indicate that there was an increased risk of material misstatement. With a decrease in sales and gross profit and an increase in interest income, there should be an expectation of a decline in operating income. 16 It is possible that participants in the no explicit request condition performed a business process analysis but did not document it in our case materials. If the participants did perform such an analysis, it would work against finding significant results.
13
-
product/service delivery process while the supporting business process was operationalized as
the human resource management process. The classification of the processes into core versus
supporting was based on the source case developed by Greenwood and Salterio (2002) (see next
section). The case materials clearly labeled the two processes as core or supporting.
Case Materials
The case used in this study was a hypothetical grocery retailer, National Foods, located in
the Southeastern United States. The profile of the client was based on the Loblaws case
developed by Greenwood and Salterio (2002).17 The case materials contained three parts. Part 1
included the strategic analysis of National Foods; including background on the entity, its external
environment and related risks, Porters five-forces framework for the food distribution industry,
information on the bargaining power of suppliers and buyers, threat of new entrants, competition,
and a summary of key risks. It presented National Foods strategies, objectives, and an entity-
level business model. Lastly, the materials contained an audited balance sheet and income
statement for two years and three years, respectively. Each of these statements was also
converted to common size financial statements. After this information, all participants were
asked:
(1) Based on your review of the information about National Foods, Inc., what are the key
business risks faced by National Foods, Inc.? How many of these risks affect National
Foods financial statements? What areas of National Foods financial statements should
be subject to special audit focus? The case provided a place for the participants to list (1)
key business risks and (2) potential effects on the financial statements and areas of audit
focus. The number of key business risks was used to develop the NBR-E variable while
the potential effects were used to develop the NFSI variable. 17 We thank the authors and KPMG for permission to use the case.
14
-
(2) To assess the business risk for National Foods, Inc. at the entity level on a 7 point scale (1
= very low risk, 5 = moderate risk, 9 = very high risk). This represents the auditors
assessment of client business risk at the entity level (CBR-E).
(3) To assess the risk of material misstatement for National Foods, Inc., at the entity level (1
= very low risk, 5 = moderate risk, 9 = very high risk). This represents the auditors
assessment of the risk of material misstatement at the entity level (RMM-E).
Part 2 of the case contained information on two of National Foods business processes.
The two types of processes were introduced to the participants using the following information:
You will notice that the core business processes for the company include the following: Brand and Image Delivery, Product and Service Delivery, and Customer Service Delivery. The resource management (non-core, or supporting) processes consist of Human Resource Management, Property Management, Regulatory Management, Financial/Treasury Management, and Information Management. Your manager has asked you (BPA condition = to perform a detailed analysis of; No BPA condition = to review) one core process (Product and Service Delivery), and one supporting process (Human Resource Management).
The case materials for the product/service delivery process also included information on:
category management, supplier selection, logistics and distribution, stock management, and price
management. Similarly, the case materials for the human resource management process included
information on functions such as human resource policy, compliance with labor regulations;
employee recruitment, hiring, training, and development; performance reviews, mentoring, and
counseling; and union contract and grievance procedures. After each process, the participants in
the BPA condition were asked:
(1) Based on the description of the Product/Service Delivery (Human Resource
Management) process at National Foods, Inc. and information in Part 1, please indicate
the extent to which you agree that the following items represent the key objectives of this
15
-
process. This was followed by a list of 9 objectives for each process (1 = disagree to 7 =
agree).
(2) In your opinion, what are the key activities and associated risks of National Foods
Product/Service Delivery (Human Resource Management) process? Please list at least
one key activity and related risk for each sub-process. You may refer to the process
description if necessary.
(3) Please indicate the significant classes of transactions or accounts affected by the
National Foods Product/Service Delivery (Human Resource Management) process. The
classes of transactions were categorized as routine, non-routine, and estimates.
(4) Based on your review of information about National Foods, Inc., and your analysis in
Parts 2a and 2b, please make an assessment of the overall risk that the objectives of the
Product/Service Delivery Process (Human Resource Management) at National Foods are
not achieved (1 = very low risk, 5 = moderate risk, 9 = very high risk). This represents
the auditors assessment of the business process risk for each process (BPR-C = core
business process or BPR-S = supporting business process).
(5) Based on your review of information about National Foods, Inc., and other analyses you
performed in Parts 2a and 2b, please assess the risk of material misstatement for the
classes of transactions/accounts affected by the National Foods Product/Service
Delivery (Human Resource Management) Process (1 = very low risk, 5 = moderate risk,
9 = very high risk). This represents the auditors assessment of the risk of material
misstatement for the accounts/transactions in each process (RMM-C = core business
process or RMM-S = supporting business process).
16
-
The completion of questions (1) (3) represent the way we requested the auditors to perform and
document the analyses of both processes. The participants in the condition that did not require a
formal business process analysis only answered questions (4) and (5).
The third part of the materials requested the participants to provide demographic
information, task specific information, and responses about the case materials. Figure 1 provides
a graphical representation of the points where each variable was assessed.
Experimental Procedures and Participants
The experiment was administered at training sessions of the Big 4 accounting firms18
held in the United States and in Norway.19 One hundred thirty-four fully completed, usable
responses were obtained from audit seniors. On average, the participants had 45 months (3.75
years) of auditing experience (27 engagements, regardless of size). Their audit planning
experience, on average, was 16 engagements. The most frequently reported areas of
specialization included: financial services, insurance, and real estate (38%), manufacturing
(14%), energy, oil and mining (12%), information technology, software, and communications
(12%), followed by transportation and shipping (11%), and communications (6%). The
remaining 7% of participants specialized in other services, such as healthcare, higher education,
non-for-profit entities, travel agencies, and international trade. In summary, none of the
participants planned most of their engagements in the industry of the client depicted in the
experimental case- grocery retail. Therefore, collectively, our sample can be viewed as a
18 Firm affiliation is not a statistically significant factor in our analyses. 19 Country is not statistically significant in any of the analyses. For Norwegian participants, we include only responses where self-rated knowledge of English was 5 or above on a 7-point scale anchored at 1=very poor knowledge of English and 7=very good knowledge of English.
17
-
generalist auditor with respect to the experimental case, corresponding to the type of auditors
who tend to work on such engagements in practice.20
Task-Specific Experience
Participants reported a high level of familiarity with the risk analysis task. All of the
participants had performed analyses of client business processes during previous audit
engagements. When asked about the methods used to perform an analysis of a business process,
the most often mentioned methods included the following: interview, discussions, inquiry of
client (40%); research about the client company and its processes (18%); research regarding the
business environment, industry risks, and competitors (13%); review of the process-level
controls (13%); review of the previous years files and analyses (9%); flowcharting, process or
cycle mapping (7%); and consultations with other members of the audit team (7%).21 Thus, the
auditors in this study were familiar with the approach provided in our experimental materials.
The auditors also indicated that they document the results of the strategic analysis using
the following means: databases (13%), memos (10%), controls section of the working papers
(7%), flowcharts (6%), summary in the files (2%), and diagrams (1%). When asked what is
being documented in the course of business process analysis, the items cited were: narratives or
process descriptions (9%); issues and risks (9%); conclusions and explanations (7%), areas tested
(4%), people interviewed and management responses (4%), and expectations (3%).
Finally, participants were asked to indicate whether they perform business process
analysis and document its results on a typical engagement using a 7-point scale, where 1
20 Our information regarding generalist auditors work on grocery clients was confirmed with authors of the experimental case who in turn consulted with several Big 4 partners. 21 The sum of percentages for the methods used to analyze client business processes exceeds 100% because some of the participants reported more than one method. Other methods that were mentioned less often included review of the business process strategy analysis conducted by the client; logical inference; analysis of the correlation between business risk, transaction types, and audit risk; analysis of key routines and transactions; and following the managers guidance.
18
-
corresponded to never, and 7 corresponded to always. The mean response was 4.68 for
performance of business process analysis and 4.97 for documentation. Thus, it appears that
auditors typically perform analyses of business processes on their audit engagements.
Auditors indicated that case materials and instructions were understandable (mean
response of 5.32 on a 7 point scale where 1 was not at all understandable and 7 was very
understandable) and realistic (mean response of 5.81 on a 7-point scale where 1 was not at all
realistic and 7 was very realistic). The average self-reported time to complete the
experimental task was approximately 43 minutes.22
IV. RESULTS
Manipulation Check
We had originally planned on using the auditors assessment of CBR-E as a manipulation
check on RSR since one would expect that the more residual risk, the higher the assessment of
the clients business risk. Unfortunately, CBR-E did not vary across the two levels of RSR.23
In an attempt to get a measure of the two levels of RSR, we partially re-administered the
experiment to 86 participants in a graduate auditing class in Canada. These participants have,
one average, 16 months of audit experience, and become senior associates immediately
following completion of the class. The participants were presented with background information
about the client, National Foods, Inc., and results of strategic analysis, as Part I of the instrument
used in the main experiment, and were asked to provide their assessment of RSR on a 9-point
scale, anchored at 1 as very low risk, and 9- very high risk. Next, they were provided with
description of two business processes, as in Part 2 of the main experiment, and were asked to
22 When we analyzed time in relation to BPA and RSR factors, neither was statistically significant (p=.579 for BPA and p=.068 for RSR). 23 See the ANOVA results reported for H1.
19
-
classify these processes as core or supporting to the clients operations. Finally, participants
provided demographic information.
Analysis of participants RSR assessments indicated that they perceived the difference
between the low and high conditions as intended: the mean assessment of RSR was 5.14 in
the low condition, and 6.64 - in high; these assessments were statistically significantly
different (F=27.901, p=.000). In addition, 98% of participants correctly classified Product and
Service Delivery process as core, and 63% classified Human Resources Management process as
supporting to the clients operations. Therefore, we obtained ex-post comfort in our
manipulations of RSR and ToP.24
Descriptive Statistics
Table 1 presents descriptive statistics for all measured and dependent variables. Table 2
(Panel A) shows the correlations between all variables. NBR-E and CBR-E do not appear to
correlate significantly with RSR (p>.05 for both variables), casting doubt on prediction in H1.
However, there is a positive, significant correlation between NBR-E and NFSI (p=.000 in Table
2, Panels A and B; p=.032 in Panel C). Also, we observe a positive, significant correlation
between RMM-E and CBR-E (p=.001 in Table 2, Panel A; p=.063 in Panel B, and p=.016 in
Panel C). Therefore, this provides preliminary support for H2. BPR-C and BPR-S are correlated
with the independent variable BPA (p
-
analysis factor: p!.05 for corr(CBR-E; BPR-C) and corr(CBR-E; BPR-S) at BPA=0 (Panel B,
Table 2) and BPA=1 (Panel C, Table 2). This provides partial support for H3 because we also
observe that none of the process-level business risk assessments are significantly correlated with
the independent variable RSR. Specifically, Panel B (Table 2) indicates that in the absence of
performing a business process analysis, process-level business risk assessments are not
statistically significantly correlated with residual strategic risks (p=.549 for corr(RSR; BPR-C);
p=.175 for corr(RSR; BPR-S)). Finally, significant correlations between process-level
assessments of business risk (BPR-S, BPR-C) and risk of material misstatement at the entity
level (RMM-E) (p
-
Once the business risks have been identified, the auditor assesses the risk of material
misstatement and what can go wrong in the financial statements. H2 tests whether the
identification of a greater number of business risks (NBR-E) and higher assessments of client
business risk at the entity level (CBR-E) result in a higher assessment of the risk of material
misstatement at the financial statement level (RMM-E) and in the identification of a greater
number of financial statement implications (NFSI). We ran a MANOVA with RSR, CBR-E and
NBR-E as independent variables, and RMM-E and NFSI as dependent variables. Table 3 (Panel
A) presents the result. The MANOVA shows that RSR is not significant (F=.95, p=.389) while
CBR-E (F=5.50, p=.005) and NBR-E (F=12.44, p=.000) are significant. Panel B presents the
individual ANOVAs. Using RMM-E as the dependent variable, Panel B shows that CBR-E is
significant (F=10.45, p=.002) while RSR (F=1.20, p=.275) and NBR-E (F=1.37, p=.243) are not
significant. The ANOVA with NFSI as the dependent variable indicates that NBR-E is
significant (F=23.14, p=.000) while RSR (F=.80, p=.373) and CBR-E (F=.43, p=.530) were not
significant. Generally, these results support H2 but suggest that the linkage is more specific than
we hypothesized.
[Insert Table 3]
To examine these findings further, we ran two regressions. First, we regressed NFIS on
NBR-E and CBR-E.25 Panel C shows that NBR-E is significant (t=4.73. p
-
the higher the auditors assessment of client business risk (CBR-E), the higher the auditors
assessment of RMM-E. Therefore, we conclude overall support for H2.
The Effect of Performing a Business Process Analysis
Table 4 presents the repeated measures ANCOVA used to test H3. The results show that
the BPA variable is highly significant (p=.002). 26 In the core business process, the auditors
business process risk assessments were higher in the condition where the auditors performed a
business process analysis (5.42) than in the condition where they did not (4.60). In the
supporting business process, we find a similar result (4.26 v. 3.53). This provides overall
support that performing a formal business process analysis affects the auditors business process
risk assessments.
[Insert Table 4]
To test the predictions for H3, we split the sample by the two levels of BPA and analyze
the sub-samples separately. Table 5 presents the results of these analyses. In each analysis, RSR
is a between-subject factor, CBR-E is a covariate, and ToP is a within-subject factor. H3
predicts that for the auditors who perform a business process analysis will not rely on RSR and
that their assessments of business risks for both processes will be associated with their own
entity-level client business risk assessment (CBR-E). As shown in Panel A, RSR is not
significant (p=.106) and CBR-E is significant (p
-
business processes. Therefore, our predictions for performing a business process analysis are
supported.
[Insert Table 5]
H3 also predicts that auditors who do not perform a business process analysis will rely on
the RSR to develop their process-level business risk assessments. As shown in Panel B, the RSR
variable is not significant (p=.452). Thus, our prediction that the auditors business process level
risk assessments would be associated with RSR for those auditors who did not perform a formal
business process analysis was not supported. However, they are consistent with the results for the
auditors who performed a formal business process analysis, their business process-level risk
assessments were associated with their entity-level client business risk assessment (CBR-E; p=
.004). The significance of CBR-E in both conditions suggests that the auditors used their CBR-E
judgments in their assessment of BPR regardless of whether the auditors performed a BPA or
not. We interpret this finding as an indication that auditors utilize primarily an on-line strategy;
i.e., they rely on a series of their own diagnostic judgments throughout the risk assessment
process, as opposed to applying a memory-based judgment that would imply plain transfer of
information about residual strategic risk to the process level judgments.
H4 predicts an interaction effect (ToP * BPA) - the process level business risk
assessments (BPR-C and BPR-S) of auditors who perform a formal analysis of the entitys
business processes will not differ across the core and supporting processes while the auditors
who do not perform a formal analysis of the entitys business processes will have higher process
level business risk assessments for the core process than the supporting process. Table 4 shows
that the ToP * BPA interaction is not significant (p=.406).
24
-
To investigate this result further, we examine the analyses reported in Table 5 where the
dataset is split by whether a formal business process analysis was, or was not, performed. Panel
A of Table 5 shows that ToP is not significant (p=.679). Specifically, when BPA=1, mean BPR-
S assessment is 4.26, while mean BPR-C is 5.42, and they do not appear to be statistically
significantly different. Thus, these auditors business process risk assessments do not differ
across core and supporting process. Panel B of Table 5 shows a significant main effect for ToP
(p=.029, one tailed).27 The means for the main effect of ToP show that the auditors BPR
assessments were higher in the core process compared to the supporting process (4.60 v. 3.53).
Therefore, while the overall analysis did not support the predicted interaction effect, the results
of the individual ANCOVAs do. We interpret these results to support H4 and to suggest that a
formal, documented analysis of a business process potentially overcomes the auditors tendency
to look for problems in the core business process while overlooking the supporting process (see
Ballou et al. 2004, ODonnell and Schultz 2005).
Assessment of Risk of Material Misstatement at the Process Level
The last two hypotheses are tested in a repeated measures ANCOVA that includes all of
the previous independent variables and covariates, with RMM-C and RMM-S as the dependent
variables. Table 6 presents the results.
[Insert Table 6]
H5 predicts that the auditors assessments of the risk of material misstatement at the
financial statement account level (RMM-C and RMM-S) are directly related to the auditors
assessments of the risk of material misstatement at the entity level (RMM-E). As Table 6 shows,
RMM-E is significant (F=11.76, p=.001). The ToP*RMM-E interaction term was also significant
27 The ToP*RSR interaction was significant (p=.046). Simple effects tests of the ToP*RSR interaction indicate process level business risk assessments are marginally sensitive to RSR in the supporting process (p=.087, 2-tailed), but not for the core process (p=.633, 2-tailed).
25
-
(F=5.36, p=.022). Simple effects tests show that RMM-E is significant for the critical process
(p=.000) but not for the supporting process (p=.368). We interpret this result to indicate that
auditors incorporate their entity-level RMM assessments more readily in account-level
assessments for the core, but not for the supporting process. Thus, H5 is supported.
H6 predicts that when auditors assess the risk of material misstatement in the accounts
and transactions affected by the business process their judgments will be influenced by their
process-level business risk assessments (BPR-C and BPR-S) and not by their assessments of the
entity-level client business risk (CBR-E), or by residual strategic risk (RSR). Table 6 shows that
the covariates measuring process-level business risks for both processes (BPR-C: p
-
association with their earlier assessments of entity-level client business risk, but not with
evidence inputs about residual strategic risk. Taken together, these findings suggest that auditors
tend to rely upon their own assessments of client-level business risk in making process-level
assessments, rather than upon conclusions about residual strategic risk communicated to them in
a strategic analysis summary. Our findings lend credence to the standard-setters and accounting
firms objective of connecting global (entity-level) and process-level risk assessments by means
of explicitly requiring auditors to perform and document work performed, including analyses of
the clients business processes (AICPA 2005; PCAOB 2004, A8-A10). Our result showing that
process-level assessments of risk of material misstatement are influenced mostly by assessments
of business risks at the same level, as well as entity-level risk of material misstatement, further
corroborate such connectivity in auditors judgments.
We find that the requirement to perform an analysis of a business process is associated
with auditors process-level risk assessments; and that auditors process-level business risk
assessments are not associated with the type of the process (core versus supporting) in the
presence of the requirement to perform an analysis of the entitys business processes. Taken
together, these findings provide support for the business risk audit approach. They further
suggest that concerns that auditors who perform and document process analysis may overlook
risks embedded in the support process while focusing solely on the core process may be
unwarranted when auditors perform a formal business process analysis (Ballou et al. 2004).
Limitations and Future Research
This study is subject to limitations typically associated with experimental work that uses
audit professionals as participants. In addition, our experimental manipulation of business
process analysis was generic, and not tailored specifically to any of the participants firms
27
-
methodologies. Also, we examine risk assessments made by auditors individually, thereby
ignoring the highly interactive nature of audit team discussions that are likely to surround such
tasks in practice. Finally, due to the nature of the experimental task, auditors participating in the
experiment did not have an option of obtaining additional information beyond what was
provided in the materials, and thus they may have based their risk assessments on what they
would normally consider incomplete information.
Future studies may investigate whether business risk audit approaches also provide for
connectivity between the multiple-level (entity, process, and assertion) risk judgments and
auditors choices regarding the nature, timing, and extent of audit procedures. Results of our
study suggest that structuring an experiment whereby risk assessment, process analysis, and
planning judgments are set as a series of on-line judgment tasks may be an appropriate way of
testing such connectivity.
28
-
REFERENCES
American Institute of Certified Public Accountants. 1993. The Expectation Gap Standards: Progress, Implementation Issues, Research Opportunities. New York, NY: AICPA.
__________. 2005. Audit Documentation. New York, NY: AICPA. __________. 2006. Statements on Auditing Standards Nos. 104-111. (March) New York, NY:
AICPA. Ballou, B., Earley, C. E., and J. Rich. 2004. The impact of strategic positioning information on
auditor judgments about business process performance. Auditing: A Journal of Practice & Theory 23(2): 71-88.
Bell, T., Marrs, F., Solomon, I., and H. Thomas. 1997. Auditing Organizations through a
Strategic Systems Lens: The KPMG Business Measurement Process. KPMG Peat Marwick LLP.
__________, M.E. Peecher, and I. Solomon. 2002. The Strategic-Systems Approach to Auditing.
In Cases in Strategic-Systems Auditing. Bell, T.B., and I. Solomon (Eds.). KPMG LLP. __________. 2005. The 21st Century Public Company Audit: Conceptual Elements of KPMGs
Global Audit Methodology. KPMG LLP. CICA. 2005. CICA Assurance Handbook Section 5141 Understanding the Entity and Its
Environment and Assessing the Risks of Material Misstatement. Available at http://www.cica.ca/index.cfm/ci_id/7966/la_id/1.htm.
Carlston, D. E. 1980. The recall and the use of traits and events in social inference processes.
Journal of Experimental Social Psychology 16 (July): 303-328. Choy, A. K., and R. R. King. 2005. An experimental investigation of approaches to audit decision
making: An evaluation using systems-mediated mental models. Contemporary Accounting Research 22(2): 311-350.
Coupey, E., O. Bodur, and D. Brinberg. 1998. Pre-decision processes in consumer choice: Effects
of prior knowledge on aspects of decision structuring. Advances in Consumer Research 25:226-232.
Feldman, J. M., and J. G. Lynch, Jr. 1988. Self-generated validity and other effects of
measurement on belief, attitude, intention, and behavior. Journal of Applied Psychology 73(3): 421-435.
Friedman, L., Howell, W.C., and C. R. Jensen. 1985. Diagnostic judgment as a function of the
preprocessing of evidence. Human Factors 27(6): 665-673.
29
-
Greenwood, R., and S. Salterio. 2002. Loblaw Companies Ltd. In Cases in Strategic-Systems Auditing. Eds., Bell, T. B., and I. Solomon. U.S.A.: KPMG LLP.
Haberstroh, S., and T. Betsch. 2002. Chapter 13: Online strategies versus memeory-based
strategies in frequency estimations. In Sedlmeier, P., and T. Betsch (Eds.), Etc. Frequency Processing and Cognition New York, NY: Oxford University Press, Inc.
Hammond, K., Hamm, R.M., Grassia, J., and T. Pearson. 1987. Direct comparison of the efficacy
of intuitive and analytical cognition in expert judgment. IEEE Transactions on Systems, Man, and Cybernetics 17(5): 753-770.
Hastie, R., and B. Park. 1986. The relationship between memory and judgment depends on
whether the judgment task is memory-based or on-line. Psychological Review 93(3): 258-268.
Hastie, R., B. Park, and R. Weber. 1984. Social memory. In Handbook of Applied Cognition.
Wyer, R. S., Jr., and T. K. Skrull (Eds.). Lawrence Erlbaum Associates: Hillsdale, NJ. IAASB. 2005. Handbook of International Auditing, Assurance, and Ethics Pronouncements.
International Federation of Accountants: New York. Jeppesen, K.K. 1998. Reinventing auditing, redefining consulting and independence. The
European Accounting Review 7(3): 517-539. Kardes, F. R. 1986. Effects of initial product judgments on subsequent memory-based judgments.
Journal of Consumer Research 13(1): 1-11. Kotchetova, N., and W.F. Messier, Jr. 2006. Strategic analysis and auditors risk assessment.
Working paper. University of Waterloo. Lemon, W. M., K.W. Tatum, and W. S. Turley. 2000. Developments in the Audit Methodologies
of Large Accounting Firms. UK: ABG Professional Information. Loken, B., and R. Hoverstad. 1985. Relationships between information recall and subsequent
attitudes: Some exploratory findings. Journal of Consumer Research 12(2): 155-168.
Lynch, J. G., Jr., H. Marmosrstein, and M. F. Weigold. Choices from sets including remembered brands: use of recalled attributes and prior overall evaluations. Journal of Consumer Research 15(2): 169-184.
Messier, W.F., Jr., Glover, S. M., and D. F. Prawitt. 2006. Auditing and Assurance Services: A
Systematic Approach. 4th Ed. McGraw-Hill/Irwin: New York, NY. Nisbett, R., and L. Ross. 1980. Human Inference: Strategies and Shortcomings of Social
Judgment. Prentice Hall: Englewood Cliffs, NJ.
30
-
ODonnell, E. and J. Schultz. 2005. The halo effect in business risk audits: Can strategic
assessment bias auditor judgment about accounting details? The Accounting Review 80(3): 921-939.
Piercey, M. D. 2006. Somewhat possible or substantial doubt? Documentation requirements,
persuasion tactics, and verbal (vs. numerical) audit risk assessment. Working paper. University of Illinois at Urbana-Champaign.
Public Oversight Board - Panel on Audit Effectiveness. 2000. The Panel on Audit Effectiveness:
Report and Recommendations. POB: Stamford, CT. Public Company Accounting Oversight Board. 2004. Auditing Standard No. 3 - Audit
Documentation. PCAOB: Washington, DC. Rao, A. R., and K. B. Monroe. 1988. The moderating effect of prior knowledge on cue utilization
on product evaluations. Journal of Consumer Research 15(2): 253-264. Salterio, S.E., W. R. Knechel, and N. Kotchetova. 2006. Performance measurement systems and
strategic analysis extensiveness: Auditors usage of balanced scorecards and performance benchmarks. Working paper. Queens University.
Weil, J. 2004. Behind the wave of corporate fraud: A Change in how auditors work. Wall Street
Journal (March 25): A1, A14. Wiggins, M., and D. OHare. 2003. Weatherwise: Evaluations of cue-based training approach for
the recognition of deteriorating weather conditions during flight. Human Factors 45(2): 337-345.
31
-
Figure 1 A Graphical Representation of the Case Materials and the Elicitation of Measured and
Dependent Variables
Perform Entity-Level Strategic Analysis
(Part 1)
Identify and Assess Entity Level Risks
Perform (or do not perform) Business Process-Level Analysis
(Part 2a)
Assess Business Process Level Risks for a Critical
and a Supporting Process
Analyze Related Financial Statement Accounts
(Part 2b)
CBR-E NBR-E
RSR (high/low)
RMM-C RMM-S
Assess Risk of Material Misstatement at the Financial Statement Account Level
Identify Financial Statement Implications of
Business Risks
AssessRisk of Material Misstatement at the
Entity Level
BPR-C BPR-S
RMM-E
BPA (yes/no)
NFSI
Entity Level
Process Level
-
Relationship to Experimental Materials (see text for a more detailed description): Part 1: Participants were presented with information about the entity that was used for understanding the entity and its environment. One independent variable, residual strategic risk (RSR) was manipulated between-subjects at two levels (high and low) at this point in the experiment. The participants then listed the business risks identified (NBR-E) and the financial statement implications (NFSI) and assessed CBR-E and RMM-E. Part 2a: Participants were required to perform (or not perform) a formal business process analysis on two business process (core and supporting). The requirement to perform or not perform a formal business process analysis was the second between-subjects independent variable (BPA). The two business processes represent the within-subjects independent variable: type of process (ToP). The participants then assess BPR-C and BPR-S. Part 2b: Participants were then asked, based on the previous information, to assess the risk of material misstatement for the classes of transactions/accounts affected by each process. This resulted in the RMM-C and RMM-S assessments. Measured and Dependent Variables: CBR-E = The auditors assessment of client business risk at the entity level. NBR-E= Number of business risks at the entity level documented by the participants. NFSI = Number of financial statement implications of identified based on documented business risks. RMM-E = The auditors assessment of the risk of material misstatement at the entity level. BPR-C = The auditors assessment of business process risk for the core process (product/service delivery). BPR-S = The auditors assessment of business process risk for the supporting process (human resource
management). RMM-C = The auditors assessment of the risk of material misstatement for the classes of transactions/accounts
affected by the core process (product/service delivery). RMM-S = The auditors assessment of the risk of material misstatement for the classes of transactions/accounts
affected by the supporting process (human resource management).
33
-
Table 1 Descriptive Statistics
Panel A: Number of business risks identified (NBR-E) NBR-E BPA RSR no explicit request explicit request Marginal Means
Low 3.58
(1.500) n=38
3.83 (1.200) n=35
3.70 (1.361) n=73
High 3.63
(1.012) n=19
4.31 (1.370) n=42
4.10 (1.300) n=61
Total n=57 n=77 n=134 Panel B: Client business risk at the entity level (CBR-E) CBR-E BPA RSR no explicit request explicit request Marginal Means
Low 5.50
(1.247) n=38
5.60 (1.479) n=35
5.55 (1.354) n=73
High 5.26
(1.098) n=19
5.81 (1.254) n=42
5.64 (1.225) n=61
Total 5.42
(1.194) n=57
5.71 (1.356) n=77
5.59 (1.293) n=134
Panel C: Number of Financial Statement Implications Identified (NFSI) NBR-E BPA RSR no explicit request explicit request Marginal Means
Low 3.16
(2.034) n=38
3.23 (1.395) n=35
3.19 (1.745) n=73
High 2.74
(1.558) n=19
3.33 (1.843) n=42
3.15 (1.769) n=61
Total 3.02
(1.885) n=57
3.29 (1.645) n=77
3.17 (1.749) n=134
-
Panel D: Risk of material misstatement at the entity level (RMM-E) RMM-E BPA RSR no explicit request explicit request Marginal Means
Low 4. 79
(1.379) n=38
5.29 (1.405) n=35
5.03 (1.404) n=73
High 5.11
(1.329) n=19
5.43 (1.016) n=42
5.33 (1.121) n=61
Total 4.89
(1.359) n=57
5.36 (1.202) n=77
5.16 (1.287) n=134
Panel E: Business process risk for core process (BPR-C) BPR-C BPA RSR no explicit request explicit request Marginal Means
Low 4.68
(1.646) n=38
5.63 (1.610) n=35
5.14 (1.686) n=73
High 4.42
(1.346) n=19
5.24 (1.206) n=42
4.98 (1.297) n=61
Total 4.60
(1.545) n=57
5.42 (1.408) n=77
5.07 (n=1.518)
n=134 Panel F: Risk of material misstatement for the core process (RMM-C) RMM-C BPA RSR no explicit request explicit request Marginal Means
Low 4.61
(1.717) n=38
5.49 (1.502) n=35
5.03 (1.666) n=73
High 4.58
(1.346) n=19
5.07 (1.237) n=42
4.92 (1.282) n=61
Total 4.60
(1.591) n=57
5.26 (1.371) n=77
4.98 (1.499) n=134
35
-
Panel G: Business process risk for the supporting process (BPR-S) BPR-S BPA RSR no explicit request explicit request Marginal Means
Low 3.32
(1.416) n=38
4.37 (1.308) n=35
3.82 (1.456) n=73
High 3.95
(2.013) n=19
4.17 (1.080) n=42
4.10 (1.422) n=61
Total 3.53
(1.649) n=57
4.26 (1.185) n=77
3.95 (1.442) n=134
Panel H: Risk of material misstatement for the supporting process (RMM-S) RMM-S BPA RSR no explicit request explicit request Marginal Means
Low 3.66
(1.744) n=38
4.06 (1.434) n=35
3.85 (1.604) n=73
High 4.16
(1.604) n=19
4.05 (1.464) n=42
4.08 (1.498) n=61
Total 3.82
(1.602) n=57
4.05 (1.441) n=77
3.96 (1.555) n=134
_________________________________________ Description of Independent and Dependent Variables: RSR = Residual Strategic Risk - between subjects, independent variable. Coded as 0 if "low", 1 if "high". BPA = Business Process Analysis - between-subjects, independent variable. Coded as 0 if participants did not perform a business process analysis; coded as 1 if they did perform a business process analysis. NBR-E= Number of business risks at the entity level documented by the participants. CBR-E = Client business risk at the entity level an assessment of the risk from 1 (very low) to 9 (very high). NFSI = Number of financial statement implications of identified based on documented business risks. RMM-E = Risk of material misstatement at the entity level an assessment of the risk from 1 (very low) to 9 (very high). BPR-S = Business process risk for the supporting process (human resources) an assessment of the risk from 1 (very low) to 9 (very high). RMM-S = Risk of material misstatement for the supporting process (human resources) an assessment of the risk from 1 (very low) to 9 (very high). BPR-C = Business process risk for core process (product and service delivery) an assessment of the risk from 1 (very low) to 9 (very high). RMM-C = Risk of material misstatement for the core process (product and service delivery) an assessment of the risk from 1 (very low) to 9 (very high). ToP independent variable Type of a Process, manipulated within subjects, coded as type 1 if business process is supporting in nature (human resources), coded as 2 if business process is of core nature (product and service delivery).
36
-
Table 2 Correlations
Panel A: Full sample (N= 134) RSR BPA NBR-E CBR-E NFSI RMM-E BPR-S RMM-S BPR-CNBR-E .149
(.086) .183
(.035) 1
CBR-E .035 (.685)
.113 (.195)
.028(.750)
1
NFSI
-.013 (.885)
.076 (.382)
.383(.000)
.061(.482)
1
RMM-E .117 (.179)
.181 (.037)
.120(.167)
.276(.001)
.014(.872)
1
BPR-S .096 (.271)
.252 (.003)
.067(.445)
.339(.000)
.021(.806)
.236(.006)
1
RMM-S .075 (.391)
.073 (.405)
.055(.528)
.279(.001)
.008(.924)
.214(.013)
.636 (.000)
1
BPR-C -.051 (.562)
.268 (.002)
.163(.061)
.351(.000)
.013(.885)
.241(.005)
.448 (.000)
.237(.006)
1
RMM-C -.036 (.676)
.220 (.011)
.185(.032)
.240(.005)
.030(.730)
.423(.000)
.354 (.000)
.287(.001)
.708(.000)
Panel B: Sub-sample where BPA=0 (N = 57) RSR NBR-E CBR-E NFSI RMM-E BPR-S RMM-S BPR-C NBR-E .019
(.891) 1
CBR-E -.094 (.485)
.074 (.584)
1
NFSI
-.106 (.432)
.530 (.000)
.274(.039)
1
RMM-E .111 (.413)
.015 (.909)
.248(.063)
-.041(.761)
1
BPR-S .182 (.175)
.065 (.630)
.366(.005)
.141(.297)
.137(.310)
1
RMM-S .140 (.300)
.148 (.273)
.388(.003)
.118(.383)
.185(.168)
.657 (.000)
1
BPR-C -.081 (.549)
.221 (.099)
.258(.053)
.162(.229)
.115(.392)
.414 (.001)
.197(.143)
1
RMM-C -.008 (.954)
.206 (.124)
.166(.217)
.133(.322)
.286(.031)
.219 (.102)
.283(.033)
.637(.000)
37
-
Panel C: Sub-sample where BPA=1 (N = 77) RSR NBR-E CBR-E NFSI RMM-E BPR-S RMM-S BPR-C NBR-E .184
(.109) 1
CBR-E .077 (.503)
-.037 (.749)
1
NFSI
.032 (.783)
.244 (.032)
-.110(.339)
1
RMM-E .060 (.607)
.154 (.180)
.274(.016)
.040(.730)
1
BPR-S -.087 (.454)
-.024 (.837)
.301(.008)
-.160(.165)
.274(.016)
1
RMM-S -.003 (.977)
-.051 (.657)
.190(.099)
-.112(.333)
.224(.050)
.624 (.000)
1
BPR-C -.139 (.228)
.036 (.754)
.394(.000)
-.166(.150)
.283(.013)
.407 (.000)
.255(.025)
1
RMM-C -.151 (.188)
.104 (.368)
.267(.019)
-.103(.371)
.509(.000)
.436 (.000)
.273(.016)
.741(.000)
________________________________________ Note: Values in cells are correlations (two-tailed p-values). Refer to table 1 for a description of the independent and dependent variables.
38
-
Table 3 Multivariate Analysis of Covariance and Regressions
Panel A: Multivariate Tests
Effect Value F df p-value RSR Wilks' Lambda .985 .951 2.000 .389
CBR-E Wilks' Lambda .921 5.502 2.000 .005 NBR-E Wilks' Lambda .838 12.438 2.000 .000
Panel B: Tests of Between-Subjects Effects
Source Dependent Variable
SS df F p-value
RMM-E 21.374 3 4.654 .004Corrected Model NFSI 62.880 3 7.917 .000RMM-E 1.840 1 1.202 .275RSR NFSI 2.111 1 .798 .373RMM-E 15.994 1 10.448 .002CBR-E NFSI 1.139 1 .430 .530RMM-E 2.105 1 1.375 .243NBR-E NFSI 61.261 1 23.140 .000RMM-E 199.014 130 Error NFSI 344.173 130
Panel C: Regression Dependent Variable: Number of Financial Statement Implications of Identified Business Risks (NFSI)
Model
SS df MS F p-value
60.768 2 30.384 11.494 .000346.284 131 2.643
Regression Residual Total 407.052 133
Model
Standardized Coefficients
t
p-value
1.167 .245.382 4.734 .000
Constant NBR-E CBRE .051 .628 .531
39
-
Panel D: Regression Dependent Variable: Risk of Material Misstatement at the Entity Level (RMM-E)
Model SS df MS F p-value
19.534 2 9.767 6.370 .002200.855 131 1.533
Regression Residual Total 220.388 133
Model
Standardized Coefficients
t
p-value
5.752 .000.113 1.348 .180
Constant NBR-E CBRE .273 3.266 .001
_____________________________ Refer to table 1 for a description of the independent and dependent variables.
40
-
Table 4 Repeated Measures Analysis of Covariance
Dependent Variables: Business Process-level Assessments of Business Risk (BPR-S, BPR-C)
Source SS df F p -value Between-Subjects Effects:
Intercept 76.109 1 31.040 .000RSR .162 1 .066 .798BPA 24.382 1 9.944 .002RSR * BPA 6.392 1 2.607 .109CBR-E 61.969 1 25.273 .000Error 187.106 129
Within-Subjects Effects: ToP 2.842 1 2.358 .127ToP * RSR 4.439 1 3.684 .044ToP * BPA 1.000 1 .694 .406ToP * RSR * BPA 1.848 1 1.534 .218ToP * CBR-E .053 1 .044 .834Error(ToP) 155.472 129
________________________________________ Refer to table 1 for a description of the independent and dependent variables.
41
-
Table 5 Analysis of Covariance by BPA
Dependent Variables: Business Process-level Assessment of Business Risk (BPR-S, BPR-C) Panel A: Business Process Requirement (BPA = 1)
Source SS df F p -value Between-Subjects Effects: Intercept 66.790 1 34.391 .000 RSR 5.202 1 2.678 .106 CBR-E 33.345 1 17.170 .000 Error 82.777 74
Within-Subjects Effects: ToP .175 1 .173 .679 ToP * RSR .449 1 .443 .508 ToP * CBR-E 1.611 1 1.590 .211 Error (ToP) 75.122 74
Panel B: No Business Process Requirement (BPA = 0)
Source SS df F p -value Between-Subjects Effects: Intercept 16.365 1 5.141 .027 RSR 2.056 1 .646 .425 CBR-E 29.333 1 9.216 .004 Error 171.877 54
Within-Subjects Effects: ToP 5.364 1 3.757 .057 ToP * RSR 5.590 1 3.915 .046 ToP * CBR-E 1.692 1 1.185 .281 Error (ToP) 77. 098 54
_____________________________________ Refer to table 1 for a description of the independent and dependent variables.
42
-
Table 6 Repeated Measures Analysis of Covariance for Process-level Assessment of
Risk of Material Misstatement
Dependent Variables: Risk of Material Misstatement for each Business Process (RMM-S, RMM-C)
Source SS df F p -value
Between-Subjects Effects: Intercept 2.170 1 1.514 .221RSR .020 1 .014 .907BPA 1.283 1 .895 .346RSR * BPA .088 1 .061 .805Covariates: CBR-E .003 1 .002 .963RMM-E 16.886 1 11.762 .001BPR-S 50.584 1 35.276 .000BPR-C 35.626 1 24.845 .000Error 180.680 126 Within-Subjects Effects: ToP .111 1 .107 .744ToP * RSR .301 1 .289 .592ToP * BPA 1.258 1 1.210 .273ToP * RSR * BPA .213 1 .205 .652ToP * CBR-E 2.500 1 2.405 .123ToP * RMM-E 5.577 1 5.365 .022ToP * BPR-S 45.878 1 44.139 .000ToP * BPR-C 55.610 1 53.501 .000Error(ToP) 130.967 126
________________________________________ Refer to table 1 for a description of the independent and dependent variables.
43
