Audit of Lookouts – Traveller Mode June 2013
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Audit of Lookouts – Traveller Mode
June 2013
2
DRAFT PROTECTED A
TABLE OF CONTENTS
1.0 INTRODUCTION ......................................................................................................................................... 3
2.0 SIGNIFICANCE OF THE AUDIT .............................................................................................................. 4
3.0 STATEMENT OF CONFORMANCE ........................................................................................................ 4
4.0 AUDIT OPINION ......................................................................................................................................... 4
5.0 KEY FINDINGS ............................................................................................................................................ 5
6.0 SUMMARY OF RECOMMENDATIONS ................................................................................................. 5
7.0 MANAGEMENT RESPONSE ..................................................................................................................... 6
8.0 AUDIT FINDINGS ....................................................................................................................................... 7
8.1 LOOKOUTS ISSUANCE AND MAINTENANCE .................................................................................................. 7 8.2 LOOKOUTS INTERCEPTION AT THE BORDER .............................................................................................. 13 8.3 MONITORING, REPORTING AND ACCOUNTABILITY FOR LOOKOUTS .......................................................... 15
APPENDIX A – ABOUT THE AUDIT ................................................................................................................... 18
APPENDIX B – SAMPLING APPROACH ............................................................................................................ 20
APPENDIX C – ACRONYMS ................................................................................................................................. 21
3
{*} An asterisk appears where sensitive information has been removed in accordance with
the Access to Information Act and Privacy Act.
1.0 INTRODUCTION
The Canada Border Services Agency (CBSA or the Agency) provides integrated border services
that support national security and public safety priorities while facilitating the movement of
legitimate travellers and goods. The Agency mainly enforces the provisions of the Customs Act
and the Immigration and Refugee Protection Act, and is responsible for administering more than
90 acts, regulations and international agreements.
All travellers and goods entering Canada are required to report to the Agency through a
designated port of entry (POE). The Agency provides services at approximately 1,200 locations
across Canada, including 13 international airports, 117 land border crossings, 4 major marine
port facilities, and 27 rails sites.1 Each year the Agency processes nearly 100 million travellers
and goods at the Canadian POEs across Canada.2
The Agency has designed and implemented lookout processes and systems to manage and
intercept high-risk travellers and goods connected to terrorism, organized crime, contraband,
proceeds of crime, strategic exports, irregular migrants, and biological threats entering Canada.
Lookouts are intelligence products designed to identify a person, corporation, conveyance or
shipment that, according to available information and intelligence, may pose a threat to the
health, safety, security, economy, or environment of Canada and Canadians. Lookouts may be
issued for a number of reasons, including intelligence information and national security risks.
Border services officers (BSOs) scan or manually input data from travel documents at the
primary inspection line (PIL) and travellers’ information is verified against various databases and
systems. Through automated controls at the primary inspection line, BSOs are notified about
which travellers need to be intercepted and examined. High-risk travellers are referred for
secondary examination in customs or immigration or both. Travellers found to be violating
Canadian legislation may be subject to an enforcement action such as a seizure, an arrest, a
monetary penalty or denial of entry to Canada.
The audit includes examination of controls around the lookout “lifecycle” process for Category 1
and 2 lookouts.3 Category 1 lookouts have information being put through the intelligence
process, and are prepared and issued by Intelligence officers. Category 2 lookouts are the product
of CBSA officers other than Intelligence officers, including BSOs, officers at the National
Targeting Centre (NTC) and the Border Operations Centre (BOC), as well as criminal
investigators and Inland Enforcement officers. Intelligence officers may also enter a Category 2
1 Source: Report on Plans and Priorities, 2012-13. In addition to the above-mentioned CBSA service locations, there are 440
small vessel marina reporting sites, 12 ferry terminals, 3 postal processing plants, 3 detention facilities, 48 international locations
staffed with CBSA liaison officers, and 4 major marine port facilities. 2 CBSA, DPR 2011-2012, Section: Organizational Priority, Summary of Progress Against Priority.
3 Category 3 lookouts are issued by the CBSA on behalf of other government departments (OGDs) and were excluded from the
audit scope.
4
DRAFT PROTECTED A
lookout when information has not yet been put through the full intelligence process to warrant a
Category 1 lookout.
A lookout “lifecycle” encompasses three key stages of:
Issuance;
Maintenance; and
Reporting/closing.
2.0 SIGNIFICANCE OF THE AUDIT
Each year the Agency processes nearly 100 million travellers at Canadian ports of entry across
Canada.4 The Agency priorities include facilitating travel across Canada’s border while at the
same time protecting Canada’s population from border-related risks. The Agency’s controls
around the lookout process are integral in the interception of high-risk travellers and goods
connected to terrorism, organized crime, irregular migrants, and biological threats. If the
Agency’s controls around the lookouts processes are not functioning as intended, high-risk
travellers and goods may enter Canada and threaten its security, safety and economy.
The audit objective was to provide assurance that the Agency lookouts are appropriately
managed and processed. More specifically, the objective was to determine whether controls
around the lookout process are well-designed and effective in identifying and intercepting high-
risk travellers and goods upon entering Canada.
3.0 STATEMENT OF CONFORMANCE
The audit conforms to the Internal Auditing Standards for the Government of Canada, as
supported by the results of the quality assurance and improvement program. The audit approach
and methodology followed the International Standards for the Professional Practice of Internal
Auditing as defined by the Institute of Internal Auditors and the Internal Auditing Standards for
the Government of Canada as required by the Treasury Board Internal Audit Policy.
This audit provides a high level of assurance that the opinion provided therein is appropriate and
reflects conditions that existed at the time of the audit.
4.0 AUDIT OPINION
The Agency has processes and systems to manage and intercept high-risk travellers. Some gaps
in the design of controls in lookouts’ issuance and maintenance were noted. Controls have been
implemented at ports of entry to identify and intercept lookouts and report on the examination
results. However, these controls are not always working as intended and effective monitoring at
the corporate level is not in place to assess the functioning of the lookout program and gauge
whether the Agency’s priorities are achieved and risks appropriately managed. This results in a
moderately high-risk exposure with respect to some of the Agency’s enterprise risks.
4 CBSA, DPR 2011-2012, Section: Organizational Priority, Summary of Progress Against Priority.
5
5.0 KEY FINDINGS
The Agency has processes and systems to manage and intercept high-risk travellers and goods.
Intelligence officers issue and maintain their lookouts after applying an intelligence cycle;
however, there is limited management oversight and ongoing monitoring in the course of normal
operations to ensure lookouts are accurate, and valid.
The Agency does not have reasonable assurance that all lookouts are maintained in a timely
manner. {*}. Without documented rationale in the intelligence file, it is unknown {*}.
Immigration lookouts are not consistently reviewed to determine if the information in the system
warrants the continuation of the lookout.
Intelligence information for lookouts is documented in the Intelligence Management System
(IMS) for most lookout files reviewed. However, the IMS files do not always contain sufficient
information to demonstrate the application of the intelligence cycle and to facilitate quality
review. A routine management review process for lookouts’ quality is not in place.
The electronic scanning of travel documents has been mandatory at all airports since at least
2001. This practice, recently expanded to land borders, will improve the robustness of key
controls for lookouts. Practices for referring close matches for secondary examination vary
across ports. Recent system enhancements and guidance on dealing with close matches issued to
the frontline officers should improve the interception of high-risk travellers.
Examination results are not consistently entered in the systems by BSOs and the quality of
narrative reports varies. The completion and the quality of examination results are not
systematically reviewed by management. Without sufficient information and timely examination
results, reporting on the results of the lookout program and lookouts maintenance is challenging.
Ongoing monitoring is a key element of the risk management approach. Formal monitoring at the
corporate level is not performed. Statistics on the lookouts issued, intercepted, un-acquitted and
missed are not gathered and the Agency does not regularly assess the design, functioning and
effectiveness of controls through periodic reviews. National oversight and monitoring are key to
the assessment of the functioning of the lookout program and ensuring that the Agency’s
priorities and risks are appropriately managed.
The CBSA management established a Lookouts Working Group in September 2012 to assess and
develop new strategies for the management of lookouts. The objectives of the Group include the
improvement of monitoring, documentation and follow-up of lookouts; the review of lookouts
Standard Operating Procedures (SOPs) and guidance; the review of training on the issuance and
maintenance of lookouts; the examination of the approval process in lookouts’ issuance; and the
establishment of a quality assurance and review process, among others. The Group is progressing
in the achievement of its objectives.
6.0 SUMMARY OF RECOMMENDATIONS
This audit makes four recommendations:
6
DRAFT PROTECTED A
Strengthen supervisory review in the lookouts’ issuance and maintenance process and to
ensure that all lookouts are documented in the Intelligence Management System to enable
the review of lookouts’ quality.
Review and improve system controls to enhance data integrity and enable timely
maintenance of lookouts.
Establish a performance measurement system to systematically track, monitor, and report
on missed and un-acquitted lookouts and implement a quality assurance process to
strengthen the quality of examination results.
Review and improve the monitoring and reporting function for the overall lookout
process.
7.0 MANAGEMENT RESPONSE
The Agency agrees with the recommendations of this audit, and its findings are consistent
with the conclusions of the CBSA's Lookouts Working Group. Formed in September 2012 to
assess, develop and implement new strategies for the management of lookouts, the working
group has already accomplished and initiated a great deal of work that addresses many of the
audit's recommendations.
Specifically, the CBSA undertook revisions to the Lookouts Policy and Standard Operating
Procedures to clarify processes and outline enhanced accountabilities and responsibilities. A
National Directive was issued in February 2013 which mandates the consistent and timely
reporting and reconciliation of lookouts.
Development of an overall performance framework is underway for the ongoing monitoring of
lookouts, with regular reporting requirements to senior and executive level management
committees. The Agency has also implemented a quality assurance review of its existing
100,000 lookouts and will complete the review by March 2014.
The CBSA will implement, by March 2014, measures to strengthen the issuance and
maintenance of lookouts, the recording of related examination results, quality assurance, and
performance measurement.
Business requirements have been developed that would improve Information Technology (IT)
system controls to support management oversight and ongoing monitoring and to ensure the
accuracy and validity of lookouts. These systems changes will be implemented on a priority
basis, commencing in July 2013. In the long term, the CBSA recognizes that the full
realization of lookout system controls would require significant investment in a modern and
integrated lookouts system that leverages and optimizes enterprise IT tools. To this end, the
CBSA will develop a strategy to integrate lookouts into the Agency’s systems modernization
initiatives.
7
8.0 AUDIT FINDINGS
8.1 Lookouts issuance and maintenance
Audit Criteria:
Controls are in place to ensure that lookouts are issued appropriately.
Controls are in place to ensure that lookouts are reviewed, maintained, and closed
appropriately.
8.1.1 Issuance of Lookouts
Intelligence officers are responsible for preparing and issuing Category 1 and 2 lookouts. Before
a lookout can be issued, the reliability of the source and the accuracy of the information are
evaluated to determine if there is sufficient basis for the issuance of a lookout.
Intelligence officers place lookouts under the appropriate category and within the appropriate
systems:
Integrated Customs Enforcement System (ICES) for customs-related lookouts (for
travellers), and
Field Operational Support System (FOSS) for immigration-related lookouts.
Subjects of customs lookouts are individuals who are considered high cross-border risks in areas
such as transnational organized crime, the illegal movement of currency and the illicit movement
of drugs, firearms and other contraband. Subjects of immigration lookouts are foreign nationals
who are or may be inadmissible to Canada, are a potential security risk due to prior visa refusals,
serious criminality or arrests, organized crime, or suspected national security threat, among
others.
{*}. Upon further examination, a denial of visa, seizure of goods or possible arrest may occur.
For the audit period between January 1 and September 30, 2012, the Agency issued
approximately 6,6005 customs-related lookouts in ICES6 and 4,000 immigration-related lookouts
in FOSS.7 The majority
8 of lookouts issued were Category 1.
A creation of a lookout involves judgment in applying the intelligence cycle (i.e. steps that
transform raw data and information into intelligence). Nevertheless, key controls should be in
5 Some Category 3 lookouts may have been included in this total due to inconsistent application of the categories by
the Intelligence officers. 6 Based on an extract from the ICES database provided by the auditee. This does not include lookouts issued in
previous years and extended in the audit period. 7 Based on an extract from FOSS obtained from Citizenship and Immigration Canada. This does not include
lookouts issued in previous years and extended in the audit period. 8 82% of lookouts were labelled Category 1 in ICES, and 97% in FOSS.
8
DRAFT PROTECTED A
place to ensure that lookouts are issued appropriately. For quality, lookouts need to be reliable,
accurate and relevant. {*}.
The Lookouts SOPs identify key controls during the issuance stage of the lookout process. More
specifically, prior to the issuance of a lookout, Intelligence officers are required to: apply an
intelligence cycle to the incoming information; establish that the CBSA is acting within its
jurisdiction; evaluate the accuracy and reliability of the source of information; and perform
various database checks for any record of past enforcement, intelligence, or other data that exists
on an individual. They also reach out to the originator of the information and any other relevant
law enforcement or customs/immigration partners.
To assess the controls during the issuance and maintenance process, a total of 649 files (32 from
ICES and 32 from FOSS) were randomly selected. Most files reviewed did not contain sufficient
information that would demonstrate how incoming information was evaluated by the Intelligence
officers. As a result, we could not assess the extent to which the intelligence cycle was applied
prior to the issuance of lookouts.
Some inconsistencies across the Regions visited and among Intelligence officers in the lookouts
issuance may impact the effectiveness of lookouts and data integrity. {*}.
{*}.
{*}.
{*}.
{*}.
Duplicate System Entry of Immigration Lookouts: FOSS is a Citizenship and Immigration
Canada (CIC) legacy mainframe system used mainly by the CBSA and CIC. CIC plans to
decommission the system in December 2014. In the interim, some Intelligence officers issue
immigration lookouts in both ICES and FOSS systems. {*}. However, the duplicate system entry
of lookouts would require BSOs at the border to input their examination results into both systems
(ICES and FOSS), to successfully close the loop on their intercepted lookouts. The practice of
double issuance of lookouts creates problems in closing the loop and impacts data integrity of the
Agency lookout systems due to double counting of these lookouts. The Lookouts SOPs do not
provide clear direction on this practice and there was no directive from Headquarters on whether
this practice was acceptable.
Lookout Categories: Identification of appropriate category of lookout is important to determine
the degree of intelligence cycle to be applied as well as for planning and performance
measurement reporting purposes. The Lookouts SOPs define the different lookouts categories
(Category 1, 2 and 3). However, the interpretation of the definitions was not consistent amongst
9 A non-statistical sampling method was used. Additional information can be found in the ‘About the Audit’ section
of the report.
9
the Intelligence officers interviewed in that some officers categorized lookouts created on behalf
of other departments as Category 1 instead of Category 3. This leads to inconsistent application
of the categories and contributes to data integrity issues.
The Lookouts Working Group, established during the audit, is assessing and developing new
strategies for the management of lookouts, including the development and provision of training
on the issuance and maintenance of lookouts to ensure integrity and national consistency.
Field Operational Support System (FOSS)
For this audit, it is important to provide some background information on FOSS. FOSS is a
Citizenship and Immigration Canada (CIC) legacy system used by the CBSA and CIC users for
functions associated with the immigration continuum (i.e. the CIC facilitation function, the
CBSA-CIC shared functions, and the CBSA enforcement function). The most current version of
FOSS III, which is now over 30 years old, is a document management system containing all
document-related evidence of the immigration client continuum for the administration of the
Immigration and Refugee Protection Act (IRPA). Nearly all the CBSA systems related to
traveller issues or immigration processing rely on FOSS immigration enforcement, facilitation,
lookouts and lost, stolen, fraudulent document data, either directly or via system-to-system
interfaces.
The Agency has identified FOSS as being an obsolete system {*} and the expiry of the FOSS
support contract in December 2014. It is important to note that FOSS was not built specifically
for managing lookouts. {*}. The Agency has determined that FOSS does not meet the evolving
business needs and is viewed as an impediment to managing an effective and efficient lookout
program.
As part of CIC’s modernization project, FOSS is scheduled to be decommissioned in December
2014. CIC has been developing and implementing its transition to the Global Case Management
System (GCMS) to support the CIC facilitation function and some of the CBSA-CIC shared
functions of the immigration continuum. However, the CBSA enforcement function was
removed from the scope of CIC’s GCMS project in 2008. Since then, the Agency has been
working towards finding a solution to replace the enforcement functionality currently provided
by FOSS and its interfaces. The audit did not examine whether the Agency has adequately
identified and managed the risks related to FOSS replacement.
8.1.2 Lookout Maintenance
Lookouts are instructional products developed from information or intelligence, and as such,
require ongoing maintenance to provide maximum value to users.
Intelligence officers are responsible for maintaining their respective lookouts. Regular
maintenance and review help ensure the integrity of the Agency lookouts. The review should be
initiated in a timely manner to ensure that lookouts are continuously updated and remain active to
intercept high-risk travellers and their goods at Canadian ports of entry. If lookouts are not
reviewed in a timely manner {*}. Lookouts that are no longer relevant should be expired or
cancelled in the system(s) and any errors should be noted and corrected to ensure lookouts
continue to be of high quality. Otherwise, travellers may be unnecessarily subject to closer
10
DRAFT PROTECTED A
questioning and searches at the ports of entry, potentially impacting service delivery and border
wait times. In addition, CBSA resources may be used inappropriately by examining travellers
based on lookouts that are no longer current or valid.
Automatic Expiry of Lookouts
The ICES system has an automated control that notifies Intelligence officers in advance of the
system expiry of their customs-related lookouts. The expiry of lookouts can be set up to 180
days.10
Lookouts that are ‘pending expiry’ should be reviewed by the Intelligence officers for
their continued reliability, accuracy and pertinence to CBSA jurisdiction. {*}.
{*}. There is a requirement under the SOPs to document the details of the lookout’s expiry or
cancellation in the Intelligence Management System. In the sample of 16 lookouts reviewed for
maintenance, {*}. In all cases, the reason or rationale for expiry was not documented. {*}.
{*}. The review date of immigration lookouts can be set for up to 10 years in FOSS. Immigration
lookouts in FOSS were not consistently reviewed by Intelligence officers to determine if the
information warrants the continuation of the lookout. Interviews confirmed that some FOSS
lookouts have exceeded their validity date and should be removed from the system.
The CBSA management established a Lookouts Working Group in September 2012 with a goal
of assessing and developing new strategies for the management of lookouts. As part of its several
objectives, the Group has initiated a review of all active lookouts in applicable systems11
to
determine their validity, or if the lookouts should be archived or deleted. This review is
particularly important to ensure all data in FOSS is accurate, current and valid prior to the
migration of this information into an enhanced CBSA system once FOSS is decommissioned in
2014. The CBSA plans to complete this review by March 2014 with substantial completion by
October 2013, as more than 100,000 lookouts need to be reviewed, including those issued by the
CBSA on behalf of other government departments (Category 3 lookouts).
Intelligence officers judge the validity of the lookouts in light of events that may have happened
over the passage of time and the receipt of additional information. Based on the examination
results entered by the BSOs at the port of entry, a decision may be taken to extend, modify,
expire or archive it. Therefore, a timely feedback to the Intelligence officers is key for facilitating
lookouts maintenance.
{*}.
{*}. If the quality of CBSA lookouts as intelligence products is not rigorously maintained, the
lookout effectiveness can be reduced.
10
The automatic expiry of Security and Missing Children lookouts can be set to 365 days. 11
ICES, FOSS and Accelerated Commercial Release Operations Support System (ACROSS).
11
8.1.3 Review of Lookouts for Quality
Management supervision and reviews are also an important control activity to ensure the
integrity of lookouts. The quality of Agency lookouts relies not only on sound issuance and
maintenance processes, but also on a regular quality review of lookouts and timely feedback to
Intelligence officers. According to the SOPs, Intelligence managers are responsible for
monitoring lookouts created and maintained by their staff.
A routine management review process for lookouts quality and adherence to the Lookouts Policy
and Procedures is not in place. The current guidance does not require that lookouts issued by
Intelligence officers be authorized by their managers. Accordingly, Intelligence officers issued
and also authorized their own lookouts. Occasionally, they discussed more difficult or sensitive
cases with their managers prior to issuing or extending lookouts but this is an informal practice.
{*}. By integrating supervisory review of lookouts quality into the issuance and maintenance
process, the Agency would be able to detect and correct deficiencies in a timely manner.
Appropriate and consistent documentation of lookout information in the right systems enables
management to conduct monitoring and quality review of lookouts to assure high-quality
intelligence products. {*}. This means that not all lookouts need to have a corresponding IMS
file. Intelligence information for lookouts was documented in IMS in most lookout files reviewed
(59 out of 64 ICES and FOSS files reviewed for issuance and maintenance). However, the IMS
files did not always contain sufficient information to demonstrate the application of the
intelligence cycle and to facilitate the review of lookouts for quality. The creation of an IMS file
(and STS file where required) for all lookouts with supporting documentation is important as it
would enable management to review how incoming information was evaluated by the
Intelligence officers and whether the lookout is of satisfactory quality.
Recommendation 1: The Vice-President of the Programs Branch, in collaboration with the
Vice-President of Operations Branch, should ensure that appropriate level of supervisory review
in the lookouts’ issuance and maintenance process is carried out, and that all lookouts are
documented in the Intelligence Management System (IMS) to enable the review of lookouts
quality.
Management Response Completion Date
Agreed. With respect to the recommendation of ensuring that an
appropriate level of supervisory review for the lookouts’ issuance and
maintenance process is carried out, the CBSA will leverage the work
completed by the Lookouts Working Group, including the issuance of a
National Directive on improved reporting of lookout outcomes, revisions
to the Lookouts Policy and Standard Operating Procedures (SOPs), and
development of a training module to address the quality of lookouts.
In 2010, a comprehensive Lookouts Policy was developed in response to
the 2007 report of the Office of the Auditor General (“Keeping the Border
Open and Secure”). This Lookouts Policy was updated again in 2012 to
reflect program improvements and organizational changes to the
Enforcement and Intelligence program as a whole. In June 2013, further
December 2013
12
DRAFT PROTECTED A
revisions will be made to the Lookouts Policy and SOPs in order to clarify
the accountabilities and responsibilities of regional officers and managers,
as well as for managers at National Headquarters.
These revisions will also reinforce the requirement for the mandatory
documentation of lookout supporting information and examination results
in the Intelligence Management System (IMS) or the Secure Tracking
System (STS) and related systems to enable the review of lookouts
quality. A review of compliance with these directives will be completed
in December 2013.
Recommendation 2: The Vice-President of the Programs Branch, in collaboration with the
Operations and Information, Science and Technology (ISTB) branches, should review and
improve system controls to strengthen data integrity and enable timely maintenance of lookouts.
This should include data entry, {*}.
Management Response Completion Date
Agreed. Programs Branch, in collaboration with Operations and ISTB,
will review and improve system controls to strengthen data integrity and
enable the timely maintenance of lookouts.
Enhanced business requirements for the lookout management systems
(IMS and ICES) have been developed and will become priorities by June
2013. These comprehensive business requirements include improvements
to data entry, {*} opportunities to reduce duplicate system entry of
immigration lookouts, lookout categories, and expiry of lookouts. As
well, requirements have been developed for a robust reporting
functionality to assist in the monitoring of the lookout program.
In the short term, implementation of these systems changes will begin in
July 2013. In fiscal year 2013-14 and 2014-15, in line with scheduled
releases, a number of priority change requests will be implemented for
both the ICES and IMS. In addition, the CBSA will develop a long-term
IT strategy to integrate improved systems controls for lookouts into the
Agency’s enterprise-wide systems modernization initiatives. This strategy
will be completed by March 2014.
March 2014
13
8.2 Lookouts Interception at the Border
Audit Criteria:
Controls at the Canadian border are in place to ensure that lookouts are intercepted as
intended.
Controls are in place to ensure the results of lookouts examinations are recorded and
appropriately communicated.
When travellers arrive at Canadian ports of entry, they follow a two-stage control process
referred to as primary and secondary inspection. At primary inspection, BSOs scan travel
documents ({*}), ask questions and make a decision whether to refer travellers and their goods to
secondary examination for more detailed checks in customs, immigration, or both. This decision
is based on several factors, including information provided by ICES and FOSS. {*} This
increases the risk of missed lookouts.
{*}. In these cases, management is required to record all relevant details.
Lookouts can be missed at primary inspection because of incorrect matching to a traveller.
Practices for referring travellers to secondary examination varied among ports visited and among
individual BSOs. Some BSOs referred only individuals whose travel documents perfectly
matched key data in the lookout {*}. The Lookout Policy and SOPs do not provide guidance on
dealing with close matches.
Recently, system improvements were implemented to include mandatory referral of close
matches and BSOs were reminded of their accountabilities related to close matches.
Generally, missed lookouts (i.e. those intercepted at primary inspection but for whatever reason
did not report to secondary examination) are not tracked and reported on. The Southern Ontario
Region is one exception as it does track and report on missed lookouts. The feedback resulting
from the analysis of events that led to a missed lookout would enable management to identify
potential control weaknesses and lessons learned.
8.2.1 Examination Results
According to the SOPs, BSOs conducting secondary examinations of lookouts are required to
record results into appropriate systems immediately following the event or as soon as possible
thereafter, regardless of whether or not the interview or examination yields results. Supervisors
and managers are responsible for ensuring that examination reports are completed in the
appropriate systems. This review is important for confirming the ‘closing of the loop’ for all
intercepted lookouts (i.e. lookouts are tracked, monitored, and followed up to ensure that they are
appropriately acquitted in the systems).
The examination results for intercepted customs and immigration lookouts were not always
documented in ICES and FOSS by the intercepting officers. In the sample of 56 intercepted ICES
14
DRAFT PROTECTED A
lookouts,12
seven did not contain examination results in ICES. Information related to interception
and examination results for immigration lookouts is not tracked by FOSS and due to this system
limitation could not be tested.
Examination results should be detailed enough to provide Intelligence officers with sufficient
information to determine whether their lookouts need to be modified, extended, or closed. This
includes, but is not limited to, the search results, interview notes, identity of any travelling
companions, action taken, etc.
While the SOPs provide broad direction on what information should be captured in the
examination results, the understanding and practice varied among BSOs of what information
should be included in the results to ensure thorough reporting. Intelligence officers informed us
that examination results were not consistently documented in the systems by the intercepting
officers, and the quality of reports varied.
Superintendents are responsible for overseeing the completion and the quality of the examination
results. Management informed us that they do not systematically review examination results. In
terms of customs-related lookouts, ICES’ functionality allows tracking of the interception and
documentation of examination results. Therefore, superintendents are able to monitor the lookout
process in ICES to ensure that all intercepted lookouts are closed by their BSOs. However, this
monitoring was not done consistently across the sites visited.
Some regions visited track all un-acquitted13
lookouts and communicate them to the ports of
entry for action. We noted that the frequency of reporting was not always sufficient to allow
timely follow-up by the superintendents and BSOs. {*}.
As mentioned earlier in this report, {*}.
The frontline supervisors at the ports told us that they generally do not review the quality of the
examination results. The Intelligence officers occasionally provide feedback on some reports to
frontline supervisors, but this is an ad hoc practice. By monitoring and improving the quality of
examination results entered by frontline officers, Intelligence officers would possess more
complete information to maintain their lookouts.
It should be noted that while the audit examined a sample of 56 intercepted lookouts, the
individuals who were subject to these lookouts in some cases were intercepted multiple times
during the audit period. Based on the information from ICES, there were 266 examination results
recorded across Canada for these individuals. Of the 266 examinations, 19 had a resultant exam.
14 The audit did not examine the effectiveness of the examination process as this would be a
12
One of the 56 intercepted customs lookouts at primary inspection was not examined at secondary (i.e. missed
lookout). 13
Un-acquitted lookouts refer to lookouts that hit at the primary inspection line (PIL), are referred for secondary
examination, but have no examinations results recorded in the systems. 14
The information presented in this paragraph is based on data extracted from ICES at the time of the audit; this
information was not audited.
15
program evaluation issue. This information is included in this report to provide some context as
to the information that is available to management.
Recommendation 3: The Vice-President of the Operations Branch should implement a quality
assurance process to strengthen the quality of examination results and establish a performance
measurement system to systematically track, monitor, and report on missed and un-acquitted
lookouts.
Management Response Completion Date
Agreed. With respect to the implementation of a quality assurance process
to strengthen the quality of examination results, the CBSA is currently
undertaking a comprehensive quality review of more than 100,000
existing lookouts to ensure compliance with the Agency’s policies and
procedures. For new lookouts, the Agency will implement a quality
assurance framework to strengthen the quality of examination results and
to ensure national consistency.
In addition, the Operations Branch, in cooperation with Programs Branch,
will establish a system to track, monitor and report on missed and un-
acquitted lookouts. The Agency will revisit and build upon existing
lookout monitoring processes, such as the best practices undertaken in
regions such as Southern Ontario Region, and develop training to ensure
national consistency.
The monitoring of examination results and missed or un-acquitted
lookouts will include monthly reports to the Operations Branch Executive
Committee (OBEC) and regular monitoring of overall program
performance by the relevant Program Management Committee. These
measures, including the review of existing lookouts, will be fully
completed by March 2014.
December 2013
8.3 Monitoring, Reporting and Accountability for Lookouts
Audit Criteria:
Roles, responsibilities, and accountabilities of key stakeholders at the entity level for
lookouts have been formally defined, documented and communicated.
The Agency monitors lookouts results to continuously improve the lookout process for
intercepting high-risk travellers entering Canada.
Results are reported to senior management and appropriate oversight bodies for
strategic direction, risk assessment and decision making.
16
DRAFT PROTECTED A
Roles, responsibilities and accountabilities15
of all key stakeholders involved with the Agency’s
lookout process were defined and documented in the Agency Lookout Policy, SOPs, and other
documents.
With respect to corporate monitoring of the lookout program, a broad definition of the roles and
responsibilities indicates that the Programs Branch is responsible for the development of the
performance management framework and the Operations Branch is tasked with performance data
collection and reporting.
There was a lack of formal monitoring at the corporate level. Statistics on the lookouts issued,
intercepted, un-acquitted and missed were not gathered, and analysis of controls’ design and
effectiveness was not performed. The Lookouts Working Group, established during the audit,
was tasked with assessing and developing new strategies for the management of lookouts,
including the improvement of monitoring.
As a key element of the regional monitoring function, the SOPs assigned an “auditing” role to
Intelligence Operations and the Regional Intelligence managers, where “auditing” is defined as
“implementing a performance measurement regime that will ensure that the lookout system is
properly evaluated and that the quality of CBSA lookouts is rigorously maintained.” While
“auditing” was expected to be in place, a corporate direction on the methodology was not
provided and the auditing function at the Regional level was not carried out.
With respect to reporting, the Programs Branch reported in the Agency’s Quarterly Performance
Report on the percentage value of goods, shipments and conveyances seized as a result of a
lookout, which is a measure related to the effectiveness of lookouts.
This performance indicator, while important for assessing the effectiveness of lookouts, does not
provide a full picture of the lookouts’ management. Reporting on other measures, including
missed and un-acquitted lookouts, would provide useful information to determine the functioning
of the lookouts program.
With regular monitoring, the performance of the lookout program will be easier to gauge, and
management will have information required to improve the lookout processes and controls.
National oversight and monitoring are key to the assessment of the lookouts program and
ensuring that the Agency’s priorities are achieved and risks are appropriately managed.
Recommendation 4: The Vice-President of the Programs Branch, in collaboration with the
Operations Branch, should review and improve the monitoring and reporting function for the
overall lookout process.
15
Responsibility is defined as the duty to perform, and accountability as the obligation to report on the fulfillment of
the responsibility; OCG, 2011, Audit Criteria related to the Management Accountability Framework, Section
Accountability, p. 48.
17
Management Response Completion Date
Agreed. Programs Branch, in collaboration with the Operations Branch,
will review and improve the monitoring and reporting function for the
overall lookout process.
The Agency has already issued a National Directive to require regional
reporting on lookouts and their outcomes, and will establish a framework
for national reporting to the Operations Branch Executive Committee
(OBEC) and the Enforcement and Intelligence Program Management
Table (see action under Recommendation 1).
The Agency will develop improved performance indicators for the overall
lookout process as part of the annual review of the CBSA’s Performance
Measurement Framework, and will integrate these indicators into its
Performance and Service Standards Report (PSSR) and quarterly program
performance reports. These measures will be completed by October
2013.
In addition, the CBSA will implement improved reporting functionality in
lookout-related systems changes and modernization initiatives, in order to
strengthen performance measurement and operational monitoring.
October 2013
18
DRAFT PROTECTED A
APPENDIX A – ABOUT THE AUDIT
AUDIT OBJECTIVES AND SCOPE
The audit objective was to provide assurance that Agency responsibilities for lookouts are
appropriately managed and processed. More specifically, the objective was to determine whether
controls for managing lookouts are well-designed and effective to intercept high-risk people
entering Canada.
The audit scope included the review of lookout controls in the air traveller and land streams
during the period of January 2012 to April 2013. The audit included visits to the three highest-
risk airports and the three highest-risk land borders identified in the CBSA’s 2010 National Port
Risk Assessment.
The audit excluded lookouts issued on behalf of other government departments and law
enforcement agencies (i.e. Category 3 lookouts), commercial lookouts, lookouts of low value
shipments (i.e. postal and courier lookouts), and targets.
The audit did not examine whether the Agency has adequately identified and managed the risks
related to FOSS replacement.
RISK ASSESSMENT
The risk assessment conducted during the planning phase identified a key risk: If the Agency’s
controls around lookout processes are not functioning as intended, there is a risk that high-risk
travellers and goods connected to terrorism, organized crime, irregular migrants, and biological
threats could enter Canada and threaten its security, safety and economy.
APPROACH
To assess the effectiveness of controls during the issuance, maintenance and interception stages
of the lookouts process, a sample of 120 lookouts was randomly selected from ICES and FOSS
from a population of approximately 10,000 active lookouts issued between January 1 and
September 30, 2012. {*} lookouts (Category 1 and 2) across Canada. In addition, the Regional
Intelligence Offices associated with these ports issued more than one-third of all customs and
immigration lookouts.
The results from the file review cannot be extrapolated to the population because the sample was
non-statistical. The results from the interviews and site visits cannot be generalized to ports
nationwide; however, visits to these locations allowed us to directly observe the lookouts process
and provided insight into the design and effectiveness of the lookouts-related border controls.
The following approach was used:
Reviewed and analyzed information from various sources, such as, but not limited to the
Lookout Policy, the SOPs, relevant bulletins, directives and reports;
19
Interviewed various stakeholders within the Programs and Operations Branches on their
roles and responsibilities, oversight function and monitoring in relation to lookout
activities;
Collected, reviewed and analyzed documents relating to lookout monitoring processes
and monitoring reports;
Collected and analyzed lookouts-related information reported to senior management
nationally and regionally;
Analyzed a random sample of active lookouts from ICES and FOSS issued and
intercepted in the Regions included in the audit scope to assess controls related to the
issuance, maintenance and interception of lookouts;
Analyzed and assessed the design and adequacy of management controls in place at the
CBSA for the management of lookouts.
AUDIT CRITERIA
The following audit lines of enquiry and criteria were selected.
Lines of Enquiry Audit Criteria
1. Governance &
Accountability for
Lookouts
1.1 Roles, responsibilities, and accountabilities of key stakeholders at the
entity level for lookouts have been formally defined, documented and
communicated.
2. Lookouts Creation
and Maintenance
2.1 Controls are in place to ensure that lookouts are issued appropriately.
2.2 Controls are in place to ensure that lookouts are reviewed, maintained,
and closed appropriately.
3. Lookouts
Interception at the
Border
3.1 Controls at the Canadian border are in place to ensure that lookouts are
intercepted as intended.
3.2 Controls are in place to ensure the results of lookouts examinations are
recorded and appropriately communicated.
4. Monitoring and
Reporting
4.1 The Agency monitors lookouts results to continuously improve the
lookout process for intercepting high-risk travellers entering Canada.
4.2 Results are reported to senior management and appropriate oversight
bodies for strategic direction, risk assessment and decision making.
20
DRAFT PROTECTED A
APPENDIX B – SAMPLING APPROACH A sample of lookouts was selected to assess the effectiveness of controls during the issuance,
maintenance and interception stages of the lookouts, and identify gaps and areas in need of
improvement. The sample was drawn from ICES and FOSS data systems for the period of
January 1, 2012 to September 30, 2012. The sample was linked to the ports scoped in the audit
(i.e. stratified) to allow review of lookouts issued, maintained and intercepted at the six highest-
risk ports included in the audit scope. The results from our file review cannot be extrapolated to
the population because the sample was non-statistical.
Table 1: Sample Size
Region GTA
SOR
QUEBEC PACIFIC Sample
Size Population
Port of
Entry
Pearson
International
Airport
Ambassador
Bridge –
land border
Pierre
Elliott
Trudeau
Airport
Lacolle
15 – land
border
Vancouver
International
Airport
Pacific
Highway
– land
border
Issuance &
Maintenance
(ICES)
8 8 8 8 32 6,587
Issuance &
Maintenance
(FOSS)
8 8 8 8 32 4,030
{*}
{*} {*} {*} {*} {*} {*} {*} {*}
{*}
{*} {*} {*} {*} {*} {*} {*} {*}
TOTAL
SAMPLE
SIZE
32 24 32 32 120
{*}.
{*}.
21
APPENDIX C – ACRONYMS
BSO – Border Services Officer
CBSA or the Agency – Canada Border Services Agency
CIC – Citizenship and Immigration Canada
FOSS – Field Operational Support System
GCMS – Global Case Management System
GTA – Greater Toronto Region
ICES – Integrated Customs Enforcement System
IMS – Intelligence Management System
PIL – Primary Inspection Line
SOPs – Standard Operating Procedures
SOR – Southern Ontario Region
STS – Secure Tracking System
Related Documents
