Performance Audit: Atlanta Police Department Body-Worn Cameras December 2018 City Auditor’s Office City of Atlanta File #18.06 H.11.b Packet Pg. 155 Attachment: B10 APD BWC Final Report (18-C-5172 : Performance Audit: Atlanta Police Department Body-Worn Cameras)
41
Embed
Performance Audit: Atlanta Police Department Body-Worn Cameras · Attachment: B10 APD BWC Final Report (18-C-5172 : Performance Audit: Atlanta Police Department Body-Worn Cameras)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Performance Audit:
Atlanta Police Department
Body-Worn Cameras
December 2018
City Auditor’s Office
City of Atlanta
File #18.06
H.11.b
Packet Pg. 155
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
H.11.b
Packet Pg. 156
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
CITY OF ATLANTA City Auditor’s Office
Amanda Noble, City Auditor
404.330.6750
December 2018
Performance Audit:
Why We Did This Audit
We undertook this audit because body-worn cameras
enhance the transparency and accountability of
interactions between citizens and the police. The chief
of police requested we conduct a performance audit to
assess compliance with the department’s body-worn
camera policy and recommend metrics for monitoring
program compliance.
What We Recommended
To ensure compliance with the department’s body-
worn camera policies and best practices, we
recommend that the chief of police:
• clarify the policy regarding recording all
incidents
• update policy to remove requirement that
supervisors upload use-of-force incidents
and to require supervisors review audit trails
to ensure involved officer has not accessed
the video
• clarify criteria for categorizing videos in the
policy
• establish a formal process for zone
supervisors’ reviews
• conduct monthly reviews of user roles and
permissions
• enforce policies requiring the compliance
team to review deleted footage prior to
deletion
• develop standard justifications for access to
videos
To ensure accountability and transparency, we
recommend the chief of police monitor and track the
following performance metrics :
• videos captured compared to dispatched calls
• videos uploaded within one day
• uncategorized videos
• videos streamed by supervisors
• videos deleted before the retention schedule
• videos audited by the compliance team
• audited videos that complied with activation
procedures
• audited videos that complied with deactivation
procedures
• audited videos that were accurately categorized
For more information regarding this report, please use the
“contact” link on our website at www.atlaudit.org
APD Body-Worn Cameras
What We Found
The Atlanta Police Department’s officers risk the
potential loss of evidentiary data and public trust by
failing to consistently use body-worn cameras to record
interactions with the public. Officers assigned body-
worn cameras captured video for 33% of officer-
dispatched calls from November 2017 to May 2018.
Officers also delayed activation and prematurely
deactivated the body-worn cameras for many incidents.
In our random sample of 150 videos, 61% were
activated and 47% were deactivated according to
policy. Overall, we estimated that 30%–46% of videos
complied with both activation and deactivation
procedures.
Officers uploaded 74% of videos according to the
department’s procedures within one day of the date
they were recorded. Officers also categorized almost
all videos but could improve accuracy of categories
assigned. In our sample, officers miscategorized 22
videos, including one that the department agreed
should have been categorized as a use of force
incident. Miscategorized videos may be deleted
prematurely, which may not comply with state law.
Supervisors are responsible for ensuring that officers
comply with camera policies; however, supervisors
reviewed only 2% of all videos uploaded between
November 2016 and May 2018. Departmental
procedures do not specify a formal process regarding
the number of videos to review or include criteria to
ensure compliance with recording policies.
Compliance staff are not reviewing videos as required
to monitor compliance with camera policies and ensure
video footage is not prematurely deleted. The team
reviewed less than 1% of videos prior to deletion
between November 2016 and May 2018. We also
identified 64 videos that were deleted by users who
should not have been authorized to delete videos from
the system.
H.11.b
Packet Pg. 157
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Management Responses to Audit Recommendations
Summary of Management Responses
Recommendation #1: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police clarify the policy to state whether all officers responding to an incident must record body camera video.
Response & Proposed Action:
The existing SOP does not provide the level of clarity required for a large organization. The SOP will be modified to eliminate those factors causing confusion.
Agree
Timeframe: December 2018
Recommendation #2: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police update the policy to remove the requirement that supervisors upload videos of use-of-force incidents and to require supervisors to review the audit trail to ensure the involved officer has not accessed the video prior to writing the report.
Response & Proposed Action:
Due to a more complete audit trail, the policy will be revised to remove the requirement that supervisors upload videos of use-of-force incidents. The policy will now require the immediate supervisor to review the audit trail for compliance by the involved officer with the policy not to access the video prior to completing the initial incident report.
Agree
Timeframe: December 2018
Recommendation #3: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police clarify criteria for labeling and categorizing videos in standard operating procedures.
Response & Proposed Action:
The SOP has too many coding options, which has caused confusion and diminished performance. Initially there were 32 coding options, this has been reduced to 19, and the goal is 10 options. This reduction of choices should provide the necessary adjustments to allow for greater efficiency and consistency.
Agree
Timeframe: December 2018
Recommendation #4: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police establish a formal process for zone supervisors’ periodic reviews, including the number and selection of videos, frequency, and required documentation.
Response & Proposed Action:
Each Zone is going to be required to have their administrative sergeant audit 25 recordings every 2-weeks. They will be required to document the specific recordings examined, whether they were properly labeled, whether the officer met stated recording requirements, and whether the Zone’s UAF footage corresponds to the number of UAF incidents captured in the 911 Center. The Audit Team will be responsible for managing compliance with this requirement.
Agree
Timeframe: January 2019
H.11.b
Packet Pg. 158
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Recommendation #5: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police conduct monthly reviews of user roles and permissions to determine if non-administrator users can delete videos.
Response & Proposed Action:
The department recognizes the need to regularly update access controls in the Evidence.com platform. There are personnel that depart, and others who change roles within our agency. Those that depart must be removed from Evidence.com access, and those who change roles need to have access appropriate to their assigned position. Tracking access also allows the BWC Team to track the status of BWCs no longer assigned and enables the team to put them back into circulation. The updating of access controls within Evidence.com will occur regularly, at least monthly.
Agree
Timeframe: December 2019
Recommendation #6: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police enforce policies requiring the compliance team to review all deleted footage prior to deletion for miscategorization.
Response & Proposed Action:
The retention period was modified, and all videos are saved for 5-years. Additionally, there are only 2 individuals with the authority to delete videos, and their accounts will be cross-checked by supervisory personnel. This policy amendment was made during the audit.
Partially Agree
Timeframe: Complete
Recommendation #7: To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief of police develop standard justifications for accessed footage to ensure compliance with the policy.
Response & Proposed Action:
The requirement to provide a specific reason for viewing a video is not an essential metric. The “notes box” is only available in Evidence.com after the video is uploaded, you are unable to add notes as to why you viewed a video in the Axon View App or in Axon Sync on the MDT, so it is not trackable across the range of viewing options.
We are requiring that administrative sergeants and supervisors review BWC video to ensure the BWCs are activated and deactivated in accordance with policy, and not interrupted during recording. The supervisors will also audit to ensure the BWC is used according to policy. Only the officer assigned the BWC, his immediate supervisor(s), and the administrative sergeant can access the individual officer’s BWC in Evidence.com. The requirement to label why a video is accessed only in Evidence.com, and not in the other viewing options makes this metric impossible to accurately track and enforce. The SOP will be amended accordingly.
Agree
Timeframe: January 2019
H.11.b
Packet Pg. 159
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Recommendations
#8–16:
To ensure accountability and transparency, we recommend the chief of police monitor and track the following performance metrics for the number and percentage of:
8. videos captured compared to the number of dispatched calls,
using the 80% threshold as a comparative benchmark
9. videos uploaded to the system within one day
10. uncategorized videos
11. videos streamed by supervisors
12. videos deleted before the retention schedule
13. videos audited by the compliance team
14. audited videos that complied with activation procedures
15. audited videos that complied with deactivation procedures
16. audited videos categorized accurately
Response & Proposed Action:
Metrics must be developed that allow for improved performance. The Zone audits will encompass much of the above-referenced, while a final audit checklist is still being identified.
Agree
Timeframe: January 2019
H.11.b
Packet Pg. 160
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
December 3, 2018
Honorable Mayor and Members of the City Council:
We undertook this audit because body-worn cameras enhance the transparency and
accountability of interactions between citizens and the police. The chief of police
requested we conduct a performance audit to assess compliance with the department’s
body-worn camera policy and recommend metrics for monitoring program compliance.
The Audit Committee has reviewed this report and is releasing it in accordance with
Article 2, Chapter 6 of the City Charter. We appreciate the courtesy and cooperation of
city staff throughout the audit. The team for this project was Rebecca Robinson, Ivy
retention periods, and chain of custody. We noted that some
supervisors appear to be unaware of standard operating procedures
related to the body-worn camera program. To ensure policy
compliance, we recommend that the compliance administrator hold
department-wide body-worn camera refresher trainings based on
the most recent policy updates.
016
30
5563
8
48 52
21 18
101
192
157
57
102
80
100
150
75
0
25
50
75
100
125
150
175
200
No
vem
ber
-16
Dec
em
ber
-16
Jan
uar
y-1
7
Feb
ruar
y-1
7
Mar
ch-1
7
Ap
ril-
17
May
-17
Jun
e-1
7
July
-17
Au
gust
-17
Sep
tem
ber
-17
Oct
ob
er-
17
No
vem
ber
-17
Dec
em
ber
-17
Jan
uar
y-1
8
Feb
ruar
y-1
8
Mar
ch-1
8
Ap
ril-
18
May
-18
Total Videos Viewed Average Monthly Goal
H.11.b
Packet Pg. 188
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 25
Recommendations
To ensure compliance with the department’s body-worn policies and
best practices, we recommend that the chief of police:
1. clarify the policy to state whether all officers responding to
an incident must record body camera video
2. update the policy to remove the requirement that
supervisors upload videos of use-of-force incidents and to
require supervisors to review the audit trail to ensure the
involved officer has not accessed the video prior to writing
the report
3. clarify criteria for labeling and categorizing videos in
standard operating procedures
4. establish a formal process for zone supervisors’ periodic
reviews, including the number and selection of videos,
frequency, and required documentation
5. conduct monthly reviews of user roles and permissions to
determine if non-administrator users can delete videos
6. enforce policies requiring the compliance team to review all
deleted footage prior to deletion for miscategorization
7. develop standard justifications for accessed footage to
ensure compliance with the policy
To ensure accountability and transparency, we recommend the chief
of police monitor and track the following performance metrics:
The number and percentage of:
8. videos captured compared to the number of dispatched calls,
using the 80% threshold as a comparative benchmark
9. videos uploaded to the system within one day
10. uncategorized videos
11. videos streamed by supervisors
12. videos deleted before the retention schedule
13. videos audited by the compliance team
14. audited videos that complied with activation procedures
15. audited videos that complied with deactivation procedures
16. audited videos categorized accurately
H.11.b
Packet Pg. 189
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 26
H.11.b
Packet Pg. 190
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 27
Appendix
H.11.b
Packet Pg. 191
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 28
H.11.b
Packet Pg. 192
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 29
Appendix A: Management Review and Response to Audit Recommendations
Report # 18.06 Performance Audit: Body-Worn Camera Program Date: 11/ 2018
Recommendation 1:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police clarify the policy to state whether all officers responding to an incident must record body camera
video.
Proposed Action: The existing SOP does not provide the level of clarity required
for a large organization. The SOP will be modified to eliminate those factors
causing confusion.
Response:
Agree
Person Responsible: Deputy Chief L. Hagin Implementation Date:
12/2018
Recommendation 2:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police update the policy to remove the requirement that supervisors upload videos of use-of-force incidents and
to require supervisors to review the audit trail to ensure the involved officer has not accessed the video prior to
writing the report.
Due to a more complete audit trail, the policy will be revised to remove the requirement that supervisors upload videos of use-of-force incidents. The policy will now require the immediate supervisor to review the audit trail for compliance by the involved officer with the policy not to access the video prior to completing the initial incident report.
Response:
Agree
Person Responsible: Deputy Chief Lane Hagin Implementation Date:
12/2018
Recommendation 3:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police clarify criteria for labeling and categorizing videos in standard operating procedures.
Proposed Action: The SOP has too many coding options, which has caused confusion and diminished performance. Initially there were 32 coding options, this has been reduced to 19, and the goal is 10 options. This reduction of choices should provide the necessary adjustments to allow for greater efficiency and consistency.
Response:
Agree
Person Responsible: DC L. Hagin Implementation Date:
12/2018
H.11.b
Packet Pg. 193
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 30
Recommendation 4:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police establish a formal process for zone supervisors’ periodic reviews, including the number and
selection of videos, frequency, and required documentation.
Proposed Action: Each Zone is going to be required to have their administrative sergeant audit 25 recordings every 2-weeks. They will be required to document the specific recordings examined, whether they were properly labeled, whether the officer met stated recording requirements, and whether the Zone’s UAF footage corresponds to the number of UAF incidents captured in the 911 Center. The Audit Team will be responsible for managing compliance with this requirement.
Response:
Agree
Person Responsible: DC L. Hagin
Implementation Date:
01/2019
Recommendation 5:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police conduct monthly reviews of user roles and permissions to determine if non-administrator users
can delete videos.
Proposed Action: The department recognizes the need to regularly update access controls in the Evidence.com platform. There are personnel that depart, and others who change roles within our agency. Those that depart must be removed from Evidence.com access, and those who change roles need to have access appropriate to their assigned position. Tracking access also allows the BWC Team to track the status of BWCs no longer assigned and enables the team to put them back into circulation. The updating of access controls within Evidence.com will occur regularly, at least monthly.
Response:
Agree
Person Responsible: N/A Implementation Date:
12/2019
Recommendation 6:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police enforce policies requiring the compliance team to review all deleted footage prior to deletion for
miscategorization.
Proposed Action: The retention period was modified, and all videos are saved for 5-years. Additionally, there are only 2 individuals with the authority to delete videos, and their accounts will be cross-checked by supervisory personnel. This policy amendment was made during the audit.
Response:
Partially Agree
Person Responsible: N/A Implementation Date:
Complete
H.11.b
Packet Pg. 194
Att
ach
men
t: B
10 A
PD
BW
C F
inal
Rep
ort
(18
-C-5
172
: P
erfo
rman
ce A
ud
it:
Atl
anta
Po
lice
Dep
artm
ent
Bo
dy-
Wo
rn C
amer
as)
Atlanta Police Department Body-Worn Cameras 31
Recommendation 7:
To ensure compliance with the department’s body-worn policies and best practices, we recommend that the chief
of police develop standard justifications for accessed footage to ensure compliance with the policy.
Proposed Action: The requirement to provide a specific reason for viewing a video is not an essential metric. The “notes box” is only available in Evidence.com after the video is uploaded, you are unable to add notes as to why you viewed a video in the Axon View App or in Axon Sync on the MDT, so it is not trackable across the range of viewing options. We are requiring that administrative sergeants and supervisors review BWC video to ensure the BWCs are activated and deactivated in accordance with policy, and not interrupted during recording. The supervisors will also audit to ensure the BWC is used according to policy. Only the officer assigned the BWC, his immediate supervisor(s), and the administrative sergeant can access the individual officer’s BWC in Evidence.com. The requirement to label why a video is accessed only in Evidence.com, and not in the other viewing options makes this metric impossible to accurately track and enforce. The SOP will be amended accordingly.
Response:
Agree
Person Responsible: N/A
Implementation Date:
01/2019
Recommendations 8-16:
To ensure accountability and transparency, we recommend the chief of police monitor and track the following
performance metrics for the number and percentage of:
8. videos captured compared to the number of dispatched calls, using the 80% threshold as a
comparative benchmark
9. videos uploaded to the system within one day
10. uncategorized videos
11. videos streamed by supervisors
12. videos deleted before the retention schedule
13. videos audited by the compliance team
14. audited videos that complied with activation procedures
15. audited videos that complied with deactivation procedures
16. audited videos categorized accurately
Proposed Action: Metrics must be developed that allow for improved performance. The Zone audits will encompass much of the above-referenced, while a final audit checklist is still being identified.
Response:
Agree
Person Responsible: DC L. Hagin Implementation Date: