08 NATO Talk

8/12/2019 08 NATO Talk

1/23

Automated video surveillance:challenges and solutions.

ACE Surveillance (Annotated Critical Evidence)

case study.

Dmitry Gorodnichy and Tony Mungham

Laboratory & Scientific Services DirectorateCanada Border Services Agency

www.videorecognition.com/ACE

8/12/2019 08 NATO Talk

2/23

2

Outline

Problems with status-quo Video Surveillance

Real-time and archival problems

Operational considerations

Next generation solution - Video Analytics based Motion detection myth and problem

Object detection as example of real intelligence

ACE Surveillancefirst fully-functional object-detection-based prototype

Year long tests with different levels of complexity

What that means for future of Video Surveillance

Conclusions

8/12/2019 08 NATO Talk

3/23

3

Role of Video Technology (VT)

In the context of enhancing security, VideoTechnology (VT) is one of the most demandedtechnologies of the 21st century

It is publicly acceptable It provides rich in content data

Multi-million funding in Canada and worldwide:

CBSA Port Runner project invested 10s of Millions inCCTV upgrades

Transport Canada opens $35M of funding towardsprocurement of CCTV

8/12/2019 08 NATO Talk

4/23

4

VT at CBSA

CBSA is a major user of CCTV systems at POEs

Most major CCTV installations start to leverage VT

Current task: to lead applied R&D to push VT to help

CBSA apply S&T innovative approaches to bordermanagement:

Event detection and notification to provide effectiveresponse to events

Traffic trends analysis to assist with border management

Video storage management to manage the cost of storageand meet obligations under the privacy act

Data integration/fusion of contextualised video information

8/12/2019 08 NATO Talk

5/23

5

Problem with status quo use of CCTVsurveillance

Modes of operation:

1. Active - personnel watch video at all times

2. Passive - in conjunction with other duties

3. Archival - for post-event analysis

Current systems and protocols are not efficient

in either mode!

Problem in real-time modes: an event may easilypass unnoticed .

due to false or simultaneous alarms,

lack of time needed to rewind and analyse all video

streams.

8/12/2019 08 NATO Talk

6/23

6

Problems in Archival mode:

Due to temporal nature of data:1. Storage space consumption problem

Typical assignment:2-16 cameras, 7 or 30 days of recording, 2-10 Mb / min.1.5 GB per day per camera / 20 - 700 GB total !

2. Data management and retrieval problem

London bombing video backtracking experience:

Manual browsing of millions of hours of digitized video fromthousands of cameras proved impossible within time-sensed period

[by the Scotland Yard trying to back-track the suspects]

8/12/2019 08 NATO Talk

7/237

Operational considerations

Lots of CCTV infrastructure: Many local initiatives, notcoordinated

Most video technology decisions are influenced byvendors - short-term solutions

Over 30 different video systems within the same dept. (atRCMP)

A national program with proper benchmark-based planningand evaluation of VT is required

Leveraging advances recently made in S&T

Technical standards for capturing /saving video data.

Policies in when, where and how VT should be used.

8/12/2019 08 NATO Talk

8/238

Video Technology today

Video Analytics (Video Recognition)

Analog

20th

century

21st century

First video recording

Digital

Wireless, Network Connected (IP)

8/12/2019 08 NATO Talk

9/239

Next generation Video Technology

Is Video Analytics based

also identified as:

Video Recognition,

Intelligent Video, Smart Video / Smart Camera

Video Analysis & Content Extraction

Perceptual Vision is not much about capturing better data (better

lenses, grabbers, coders, transmitters)

but about understanding captured data (better

theory)

8/12/2019 08 NATO Talk

10/2310

Status-quo video intelligence

Transport Canada CCTV Reference Manual forSecurity Application .

Australian Government National code of practice for

CCTV applications in urban transportUSA Government :recommended security Guidelines

for Airport Planning, Design and Construction.

. refer toMotion-based capture as IntelligentSurveillance Technology, and make theirrecommendations based on thereon.

8/12/2019 08 NATO Talk

11/2311

Motion-detection is not intelligent!

Term Motion-based is coined to make people believe thatvideo recognition is happening, which is not!

Its actually illumination-change-based, as it uses simplepoint brightness comparison:

Which often happens not because of motion! Changing light / weather (esp. in 24/7 monitoring)

Against sun/light, out of focus, blurred, thru glass

Reflections, diffraction, optical interferences

Image transmission, compression losses

8/12/2019 08 NATO Talk

12/2312

Object-detection is intelligent

but few can do it, since necessary advances in videorecognition theory became possible only recently (>2002).

In 2002 National Research Council of Canada (NRC)starts

developing Video Recognition Systems to leverage itsscientific Video Recognition expertise for the industry.

In 2005, it develops ACE Surveillance:

an object-detection-based Automated surveillanCEprototype capable of automatically extractingAnnotated Critical Evidence from live video.

NRC becomes also the organizer of the first Canadian academic workshopsdedicated to Video Processing for Security (since 2004)

8/12/2019 08 NATO Talk

13/2313

What is ACE Surveillance?

A Windows software that performs real-time video analytics byintegrating best object detection and tracking algorithms.

Replaces video clips with annotated still images:

Compresses 1 Gb of video into 2 Mb of easy to browse and

analyze still images ACE Surveillance output:

A 7-hour activity from day tonight (17:00 - 24:00) is

summarized into 2 minutes(600Kb) of Annotated CriticalEvidence snapshots.

Note illumination changes! - Watch treeshadows and sun light.

8/12/2019 08 NATO Talk

14/23

8/12/2019 08 NATO Talk

15/2315

Adds on top of existing infrastructure

8/12/2019 08 NATO Talk

16/2316

Status quo Motion-based capture(Courtesy: NRC-IIT Video Recognition Systems project)

1. Many captured snapshots are

useless: either noise or

redundant

2. Without visual annotation,

motion information is lost.

3. Hourly distribution ofsnapshots is not useful

8/12/2019 08 NATO Talk

17/2317

ACE Surveillance Object-based capture(Courtesy: NRC Video Recognition Systems project)

1. Each captured shot is useful.

2. Object location and velocity

shown augmentent.

3. Hourly distribution of shots is

indicative of what happened in

each hour, provides good

summarization of activities overlon eriod of time.

8/12/2019 08 NATO Talk

18/2318

ACE Surveillance testing benchmarks

Tested in different levels of complexity:

lighting conditions,

object motion patterns,

camera location environmental constraints.

most difficult - outdoors in unconstrained environments withlittle or no object motion consistency (as around a private

house in a regular neighbourhood).most easy - in controlled indoor environment where minimaldirect sunlight is present and where all objects are ofapproximately the same size and exhibit similar motionpattern (as at access gate inside the business building).

Outdoor wireless eye-level Outdoor, webcam, overview Indoor with sunlight, CCTV Indoor w/o sunlight, CCTV

8/12/2019 08 NATO Talk

19/23

VT within CBSA 19

Outdoor, wireless, eye-level Outdoor, webcam, overview Indoor with sunlight, CCTV Indoor w/o sunlight, CCTV

8/12/2019 08 NATO Talk

20/2320

Enables efficient detection of abnormalactivities Delivery EntryBack Door Entry

On

week-day

Onweek-e

nd

More than usual

8/12/2019 08 NATO Talk

21/2321

ACE Surveillance results

In real-time mode: alarm sounds & last captured evidence(time-stamped) is shown.

In archival mode: Zoom on the evidence browsing of captured

evidences zoom on a day, on hour, then on event - point

and click (for high res as needed)Made Commissioners much more aware of activities.

8/12/2019 08 NATO Talk

22/2322

Conclusions

Affordable automated (intelligent) video surveillance(AVS) is possible! To replace traditional DVR

OR to supplement them: DVR for 1 month + AVS for 1 year

However: Requires extra training from security officers.

Requires new protocols to handle automatically extractedevidence. - From forensic prospective, data that are not original and have

been processed by a computer can not be considered as evidence.

Requires new privacy policies. - Surveillance data are normally not kept for a long period of time

(

8/12/2019 08 NATO Talk

23/23

ACE surveillance case study outcome

ACE Surveillance(which is developed by a research lab)provides a reference standard against which can bemeasured solutions coming from industry.

It deals with common misconceptions related to deploying

intelligent video surveillance systems (IVS): motion detection myth vs real object detection and tracking.

The one-fit-all myth. - Extra video analytics expertise is required toset and operate IVS.

better video data (better resolution or compression) do not implybetter video intelligence. - ACE Surveillance is shown to work withregular TV quality data (320 by 240 pixels).

Howeverbetter quality of video image is needed for forensicpurposes as evidence

Due to closing of the project by NRC CBSA takes lead on it