Street Crossing Tracking from a moving platform Need to look left and right to find a safe time to cross Need to look ahead to drive to other side of road.

Street Crossing

marked crosswalk

mobile robot curb cut

• Tracking from a moving platform

• Need to look left and right to find a safe time to cross

• Need to look ahead to drive to other side of road

• Must stay in crosswalk

Algorithm for Tracking Cars

1. Use image differencing method to extract motion regions

2. Noise filter using 3x3 median filter; effective for typical CCD sensor noise

3. Compute edges of motion regions using Canny edge detection

4. Use Mori’s “sign pattern” to find bottoms of cars [Mori 1994]

5. Find bounding boxes of moving objects6. Use knowledge from prior frames to mark

direction of travel of each bounding box

Mori Sign Pattern

Tracking algorithm uses “Mori Scan” to reliably detect undersides of cars

The Mori “sign pattern” for vehicle detection says: the shadow underneath a vehicle is darker than any other spot on the paved road

The Mori result is invariant to lighting and holds for wet and dry roads

Use of the Mori result obviates the need for explicit shadow detection and/or removal; previously, prominent shadow edges caused oversize bounding boxes

Mori Sign Pattern

Mori Sign Pattern

Street Crossing

Six frames of a tracking sequence

System Validation

• When it is safe to cross, a person monitoring the trafficscene presses a button

• A second button press means it is no longer safe to cross

• The time between button presses specifies a safe crossing window

• Use more than one person to compensate for individual risk tolerance

• Button press data is synchronized with the video data• Compare system safety estimates to human safety

judgments

System Validation

• When it is safe to cross, a person monitoring the trafficscene presses a button

• A second button press means it is no longer safe to cross

• The time between button presses specifies a safe crossing window

• Use more than one person to compensate for individual risk tolerance

• Button press data is synchronized with the video data• Compare system safety estimates to human safety

judgments

Crosswalk Traversal

• While crossing, devotes more processing to the right-looking video stream

• Uses a forward-looking camera to detect and stay on the marked (zebra striped) crosswalk

• Uses sonar to avoid pedestrians and stopped cars on the crosswalk

• Uses a laser range finder to detect the curb cut; driving over curbs is possible, but undesirable

Crosswalk Traversal

• While crossing, devotes more processing to the right-looking video stream

• Uses a forward-looking camera to detect and stay on the marked (zebra striped) crosswalk

• Uses sonar to avoid pedestrians and stopped cars on the crosswalk

• Uses a laser range finder to detect the curb cut; driving over curbs is possible, but undesirable

Related Work

• Automated driving systems: CMU’s Navlab project, Dickmanns’ autonomous Autobahn vehicle, DARPA Challenge

• Traffic scene monitoring systems that analyze traffic conditions

• Camera orientation and assumptions of existing vision-based, car-tracking systems do not apply to street crossing

• Robotic street crossing has not been done previously

Reasoning about Bounding Boxes

• Larger, lower (in the image plane) bounding boxes correspond to close cars

• Smaller, higher bounding boxes denote distant cars

• Tracking in real time using Phission so cars move very little from frame to frame

• Track individual cars over time to determine speed and travel direction

• Need to smooth results over time since CCD cameras produce noisy data

Research conducted under the auspices of Dr. Holly A. Yanco, the Robotics Lab, and the Computer Science Department.

Contact [email protected] for additional information.