Player Highlighting and Team Classification in Broadcast Soccer Videos for the Next Generation TV Yu Huang Multimedia Content Networking Lab Core Network Research Dept., Huawei Technology (USA) 400 Somerset Corporate Blvd., Bridgewater, NJ08807 WOCC’09, NJ, USA
27
Embed
Player Highlighting and Team Classification in Broadcast ...yuhuang/papers/WOCC09Slide.pdf · Player Highlighting and Team Classification in Broadcast Soccer Videos for the Next Generation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Player Highlighting and Team Classification in Broadcast Soccer Videos for the Next Generation TV
• Background• Player Detection• Player Tracking• Use Case 1: Object Highlighting• Use Case 2: Team Classification• Demo of Use Cases • Conclusion
Yu Huang, Huawei Technologies
Background• The Next Generation TV (such as IPTV, Interactive TV and
Mobile TV etc.) offers appealing services:– Enhancement;– Personalization;– Interactivity.
• TV contents require to be analyzed and annotated.– Different levels: objects, scenes and events;– Different domains: spatial, temporal and luminance.
• Sports game (such as soccer) is a good application scenario to depict the viability of the next generation TV:
– Players are important clickable objects in the scene with linked rich media information for viewers;
– Scenes/events can be nonlinearly skimmed /browsed with summarization.
• Player localization/team classification provide a nice bridge for object level interactivity in sports programs (work focus).
Discriminant Similarity Function• Choose the “Center-Context” window;
– The inward window size wxh while the outward window size 3wx3h.
• Build histogram from the surrounding background of the target;
• Similarity measure weighted by the discrimination metric between the
target and its surrounding background.
== otherwise
yp
ypif
v
ub
ub
u ,)(
0)( ,0
,
,
ϑ
)(ypb
)(yp f ∑ =−−= hn
i iiufuf uxbhxykvCyp1
2
, ])([)/)(()(' δCandidate
Target
Similarity function
.)('')](','[max1 ,,∑ =
= m
u ufutft ypqypqρ
∑ =−= kn
i iiuqut uxbxkvCq1
*2*, ])([)(' δ
Yu Huang, Huawei Technologies
The Target Model Update
• Given the estimated location yt and scale st at the current frame;
• Run mean shift iteration again to disturb the estimated location yt , where
the initial target histogram q0 (u) takes place of the current target
histogram qt (u).
• The update strategy act conservatively like
– If the new estimated location yt’ in the second iteration does not diverge too
far from yt, , i.e. , update the current target histogram with the
candidate histogram at location yt’ and scale st as qt+1 (u) = p(yt’ ), and
update the estimated location as yt’ ;
– Otherwise, keep the old target histogram qt+1 (u) = qt (u) and the estimated
location in the first iteration yt .
ε≤− tt yy'
Yu Huang, Huawei Technologies
Case 1: Object Highlighting
• Ellipse or rectangle window for the object of interests;
• Dimming the other areas except the object window:– Decrease exponentially the Y component in YUV format.
Yu Huang, Huawei Technologies
Case 2: Team Classification• Label the Players/Referees with Team A, Team B and
Referee:– Goalies for both teams are seldom present in the videos, so
temporally not considered.
• Defined Feature for Classification:– a multi-histogram representation (bi-histograms for jersey and
shorts regions).
(To be continued)
Yu Huang, Huawei Technologies
Case 2: Team Classification• Target histogram acquisition for each class:
– For each class, click on multiple samples in the video frames;
– Each object blob is separated into upper (jersey region) and lower (shorts region) parts;
– Collect all pixels for jersey and shorts regions, then build corresponding normalized color histograms respectively;
• Team Classification when the isolated player being clicked:– Acquire its bi-histogram feature as well;– Identify its class by bi-histogram matching.
Yu Huang, Huawei Technologies
Bi-Histogram Matching• Search the best cutting line to separate the object (player/referee) blob into two
parts vertically;
• The maximized objective function (Bhattacharyya distanceBhattacharyya distance) is to realize the most discrimination between jersey and shorts regions.
• Bi-histogram matching for team classification
∑ =−= m
u uudownupper qpqp1
0.1],[ρ
Normalized Color Histogram
Search range
Cutting line
∑∑ ==⋅−+⋅= m
u uiu
m
u uiuiii
qpqpqqpp1 221 112121 )1()],(),,[(max λλρ
Yu Huang, Huawei Technologies
Demo of Use Cases
• GUI implemented with MS Visual Studio VC++;• User clicks the object by the mouse on the played video;• Real-time segmentation, identification and tracking:• Use Case 1: Object Highlighting;• Use Case 2: Team Classification.
Conclusions• Gaussian model-based playfield model + dominant color
detection for player detection; – A Semi-supervised method.
• Modified mean shift-based with soft constraints from the foreground map for player tracking;– Reduce the risk of tracking failure for fast-moving players.
• Scale adaptation based on discrimination between the target and its surrounding background in tracking;– avoid the “shrinkage” problem;
• Target histogram updating in a conservative way in tracking;– alleviate drifting problems.
• Two use cases for viability of interactive services:– “Object highlighting”;– “Team classification”: bi-histogram matching.
Yu Huang, Huawei Technologies
Acknowledgement
• Collaborators at Huawei Technologies:– Dr. Hongbing LI;– Dr. Jun TIAN;– Dr. Heather YU.