Top Banner

Click here to load reader

of 40

Computer vision techniques for interactive art

Jan 19, 2015

ReportDownload

Education

Presentation about computer vision using Processing and Jitter for the Interactive Art PhD program at the School of Arts (http://artes.ucp.pt)

  • 1. Computer Vision Techniques forInteractive ArtUsing Processing and Max/Jitter
    Interactive Art Doctoral Program
    http://www.artes.ucp.pt/si/doutoramento/index.html
    5/27/11
    Jorge Cardoso
    1

2. Topics
Techniques
Framedifferencing(for motiondetection)
Color detection(whereintheimage, does a given color appear can beused for trackingobjects)
Brightnessdetection(whatisthepositionofthebrightestpixels)
Background subtraction(for objectsegmentation)
Blobextraction(applied to theprevioustechniques to extractshapeparametersandvectorizingsilhuettes)
Movementestimation (whatdirection are theobjectsmoving)
Face detection(isthere a face intheimage? where?)
5/27/11
Jorge Cardoso
2
3. What
Overviewof some computervisiontechniques
Howtheyworkandwhat are themainlimitations
How to use theminProcessingandJitter
Some verysimple(istic) examplesofhowthey can beused for interactivework
5/27/11
Jorge Cardoso
3
4. How
General overviewofhowthose CV techniqueswork
Examplesofwhat can beaccomplishedwiththetechniques
Vdeos
Live demos

  • CodeexamplesofthesetechniquesinProcessingandJitter

5/27/11
Jorge Cardoso
4
5. Computer Vision for Interaction
Use a computer and camera to extract information about what people are doing and use that as an implicit or explicit form of interaction
Computer visionis the science and technology of machines that see. As a scientific discipline, computer vision is concerned with the theory for building artificial systems that obtain information from images Computer vision. (2009, April 24). InWikipedia, The Free Encyclopedia. Retrieved 11:53, May 9, 2009, fromhttp://en.wikipedia.org/w/index.php?title=Computer_vision&oldid=285848177
5/27/11
Jorge Cardoso
5
6. Image
Computer vision are processes applied to sequences of images
An image is just
5/27/11
Jorge Cardoso
6
7. 5/27/11
Jorge Cardoso
7
Image
Computer vision are processes applied to sequences of images
An image is just a sequence of number (usually arranged in a matrix form)
Eachnumberrepresents a colour
50 44 23 31 38 52 75 52
29 09 15 08 38 98 53 52
08 07 12 15 24 30 51 52
10 31 14 38 32 36 53 67
14 33 38 45 53 70 69 40
36 44 58 63 47 53 35 26
68 76 74 76 55 47 38 35
69 68 63 74 50 42 35 32
8. Image
Each pixel (i.e., number in the matrix) isusuallystoredin RGB or ARGB formatso
It can be decomposed:
5/27/11
Jorge Cardoso
8
or
9. Frame Differencing
If we subtract two consecutive video frames, threshold and binarize it, we get a rough view of the movement in the video
Procedure:
Go through the current frame image, pixel by pixel
Calculate the distance between the color of the current frames pixel and the previous frames pixel
If distance is smaller than a given threshold, assume no movement (black), otherwise assume movement (white)
5/27/11
Jorge Cardoso
9
10. Frame Differencing
Color difference
Q: How do wecalculatethe distancebetweentwocolors?
A: Simpleversion: euclideandistance.
Take a color as a pointin 3D space (RGB -> XYZ)
Calculatethedistancebetweenthetwopoints
Subtracteachcomponent (dR = R1-R2, dG = G1-G2, dB = B1-B2)
Dist = Sqrt(dR*dR + dG*dG + dB*dB)
A1: Slightly (perceptively) betterversion: weighteachcomponentdifferently:
Dist = Sqrt(2*dR*dR + 4*dG*dG + 3*dB*dB)
5/27/11
Jorge Cardoso
10
11. Frame Differencing
Q: Whatsitgood for?
A:
KALOETHOGRAPHIA (2010)
Pedro Cascalheira
http://artes.ucp.pt/blogs/index.php/vai/2011/02/14/alunos-2009-2010
aB (2011)
Jorge Coutinho
https://picasaweb.google.com/lh/photo/x9bGyxFsTLVFbkk9kD3xLA?feat=directlink
WebCamPiano
MemoAkten
[video]
5/27/11
Jorge Cardoso
11
12. Frame Differencing - Code
ProcessingCode
FrameDifferencing
FrameDifferencingWithMotionDetection
FrameDifferencingForAirPiano
JitterCode
FrameDifferencing.maxpat
FrameDifferencingWithMotionDetection.maxpat
5/27/11
Jorge Cardoso
12
13. Color detection
Isolate color regions in an image
Procedure:
Go through the image, pixel by pixel
Calculate the distance between the color of the current pixel and the reference color
If distance is smaller than a given threshold, keep the pixel
5/27/11
Jorge Cardoso
13
14. Color Detection
Color Difference
For a betterapproachthan RGB distance,use HSB
Give more weight to theHue
Dist = Sqrt(3*dH*dH + dS*dS + 2*dB*dB)
5/27/11
Jorge Cardoso
14
15. Color Detection
Q: Whatsitgood for?
A:
Catchoftheday [video]
PlayDohAsInterfaceKeyboard[video]
5/27/11
Jorge Cardoso
15
16. Color Detection - Code
ProcessingCode
ColorDifference
ColorDifferenceSequencer
JitterCode
ColorDifferenceWithFindbounds.maxpat
ColorDifference.maxpat
5/27/11
Jorge Cardoso
16
17. Brightenessdetection
17
Slightvariationon color detectionwhenwereonlyinterestedintrackingbrightpixels
For example, to track a light
Procedure:
Same as color detectionbutextractthebrightnessofeach pixel andkeeponlythehighest
Jorge Cardoso
5/27/11
18. 5/27/11
Jorge Cardoso
18
Brighteness detection

  • Q: How do wecalculatethe brightness of a color?

19. A: 20. In RGB: 21. (perceptive) Brightness = (0.2126*R) + (0.7152*G) + (0.0722*B) 22. (physical/energy) Brightness = (R+G+B)/3 23. 24. In HSB: 25. Brightness = B 26. Just use thebrightness() functioninwhateverlanguage