Top Banner
Processing Big Data Compressed sensing
20

Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Jul 14, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Processing Big DataCompressed sensing

Page 2: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Dimensionality reduction• Linear

• Principal Component Analysis: SVD-based

• Compressed sensing

• Matrix sketching

• Non-linear

• Kernel PCA

• Isometric mapping

Page 3: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing

Consider a sparse vector

i.e., a d-dimensional vector with only m non-zero entries

d = 19, m = 4

f = (0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0)<latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit>

Page 4: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing

Let a measurement be a random vector xi 2 {�1, 0, 1}d<latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit><latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit><latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit><latexit sha1_base64="7jApKt3CNHGN4HereefsAGduPAA=">AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=</latexit>

xi = (0,�1, 0, 1, 0, 0, 1, 0, 0,�1, 1, 0, 0, 0, 1, 0, 0, 1, 0)<latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit><latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit><latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit><latexit sha1_base64="Q8XureH8/qzqRBy/lYGDSTAV2Xo=">AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=</latexit>

Taking a measurement is

f = (0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0)<latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit><latexit sha1_base64="CMPizmkPd2D6NHAe9rDfZHIzG5w=">AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=</latexit>

hf, xii = 2<latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit><latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit><latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit><latexit sha1_base64="TRsC3i0iYgFrQJQAhSKDgpd3DFo=">AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=</latexit>

X = [x1 x2 · · · xN ]<latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit>

in general, we have a measurement matrix

Page 5: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing

X = [x1 x2 · · · xN ]<latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit><latexit sha1_base64="36l9paWuaKl0+bBV4lmCwShAMJY=">AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==</latexit>

in general, we have a measurement matrix

y = Xf<latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit><latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit><latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit><latexit sha1_base64="LHRD23RPIEfUyMZq5FCLL1bM3Ag=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=</latexit>

and a measurement

N = Km log

✓d

m

<latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit><latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit><latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit><latexit sha1_base64="OEtNCMh+b8pW0pBIo2UDP5cqQQE=">AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl</latexit>

Our goal is to make only N measurements

where K in some settings is 4 or max 20

Page 6: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensingCompression: y takes N logN

<latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit><latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit><latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit><latexit sha1_base64="yBK2voB0QqZtCL5ROVK8BCSFm7w=">AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y</latexit>

Application: Hubble telescope

Has high-res camera in space, for less interference. But communication to Earth is expensive. With an initially random matrix X in space used as a mask, the images are compressed. Matrix X is known on Earth.

Data is often noisy, and have more attributes than the real underlying process. CS helps finding the true structure of the data.

Noisy Data

Page 7: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

How to recover the compressed signal?

One of the simplest ways: Orthogonal Matching Pursuit

Simple and greedy method

Greedy algorithm: makes the locally optimal choice at each stage, hoping to find a global solution

Example: reaching the path of largest sum

https://en.wikipedia.org/wiki/Greedy_algorithm

A greedy strategy is not guaranteed to reach the optimal solution.

Page 8: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

Xj<latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit><latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit><latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit><latexit sha1_base64="6E9Oi48tZeFkKRsbB82UZln6sLc=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=</latexit>

Find the measurement column

Note: we are not finding the row xi<latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit><latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit><latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit><latexit sha1_base64="AVmfx+YeFaOOyFtkiB8CIxCujYQ=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==</latexit>

j is the index of f that better explains y

Step 1

Page 9: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

Reconstruction residual between measurement and what is our estimate for the entry j of f

Step 2

Page 10: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

this is what remains to be explained by other elements of f

Step 3

compute the residual

and return to step 1, until r is small (there is not that much to explain) oris small (further explanation is not useful)

�<latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit><latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit><latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit><latexit sha1_base64="+03BdUB6TWqLjuCfwSU3OIMjF3A=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p</latexit>

y = r<latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit><latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit><latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit><latexit sha1_base64="6oq/mpUf2eN9xBPKMOeoPhxeCvI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o</latexit>

do

Page 11: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Orthogonal Matching pursuit

r = y<latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit><latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit><latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit><latexit sha1_base64="BZQIY4Rd6Cwz0+aWq3TLHaYHud4=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o</latexit>

Xj = argmaxXj0

|hr,Xj0i|<latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit><latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit><latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit><latexit sha1_base64="2onlBeauZORnbY/TioPQ77F+beQ=">AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=</latexit>

�j = argmin�

kr �Xj�k<latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit><latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit><latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit><latexit sha1_base64="7aTTb7M8M5uMDXtbsUbkY1QxtOM=">AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==</latexit>

r = r �Xj�j<latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit><latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit><latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit><latexit sha1_base64="U5oaMI2JZkX4pYDqkd9mXrbG+us=">AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=</latexit>

gj = �j<latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit><latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit><latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit><latexit sha1_base64="kLC609921l1PgIXwWoQVISafiwg=">AAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv</latexit>

Set

Return

S1:

S2:

S3:

Page 12: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Examplef = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]

<latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit><latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit><latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit><latexit sha1_base64="orN/A16ZqzHYB4nfSe/ir16mQXA=">AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==</latexit>

y = XfT<latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit><latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit><latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit><latexit sha1_base64="pRN4evLHOreRm5ER1zkFskUF844=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==</latexit>

y = [0, 0, 0, 1, 1,�2]T<latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit><latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit><latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit><latexit sha1_base64="8nLsPuAsSL+BjApHEaL/aihB+zA=">AAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm</latexit>

Page 13: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Compressed sensing: full reconstructionWe can fully reconstruct any sparse signal if it was compressed by a matrix satisfying the Restricted Isoperimetric Property (RIP)

RIP guarantees a low distortion of the norm of any sparse representable vector

Full reconstruction is done by a solving linear problem: polynomial time

A random matrix with the characteristics of X is likely to satisfy the RIP condition

Page 14: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

CS: problem formulation

f̂ 2 argming:Xg=y

kgk0<latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit><latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit><latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit><latexit sha1_base64="pmHqRjyfxPvrobBarcsT/QYPvc4=">AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO</latexit>

If X is RIP

kfk0 m<latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit><latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit><latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit><latexit sha1_base64="am+yA3MWYdKVmF202w6Dt/5pJNA=">AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=</latexit>

)<latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit><latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit><latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit><latexit sha1_base64="apOjzIa4RvBwcVe1EmrYuZsBwiw=">AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==</latexit>

f̂ = f<latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit><latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit><latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit><latexit sha1_base64="qLac7PGSar98RwOyckxNI39rl9k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==</latexit>

Page 15: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

CS: convex relaxation is tight

We can prove, under mild conditions, that

f = argming:Xg=y

kgk0 = argming:Xg=y

kgk1<latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit><latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit><latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit><latexit sha1_base64="dcyv8us7XcNtJNtodiWFL7dFCYM=">AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==</latexit>

More information in Shai Shalev-Shwartz, Compressed Sensing: Basic results and self contained proofs

Page 16: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Matrix sketching

Going back to a dense data matrix A

We are going to see a way to “improve” PCA and the SVD

Page 17: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Matrix sketching: the SVD

Jeff M. Phillips, University of Utah

Page 18: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Matrix sketching: the SVD

Jeff M. Phillips,

PCA:

Page 19: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Approximation of A using the SVD

Columns of V are linear combinations of features

Interpretability of V

What is a linear combination of purchased books?

What is a linear combination between number of calls and being male or female?

Computational efficiency

Computing the SVD demands loading all A into memory and requires time

What about really big data?

What about streamed data?

Page 20: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data Compressed sensing. Dimensionality reduction ... What about really big data? What about

Row sampling

Interpretability: choose V so that columns of V are columns of A