Parallel Programming Henri Bal (bal@ cs .vu. nl ) Vrije Universiteit Faculty of Sciences Amsterdam
Dec 21, 2015
Overview
• What is parallel programming?
• Why do we need parallel programming?
• Organization of this course
• Practicum Parallel Programming
ICWall
•A large display suitable for:– Education (classroom)– Modern scientific applications thatneed to visualize large data-sets
– 3D applications
ICWall: an "off-the-shelf" video wall
•High-end graphics is very expensive•Use tiled display with many projectors•Use standard (inexpensive) components
– 9 PCs, graphic and network cards, 16 projectors
•Similar idea as cluster computing, but applied to graphics
•Current ICWall is strongly simplified version–1 PC with 2 projectors
Parallel Programming
• Sequential programming
Single thread of control
• Parallel programming
Multiple threads of control
• Why parallel programming?
Eases programming? Not really.
Performance? Yes!
Famous quote
• “Parallel programming may do something to revive the pioneering spirit in programming,which seems to be degenerating intoa rather dull and routine occupation’’
• S. Gill, Computer Journal, 1958
Why do we need parallel processing?
• Many applications need much faster machines
• Sequential machines are reaching their speed limits
• Use multiple processors to solve large problems fast
• Microprocessors are getting cheaper and cheaper
• Cheap multiprocessors and multicore CPUsbring parallel processing to the desktop!
History
• 1950s: first ideas (see Gill’s quote)
• 1967 first parallel computer (ILLIAC IV)
• 1970s programming methods, experimental machines
• 1980s: parallel languages (SR, Linda, Orca), commercial supercomputers
• 1990s: software standardization (MPI), clusters, large-scale machines (Blue Gene)
• 2000s: grid computing: combining resources world-wide (Globus)
Large-scale parallel machines
• Many parallel machines exist
• See http://www.top500.org
• Current #1 is IBM BlueGene/L with 131072 CPUs
131072 131072
Challenging Applications• Modeling ozone layer, climate, ocean• Quantum chemistry• Protein folding• General: computational science• Aircraft modeling• Handling use volumes of data from scientific instruments
– Lofar (astronomy)– LHC (CERN, high-energy physics)
• Computer chess• Analyzing multimedia content• Generating movies
About this Course
Goal: Study how to write programs that run in parallel on a large number of machines.
Focus on programming methods, languages, applications
Prerequisites:
Some knowledge about sequential languages
Little knowledge about networking and operating systems
Aspects of Parallel Computing
Algorithms and applications
Programming methods, languages, and environments
Parallel machines and architectures
Course Outline• Introduction in algorithms and applications• Parallel machines and architectures
Overview of parallel machines, trends in top-500Cluster computers, BlueGene
• Programming methods, languages, and environmentsMessage passing (SR, MPI, Java)Higher-level language: HPF
• ApplicationsN-body problems, search algorithms, bioinformatics
• Grid computing Multimedia content analysis on Grids (guest lecture Frank Seinstra)
Course Information
ExaminationWritten exam based on:- Reader: available electronically from Blackboard- Lectures
More information (and slides):http://www.cs.vu.nl/~bal/college07.html
Practicum Parallel Programming
Separate practicum (6 ECTS)
• Implement ASP + SOR algorithms in C/MPI
• Implement IDA* search algorithm in Java/RMI
• Test and measure the programs on our DAS cluster
More information
Register by email to [email protected] (Niels Drost) now
Starts after MPI lecture (mid October)
See http://www.cs.vu.nl/pp-cursus/ for a complete description