Hot Topics in Parallel Computing Prof. Richard (Rich) Vuduc CSE 8803-HPC [01] Tuesday, August 23, 2010 Tuesday, August 24, 2010
Hot Topics in Parallel ComputingProf. Richard (Rich) Vuduc
CSE 8803-HPC
[01] Tuesday, August 23, 2010
Tuesday, August 24, 2010
Introductions
Who?
Program & year?E.g., CS PhD, AE MS, …
One interest?E.g., dissertation topic, “compilers,” bowling, …
2
Tuesday, August 24, 2010
Origins & philosophy:
This course gets its name from an annual workshop called, “Hot Topics in Parallelism (HotPar).”
The spirit of HotPar is to discuss and debate the path forward for parallel computing, which many believe is at a kind of turning point.
Of course, this depends strongly on YOU!
Tuesday, August 24, 2010
Position:Parallel computing is a 50 year old field. Yet, now is the time to study it.
Question:Really? Why or why not?
Tuesday, August 24, 2010
Position:Parallel computing is a 50 year old field. Yet, now is the time to study it.
Question:Really? Why or why not?
Applications!– Data & compute intensive
Hardware!– Latency and power
Q: Software in-between?
Tuesday, August 24, 2010
“Extreme scale” systems:Climate modeling
This laptop ~ 1010 ops / s
Today: “Petascale” systems~ 1015 ops/s
To predict climate change 10-50 years into the future, need ~ 1018 ops/s system –“Exascale”
Need 108 = 100 million laptops
6No one yet knows how to build or the “best” way to program such a system!
Tuesday, August 24, 2010
Meeting Diarist Application“Parlab All”
3
Opportunities and Challenges of Parallelizing Speech Recognition.Jike Chong, UC Berkeley; Gerald Friedland, Adam Janin, Nelson Morgan, and Chris Oei, ICSI @ BerkeleyIn HotPar, June 2010. – http://www.usenix.org/events/hotpar10/tech/
Tuesday, August 24, 2010
Tuesday, August 24, 2010
2-core CPU
Tuesday, August 24, 2010
2-core CPU+ 32-core GPU (↑perf+power)
Tuesday, August 24, 2010
2-core CPU+ 32-core GPU (↑perf+power)+ 16-core GPU (↓perf+power)
Tuesday, August 24, 2010
26-core smartphone. Chinese entrepreneur Steve Chao to showcase the Ouidoo at the Shanghai World Expo. [engadget.com]
12
Tuesday, August 24, 2010
Phones for HPC?
13
Tuesday, August 24, 2010
Position:Parallel computing is a 50 year old field. Yet, now is the time to study it.
Question:Really? Why or why not?
(from H&P 4th, App. K)
“The turning away from the conventional organization came in the middle 1960s … Electronic circuits are ultimately limited in their speed of operation by the speed of light …”– W. Jack Bouknight, Illiac IV System (1972)
“… sequential computers are approaching a fundamental physical limit on their potential computational power. Such a limit is the speed of light …”– Angela L. DeCegama, The Technology of Parallel Processing, Volume I (1989)
“… today’s multiprocessors … are nearing an impasse as technologies approach the speed of light …”– David Mitchell, The Transputer: The Time is Now (1989)
Tuesday, August 24, 2010
We’ve been here before:The Dead Supercomputer Society
ACRIAlliantAmerican SupercomputerAmetekApplied DynamicsAstronauticsBBNCDCConvexCray ComputerCray ResearchCuller-HarrisCuller ScientificCydrome
Dana/Ardent/Stellar/StardentDenelcorElxsiETA SystemsEvans and Sutherland Computer DivisionFloating Point SystemsGalaxy YH-1Goodyear Aerospace MPPGould NPLGuiltech
Intel Scientific ComputersInternational Parallel MachinesKendall Square ResearchKey Computer LaboratoriesMasParMeikoMultiflowMyriasNumerixnCubePrisma
Thinking MachinesSaxpyScientific Computer Systems (SCS)Silicon Graphics Inc (now Rackable)Soviet SupercomputersSun (bought by Oracle)SupertekSupercomputer Systems (SSI)SuprenumVitesse Electronics
15Industry is placing a huge bet on parallelism. Are we ready? Will we succeed?
Tuesday, August 24, 2010
Course Logistics
Rich Vuduc (instructor), richie@gatech or AIM: VuducOfficeHoursJonathan Brownsworth (TA), j.brownsworth@gatech
Sign up for course website: gtcse8803hpc.ning.com
Read 2–4 papers per week.
Submit two paragraph written summaries as “blogs” on ning site.
We all take turns leading discussions each week and “blogging” a summary of it (again, on ning site)
Do a research-oriented final project; think “publication”
16
Tuesday, August 24, 2010
Grading
Class participation: 25%
Write-ups: 25% – May miss 3 days without penalty
Final project: 50%
17
Tuesday, August 24, 2010
Read an submit write-up before class
Write-up format: Concise (2 paragraph) summary & critique
What is the main claimed contribution?What do you think is the real contribution?
Strengths or weaknesses in the methodology?
What new questions are raised by this paper?
Practice for “real” life research and reviewing
18
Readings and your write-ups
Tuesday, August 24, 2010
Thursday 8/26: No class. Instead, go to theNVIDIA CUDA Center of Excellence Kick-off in Klaus 1116.
Assignment: Go any time between 9am to 3:30pm, and write-up a one to two-paragraph of what you learn
Post on your ning.com blog before next class (8/31)
19
Tuesday, August 24, 2010
Week Tuesday
Aug 24: Intro Aug 26: NVIDIA CCOE Kick-off
Aug 31: Rich Sep 2: Rich / Jonathan
Sep 7: NO CLASS Sep 9: NO CLASS
Sep 14: Regular class Sep 16: Regular class
Tuesday, August 24, 2010