Lecture 8 – Collective Pattern Collectives Pattern CS 472 Concurrent & Parallel Programming University of Evansville Selection of slides from CIS 410/510 Introduction to Parallel Computing Department of Computer and Information Science, University of Oregon http://ipcc.cs.uoregon.edu/curriculum.html
32
Embed
CS 472 Concurrent & Parallel Programming University of ...uenics.evansville.edu/~hwang/f17-courses/cs472/lecture05-collectiv… · CS 472 Concurrent & Parallel Programming Lecture
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Lecture 8 – Collective Pattern
Collectives PatternCS 472
Concurrent & Parallel ProgrammingUniversity of Evansville
Selection of slides from CIS 410/510 Introduction to Parallel Computing
Department of Computer and Information Science, University of Oregon
http://ipcc.cs.uoregon.edu/curriculum.html
Lecture 8 – Collective Pattern
Announcements
• No class next Thursday, September 21. Instructor will be traveling to a conference. It will be the second day of working on the lab project associated with today’s topic.
• Collective operations deal with a collection of data as a whole, rather than as separate elements
• Collective patterns include: • Reduce
• Scan
• Partition
• Scatter
• Gather
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives3
Lecture 8 – Collective Pattern
Collectives
• Collective operations deal with a collection of data as a whole, rather than as separate elements
• Collective patterns include: • Reduce
• Scan
• Partition
• Scatter
• Gather
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives4
Reduce and Scan will be covered in this lecture
Lecture 8 – Collective Pattern
Reduce
• Reduce is used to combine a collection of elements into one summary value
• A combiner function combines elements pairwise
• A combiner function only needs to be associative to be parallelizable
• Example combiner functions:• Addition
• Multiplication
• Maximum / Minimum
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives5
Lecture 8 – Collective Pattern
Reduce
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives6
Serial Reduction Parallel Reduction
Lecture 8 – Collective Pattern
Reduce
• Vectorization
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives7
Lecture 8 – Collective Pattern
Reduce
• Tiling is used to break chunks of work up for workers to reduce serially
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives8
Lecture 8 – Collective Pattern
Reduce – Add Example
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives9
1 2 45 9 7 0 1
Lecture 8 – Collective Pattern
Reduce – Add Example
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives10
1 2 45 9 7 0 1
28
12
3
8
21
29
28
29
Lecture 8 – Collective Pattern
Reduce – Add Example
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives11
1 2 45 9 7 0 1
Lecture 8 – Collective Pattern
Reduce – Add Example
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives12
1 2 45 9 7 0 1
3 9 116
12 17
29
29
Lecture 8 – Collective Pattern
Reduce
• We can “fuse” the map and reduce patterns
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives13
Lecture 8 – Collective Pattern
Reduce
• Precision can become a problem with reductions on floating point data
• Different orderings of floating point data can change the reduction value
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives14
Lecture 8 – Collective Pattern
Reduce Example: Dot Product
• 2 vectors of same length
• Map (*) to multiply the components
• Then reduce with (+) to get the final answer
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives15
Also:
Lecture 8 – Collective Pattern
Dot Product – Example Uses• Essential operation in physics, graphics, video games,…
• Gaming analogy: in Mario Kart, there are “boost pads” on the ground that increase your speed• red vector is your speed (x and y direction)
• blue vector is the orientation of the boost pad (x and y direction). Larger numbers are more power.
CS 472 Concurrent & Parallel Programming
Lecture 5 Collectives16
Photo source
How much boost will you get? For the analogy, imagine the pad multiplies your speed:• If you come in going 0, you’ll get nothing• If you cross the pad perpendicularly, you’ll
get 0 [just like the banana obliteration, it will give you 0x boost in the perpendicular direction]