3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1.

3/12/2013 Computer Engg, IIT(BHU) 1

PARALLEL COMPUTERS-1

INTRODUCTION

•A parallel computer is either a single computer with more than one processor or group of computers to run any application

TYPES OF PARALLEL COMPUTERS•There are basically two types of parallel computers

-multiprocessor system

-multicomputer system

MULTIPROCESSOR SYSTEM

•Through interconnection networks multiple processors are joined

•Each processor shares a global address space

•Each processor has a local memory

•For communication between Pi and Pj , Pi first writes in the shared address space then Pj reads it from the shared address space

A MULTIPROCESSOR MODEL

Shared Address Space

INTERCONNECTION NETWORK

P1 P2 P3 Pn

•A high level language with parallel programming constructs and compiler directive can be used for programming in such a multiprocessor system

•Threads(regular high level code sequences for individual processors) can also be used for the programming

EXAMPLES OF MULTIPTOCESSORS•Dual Pentium

•Quad Pentium etc.

ADVANTAGE

-Data is easily shared DISADVANTAGE

-Hardware implementation is difficult

•Large shared memory multiprocessor system may have hierarchical and distributed memory

•In hierarchical memory arrangement accessing nearby memory locations is much faster, hence termed as NUMA “Non Uniform Memory Access”

•Similarly UMA provides “Unform Memory Access”

MULTICOMPUTER SYSTEM

•Complete computers are connected through interconnection networks

•Each computer has a processor and local memory

•With the help of interconnection networks, computer communicate with each other

•Memory of a computer is not accessible by other processor

•Computers communicate by passing messages

•A message is nothing but data or instruction to be sent

Chip Multiprocessor

•There is considerable diversity among parallel machines

•One of the today’s popular parallel architectures is Multi Core architecture

•Multiple instruction execution engines are fabricated on a single chip

•These engines are known as Cores

Features of Intel Core Duo

•Two 32-bit Pentium processors on a single chip

•Each processor has its own L1 data and instruction cache

•Shared L2 cache

•Shared memory controller, I/O controller

•Fast communication between two through shared memory

LOGICAL ORGANIZATION OF INTEL CORE DUO

FRONT SIDEBUS

MEMORY BUS CONTROLLER

L2 CACHE

P1 P2

L1-I L1-D L1-I L1-D

Other Parallel Computing Scenarios•Cluster computing

•Distributed computing

•Grid computing

•Cloud computing

Cluster computing

Cluster: Multiple interconnected personal computers as parallel computing platform

- Interconnects are available in several forms including Gigabit ethrrnet, Myrinet, Infinibands, Fiber channels etc.

- Processor communicates with other processor by passing messsages

A TYPICAL CLUSTER

Connection to other computers & switches in network

COMPUTERS

ETHERNET SWITCH

Advantages

Advantages over most other forms of High Performance Computing:

•High scalability (Latest processors can easily be incorpoated)

•Fault tolerance

•Memory is not shared among the machines

Distributed Computing

•Multiple autonomous computer work together to achieve a common goal

•Autonomous computers communicate through a computer network

•It runs as a single system

A DISTRIBUTED SYSTEM

: NODE

•In distributed computing each processor has its own private memory compared to parallel computers which has shared memory.

Shared Memory

P1 P2 Pn

Benefits Over Centralized System•Scalability: System is easily expanded by adding more machines

•Redundancy: Several machines solve same problem

Grid Computing

•Notion of computing grid arose in the early 1990’s

•Like Power Grid where anyone can tap into, without worrying how the power is obtained and transmitted, A computing grid provides computational power hiding other details from the user

•Resources located at different place participate in a grid

•Grid computing harnesses the CPU cycles of the computers in a network to solve a problem

Primary Attributes of a Grid

Ian foster listed 3 primary attributes of a grid:

•Computing resources are not administered centrally

•Open standards are used

•Non-trivial quality of service is achieved

GARUDA Grid

•India’s first national grid initiative

•Academicians, researchers together develop their data and compute intensive applications

•GARUDA is a SOA based cyber infrastructure connecting computational nodes, mass storage and scientific instruments

Cloud Computing

•Cloud uses internet and central remote servers to maintain data and applications

•Provides computation, software, data access and storage resources

•A cloud can be used through a web browser or simple desktop/mobile apps.

Cloud Computing Service Models

Application

Infrastructure

Platform

Server

Tablet

Laptop

Mobile

Desktop

REFERENCES

•Wilkinson