8/16/2019 PP16-lec1-intro1

1/18

1.1

Parallel Processingsp2016

lec#1

Dr M Shamim Baig

8/16/2019 PP16-lec1-intro1

2/18

1.2

Course Information• Prereq: Computer Architecture, Programming in C C!!

• "e t $ef:% Parallel Computers: Architecture & Programming by Rajaraman% Introduction to Parallel Computing ! nd ed" by #rama$ Programming Massi%ely Parallel Processors ! nd d" by 'ir(

• &ra'ing Polic(

) *ui++es : 10 ) Assignments Pro-ect : 1 ) /i'term am : 0 ) inal am : 3

•Instructor: 4r / 5hamim aig ) mail: ms7aig8case.e'u.p9

• "A: mer arooq % mail: umer.farooq8case.e'u.p9 Cell# 0 3 % ;

8/16/2019 PP16-lec1-intro1

3/18

1.

Course Intro'uction

• )ith ad%ances in computer architecture* high per+ormance multiprocessor computers ha%e becomereadily a%ailable & a++ordable, As a result* high

per+ormance & supercomputing is accessible to alarge segment o+ industry that -as once restricted tomilitary research & large corporations , .he course iscomprised o+ architecture* algorithms & programming

o+multicore & Parallel computing systems, It +ocuseson design concepts* principles* paradigms* models* per+ormance e%aluation and real li+e applications,

8/16/2019 PP16-lec1-intro1

4/18

1.3

Parallel ComputingArchitectures

• Current computer architectures areincreasingl( rel(ing upon parallelism toimpro?e performance: ) Pipeline' processing ) /ultiple e ecution units ) /ulti%core processor @Cores ) /an(%core processor @&P ) /utiprocessors @CP s B

) /ulticomputers @PCs or9stations

8/16/2019 PP16-lec1-intro1

5/18

1.

Course Contents• Parallel Architecture /o'els:

) Implicit: Pipelining, 5uperscalar B DEI . ) plicit: 5I/4,/I/4, multicore 5/P,

?ector arra( processor, /PP B Clusters

• Parallel /emor( /o'els : ) 5/, 4/, 45/, /A F /A ) Cache Coherence protocols

• Interconnection Fet=or9s @IF : ) 5tatic ?s 4(namic net=or9 topologies ) EAFs ?s 5AFs @/(rinet, Infini7an'

8/16/2019 PP16-lec1-intro1

6/18

1.6

• Collecti?e Communication Gperations: ) roa'cast, $e'uction, 5catter B &ather

• m7e''ing /apping application to IF

) e,g $ing /esh into H(percu7e & %ice %ersa

• Parallel 4ata Partitioning "echniques

) 14 24 4 ro=%=ise column%=ise

• Parallel Algorithms 4esign, Anal(sis B

Programming mo'els: 5 P/4 /P/4 B P$A/

Course Contents /cont0d

8/16/2019 PP16-lec1-intro1

7/181.J

• /ulticore 5/P programming using Gpen/P

• /essage Passing multicomputer Cluster

programming using /PI

• /assi?el( @'ata Parallel Processing @ /PP

using &P C 4A platform

• Heterogeneous Parallel Computing using

CP B &P @/PI open/P cu'aC together

Course Contents /cont0d

8/16/2019 PP16-lec1-intro1

8/181.;

• Parallel Performance ?aluation

) Parallel Einpac9 stan'ar' test criteria

) Compute Communicate "ime Anal(sis ) 5(stems monitoring measurement tools

• Parallel Performance Ea=s

) Am'hlKs Ea= B &ustafsonKs Ea=

Course Contents /cont0d

8/16/2019 PP16-lec1-intro1

9/18

Applications of HPC

1.<

8/16/2019 PP16-lec1-intro1

10/181.10

ses for Parallel Computing:• Historically, parallel computing has been considered to be "the high end of

computing", and has been used to model difficult scientific and engineeringproblems found in the real world .

• Some examples: ) Atmosphere, Earth, En ironment ) Physics ! applied, nuclear, particle, fusion, photonics ) ioscience, iotechnology, #enetics ) Chemistry, $olecular Sciences , #eology, Seismology ) $echanical Engineering ! from prosthetics to spacecraft ) Electrical Engineering ! Circuit %esign, $icroelectronics ) Computer Science & $athematics

8/16/2019 PP16-lec1-intro1

11/181.11

"he ni?erse is Parallel:• Parallel computing is an e?olution of serial computing that

attempts to emulate =hat has al=a(s 7een the state of affairsin the natural =orl':

• man( comple , interrelate' e?ents happening at the sametime, (et =ithin a sequence.

•or e ample: ) #alaxy formation

) Planetary mo ement ) 'eather and ocean patterns

) (ectonic plate drift ) )ush hour traffic ) Automobile assembly line

) uilding space shuttle ) *rdering a hamburger

8/16/2019 PP16-lec1-intro1

12/181.12

ses for Parallel Computing:"o'a(, commercial applications pro?i'e an equal or greater 'ri?ing force in the 'e?elopment of fastercomputers.

"hese applications requireprocessing of large amount of 'atain sophisticate' =a(s .

+or example:•%atabases, data mining•'eb search engines,

Cloud computing•Ad graphics & irtual reality, esp entertainment industry•*il exploration•$edical imaging and diagnosis•Pharmaceutical design

•$anagement of national & multi!national corporations•+inancial & economic modeling• etwor-ed ideo & multi!media technologies•Collaborati e wor- en ironment

•Computational +luid %ynamics•% A research

8/16/2019 PP16-lec1-intro1

13/181.1

ho an' hat . • (op/001org pro ides statistics on parallel computing users in the charts below • Some things to note: Sectors may o erlap 2

) for example, research may be classified research1 )espondents ha e to choose between the two1 ) " ot Specified" is by far the largest application ! probably means multiple applications1

http://top500.org/http://top500.org/

8/16/2019 PP16-lec1-intro1

14/181.13

on eumann Architecture

Comprised of fourmain components:

MemoryControl UnitArithmetic Logic

UnitInput/Output

8/16/2019 PP16-lec1-intro1

15/181.1

serial computing 15

Traditionally, software has een written for serial computation:

To e run on a single computer ha!ing a single C"UA pro lem is ro#en into a discrete series of

instructions$Instructions are e%ecuted one after another$Only one instruction may e%ecute at any moment intime$

8/16/2019 PP16-lec1-intro1

16/181.16

parallel computing Problem

• "he computational pro7lem usuall('emonstrates characteristics such as the

a7ilit( to 7e: ) ro9en apart into 'iscrete pieces of =or9 that

can 7e sol?e' simultaneousl( ) ecute multiple program instructions at an(

moment in time ) 5ol?e' in less time =ith multiple compute

resources than =ith a single compute resource.

8/16/2019 PP16-lec1-intro1

17/181.1J

parallel computing 17 In the simplest sense, parallel computing is the simultaneous

use of multiple compute resources to sol!e a computationalpro lem: To e run using multiple C"UsA pro lem is ro#en into discrete parts that can e sol!edconcurrently

&ach part is further ro#en down into a series of instructionsInstructions from each part e%ecute simultaneously ondi'erent C"Us

8/16/2019 PP16-lec1-intro1

18/181 1

'hy 3se ParallelComputing.

• 4imits to serial computing: ) oth ph(sical B practical reasons pose

significant constraints to simpl( 7uil'inge?er faster serial computers @/oors Ea= :• Eimits to miniaturi+ation• "ransmission spee's

• Po=er 4issipation• nerg( consumption• conomic limitations