Trojans A Pentium/Linux PC Cluster for Internet-Based Multimedia Applications · 2000-09-07 · Trojans: A Pentium/Linux PC Cluster for Internet-Based Multimedia Applications! Pentium/Linux
Post on 21-Sep-2018
217 Views
Preview:
Transcript
Trojans : A Pentium/Linux PC Clusterfor Internet-Based Multimedia Applications
! Pentium/Linux PC Clustering
! SSI and Availability Middleware
! Superserver Consolidation
! Innovative Cluster Applications! Parallel 3-D Image Rendering
! Real-Time Video Scheduling
! Global Supply Chain Management
! Fast Gene/DNA Sequence Matching 1
Professor Kai HwangUSC Cluster and InternetComputing Laboratory
9/7/2000 , Kai Hwang, USC 2222
Research Team
and Collaborators
The USC Team:Dr. Hai Jin and Dr. Dan Meng, Visiting Scientists
Wonwoo Ro and Shu Xiao, Ph.D. candidates in Computer Engr.
Wei Yue, Ph.D. candidate in Computer Science
Kai Hwang, Professor of Elec. Engr. and Computer Science
Collaborators at Hong Kong Univ. :Dr. Ricky Kwok and Dr. Xiola Lin, Computer Engineering
Dr. Choli Wang, Computer Science and Information Systems
Sam Lin, Ph.D. candidate in Computer Engineering
Roy Ho, M. Phil. student at HKU, currently visiting USC
Clustering of Multiple PCs or Workstationsfor Distributed Supercomputing
Sept. 30, 1999, K. Hwang p.3
9/7/2000 , Kai Hwang, USC 4444
What is a Cluster of Computers?
2Physically, a cluster is a collection of computernodes interconnected by a system-area network(SAN) or by a local-area network (LAN), oftenhoused in the same building.
2 Logically, all computers in the cluster are gluedtogether with middleware support for collectiveusage as a single computing resource, inaddition to the traditional role as individualcomputers.
9/7/2000 , Kai Hwang, USC 5555
g Prototype has 16 Pentinum PCshoused in two 9-ft computer racks.
g All PCs run with the Redhat Linuxversion 6.0 (Kernel version 2.2.5)
g All 16 PC nodes are interconnectedby a 100 Mbps Fast Ethernet switch
g The cluster is ported with DQS,LSF, MPI, PVM, TreadMarks,Elias, and NAS benchmarks, etc.
g Scaling to a future system with 100’sor 1000’s of future PC nodes inter-connected by Gigabit networks
The USC Trojans Cluster ProjectInternet and Cluster Computing Lab., EEB Rm.104
Web site: http://andy.usc.edu/trojan/
9/7/2000 , Kai Hwang, USC 6666
Trojans: An I/O-Centric PC Cluster builtwith Pentium/Linux, Fast Ethernet, SSI,and Checkpointing Middleware at USC
Linux
Pentium
Linux
Pentium
Linux
Pentium
100 Mbps Fast Ethernet
Checkpointing and Availability Support
Single System Image Middleware
Programming Envi-ronments (Java, C,
Fortran, MPI, PVM)
Web WindowsUser Interface
Other Subsystems(Database, RAID,
OLTP, etc.)
9/7/2000 , Kai Hwang, USC 7777
Partitioning Trojan Nodes for Multimediaand I/O-Centric Web Server Applications
EntryPartition
(Internet/Intranet)
SAN(Fast Ethernet)
Console
Data/InfoPartition
ServicePartition
Service Flow Data Flow
9/7/2000 , Kai Hwang, USC 8888
Software Agents Running on A Clusterfor A Superserver Consolidation
LocalD atabase
H eterogeneousData Sources
PC /W orkstation
Laptop
NC
TV Top-SetBox/Page
JA VA -E nabledClient H osts
M ainfram e
Internet/Intranet
Internet/Intranet
Database
W eb site
U sing A PC /W orkstationC luster as A Supersever
U ser A gent(Applet A gent)
ServiceBroker A gent
ServiceAgent
Inform ationA gent
O therSuperserver(s)
C lient B roker Service D ata
9/7/2000 , Kai Hwang, USC 9999
Multi-Server Consolidation for DistributedMultimedia Processing with Guaranteed QoS
MM MultipointComm Server
MM ServersMM StorageServer
MM Transaction/Mgmt. Services
MM NetworkServices
MM EnablingNetworks
MM User Platforms
Desktops(Business)
Handhelds(Mobile)
Set-tops(Home)TV
PC
MAC
Workstation
Distributed SoftwareInfrastructure
Video Playback
Video Conf
Audio Conf
Media Acceleration Tech
9/7/2000 , Kai Hwang, USC 10101010
Scaled Workload leads to LinearSpeedup on The Trojan Cluster
9/7/2000 , Kai Hwang, USC 11111111
1. Parallel 3-D Image Rendering onCluster Nodes with Load Balancing
root
Node1
Node2
Noden
Node1
Node2
Noden
Node1
Node2
Noden
Node1
Node2
Noden
root
Display
SceneDescription
Pre-processing phase Transformationphase
Rasterization phase
Decomposedataset by
object
SimplifyModel
TransformModel
RasterizeModel
Collectpartialimages
Display
Geometricdatabase
Rendering Looping
Dataredistribution
Innovative Cluster Applications :Distributed Multimedia Processing
3-D Image Rendering Using AdaptiveSupersampling on Polygon Edges
• Data Structuretypedef struct {
Color rgb;Boolean SuperPixel;union {
int z;Subpixel * pblock;
} zOrpblock;short dxz, dyz;
} Pixel;
Polygon edge
16 blocks of Subpixel
1 Pixel
typedef struct {Color rgb;int z;
} Subpixel;Preallocated array of Subpixel
The memory usageon a supersampledpixel is fixed nomatter how manypolygons gothrough the pixel.
12
Parallel 3-D Image Rendering over 10Computer Nodes with Load Balancing
0
1
2
3
4
P1
P2
P3
P4
P5
P6 P7
P8
P9
P10 P
1
P2
P3
P4
P5
P6
P7 P8
P9
P10 P
1
P2
P3
P4 P5
P6
P7
P8 P9
P10 P
1 P2
P3
P4 P5
P6
P7
P8
P9
P10
Node Number
Ren
der
ing
Tim
e(s
ec)
Parallel Image Rendering without Load Balancing
Transformation Phase Data Redistribution Rasterization Phase
0
1
2
3
4
5
6
7
P1
P2
P3
P4 P5
P6
P7
P8
P9
P10 P
1 P2
P3 P4
P5
P6
P7
P8
P9
P10 P
1 P2
P3
P4
P5
P6
P7
P8
P9
P10 P1
P2
P3 P4
P5
P6 P7
P8
P9
P10
Node Number
Ren
der
ing
Tim
e(s
ec)
13
A c c e s sn o d e
s w itc h
s w itc h
s w itc hB a c k b o n e
n e tw o rk
S e r v ic eg a t e w a y
L o c a lA rc h iv e s
A c c e s s N e tw o r k
c lie n t
c lie n t
c l ie n t
V id e o a rc h iv e s
V id e o -o n -D e m a n d S y s te m A r c h ite c tu r e
C lu s te r e dV id e o S e r v e r
c l ie n t
c l ie n t
c l ie n t
2. Real-Time Scheduling of Video Streams with High QoSGuarantee in Video-on-Demand (VoD) Services
14
Rate-Based Scheduling with Bandwidth Divisionover Multiple Video Streams To Guarantee QoS
MovieFrames
ClusterServer
Clients
BandwidthDivision
Network
15
9/7/2000 , Kai Hwang, USC 16161616
0.02
0.06
0.1
0.14
0.18
0.22
2 3 4 5 6 7 8
number of video streams
Off
set
Qo
S
Average Offset from The Expected QoSOver Multiple Video Streams
Time-Division
Legend:Bandwidth-Division
3.Supply Chain Managementin Business Globalization.
17K.Hwang,USC, April 7, 1999
Distributed Multi-Agent Computingon the Trojans Cluster
18
Internet
Agent Name Server
OracleDatabaseWeb Browser
Agent
Agent
AgentAgent
AgentDatabase Server
1
2
3
4
56
7
Solving The Traveling SalesmanProblem on The Trojans Cluster
19
4. Bioinformatics : ComputationalMolecular Biology for Health Care
" Human genome research for health care,
disease control, and new drug development
" Parallel processing in the search, sorting,
and alignment of DNA/protein sequences
" Cluster of PCs/workstations for parallel
and distributed biological sequence analysis
Sept. 30, 1999, K. Hwang, p. 20
Internet-basedHumanBiologicalSequenceAnalysis onUSC Trojans
PC Cluster
http://www.sanger.ac.uk/HGP/
21
1. Prove the scalability and programmability of future clusters using low-cost commodity components with SSI services. These PC/WS clusters may eventually replace the high-cost servers and mainframes.
22
Concluding Remarks :
K. Hwang, USC, Sept.30, 1999
2. Support Java, Internet, multi-agent, multimedia, metacomputing, and many innovative cluster, Intranet, Internet, and Web applications in science, education, business, industry, and government with significant cut in cost:
" Distributed multimedia processingin electronic commerce, data-mining, digitalentertainment, and urban management.
" Dedicated digital Libraries for distanceeducation, bioinformatics for health-care,and economic crisis management
" Innovative applications in remote services,tele-medicine, collaborative designs, andenvironmental protection, etc. Sept. 30, 1999, K. Hwang 23
9/7/2000 , Kai Hwang, USC 24242424
Recent Publications:
[1] K. Hwang , et al, “ Designing SSI Clusters withHierarchical Checkpointing and Single I/O Space”IEEE Concurrency, March, 1999, pp.60-69.
[2] K. Hwang, et al, “ Trojans: An I/O-Centric ClusterArchitecture for Consolidated Web ServerApplications ”, submitted to IEEE Int’l Paralleland Distributed Processing Symposium,Cancun, Mexico, May 1-5, 2000.
[3] K. Hwang and Z. Xu, Scalable ParallelComputing, McGraw-Hill, N.Y., 1998.
9/7/2000 , Kai Hwang, USC 25252525
Frames
Movie
ClusterServer
ClientsTime
Time-Division Scheduling of MultipleVideo Streams To Yield Higher QoS
9/7/2000 , Kai Hwang, USC 26262626
Steps to Implement Single Entry Point inTrojans Cluster for Server Consolidation
R ound-R obin D N S
N am e resolverequest
1
2
IP add ress ofincom ing node
3 S ervicerequest
4 R equest d ispatchingto an outgoing node R eply from the
outgoing node5
U ser
In ternet/In tranet
9/7/2000 , Kai Hwang, USC 27272727
Multi-Agent System for SocioeconomicAnalysis and Crisis Management
InternetInformation
Sources
InformationAgents
SEIAgents
FI Agents
EPI Agent
User Agents
Local Storage(Database orData Files)
Backup
Historical DataManagement Agents
Users
GUI GUI
Three-TierAnalysis Model
Middleware for Web ServerConsolidation on a Cluster
Implementationlevel
Managementlevel
Programminglevel
Job Management System (JMS)(GLUnix, LSF, CODINE)
Single File Hierarchy (SFH)(NFS,AFS,xFS, Proxy)
Distributed Shared Memory (DSM)
(TreadMark, Wind Tunnel, etc)
Checkpointing /ProcessMigration (CPM)
Single Process Space(SPS)
Cluster Hardware and OS Platform
User Applications
Single I/O SpaceSIOS)
9
top related