Warehouse-Scale Computers to Exploit Request-Level and ...cc.sjtu.edu.cn/Upload/20171116120138921.pdf · The substation switches from 115,000 volts to medium-voltage lines of 13,200

Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism

▪The datacenter is the computer• Luiz Andre Barroso, Google (2007)

Outline

▪ Introduction to WSCs

▪ Programming Models and Workloads

▪ Computer Architecture of WSCs

▪ Infrastructure and Costs of WSCs

▪ Cloud Computing

▪ Crosscutting Issues

▪ A Google WSC

Introduction

▪ Warehouse-scale computer (WSC)▪ Provides Internet services

▪ Search, social networking, online maps, video sharing, online shopping, email, cloud computing, etc.

▪ Differences with HPC “clusters”:▪ Clusters have higher performance processors and network▪ Clusters emphasize thread-level parallelism, WSCs emphasize

request-level parallelism

▪ Differences with datacenters:▪ Datacenters consolidate different machines and software into one

location▪ Datacenters emphasize virtual machines and hardware

heterogeneity in order to serve varied customers

Introduction

▪ Important design factors for WSC:▪ Cost-performance

▪ Small savings add up▪ Energy efficiency

▪ Affects power distribution and cooling▪ Work per joule

▪ Dependability via redundancy▪ Network I/O▪ Interactive and batch processing workloads▪ Ample computational parallelism is not important

▪ Most jobs are totally independent▪ “Request-level parallelism”

▪ Operational costs count▪ Power consumption is a primary, not secondary, constraint when designing system

▪ Scale and its opportunities and problems▪ Can afford to build customized systems since WSC require volume purchase

Introduction - Failures

▪ Outages and anomalies with the approximate frequencies of occurrences of a cluster with 2400 servers

Introduction

▪ The WSC goal is to make the hardware/software in the warehouse act like a single computer that typically runs a variety of applications.

▪ The largest cost in a conventional datacenter is the people to maintain it, whereas, in a well-designed WSC the server hardware is the greatest cost, and people costs shift from the topmost to nearly irrelevant.

Outline





▪ Cloud Computing


▪ A Google WSC

Programming Models and Workloads

▪ Batch processing framework: MapReduce

▪ Map: applies a programmer-supplied function to each logical input record▪ Runs on thousands of computers▪ Provides new set of key-value pairs as intermediate values

▪ Reduce: collapses values using another programmer-supplied function

▪ A generalization of the single-instruction, multiple-data (SIMD) operation


▪ Example:▪ map (String key, String value):

▪ // key: document name▪ // value: document contents▪ for each word w in value

▪ EmitIntermediate(w,”1”); // Produce list of all words

▪ reduce (String key, Iterator values):▪ // key: a word▪ // value: a list of counts▪ int result = 0;▪ for each v in values:

▪ result += ParseInt(v); // get integer from key-value pair▪ Emit(AsString(result));


▪ MapReduce runtime environment schedules map and reduce task to WSC nodes

▪ Availability:▪ Use replicas of data across different servers

▪ Replicas can help with read performance as well as with availability; with proper placement

▪ Use relaxed consistency:▪ No need for all replicas to always agree

▪ Workload demands▪ Often vary considerably


Annual MapReduce Usage at Google over time

Features of WSC

▪ To reduce operational costs of availability, all WSCs use automated monitoring software so that one operator can be responsible for more than 1000 servers.

▪

▪ Programming frameworks rely upon internal software services for their success ▪ Google FileSystem, BigTable, Dynamo…

Features of WSC

▪ The workload demands of public interactive services all vary considerably

▪ even a popular global service such as Google search varies by a factor of two depending on the time of day

▪ It is more important for servers in a WSC to perform well while doing little than to just to perform efficiently at their peak, as they rarely operate at their peak.

Average CPU utilization of more than 5000 servers during a 6-month period at Google.

WSC hardware and software must cope with variability in load based on user demand and in performance and dependability due to the vagaries of hardware at this scale

Outline





▪ Cloud Computing


▪ A Google WSC

Computer Architecture of WSC

▪ WSCs often use a hierarchy of networks for interconnection

Computer Architecture of WSC

▪ The 19-inch (48.26-cm) rack is the standard framework to hold servers

▪ Each 19-inch rack holds 48 1U servers connected to a rack switch▪ Bandwidth within the rack is the same for each server▪ It does not matter where the software places the sender and the

receiver

▪ Rack switches are uplinked to switch higher in hierarchy▪ Uplink has 48 / n times lower bandwidth, where n = # of uplink

ports▪ “Oversubscription”

▪ Goal is to maximize locality of communication relative to the rack

Storage

▪ Storage options:▪ Use disks inside the servers, or▪ Network attached storage (NAS) through Infiniband

▪ NAS solution is generally more expensive per terabyte of storage, but it provides many features, including reliability

▪ WSCs generally rely on local disks▪ Google File System (GFS) uses local disks and maintains at

least three replicas

Array Switch

▪ Switch that connects an array of racks▪ Array switch should have 10 X the bisection bandwidth of

rack switch

▪ Cost of n-port switch grows as n2

▪ Often utilize content addressable memory chips and FPGAs

WSC Memory Hierarchy

▪ Servers can access DRAM and disks on other servers using a NUMA-style interface▪ Each server has 16GB of memory ▪ Every pair of racks includes one rack switch and holds 80

2U servers▪ The array switch can handle 30 racks

WSC Memory Hierarchy (2)

▪ Network overhead increases latency from local DRAM to rack DRAM and array DRAM, but still have more than 10X better latency than local disk

▪ WSC needs 20 arrays to reach 50000 servers, one more level of the networking hierarchy is added.

Outline





▪ Cloud Computing


▪ A Google WSC

Infrastructure and Costs of WSC

▪ Location of WSC

▪ Proximity to Internet backbones

▪ Low electricity cost

▪ Low property tax rates

▪ Near Internet users

▪ Low risk from earthquakes, floods, and hurricanes


▪ Power distribution▪ UPS: uninterruptible power supply. Located in a

separate room from the IT equipment▪ Power conditioning, holding the electrical load when switching


▪ Efficiency of conversion▪ The substation switches from 115,000 volts to medium-voltage lines of

13,200 volts, with an efficiency of 99.7%.

▪ The efficiency of a large UPS is 94%, so the facility loses 6% of the power by having a UPS

▪ Power distribution unit (PDU) converts to low-voltage, internal, three-phase power at 480 volts. conversion efficiency is 98%.

▪ Another down step to two-phase power at 208 volts that servers can use, once again at 98% efficiency.

▪ The connectors, breakers, and electrical wiring to the server have a collective efficiency of 99%

▪ Overall efficiency▪ 99.7% * 94% * 98% * 98% * 99%=89%


▪ Cooling▪ Air conditioning used to cool server room using chilled water

▪ 18 C – 22 C▪ Keep temperature higher (closer to 22 C)

▪ fans push warm air past a set of coils filled with cold water and a pump moves the warmed water to the external chillers to be cooled down.


▪ Cooling (fans and water pumps that move air and water)


▪ Cooling towers can also be used▪ Leverage the colder outside air to cool the water before

it is sent to the chillers▪ Minimum temperature is “wet bulb temperature”

▪ Warm water flows over a large surface in the tower, transferring heat to the outside air via evaporation and thereby cooling the water. This technique is called airside economization.

▪ An alternative is use cold water instead of cold air. ▪ Google’s WSC in Belgium uses a water-to-water intercooler

that takes cold water from an industrial canal to chill


▪ Airflow is carefully planned. Efficient designs preserve the temperature of the cool air by reducing the chances of it mixing with hot air.

▪ Cooling system also uses water (evaporation and spills)▪ E.g. 70,000 to 200,000 gallons per day for an 8 MW facility

▪ Power cost breakdown:▪ Chillers: 30-50% of the power used by the IT equipment▪ Air conditioning: 10-20% of the IT power, mostly due to fans

▪ How man servers can a WSC support?▪ Each server:

▪ “Nameplate power rating” gives maximum power consumption▪ To get actual, measure power under actual workloads

▪ Oversubscribe cumulative server power by 40%, but monitor power closely


▪ Breaking down power usage inside the IT equipment itself▪ 33% of the power for processors

▪ 30% for DRAM

▪ 10% for disks

▪ 5% for networking

▪ 22% for other reasons (inside the server)

Measuring Efficiency of a WSC

▪ Power Utilization Effectiveness (PEU)▪ = Total facility power / IT equipment power▪ Median PUE on 2006 study was 1.69▪ The bigger the PUE, the less efficient the WSC

▪ Performance▪ Latency is important metric because it is seen by users▪ Cutting system response time 30% shaved the time of an

inter- action by 70%.


▪ Bing study: users will use search less as response time increases

▪ A high percentage of requests be below a latency threshold rather just offer a target for the average latency.

▪ Service Level Objectives (SLOs)/Service Level Agreements (SLAs)▪ E.g. 99% of requests be below 100 ms


Cost of a WSC

▪ Capital expenditures (CAPEX)▪ Cost to build a WSC

▪ Operational expenditures (OPEX)▪ Cost to operate a WSC

Outline





▪ Cloud Computing


▪ A Google WSC

Cloud Computing

▪ Datacenters tend to be utilized only 10% to 20% of the time. By making WSCs available to the public, uncorrelated peaks between different customers can raise average utilization above 50%.

▪ WSCs offer economies of scale that cannot be achieved with a datacenter:▪ 5.7 times reduction in storage costs▪ 7.1 times reduction in administrative costs▪ 7.3 times reduction in networking costs▪ This has given rise to cloud services such as Amazon Web

Services▪ “Utility Computing”

Cloud Computing - AWS

▪ Virtual Machines▪ Protect users from each other▪ Simplify software distribution within a WSC▪ Easy to control resource usage▪ Enable multiple price points▪ Hide the identity of older hardware▪ Introduce new and faster hardware

▪ Very low cost▪ Reliance on open source software

▪ No initial guarantee of service▪ No contract required – Pay-for-use

Cloud Computing

▪ Cloud computing providers take on the risks of over-provisioning and under-provisioning

▪ Cloud computing has made the benefits of WSC available to everyone▪ 1000 servers for a hour cost no more than 1 server for

1000 hours

Outline





▪ Cloud Computing


▪ A Google WSC

WSC network as a Bottleneck

▪ Large switches are manually configured and fragile at a large scale

▪ It is difficult to afford more than dual redundancy in a WSC using these large switches, which limits the options for fault tolerance

▪ WSC network bottlenecks constrain data placement and complicate WSC software

Energy Efficiency

▪ Another source of electrical inefficiency is the power supply inside the server

▪ Power supplies were 60%~80% efficient, thus there were greater losses inside the server▪ Supply a range of voltages to chips and disks…

Energy Efficiency

▪ Energy Proportionality – servers should consume energy in proportion to the amount of work performed

Outline





▪ Cloud Computing


▪ A Google WSC

Containers

▪ Google built WSCs using shipping containers. The idea of building a WSC from containers is to make WSC design modular

http:// www.google.com/corporate/green/datacenters/summit.html

Cooling

▪ By controlling airflow to prevent hot spots, the container can run at a much higher temperature.

▪ The cooling is below a raised floor that blows into the aisle between the racks

▪ Hot air is returned from behind the racks. The restricted space of the container prevents the mixing of hot and cold air, which improves cooling efficiency Two racks on each side of the container

Power

▪ Rather than have a separate battery room, each server has its own lead acid battery that is 99.99% efficient▪ Deployed incrementally with each machine, thus no money or power

spent on overcapacity

▪ Use standard off-the-shelf UPS units to protect network switches

▪ DVFS was not deployed to avoid quality-of-service violation for online workloads

Server

▪ Two sockets

▪ Eight DIMMs

▪ Single network interface card (NIC)

▪ Two sata disk drives

Networking

▪ 40000 servers are divided into three arrays of more than 10000 servers each

▪ The 48-port rack switches uses 40 ports to connect to servers, leaving 8 for uplinks to the array switches

▪ The number of uplink ports used per rack switch varies from a minimum of 2 to a maximum of 8▪ Applications with significant traffic demands beyond a rack tended to

suffer from poor network performance

Monitoring and Repair

▪ Use monitoring software to track the health of all servers and networking gear

▪ Diagnostics are running all the time. When a system fails, many of the possible problems have simple automated solutions

▪ Failed machines are addressed in batches to amortize the cost of repair

▪ Next Lecture: Prof. Wenli Zheng

谢谢！Thanks! Q&A

Warehouse-Scale Computers to Exploit Request-Level and ...cc.sjtu.edu.cn/Upload/20171116120138921.pdf · The substation switches from 115,000 volts to medium-voltage lines of 13,200

Documents