The Design, Implementation and Performance Evaluation of ... · The Design, Implementation and Performance Evaluation of Xylophone: A Scalable QoS-enabled WiFi Locator Service Xinyu

The Design, Implementation and Performance Evaluation of Xylophone: A Scalable QoS-enabled WiFi

Locator Service

Xinyu Xing1, Jianxun Dang

2, Shivakant Mishra

1, Xue Liu

2

1University of Colorado at Boulder,

2McGill University

Contact: [email protected]

Department of Computer Science

University of Colorado at Boulder

Technical Report CU-CS 1072-10

July 2010

Procedia Computer Science 00 (2010) 1–22

Procedia ComputerScience

The Design, Implementation and Performance Evaluation ofXylophone: A Scalable QoS-enabled Wifi Locator Service

Xinyu Xinga, Jianxun Dangb, Shivakant Mishraa, Xue Liub,

aDepartment of Computer Science, University of Colorado at Boulder, United StatesbSchool of Computer Science, McGill University, Canada

Abstract

WiFi access points that provide Internet access to users have been steadily increasing in urbanareas. Different access points differ from one another in terms of services that they provide, in-cluding available upstream and downstream bandwidths, overall network capacity, open/blockedports, security features, and so on. However, there is no reliable service available at present thatcan aid a user in selecting an access point from the many that are available. This paper presentsxylophone, a WiFi locator service that enables users to select commercial hotspots in accordancewith the current quality of service of hotspot access points. The primary research challenge inxylophone is how to accurately estimate current quality of service of various access points in anefficient manner without requiring any installation of special software on the access points, andnot burdening the WiFi subscribers to perform any communication or computation intensive task.Xylophone has been extensively evaluated via a prototype implementation in an indoor testbedand in the Amazon’s EC2 platform. The evaluation demonstrates low latencies experienced byWiFi subscribers to measure DHCP connectivity, authentication and association process, and dis-cover open/blocked ports. Also, the bandwidth measurement component of xylophone exhibitshigh measurement accuracy, low latency, high scalability, and minimal intrusiveness.

Keywords: wireless access point selection, bandwidth measurement, scalability, cloud

1. Introduction

Commercial hotspots that offer Internet access over a wireless local area network throughthe use of a router connected to an ISP have been increasingly becoming popular in most publicareas. These hotspot WiFi services are fully-featured and time-tested. However, the New YorkTimes and The Wall Street Journal reported in 2005 that these commercial hotspots may notbe able to satisfy clients’ connectivity requirements. Consider the following scenario. When abusiness traveler checks into a hotel, connects to a WiFi network in the hotel and attempts to

/ Procedia Computer Science 00 (2010) 1–22 2

upload financial statements to a centrialized file server, he realizes that his WiFi device is expe-riencing poor connectivity due to limited Access Point (AP) backhaul bandwidth1 and firewallrestrictions. So, the traveler decides to use a WiFi locator service to search for alternative WiFinetworks nearby. Generally, he can find several alternatives with ease as commercial hotspotsare available in abundance in most locations in urban areas. However, apart from location andnavigation information, current locator services do not provide any other information. As a re-sult, the traveler has no way to find out in advance, i.e. before he pays for a connection, whichAP provides the best quality of service as per his needs.

To address this problem, [3] introduced a collaborative service called WiFi-reports that pro-vides WiFi clients with historical information about hotspot AP performance before the clientsdecide to pay for an access. However, [3] depends on voluntary collaboration among users, re-quiring users to report hotspot AP performance. It requires that volunteer users measure hotspotAP performance (connectivity capacity, bandwidth etc.) by running a third-party measurementsoftware on their mobile devices while accessing the Internet for their computing needs. Theusers then need to report their measurement results to a central server that maintains a historyof performance of various APs. Utility of such a historical based mechanism is based on theassumption that either the volunteer users report hotspot AP performance quite regularly or thehotspot AP performance remains approximately constant. Unfortunately, in the absence of anyincentives, it is unlikely that users will voluntarily report hotspot AP performance quite regu-larly. Furthremore, hotspot AP performance, particularly bandwidth can change quite frequentlyin general. Finally, bandwidth measurement is a communication and computation intensive task,and so, only a few users will be willing to run such software. When only a few clients participate,such collaborative services fail to provide an accurate and up-to-date perfomance characteristicsof various APs.

This paper introduces xylophone – a system prototype for commercial hotspot services –that operates as an extension to hotspot WiFi locator service. Like a hotspot WiFi locatorservice[4][5][6], xylophone allows a hotspot subscriber to search for available hotspots in a givenarea. In addition, xylophone enables hotspot subscriber to select commercial hotspots based ontheir current quality of service. Xylophone addresses the limitations of WiFi-reports by mov-ing the computation and communication intensive tasks such as bandwidth measurement to aseparate cloud platform. This is done by introducing a novel method for estimating the currentbandwidth of APs. Specifically, xylophone has several unique features. First, it provides thecurrent quality of service of hotspot APs in terms of current bandwidth as well as connectivitycapacity, open/blocked ports, etc. Second, unlike [3], xylophone does not require clients to per-form any compute or communication intensive tasks to measure bandwidth. Third, xylophoneintroduces a novel methodology for estimating hotspot AP backhaul bandwidth that does not re-quire voluntary user participation or hotspot APs to run any special software. Fourth, xylophoneis highly scalable in terms of simultaneously providing up-to-date performance information of asignificantly large number of APs by utilizing cloud services. Finally, xylophone utilizes efficienttechniques to estimate current performance, resulting in performance measurements completedwithin a few seconds.

While the goal of xylophone is to enhance selectivity for commercial hotspot APs, it can alsobe operated as a part of hotspot management system to monitor hotspot AP performance anddiagnose network faults. The contributions of the paper are listed as follows.

1Unless otherwise stated, bandwidth refers to the available bandwidth as defined in [1][2].


(1) A novel bandwidth estimation methodology is presented. This methodology not only es-timates the upstream and downstream bandwidths accurately, but also it is efficient (low latency),scalable and less-intrusive.

(2) A prototype of xylophone has been implemented on Amazon’s EC2 platform and its clientcomponent on a mobile device.

(3) An extensive evaluation of xylophone in terms of measurement accuracy, latency, scala-bility and intrusiveness has been done.

The rest of the paper is organized as follows. In Section 2, we summarize related workin the areas of wireless AP selection and bandwidth measurement. In Section 3, we presentan overview of xylophone and in Section 4, we describe xylophone in detail. In Section 5, wepresent a detailed evaluation of xylophone. Finally, in Section 6, we conclude the paper.

2. Related Work

There are two areas of research that xylophone builds on: improving wireless network selec-tion and bandwidth estimation.

Improving Wireless Network Selection. Several researchers have explored wireless networkselection in recent years. From Judd and Steenkiste’s load-sensitive AP selection algorithm[7]to WiFi-Reports[3], the objective has been to enable WiFi clients to choose hotspot APs thatprovide better performance. Virgil[8] exploits applications mounted on reference servers andclients to collaboratively measure end-to-end bandwidth between a client and a reference server.It then uses this end-to-end bandwidth to choose the best hotspot AP that can be detected ona scan. However, Virgil does not allow users to evaluate hotspot APs that are not within theirsignal range. [9] relies on a passive bandwidth measurement to select optimal hotspot APs. How-ever, this passive measurement requires modifications in the wireless driver to estimate upstreambandwidth. Furthermore, it also does not help a client to evaluate hotspot APs that are not withintheir signal range.

WiFi-Reports[3] harnesses user-submitted historical information about hotspot AP perfor-mance to help users determine which hotspot APs will be sufficient to run their applications. Itenables users to search for the best performance hotspot AP in a certain geographical region froma centralized database. WiFi-Reports improves upon earlier approaches in that it allows users toevaluate APs that may not be within the signal range of the users. However, WiFi-Reports suffersfrom several limitations. First, WiFi-Reports relies on voluntary participation of users. The keyissue is that there is no direct incentive for users to participate. Why will users run computationand communication-intensive tasks to measure the bandwidth on their devices? Indeed, it hasbeen shown that selfish (rational) users tend not to share their resources or time unless there is anexternal incentive. Second, this lack of incentives will likely result in very few users participatingvoluntarily. As a result, the central database that contains performance information of variousAPs will very likely contain outdated information. This is problematic because performance fac-tors such as bandwidth can change quite frequently. Finally, because WiFi-Reports relies on userparticipation, it needs to deal with the difficult issues of privacy of the users, and security andtrustworthiness of the information submitted by the users.

Xylophone is designed to address the limitations of these earlier approaches. Like some ofthe earlier approaches, xylophone does not require participation of APs (in terms of runningany special software) to estimate their current performance. Also, like WiFi-Reports, xylophoneadopts the strategy of a central database that users can query to obtain the performance of an


AP that may not be within their signal range. However, unlike WiFi-Reports, xylophone doesnot require a WiFi client to perform computation and communication-intensive tasks. Instead,it runs computation and communication-intensive tasks on a cloud platform. Also, unlike [9],xylophone does not require any modification to the wireless driver.

Bandwidth Estimation. There have been several methodologies and tools developed over thelast ten years for estimating bandwidth along a path. Most prior work requires installing specialsoftware at both ends of the path to be measured [10][11][1] [2][12][13]. This requirement sig-nificantly limits their applicability for our system, because it is unrealistic to install measurementsoftware on each hotspot AP. Our design options are confined to a non-cooperative environ-ment where measurement software is mounted solely on a cloud platform. To the best of ourknowledge, only a few measurement methodologies[14][15] can be utilized in a non-cooperativeenvironment.

Abget[14] exploits the properties of TCP, forces a remote host to generate a traffic flow ata certain rate and thus approximately estimates a range of end-to-end bandwidth. However,our previous study[15] indicates that Abget usually suffers from irreparable weakness especiallywhen the storage locations of large files alter. In addition, the applicability of Abget is greatlydependent upon large accessible files and it is impractical to crawl a large accessible file fromeach hotspot AP. Therefore, Abget is not suitable for xylophone. Abode[15] exploits ICMPmessages to probe hotspot AP backhaul bandwidth. However, it suffers from two fundamentallimitations. First, several empirical-based studies have shown that most ISPs block or constrainICMP messages. This can cause a serious bandwidth underestimate. Second, Abode is designedfor only upstream bandwidth estimation.

Bandwidth measurement methodology of xylophone addresses these limitations. It does notrequire any software installation on APs, does not involve crawling large files from APs, anddoes not rely on ICMP messages. Furthermore, it can accurately estimate the bandwidth in bothupstream and downstream directions in relatively smaller time periods.

3. Xylophone Overview

3.1. Challenges

To obtain performance metrics of each commercial hotspot AP, there are several unique chal-lenges confronting xylophone.

(1) Computation and communication-intensive measurement tasks: One of the most signif-icant tasks for xylophone is to estimate hotspot AP backhaul bandwidth. Though a WiFi useris able to perfrom bandwidth measurement by running a bandwidth estimation software [14][1]on his mobile device, continuous bandwidth measurement severely depletes user’s computationand communication resources. This may give rise to a situation where a hotspot subscriber maynot be willing to sacrifice his limited resources to perform computation and communication-intensive bandwidth measurement. Furthermore, bandwidth measurement may fail if two hotspotsubscribers simultaneously estimate bandwidth via a same hotspot AP[16]. Finally, bandwidthmeasurement launched by hotspot subscribers may involve potential security threats, because theinbound traffic to clients’ mobile devices may compromise clients’ security.

(2) Measurement intrusiveness: While some measurement tasks such as SNR measurementand security measurement can be performed passively, others such as bandwidth estimation haveto be performed actively. Bandwidth estimation usually involves some probe messages. Whenmeasurement latency increases, this may significantly impact hotspot subscribers’ connectivity


Hotspot Subscriber

Hotspot AP

Centralized Database

Downstream Link

Upstream Link

CLOUD

Figure 1: The xylophone architecture.

performance. Therefore, bandwidth estimation has to be efficient with low measurement latencyand sampling rate of bandwidth measurement cannot be high.

(3) Interference from middleboxes: Middleboxes, such as load balancers, intrusion detectionsystems and web proxies have an impact on the latency and flow of probe traffic sent as partof bandwidth measurement and the corresponding response messages. As a result, they intro-duce inaccuracy in the bandwidth being measured[17]. Thus, the bandwidth measurement mustaccount for these middleboxes, and compensate for the inaccuracies introduced.

(4) Scalability: According to a recent survey from Jiwire[6], there may be hundreds ofhotspots in a certain region, e.g. there are 877 of hotspots in New York. Since xylophone assignsbandwidth measurement task to a centrialized infrastructure, it becomes challenging to estimatebandwidth of these large number of hotspot APs simultaneously. Simply using one single serverto perform bandwidth measurement can be problematic due to limited outbound bandwidth andcomputation resources.

3.2. The xylophone architectureThere are three logical entities in the xylophone architecture (see Figure 1): (1) a centralized

database; (2) wireless hotspot subscribers; and (3) one or more bandwidth measurement serversin the cloud. The centralized database stores a history of performance characteristics of all APs.It exports two sets of interfaces: one interface that bandwidth measurement servers and hotspotsubscribers can use to update its content with new performance values of various APs; and theother interface that clients may use to query the current and past performance of various APs.The centralalized database of xylophone is built using standard database techniques, and we willnot discuss it anymore in the paper.

Performance characteristics of APs provided by xylophone are classified into two categories:collaborative and non-collaborative. Performance characteristics in the collabortive categoryare those that require low communication and computation for measurement, and whose valuesremain relatively stable, e.g. set of blocked TCP ports, DHCP acquisition latency, authenticationand association latency, and wireless factors2. A subscriber generates a measurement report

2A number of tools[18] exist to estimate these wireless factors. These tools are outside the scope of this paper.


containing the values of these performance characteristics, and automatically submits it to thecentralized database. This design ensures that hotspot subscribers do not have to run computationor communication intensive software on their mobile devices. Furthermore, since they reportonly those performance characteristics whose values are relatively stable, xylophone does notrequire frequent reports from these subscribers, or even a participation from a large number ofsubscribers.

We have developed a client module to run on mobile devices. This module initiates a mea-surement when a mobile device launches a connection to a hotspot WiFi network. Performanceinformation about wireless factors, DHCP acquisition latency, authentication and association la-tency is collected passivley during the process of connection establishment. No intrusive trafficis injected into hotspot WiFi network to collect these information. In addition, information aboutconnectivity capacity – Block/Open ports – is collected by sending probes to several applicationports on the cloud under our control. Our current implementation sends probes to 127 commonapplication ports in parallel. These include FTP, SSH, IMAP, BitTorrent, World of Warcraft etc.The basic operation of port scan is as follows. A SYN packet is sent to our cloud. Since allthe common application ports are open in the cloud, the cloud should respond with a SYN-ACKpacket. However, the client module will receive a RST packet, if the corresponding hotspot APfilters packets sent to a certain port. Since port scan requires connecting to the cloud, this can bedone only after the mobile device has successfully connected to the Internet.

Performance characteristics in the non-collaborative category are those that require high com-munication and computation for measurement, and whose values change frequently. These in-clude upstream and downstream bandwidths. Bandwidth measurement servers reside on a cloudand run xylophone’s bandwidth measurement software. They update the contents of the central-ized database with new bandwidth measurements of various APs. We discuss the design andimplementation of these bandwidth servers in detail in Section 4.

3.3. Trust assumptionXylophone assumes that hotspot subscribers do not submit fraudulent measurement results.

We expect this trust to result from an existing falsification detection scheme [3]. By using thisscheme, xylophone can identify fraudulent submissions as well as corresponding counterfeiters,and thus blacklist the counterfeiters. In addition, we further assume that hotspot subscriberscannot masquerade other subscribers to submit fraudulent measurements, as commercial hotspotproviders usually assign their subscribers a unique user name and password to control their ac-cess.

3.4. Network modelMost commercial hotspots use ADSL technique to connect their local wireless networks to

the Internet3. As a result, IP addresses assigned to hotspot APs can change frequently. ISPsusually use SessionTimeout to limit the maximum lifetime of an IP address and typically setsit to 24 hours[19]. In a commercial hotspot scenario, commercial hotspots usually belong to acertain ISP. So, even though a hotspot AP has dynamic IP address, xylophone that is deployed byan ISP can still track its current address.

[20] illustrates that the bandwidth bottleneck of the Internet is usually on the edge of theInterenet (i.e. 2-hop away) and relatively persistent. This observation was further verified in

3AT&T hotspots where we conducted our experiments use ADSL technique to connect to the Internet.


0

100

200

300

400

500

600

0 20 40 60 80 100

Rou

nd tr

ip ti

me

(ms)

Packet sequence number

Sending rate R = 40KB/sSending rate R = 80KB/s

Sending rate R = 100KB/sSending rate R = 130KB/s

Figure 2: RTT variation with varying TCP ACK data rate.

the context of wireless networks in [15]. With this observation, xylophone estimates hotspot APbackhaul bandwidth instead of wireless throughput. Another reason that we estimate hotspotAP backhaul bandwidth is that the ADSL-support maximum throughput is far less than wirelessthroughput, even after taking the potential wireless signal interference into consideration. Addi-tionally, the distinguishing characteristic of ADSL technology is that bandwidth is greater in thedownstream direction than the reverse[21].

Behind hotspot APs, ISP usually deploys a firewall to block unauthorized access. We assumethat this firewall does not block our probe messages. In all of our experiments, this assumptionhas been true. In case xylophone is deployed by an ISP, the ISP may set firewall rules to ensurethat these messages are not blocked.

4. Bandwidth measurement servers in the cloud

The design, implementation and evaluation of the bandwidth measurement servers are majorcontributions of this paper. Recall that one major goal is to build a bandwidth measurementmodule that is highly accurate and scalable, and has low latency and minimal intrusiveness. Wefirst describe the methodology used to measure the upstream and the downstream bandwidthsand then describe a prototype implementation in the Amazon’s EC2 platform.

4.1. Bandwidth measurement methodology

Bandwidth measurement in xylophone is based on TCP protocol behavior. In general, when ahost sends a TCP acknowledgement (TCP ACK) packet to another host and if a TCP connectionhas not been established between them, a TCP reset (TCP RST) message with the fixed lengthof 40 bytes is sent in response regardless of the size of the TCP ACK packet. Since TCP ACKand RST packets traverse the network in two different directions (i.e. TCP ACK packets aretransmitted from the bandwidth measurement server to hotspot APs - downstream link; TCPRST packets are transmitted in the reverse direction - upstream link), we can saturate eitherdownstream or upstream link by adjusting the size of the TCP ACK packet or the interval betweenback-to-back ACK packets. Bandwidth measurement methodology consists of increasing thesending data rate either by increasing the ACK packet size or the interval between back-to-backACK packets, and observing the round-trip time (RTT). Here, we define RTT of a TCP ACKpacket as the time elapsed between the time when the ACK packet is sent and the time when


0

50

100

150

200

250

300

0 1000 2000 3000 4000 5000

Freq

uenc

y

Random Variable (Timestamp of a TCP RST message)

Uniformly Distributed Random Variable

0

2

4

6

8

10

0 20 40 60 80 100

Freq

uenc

y

Random Variable (Timestamp of a TCP RST message in a certain interval)

Uniform Distribution after mod 100

0.00

50.00

100.00

150.00

200.00

250.00

0 2000 4000 6000 8000 10000 12000 14000

Freq

uenc

y


Normally Distributed Random Variable

0

2

4

6

8

10

0 20 40 60 80 100

Freq

uenc

y


Normal Distribution after mod 100

0.00

50.00

100.00

150.00

200.00

250.00

300.00

350.00

0 1000 2000 3000 4000 5000

Freq

uenc

y


Exponentially Distributed Random Variable

0

2

4

6

8

10

0 20 40 60 80 100

Freq

uenc

y


Exponential Distribution after mod 100

Figure 3: Remainder Distribution of modulo operation over uniformly, normally, exponentially distributedrandom variable.

the corresponding RST packet is received. The key idea is that when none of the links aresaturated, RTT does not change with increase in the data rate. On the other hand, when one ofthe links is saturated, RTTs between successive TCP ACK packets will have an overall increasingtrend. This is because queuing delays start increasing when one of the link has been saturated.The bandwidth of the saturated link is then estimated as the sending data rate when RTT startsincreasing from being same.

As an example, Figure 2 illustrates the variation in RTT for a sequence of ACK packets fordifferent data rates. In this experiment, a bandwidth measurement server in the cloud sends 100TCP ACK packets at different data rate on a 75 KB/s link. As shown in the figure, when sendingrate is lower than 75 KB/s, i.e. for data rate 40 KB/s, the RTT of each TCP ACK packet is samefor ACK packets. On the other hand, RTTs increase for each successive ACK packet when datarate is above 75 KB/s, i.e. for data rates 80, 100 and 130 KB/s. Furthermore, this increase inRTT is higher for higher sending data rates.

As mentioned earlier, upstream link bandwidth is lower than downstream link bandwidth.So, to measure the upstream link bandwidth, we saturate that link by increasing the rate at which40B ACK packets are sent. As this rate is increased, the rate at which RST is sent also increases.Since upstream link bandwidth is lower than the downstream bandwidth, it saturates before thedownstream link. On the other hand, to saturate the downstream link bandwidth, we saturate thatlink by increasing the size of the ACK packet sent at some predetermined rate. This ensures that


the data rate on the upstream link doesn’t change, but the data rate on the downstream link keepsincreasing.

In practice, a traffic flow may not follow the following two assumptions that most bandwidthmeasurement tools make: (1) FIFO queuing is adopted at all routers along a path, and (2) atraffic flow follows a single routing path between the two end hosts. If the communication pathbetween two hosts varies significantly in a short space of time, e.g. flow splitting and mergingincur message out-of-order, we may see unexpected fluctuations in RTT. To address this, weadopt the concept of time centroid.

Assume a TCP RST stream, generated at a certain rate, contains n + 1 TCP RST messages:m0,m1,m2, ...,mn. Let ti(i = 0, ..., n) denote the absolute time stamp of TCP RST message mi.Hence, ∆ti = ti − t0 is the relative time stamp of mi from the time instance of the first TCP RSTmessage m0. In addition, ∆ti also represents the offset of TCP RST message mi from the startingpoint of the TCP RST stream.

Given any particular sequence of relative time stamps of TCP RST messages ∆t0,∆t1,∆t2, ...,∆tnand a random interval length T (T > 0 and T � ∆tn − ∆t0), ∆′ti = ∆ti mod T represents TCPRST message mi’s offset from the starting point of its interval. Further, ∆′ti is approximatelyuniformly distributed in the range (0,T ). For example, Figure 3 shows the empirical distribu-tions of the remainders of modulo 1000 operations over uniformly, normally and exponentiallydistributed random variables respectively. As we can see from Figure 3, the remainder of modulooperation over random variables of different distributions is approximately uniformly distributed.Consequently, for a TCP RST stream with sufficiently large number of TCP RST messages, atany interval length T which has T > 0 and T � tn − t0, the relative positions of TCP RSTmessages within their respective intervals (∆′ti) are uniformly distributed.

Since a TCP stream experiences queuing delays when sending rate of the TCP stream goesabove maximum bandwidth, estimating bandwidth is to determine whether a TCP stream expe-riences queuing delays. Hence, we define the centroid of a time interval as follow:

cent(T j) =1n j

∑∆′ti (1)

where T j is the jth time interval and j = 1, ..., ∆tnT

4; n j is the number of TCP RST messages in the

jth time interval and∑ ∆tn

Tj=1 n j = n. Furthermore, ∆′ti ranges from 0 to T . In case there is no TCP

RST messages in the interval T j, we define cent(T j) to be T2 . Extending the concept of centroid

to a complete TCP RST stream, we then have

cent =T

∆tn

∆tnT∑

j=1

cent(T j) (2)

Due to the fact that each interval T j is uniformly distributed when queuing delays do not occurand a TCP stream follows a single routing path, the centroid of any time interval is approximatelyin the middle of the interval, i.e. cent(T j) = T

2 . Furthermore, the centroid of a complete TCPRST stream is equal to T

2 . This property is stated and proved as follows.

Theorem 1. If the sending rate of a TCP ACK stream R0 is less than maximum bandwidth A (i.e.R0 ≤ A), then cent = T

2 .

4We tune the value of T and make ∆tn divisible by T .


tk+1 tk+2tk tk+3

tk’ tk+1

’ tk+3’tk+2

’

tk+(n−1)

tk+(n−1)’

Measurement server

Measurement server

Hotspot AP

TC

P A

CK

TC

P A

CK

TC

P A

CK

TC

P A

CK

TC

P A

CK

TC

P R

ST

TC

P R

ST

TC

P R

ST

TC

P R

ST

TC

P R

ST

t

t

TC

P A

CK

TC

P R

ST

...... ...... ...... ......

...... ...... ...... ......

T

Figure 4: A bandwidth measurement server in the cloud sends a TCP ACK stream to a hotspot AP, and then the hotspotAP responds back to the server with a TCP RST stream, when R0 ≤ A.

Proof. Suppose that tk is the absolute time stamp of TCP ACK message k and δk is the RTTof the TCP ACK message. Thus, the arrival time of the corresponding TCP RST message t′k =

tk + δk. When R0 ≤ A, δk = d0 for k ∈ 1, 2, .... Since the sending interval for back-to-backTCP ACK messages is τ = tk+1 − tk, the arrival interval for back-to-back TCP RST messages ist′k+1 − t′k = tk+1 + δk+1 − tk − δk = τ. Furthermore, we assume time interval T . Over the interval[t′k, t

′k + T ), n TCP RST messages arrive (see Figure 4) and so,

cent(T ) =1n·

n−1∑j=0

t′k+ j =1n·

n · (n − 1)2

· τ =(n − 1) · τ

2=

T2

(3)

Consequently, the centroid of a TCP RST stream is

cent =N · cent(T )

N=

T2

(4)

where N is the number of interval in a TCP RST stream.

Recall the assumption that we makes in Theorem 1: a TCP stream follows a single routingpath. However, this assumption is not always true in a real network [17]. A TCP stream maybe split or merged, which causes slight transmission delays are imposed upon some packets inthe TCP stream. The following theoretical proof illustrates that the centroid of a stream approx-imately remains T

2 even though stream splitting and merging happen.

Theorem 2. When the sending rate of a TCP ACK stream R0 is less than maximum bandwidthA (i.e. R0 ≤ A), stream splitting and merging will not skew the centroid of a TCP RST stream.

Proof. Suppose the centroid of an interval fluctuates, the centroid of the interval becomes T2 + o

where o is the offset of the expected centroid. Then the centroid of a TCP RST stream is

cent =T2

+T

∆tn· o (5)

If ∆tnT is large enough, the value of T

∆tn· o approaches zero.


tk+1 tk+2tk tk+3

tk’ tk+3

’tk+2’tk+1

’ tk+(m−1)’

tk+(m−1)

Measurement server

Measurement server

Hotspot AP

TC

P A

CK

TC

P R

ST

t

t

TC

P A

CK

TC

P A

CK

TC

P A

CK

TC

P R

ST

TC

P R

ST

TC

P R

ST

TC

P A

CK

TC

P R

ST

...... ...... ......

...... ...... ......

T

Figure 5: A bandwidth measurement server in the cloud sends a TCP ACK stream to a hotspot AP, and then the hotspotAP responds back to the server with a TCP RST stream, when R0 > A.

Intuitively, when a bandwidth measurement server saturates either upstream or downstreamchannel of a hotspot AP, a queuing delay is imposed upon a TCP stream. Therefore, there is anobvious centroid offset for a TCP RST stream. This property is stated and proved formally asfollows.

Theorem 3. When the sending rate of a TCP ACK stream R0 is greater than maximum band-width A (i.e. R0 > A), the centroid of a TCP RST stream skews from T

2 to T2 + d, where d is an

offset of the centroid.

Proof. In this case, the arrival time t′k for TCP RST message k is the sum of the absolute timestamp of corresponding TCP ACK message k and RTT of the TCP ACK δk, i.e. t′k = tk +δk wheretk is the absolute time stamp of corresponding TCP ACK message k (see Figure 5). Different fromTheorem 1 where δk is a constant (i.e. δk = d0 for k ∈ 1, 2, ...), δk is a variable when R0 > A,since each TCP message may experience a variable queueing delay. Therefore, the arrival timefor back-to-back TCP RST messages is denoted as follow

t′k+1 − t′k = tk+1 + δk+1 − tk − δk = τ + (δk+1 − δk) = τ + αk (6)

where τ = tk+1 − tk and αk = δk+1 − δk with αk > 0. Over the interval [t′k, t′k + T ), the centroid for

m TCP RST messages is

cent(T ) =1m·

m−1∑j=0

t′k+ j =1m· [

m · (m − 1)2

· τ +

m−1∑j=1

α j] =τ · (m − 1)

2+

1m·

m−1∑j=1

α j =T2

+ di (7)

where di = 1m ·∑m−1

j=1 α j and d =∑N

i=1 di

N (N denotes the number of intervals in a TCP RST stream).Consequently, the centroid of a TCP RST stream is

cent =N · T

2 +∑N

i=1 di

N=

T2

+ d (8)

4.1.1. VerificationTo verify our theorems, we investigate two hotspot APs that have the same downstream band-

width (450 KB/s) and different upstream bandwidths (130 KB/s and 65 KB/s). Two measurement


0

200

400

600

800

1000

10 20 30 40 50 60 70 80 90 100

Cen

troi

d sk

ew (

ms.

)

Sending rate (KB per sec.)

Figure 6: Centroid skew when bandwidth is 130 KB/s.

0

200

400

600

800

1000

10 20 30 40 50 60 70 80 90 100

Cen

troi

d sk

ew (

ms.

)

Sending rate (KB per sec.)

Figure 7: Centroid skew when bandwidth is 65 KB/s.

servers in the Amazon’s EC2 platform send TCP ACK packets of size 40 bytes to the two hotspotAPs respectively with the varying data rate, ranging from 10 KB/s to 100 KB/s. Figure 6 and 7show the fluctuation of the centroid of the TCP RST stream around the time centroid for the APwith 130 KB/s and 65 KB/s upstream bandwidths respectively. We notice that this fluctuationis on both sides of the time centroid for 130 KB/s upstream link, where the sending rate is lessthan the bandwidth. On the other hand, for the 65 KB/s upstream link, when the sending rateis increased to 70 KB/s, the centroid of the TCP RST stream experiences a sudden jump. Inaddition, the centroid skew continues an upward trend beyond this sending rate. Therefore, toestimate the bandwidth, the measuremet servers monitor the variation of the centroid of the TCPRST stream and locate the point of sudden jump.

To locate the point of sudden jump in centroid, we use a binary search algorithm. Initially, weset lower and upper bounds (Rlower and Rupper) and a termination condition (Rupper − Rlower ≤ R0,where R0 is the measurement resolution, which is the nearest bounding range between the lowerand upper bound). The specific values for R0, Rlower and Rupper are discussed in the next section.The search process iteratively adjusts the lower and upper bounds until Rupper−Rlower is less thanthe measurement resolution R0. The measurement server sends a TCP ACK stream at a certainrate R(n) = (Rupper + Rlower)/2. If a centroid skew appears, the server adjusts the upper bound toRupper = (Rupper + Rlower)/2. On the other hand, if no centroid skew appears, the server adjuststhe lower bound to Rlower = (Rupper + Rlower)/2. This process is repeated until the terminationcondition is satisfied. The sending rate at this time is the estimated bandwidth.

4.2. Prototype implementationWe have implemented and experimented with two different prototypes of our bandwidth mea-

surement servers. The first one is implemented in an indoor testbed at McGill University. Wehave used this prototype to evaluate the accuracy of our measurement methodology and estimatedthe value of measurement resolution, R0. Next, to test the scalability of our methodology, wehave implemented a prototype in the Amazon’s EC2 platform [22]. Current EC2 platform offers250 Mbps bandwidth for each instance (virtual machine), which guarantees that the bandwidthestimation component running on each instance can obtain a large enough outbound bandwidth.Since the outbound bandwidth of an instance is far larger than the outbound and inbound band-width of a hotspot AP (250 Mbps vs. 12 Mbps), an instance is capable of estimating hotspot APbackhaul bandwidth.

Our xylophone system in the cloud consists of three components: the hardware infrastruc-ture, a distributed framework and a bandwidth estimation component. The hardware infrastruc-


ture serves as the underpinning of the cloud, providing support for computing, networking andstorage. EC2 is employed in our implementation as the hardware infrastructure. We harness 11medium High-CPU instances (one instance serves as master, the other ten serve as slaves), eachof which serves as a measurement server in the cloud. Each instance is a 32-bit platform with5 EC2 Compute Units5 (2 virtual cores with 2.5 EC2 Compute Units each), 1.7 GB of Memory,350 GB of local instance storage and 250 Mbps network capability [23]. To organize and managethese computing resources, we need a distributed framework to connect them together. Thus, ourprototype implementation chooses Hadoop [24] to serve as our distributed framework. Finally,the bandwidth measurement component runs on each instance.

At a high level, our xylophone system in the cloud works as follows. We first assign mea-surement tasks to instances. In our prototype implementation, measurement tasks are describedin a set of text files. Each of the files contains the IP addresses of hotspot APs and correspond-ing measurement parameters (e.g. measurement sampling rate and measurement resolution etc.).The master instance in the cloud (Hadoop’s JobTracker) uniformly assigns the text files (i.e.measurement tasks) to those slave instances in the cloud (Hadoop’s TaskTrackers). Followingthe description in the text file that the master instance assigns, a Hadoop’s TaskTracker runsthe bandwidth measurement component, collects hotspot AP backhaul bandwidth and outputsestimation results to another text file, in which bandwidth estimation results and estimation la-tencies are listed. Since output files are stored in Hadoop Distributed File System (HDFS), theHadoop’s JobTracker is able to access these output files, incorporate them into a measurementreport and store the report into our centralized database. While the operation process is relativelystraightforward, several important practical issues must be taken into consideration.

(1) Automatic task assignment in Hadoop platform is dependent upon the size of the inputfile. By default, HDFS cuts a huge file into 128 MB chunks. Each chunk is automaticallyassigned to one TaskTracker. However, xylophone cannot utilize automatic task assignment.Recall that the purpose of our xylophone system is to estimate hotspot AP backhaul bandwidth.Assume that tens of thousands of measurement tasks are described in a single file. The sizeof the file is at most several megabytes. This can give rise to a curious situation where all themeasurement tasks are assigned to one TaskTracker to handle. To address this problem, wemanually divide all the measurement tasks into a set of files. Thus, Hadoop platform can treatthese files as separate HDFS chunks and distribute the files (i.e. measurement tasks) to multipleTaskTrackers. As a future work, we will automate this process.

(2) Since our measurement tasks are performed by instances in the cloud, it is possible thatmultiple instances share a single physical machine. Hence, instances may experience suspend-ing and resuming. Once an instance is suspended, the measurement component running on theinstance will be interrupted until the instance resumes. In practice, we observe that inter-instancesuspension times for EC2’s small default instances are large enough to invalidate our bandwidthestimation results, whereas medium High-CPU instances have much shorter suspension times.Consequently, we choose medium High-CPU instances, rather than the small default ones.

(3) The NICs (Network Interface Cards) in EC2 instances are also virtual devices. Therefore,the driver and the networking protocol stack in EC2 might be slightly different from those regulardriver and networking stack in physical machines. According to our experiment, we observethat a virtual NIC driver does not timestamp incoming packets. This can cause the incomingpackets stay at the system buffer for a long time without time stamps. To address this problem,

5One EC2 Compute Unit equals 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.


0 1 2 3 4 5 6 7 8 9

10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12

Est

imat

ion

band

wid

th (

Mbp

s)

Actual bandwidth (Mbps)

r = 0.1r = 0.5r = 1.0r = 1.5r = 2.0r = 4.0

Figure 8: Bandwidth estimation.

0 1 2 3 4 5 6 7 8 9

10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12

Cal

ibra

ted

estim

atio

n ba

ndw

idth

(M

bps)

Actual bandwidth (Mbps)

r = 0.1r = 0.5r = 1.0r = 1.5r = 2.0r = 4.0

Figure 9: Calibrated bandwidth estimation.

we implement dual threads – one thread for sending outgoing TCP ACK packets and the otherfor receiving incoming TCP RST packets. This enables us to timestamp each incoming packetinstantly and precisely.

(4) To enhance reliability of xylophone system, some environment parameters of Hadoopplatform must be set properly. In particular, mapred.task.timeout should be set to 300, indicatingthat a measurement task should be canceled by the TaskTracker if it cannot output measurementresults within 300 seconds. Note that a measurement task contains at most 5 hotspot APs in ourcurrent deployment and estimating bandwidth of one hotspot AP at most takes 60 seconds. Inaddition, we further set setNumMapTasks to 1. It indicates that at most one measurement taskruns on a TaskTacker simultaneously. This is because if multiple measurement tasks run on oneinstance simultaneously, they may experience interference with each other.

5. Evaluation

We have evaluated xylophone extensively using our two prototype implementations. We haveevaluated the following aspects – measurement accuracy, latency, scalability, intrusiveness andsampling rate.

5.1. Testbed

We have deployed an indoor testbed in McGill university to experiment with and evaluatethe bandwidth estimation component of xylophone system in terms of accuracy, latency andintrusiveness. In addition, we have used this testbed to derive optimal values of some of theparameters used. The testbed consists of a Cisco Aironet 1131 AG series wireless AP, a Linuxbox and a Dell OptiPlex 980 Mini Tower. The wireless AP is used as a hotspot AP in ourexperiments and a virtual machine running on the Mini Tower is used to run the bandwidthestimation task. To emulate an asymmetric connection, we used the linux box as a traffic shaper.It bridges the wireless AP and the Mini Tower. The traffic passing through the Linux box isshaped using the Linux bandwidth (tbf) and latency (netem) shaping modules. In addition, wealso experimented with 50 AT&T commercial hotspot APs in Denver, CO area to examine theperformance of xylophone system in the context of real commercial hotspot environment.


Resolution (r = α) Correlation coefficient ar=α br=α (Mbps)α = 0.1 1.000000 0.9447 0.0755α = 0.5 1.000000 0.9375 0.2344α = 1.0 1.000000 0.9463 0.0891α = 1.5 1.000000 0.9375 0.4687α = 2.0 0.990300 0.9572 0.3409α = 4.0 0.953400 0.9572 0.3409

Table 1: The corelation coefficients between the actual bandwidth and estimation and calibration coefficients for differentvalues of measurement resolution.

5.2. Measurement accuracy

We investigate the accuracy of bandwidth estimation using our indoor testbed. By using thebandwidth shaping module (tbf), we emulate different hotspot AP backhaul bandwidth. In thisexperiment, we set up upstream bandwidth of the AP to 500 Kbps and vary downstream band-width of the AP from 1 Mbps to 12 Mbps. The bandwidth estimation component running onthe Mini Tower sends probes to estimate downstream bandwidth with different measurement re-sultions. Figure 8 plots our estimated bandwidth as a function of actual bandwidth for differentvalues of measurement resolution. Overall, higher the measurement resolution is, less accurateour estimation is. In addition, we make an interesting observation from this figure. The estima-tion errors for each measurement resolution are consistent over bandwidth variation. Thus aninteresting question is: Are the estimations linearly correlated with each other? To answer thisquestion, we take the actual bandwidth in this experiment as the reference and calculate Pearsonproduct-moment correlation coefficients between the actual bandwidth and our estimation for dif-ferent values of measurement resolution. The calculation results are shown in the second columnof Table 1, which validates that estimation results are highly correlated with actual bandwidth.Taking the actual bandwidth as reference, we can thus calibrate our estimation results throughlinear transformations. Let B′r=α and Br=α denote the calibrated estimation result and originalestimation results for measurement resolution r = α respectively. The linear transformation fromBr=α to B′r=α is

B′r=α = ar=α · Br=α + br=α (9)

called calibration formula, where ar=α and br=α are the calibration coefficients for measurementresolution r = α. Using linear regression, we can calculate the value of ar=α and br=α. The lasttwo columns of Table 1 list the calibration coefficients. Figure 9 shows the estimated bandwidthusing this calibration.

For further comparison, we calculate the mean square error (MSE). Figure 10 plots the cumu-lative distribution of MSE before and after calibration. We can see that our estimation accuracyis significantly improved through calibration. These calibration formulas that we obtain fromour indoor testbed will be further utilized in our real deployment. In addition, we can see thatthe calibrated results do not indicate a great improvement when the measurement resolution goesabove 1.5. Therefore, our prototype implementation uses measurement resolution=1.5.

To further investigate the accuracy of our bandwidth estimation component, we have experi-mented with measuring the bandwidths of several commercial hotspot APs that are not under ourcontrol. Since these APs are not under our control, we have no way of knowing the actual hotspot


0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.5 1 1.5 2 2.5 3 3.5

Cum

ulat

ive

dist

ribut

ion

Mean Square Error (Mbps)

Before calibrationAfter calibration

Figure 10: Mean square error of bandwidth estimationbefore and after calibration.

7

7.2

7.4

7.6

7.8

8

0 1 2 3 4 5 6 7 8 9 10 11 12 13

Ban

dwid

th (

Mbp

s)

Measurement #

EstimationMRTG

Figure 11: Comparison between bandwidth measuredusing MRTG and xylophone.

AP backhaul bandwidths that we can compare with our measurements. Instead, we compare ourmeasurement results with bandwidth measured using another popular tool, MRTG[25]. We runMRTG behind a hotspot AP. MRTG measures hotspot AP backhaul bandwidth by monitoringincoming and outgoing traffic within a 5-minute time slot. Notice that bandwidth estimationcomponent of xylophone takes far less estimation latency. To compare these short-term estimateswith MRTG measurement, we continuously perform bandwidth estimation from an instance inthe cloud for 5 minutes. Based on these measurements over a 5-minute time slot, we calculatean average 5-minute-bandwidth, R̄ as follows:

R̄ =

K∑i=1

Ti

5× Ri (10)

where K is the number of times the bandwidth is estimated in the 5-minute interval; Ti and Ri

are the estimation latency and the estimated bandwidth respectively in the ith time during the5-minute interval.

Figure 11 shows the bandwidth estimated using MRTG during a 5-minute interval and theaverage 5-minute bandwidth during the same 5-minute interval for 12 different 5-minute intervalsfor one commercial hotspot AP. The main observation is that xylophone is able to accuratelyestimate AP backhaul bandwidth. MRTG measures the bandwidth right behind the hotspot AP,while our estimation probe messages traverse the Internet and finally arrive at the hotspot AP. Theobservation that these two measurements match fairly well confirms the result that the bandwidthbottleneck of a hotspot AP is usually located at the edge of the Internet [26][20].

5.3. Latency

5.3.1. Latency at WiFi subscribers endWe first evaluate the latency experienced by the hotspot subscribers when they measure per-

formance values such as DHCP connectivity latency, open/blocked ports, and authenticationand association times. Figure 12 shows the measured DHCP latency for 16 AT&T commercialhotspot APs in the Boulder, CO region over a week. The main observation is that the maximumlatency for DHCP connectivity measurement is below 15 seconds for all APs except one. Theaverage DHCP measurement latency is about 5 seconds. Also, we notice that some hotspot APs(hotspot AP 8 and 16) usually experience some DHCP problems. To measure the latency ofport scans, we performed port scan of 127 common application ports in parallel. It took at most


0

5

10

15

20

25

30

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

DH

CP

late

ncy

(sec

.)

Hotspot AP

Figure 12: DHCP measurement on 16 hotspot APsover a week.

0

3

6

9

12

15

18

21

24

27

30

30 60 90 120 150

Est

imat

ion

late

ncy

(sec

.)

Round trip time (ms)

Downstream bandwidth α = 3.5 MbpsUpstream bandwidth β = 3.5 Mbps

Figure 13: Measurement latency at varying transmis-sion delays.

170 milli-seconds to complete this port scan for the 16 commercial AT&T hotspot APs. Finally,we also investigated the latency of the authentication and association mechanisms for different802.11 security mechanisms as part of a previous project, see [27]. The maximum latency forauthentication and association is about 2.49 seconds. Overall, we observe that the latencies forcollecting the performance values on the WiFi hotspot subscribers end are quite low. Consideringthat these measurements require only minimal communication and computation, our conclusionis that xylophone design on WiFi hotspot subscribers end is quite reasonable.

5.3.2. Latency at bandwidth measurement serversThe Internet is highly dynamic in terms of transmission latency, bandwidth between the cloud

and a hotspot AP, etc. In general, there is no way to predict bandwidth estimation latency.Instead of investigating overall latency for bandwidth estimation, we examine several factorsthat cause bandwidth estimation latency variation. By using latency shaping module (netem) inour indoor testbed, we emulate different transmission delays between the cloud and a hotspot AP.In one set of experiments we measure the upstream bandwidth by setting upstream bandwidthequal to 3.5 Mbps, and in another set of experiments, we measure the downstream bandwidth bysetting downstream bandwidth equal to 3.5 Mbps. Figure 13 shows the estimation latency forestimating upstream and downstream bandwidths as a function of transmission delay betweenthe measurement server and the hotspot AP varies from these two sets of experiments. Thefirst observation here is that the estimation latencies are reasonably low, less than 27 secondsfor round trip times as high as 150 ms. Second, we notice that the measurement latency showsapproximately a linear increase with respect to increase in RTT. Based on this, we can say thatthe measurement latency will remain reasonably low even when the RTT goes beyond 150 ms.

Finally, we notice that measurement latency is significantly different for upstream bandwidthestimation and downstream bandwidth estimation. The main reason for this is that the TCPACK packets that the instance sends out have different packet sizes while measuring upstreamand downstream bandwidths. Recall that the instance uses 40-byte ACK packets to estimateupstream bandwidth but larger ACK packets (>>40 bytes) to estimate downstream bandwidth.Thus it takes much longer to send the same number of ACK packets while measuring downstreambandwidth than upstream bandwidth.

To understand the impact of packet size on bandwidth estimation latency, we conductedanother set of experiments. In these experiments, the downstream bandwidth was set to threedifferent values: 3.5 Mbps, 5.5 Mbps and 7.5 Mbps. By using different size of TCP ACK packets


4

6

8

10

12

14

16

18

20

600 800 1000 1200 1400

Est

imat

ion

late

ncy

(sec

.)

Packet size (Byte)

α = 3.5 Mbpsα = 5.5 Mbpsα = 7.5 Mbps

Figure 14: Measurement latency at varying size of aTCP ACK packet.

0

2

4

6

8

10

12

14

16

18

0 0.5 1 1.5 2 2.5 3 3.5 4

Est

imat

ion

late

ncy

(sec

.)

Resolution

α = 12Mbpsα = 5Mbps

Figure 15: Measurement latency at different resolu-tions.

to estimate downstream bandwidth in our indoor testbed with a fixed RTT, we can observe inFigure 14 that the bandwidth estimation latency shows a linear increasing trend. The reason isthat less delays are involved when an instance sends out a TCP ACK stream at a certain datarate using smaller TCP ACK packets. Another interesting observation in Figure 14 is that lowerbandwidth shows higher estimation latency when an instance uses same size of TCP ACK packetsto estimation bandwidth. The reason for this is quite straightforward – an instance spends lesstime in sending the same number of TCP ACK packets if these packets are transmitted througha high-speed link. This explanation is also validated in Figure 15.

Finally, we examine the effect of measurement resolution on bandwidth estimation latency(See Figure 15). In this experiment, we vary measurement resolution from 0.1 to 4.0. Note thatthe estimation latency exponentially decreases as resolution value increases. To understand this,recall that measurement resolution is part of the termination condition of our bandwidth esti-mation. Since bandwidth estimation is the process of a binary search, larger the resolution is,faster the search process will terminate. Therefore, we can easily reduce bandwidth estimationlatency by choosing a greater value of measurement resolution. As we discussed in Section 5.2,choosing a greater value of resolution, however, may harm the accuracy of our bandwidth esti-mation. In order to balance this tradeoff between estimation accuracy and latency, the prototypeimplementation of xylophone uses the configuration – measurement resolution = 1.5.

5.4. Scalability

One practical issue with xylophone is how well will it scale when the number of APs is veryhigh, e.g. 1000 APs? Clearly, we need multiple instances to handle such a large number ofAPs. To evaluate this, we implemented the bandwidth estimation component on Amazon’s EC2platform. In this experiment, we first perform our measurement task (i.e. estimating hotspotAP backhaul bandwidth of 50 AT&T hotspot APs in Denver and Boulder region) from a singleinstance. Then, we increase the number of instances and evenly divide bandwidth measurementtasks of 50 AT&T hotspot APs. Figure 17 shows the latency of measuring upstream and down-stream bandwidths of all 50 APs for three different configurations: 1 instance, 5 instances and 10instances. We notice that the measurement latency is significantly reduced when the number ofinstances in the cloud is increased from 1 to 10. This result is quite encouraging. In particular,with 10 instances, we can measure the upstream and downstream bandwidths of all 50 APs inless than 100 seconds. So, if we have to update the bandwidths of APs every 30 minutes, ten


1

10

100

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

RT

T v

aria

tion

(ms.

)

Time (sec.)

7.5Mbps5.5Mbps3.5Mbps

Figure 16: RTT variation during bandwidth estimation.

0

100

200

300

400

500

600

700

800

0 5 10

Ban

dwid

th e

stim

atio

n la

tenc

y (s

ec.)

# of instances

Figure 17: Xylophone scalability in terms of band-width estimation.

0

20

40

60

80

100

3 4 5 6 7 8

Per

cent

age

(%)

Bandwidth (Mbps)

packets w delayspackets w/o delays

Figure 18: The impact of bandwidth estimation oncrossing traffic.

instances can manage as many as 900 APs. The issue of how frequently the bandwidths shouldbe measured is discussed in Section 5.6.

Another observation from Figure 17 is that though the number of instances are increasedto 5 times and 10 times, the bandwidth estimation latency is not decreased by 5 times and 10times. Two major factors contribute to this. The first is the overhead of managing all instances inthe cloud. The other is the RTT between an instance and hotspot AP are not equal for differenthotspot APs, so the actual distribution of which set of APs is assigned to different instances canimpact the latency.

5.5. Intrusiveness

Since bandwidth estimation of xylophone system is based on the concept of self-inducedcongestion (i.e., an instance saturates hotspot AP backhaul bandwidth in the process of the band-width estimation.), an important question is how does the bandwidth estimation of xylophoneimpact the performance of ongoing user computations? For example, does it result in increase intransmission delays for hotspot subscribers? To quantitatively investigate this potential impacton transmission delays, we conducted an experiment in our indoor testbed. A client uses hping2to generate a TCP packet every 100 milliseconds through the AP we deployed in our testbed.Over this peroid of time, the instance probes the AP backhaul bandwidth. Figure 16 shows thetransmission delays for some TCP packets from the client side increases. Furthermore, we cansee that overall transmission delays show an obvious increasing trend when the TCP stream fromthe client side passes through a low-bandwidth link. The reason is that when TCP packets arebuffered in a queue, lower bandwidth routers need longer time to empty the packet buffered inthe queue. Another important observation from Figure 16 is that there are no drop-off packets.


0

2

4

6

8

10

0 10 20 30 40 50 60 70

Ban

dwid

th (

Mbp

s)

Time (Hour)

Figure 19: Time series plot of bandwidth at a hotspot AP in Boulder region, Colorado.

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12

CD

F

Variation between succssive samples (Mbps)

Figure 20: CDF of relative variation in bandwidth be-tween successive samples.

0

20

40

60

80

100

0 10 20 30 40 50 60

Per

cent

age

(%)

Duration (minutes)

relative deviation within 10%relative deviation within 20%relative deviation within 30%

Figure 21: Percentage of slide windows in which therelative deviation is within 10%, 20% and 30%.

To examine the overall impact of bandwidth estimation on a TCP stream sent from a client side,we investigate what percentage of TCP packets from the client side experience longer transmis-sion delays. Figure 18 indicates that only 10%–20% TCP packets generated from the client sideexperience longer transmission delays. The reasons for this relatively lower percentages are: (1)bandwidth estimation does not cause a persistent increase in queue size because a probe streamis never sent before the previous probe stream has been acknowledged; and (2) the process ofbandwidth estimation is a process of binary search, and therefore, not every probe stream willsaturate hotspot AP backhaul bandwidth.

5.6. Sampling frequency

As we discussed in Section 5.5, bandwidth estimation impacts the performance of hotspotsubscribers in terms of increasing the transmission delays for a relatively small percentage ofpackets. Therefore, xylophone must reduce the frequency of bandwidth estimation of an AP.However, lower frequency may degrade the applicability of our system. For instance, supposexylophone probes hotspot AP backhaul bandwidth every 60 mintues. During this period of time,hotspot AP backhaul bandwidth may vary significantly. Therefore, it is important to choose anoptimal sampling frequency for bandwidth estimation. To do this, we need an understanding ofhow bandwidth changes over a given time period. So, we conducted a long-term experiment toanalyze the temporal stability of bandwidth. The purpose of this analysis is to (1) explore howthe bandwidth observed at a hotspot AP varies as a function of time; and (2) identify the relativestability of bandwidth over different time scales.Data collection: We collected hotspot AP backhaul bandwidth information from 50 commercialhotspot APs in Denver and Boulder region. These hotspot APs were accessed and used daily by


hotspot subscribers. They vary widely from one another in terms of the number WiFi subscribers.Some of them are used by hundred and even thousand of hotspot subscribers daily (e.g. airporthotspot), while some others are used by less than 10 subscribers, e.g. a coffee shop. We estimatedbandwidth B(t) of these hotspot APs every 30 seconds. To describe the temporal stability ofbandwidth, we select one of the most dynamic hotspot AP in terms of bandwidth variation. Asnapshot of the time series plot of bandwidth from this hotspot AP is shown in Figure19 for a70-hour time period.Persistence: We first take a look at the amount by which successive bandwidth samples change.If many abrupt changes over short time scales occur, the bandwidth is unstable. We compute theCumulative Distribution Function (CDF) of the amount of bandwidth change between successivesamples and analyze it. Figure 20 shows the CDF of persistence for the measured hotspot AP,whose time series plot is shown in Figure 19. We find that successive samples of the hotspot APvary by less than 8 Mbps over the entire trace period. However more than 90% of the successivebandwidth samples vary by less than 0.8 Mbps. Therefore, we can conclude that the measuredbandwidth B(t) is relatively persistent.Time scales of stability: A persistent stable bandwidth B(t) might not be stationary stable, wherethe mean and variance do not change over time or space. For example, consider a stochasticprocess X(t) in which variable Xi keeps increasing by a small amount with increase in i. In thiscase, X(t) is persistent, but not stationary stable. If we take a close look at the time series plotin Figure 19, we notice that the bandwidth varies quite a lot over the entire observation period.Indeed, B(t) is not stationary stable in general. We need to verify whether B(t) is piecewisestationary, i.e. whether B(t) is stationary stable over shorter time scales.

To check the piecewise stationary stability of B(t), we set a time window w of some length,and slide this window over the measurement interval over the entire trace period. For each in-stance of slide window, we calculate mean and relative deviation. Figure 21 shows the percentageof slide windows in which the relative deviation is less than 10%, 20% and 30% for different win-dow sizes ranging from 5 minutes to 60 minutes. For example, when we set time window w = 20minutes, 81% windows on the trace have a relative deviation less than 20%. This means that thechance that the bandwidth varies by more than 20% with in a 20-minute period is about 19%.As expected, we note that the bandwidth is less stationary stable for larger window sizes. Theseresults show that B(t) is piecewise stationary for time intervals of the order of 20 to 30 minutes.

The results reported here are from a snapshot of the most dynamic hotspot AP in our ex-periment, and so this observation cannot be generalized to all the hotspot APs. However, otherhotspot APs, especially for those hotspot APs with less dynamic behaviors in terms of bandwidthvariation, show more stable behaviors. With this observation, our prototype implementation ofxylophone uses 30 minutes as our bandwidth estimation interval, i.e. duty cycle for estimatinghotspot backhaul bandwidth is 30 minutes.

6. Conclusion

This paper introduces xylophone, a hotspot WiFi locator service that allows a hotspot sub-scriber to search for available hotspots in a given area and enables hotspot subscriber to selectcommercial hotspots based on their current quality of service. The quality of service parametersinclude upstream and downstream bandwidths, connection delays in terms of DHCP connectiv-ity latency, authentication and association latencies, and open/blocked ports. A novel bandwidthestimation methodology is introduced. Extensive evaluations of xylophone are provided whichinclude evaluation from a prototype implementation in an indoor testbed as well as a prototype


implementation in the Amazon’s EC2 platform. Evaluation results demonstrate low latenciesexperienced by WiFi subscribers to measure DHCP connectivity, authentication and associa-tion process, and discover open/blocked ports. Also, the bandwidth measurement componentof xylophone exhibits high measurement accuracy, low latency, high scalability, and minimalintrusiveness.

References

[1] M. Jain, C. Dovrolis, End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcpthroughput, IEEE/ACM Transactions on Networking (TON) 11 (4) (2003) 537–549.

[2] J. Strauss, D. Katabi, F. Kaashoek, A measurement study of available bandwidth estimation tools, in: Proc. of ACMSIGCOMM conference on Internet measurement, 2003, pp. 39–44.

[3] J. Pang, B. Greenstein, M. Kaminsky, D. McCoy, S. Seshan, Wifi-reports: improving wireless network selectionwith collaboration, in: Proc. of ACM MOBISYS, 2009, pp. 123–136.

[4] http://www.att.com/gen/general?pid=5949.[5] https://content.hotspot.t-mobile.com/assetprocess.asp?asset=com.defau lt.main.001.[6] http://v4.jiwire.com/search-hotspot-locations.htm.[7] G. Judd, P. Steenkiste, Fixing 802.11 access point selection, ACM SIGCOMM Computer Communication Review

32 (3) (2002) 31–31.[8] A. J. Nicholson, Y. Chawathe, M. Y. Chen, B. D. Noble, D. Wetherall, Improved access point selection, in: Proc.

of ACM MOBISYS, 2006, pp. 233–245.[9] S. Vasudevan, K. Papagiannaki, C. Diot, J. Kurose, D. Towsley, Facilitating access point selection in ieee 802.11

wireless networks, in: Proc. of ACM IMC, 2005, pp. 293–298.[10] G. Kola, M. K. Vernon, Quickprobe: available bandwidth estimation in two roundtrips, in: SIGMETRICS Perfor-

mance Evaluation Review, 2006, pp. 359–360.[11] N. Hu, P. Steenkiste, Evaluation and characterization of available bandwidth probing techniques, IEEE JOURNAL

ON SELECTED AREAS IN COMMUNICATIONS 21 (6) (2003) 879–894.[12] S.-R. Kang, D. Loguinov, Imr-pathload: Robust available bandwidth estimation under end-host interrupt delay, in:

Proc. of PAM, 2008.[13] P. Papageorge, J. McCann, M. Hicks, Passive aggressive measurement with mgrp, in: Proc. of ACM SIGCOMM,

2009.[14] D. Antoniades, M. Athanatos, A. Papadogiannakis, E. P. Markatos, C. Dovrolis, Available bandwidth measurement

as simple as running wget, in: Proc. of PAM, 2006.[15] X. Xing, S. Mishra, Where is the tight link in a home wireless broadband environment?, in: Proc. of IEEE/ACM

MASCOTS, 2009, pp. 49–58.[16] D. Crocey, M. Melliay, E. Leonardiy, The quest for bandwidth estimation techniques for large-scale distributed

systems, in: ACM SIGMETRICS Performance Evaluation Review, 2010, pp. 20–25.[17] X. Luo, E. W. Chan, R. K. Chang, Design and implementation of tcp data probes for reliable and metric-rich

network path monitoring, in: Proc. of USENIX Annual Technical Conference, 2009, pp. –.[18] I. Broustis, K. Papagiannaki, S. V. Krishnamurthy, M. Faloutsos, V. Mhatre, Mdg: measurement-driven guidelines

for 802.11 wlan design, in: Proc. of ACM MOBICOM, 2007, pp. 254–265.[19] G. Maier, A. Feldmann, V. Paxson, M. Allman, On dominant characteristics of residential broadband internet traffic,

in: Proc. of ACM IMC, 2009, pp. 90–102.[20] N. Hu, L. E. Li, Z. M. Mao, P. Steenkiste, J. Wang, A measurement study of internet bottlenecks, in: Proc. of IEEE

INFOCOM, 2005, pp. –.[21] K. Fukuda, K. Cho, H. Esaki, The impact of residential broadband traffic on japanese isp backbones, in: ACM

SIGCOMM Computer Communication Review, 2005, pp. 15–22.[22] http://aws.amazon.com/ec2/.[23] http://en.wikipedia.org/wiki/amazon elastic compute cloud/.[24] http://en.wikipedia.org/wiki/hadoop.[25] http://oss.oetiker.ch/mrtg/index.en.html.[26] M. Dischinger, A. Haeberlen, K. P. Gummadi, S. Saroiu, Characterizing residential broadband networks, in: Proc.

of ACM SIGCOMM conference on Internet measurement, 2007.[27] X. Xing, S. Mishra, X. Liu, Arbor: hang together rather than hang separately in 802.11 wifi networks, in: Proc. of

IEEE INFOCOM, 2010.

The Design, Implementation and Performance Evaluation of ... · The Design, Implementation and Performance Evaluation of Xylophone: A Scalable QoS-enabled WiFi Locator Service Xinyu

Documents