Technical Report - cs.umn.edu · Server-Based Dynamic Server Selection Algorithms for Enterprise Networks Technical Report Department of Computer Science and Engineering University

Server-Based Dynamic Server Selection Algorithms for Enterprise Networks

Technical Report

Department of Computer Science

and Engineering

University of Minnesota

4-192 EECS Building

200 Union Street SE

Minneapolis, MN 55455-0159 USA

TR 00-061

Server-Based Dynamic Server Selection Algorithms for Enterprise

Networks

Yingfei Dong, Zhi-li Zhang, and Y. Thomas Hou

December 05, 2000

Server-Based Dynami Server Sele tion AlgorithmsYingfei Dong� Zhi-Li ZhangyTe hni al Report 00-61 Y. Thomas HouzAbstra tServer sele tion is an important problem in repli ated server systems distributed over theInternet. In this paper, we study two server sele tion algorithms under a server-based frameworkwe have developed. These algorithms utilize server load and network performan e information olle ted through a shared passive measurement me hanism to determine the appropriate serverfor pro essing a lient request. The performan e of these algorithms is studied using simulations.Comparison with two naive server sele tion algorithms is also made. The initial simulationresults show that our dynami server sele tion algorithms have superior performan e over the twonaive ones, and as a result, demonstrate the importan e of dynami server sele tion me hanismsin a repli ated server system.Keywords: server repli ation, server-based dynami server sele tion, QoS, passive measurement

�University of Minnesota, Dept. of Computer S ien e, 4-192 EE/CS Building, 200 Union Street S.E., Minneapolis,MN 55455-0159, USA. Tel. (612) 626-7526, Fax (612) 625-0572, Email: dong� s.umn.edu.yPlease dire t all orresponden e to Prof. Zhi-Li Zhang, University of Minnesota, Dept. of Computer S ien e,4-192 EE/CS Building, 200 Union Street S.E., Minneapolis, MN 55455-0159, USA. Tel. (612) 625-8568, Fax (612)625-0572, Email: zhzhang� s.umn.edu.zFujitsu Laboratories of Ameri a, 595 Lawren e Expressway, Sunnyvale, CA 94086-3922, USA. Tel. (408) 530-4529,Fax (408) 530-4515, Email: thou� a.fujitsu. om.

1 Problem Formulation and Related WorkServer repli ation (or mirroring) is a ommon te hnique that has been used to provide s alabledistributed servi e over the Internet. If done appropriately, server repli ation an avoid serveroverload and signi� antly redu e lient a ess laten y. In addition to the issues su h as where topla e repli ated servers in the global Internet and how lients an lo ate a server with the desiredservi e, an important and hallenging problem in server repli ation is how to sele t a server topro ess a lient request so as to provide the \best servi e" for the lient. This problem, referred toas the server sele tion problem, is the fo us of our paper.In addressing the server sele tion problem, two pro edures are typi ally involved. First, statisti sabout server/network performan e need to be olle ted. Based on the olle ted statisti s, a serversele tion algorithm then sele ts the \best" server among a pool of eligible servers for pro essinga lient request. Depending on where the statisti s are olle ted and the server sele tion de isionis made, existing server sele tion approa hes [5, 7, 8, 11, 14, 16, 19, 24℄ an be lassi�ed as either lient-based or server-based .Under the lient-based approa h, statisti s about the network and server performan e is typi- ally olle ted using the a tive probing method [19, 7, 8, 11, 16℄: a lient [8℄ or its \proxy" (e.g., theservi e resolver in [11℄) sends probe pa kets to measure the network performan e.1 As an ex eption,Seshan et al. [24℄ employs a novel passive measurement method where lients share their networkperforman e dis overy through performan e monitoring at the lient side. The major drawba k ofthe lient based server sele tion approa h is that it is not transparent to lients: either lients ortheir proxies (e.g., servi e resolvers) need to know the name/lo ation of all the servers providing agiven servi e. Furthermore, the lient-based approa h requires modi� ation of lient browser soft-ware and installation of network measurement tools at every lient side, and in some ases, it mayrely on the deployment of an Internet-wide measurement infrastru ture (e.g., modi� ation of DNSto in orporate servi e resolvers). In the ase where a tive probing measurement te hniques areused, the extra traÆ introdu ed by probe pa kets an lead to network bandwidth wastage or even ongestion. In the long term the lient-based approa h may have its appeal, espe ially when anInternet-wide measurement infrastru ture [15, 22, 23℄ is in pla e. However, in the immediate future,server sele tion me hanisms using the lient-based approa h are unlikely to be widely available to1In the ase of [11℄ servers also ollaborate with servi e resolvers by \pushing" their performan e statisti s toservi e resolvers periodi ally. 1

many lients to take advantage of repli ated server systems.In ontrast, the server-based approa h relies on the ooperation of servers providing a givenservi e to determine the \best" server for a lient and inform the lient the lo ation of the servervia, say, HTTP redire t me hanism. An example of the server-based approa h is HARVEST system[5℄ whi h uses a simple network metri , hop ount , as the riterion to sele t the \best" server for lients.In [14℄, we propose and develop a server-based measurement and server sele tion frameworkwhi h employs passive network measurement te hniques and performan e information sharingamong servers to dynami ally sele t the \optimal" server for pro essing lient requests. The pro-posed framework ontains three major omponents: (1) a server-based passive network measure-ment me hanism based on t pdump [18℄ and a prototype of whi h alled Woodpe ker is des ribedin [10℄, (2) a metri s ex hanging proto ol for sharing server load and network performan e infor-mation among the servers, whi h will be des ribed in a future paper, and (3) a dynami serversele tion me hanism for sele ting an \optimal" server to pro ess a given lient request, whi h is thefo us of this paper.In this paper we des ribe two server-based dynami server sele tion algorithms. These twoalgorithms utilize both the server load and network performan e information olle ted by ea h serverand shared among them. Based on these performan e statisti s, a server de ides whether itself oranother server would be the appropriate server to pro ess a lient request it re eives. In the latter ase, the lient request is redire ted to the sele ted server (see Fig. 1). The performan e of thesealgorithms is studied using simulations. Comparison with two naive server sele tion algorithmsis also made. The initial simulation results show that our dynami server sele tion algorithmshave superior performan e over the two naive ones, and as a result, demonstrate the importan e ofdynami server sele tion me hanisms in a repli ated server system.Our server-based approa h to the problem of server sele tion has the following salient features.First, it is lient-transparent. Se ond, it does not require any modi� ation to any lient softwareor relies on the availability of an Internet-wide measurement infrastru ture. Therefore it is read-ily deployable. Third, by using shared passive network performan e measurement, we avoid thewastage of network resour es of a tive probing measurement te hniques. The network bandwidthoverhead in urred by metri s ex hanges among the servers is relatively low, as the number of serversin a repli ated system is generally small, in parti ular, when ompared to the number of lients2

X

S

Redirection ACK

Redirected Request

����������������

R������

������

Server 1

Client 4

Client 3

Client 2

Client 1

Server 2Figure 1: An example system(or the sub-networks). Fourth, by taking both the server load and network ongestion status ofpaths between servers and lients, we an employ relatively sophisti ated dynami server sele tionalgorithms to hoose the \optimal" server to servi e lient requests without in urring too mu hlaten y and overhead. Our approa h is parti ularly amenable to an enterprise-based repli atedserver system, where a number of servers onne ted through an enterprise network provide ertainservi es over the Internet to \regular" lients who need a ess to the servi es frequently.The remainder of this paper is organized as follows. In Se tion 2, the two dynami serversele tion algorithms are presented. The performan e results are shown in Se tion 3. In Se tion 4,we on lude the paper and dis uss the future work.2 Server-Based Dynami Server Sele tion AlgorithmsIn this se tion, we present two dynami server sele tion algorithms.2.1 Performan e Metri sBefore des ribing our algorithms, we de�ne two major performan e metri s used in our serversele tion algorithms: server loads and fast paths, whi h indi ate the status of servers and di�erentnetwork paths from servers to lients.Server Load: For given server S, its load denoted by LS , is de�ned as the ratio of Ttotal to Rmax.Here, Ttotal is the time period required to serve all requests in the waiting queue on S, and it isrelated to the apability of S. It is omputed based on the urrent and histori al information3

of S in luding the pro essing delay on S and the transferring delay from S to a lient. Rmaxis the maximum response delay that a typi al lient an a ept for a request of average size,su h as 10 KBytes [6℄. LS is alled light if LS � 80%; otherwise, it is alled heavily loaded.Fast Path: As shown in Figure 1, if the transferring delay of a reply from a server S to a lientX is k times longer than the transferring delay from another server R to X plus the ostof redire ting the request from S to R, the path from S to X is alled slow, and the pathfrom R to X is alled fast. Here, k is an experimental parameter, su h as 2 or 4. Thetransferring delay depends on the size of the reply and the bottlene k bandwidth of the path.For simpli ity, the bottlene k bandwidth is approximated through the TCP-friendly formula[17, 25℄. The redire tion ost is the delay of an a knowledgment from S to X plus the delayof the redire ted request from X to R. Both transferring delays are estimated through theRTTs of the paths whi h are obtained through the passive measurement on servers.2.2 Server Sele tion AlgorithmsOur two simple server sele tion algorithms are given here. When a lient X sends a request to serverS, there are two situations in whi h this request may be redire ted: (1) server S is overloaded, or(2) the path from S to X is slow. We give two algorithms to hoose another server R whi h mayprovide better servi e under both situations. The �rst algorithm is alled Random Redire tion(RR) whi h randomly redire ts a request to another server R other than S. The se ond algorithmexamines all servers and sele ts the \best" one as R based on the urrent metri s on S. Be ausethe urrent metri s on server S may not be a urate due to the delay of metri s ex hange amongservers, the se ond method is alled Best-guess Redire tion (BR). RR is the simplest method whi hdoesn't required too mu h al ulation. BR, however, is one of the most ompli ated methods, inwhi h server must know the status of all servers and related network paths to lients and ompareall the possible hoi es.Figure 2 gives the basi frame of algorithms RR and BR. When a server S re eives a requestfrom a lient X, S he ks whether a epting this request into its waiting queue or redire ting thisrequest to another server R. As shown from line 1 to 3, if X is a regular lient, and S's load islight and the path from S to X is not slow, S a epts this request. Otherwise, in RR method fromline RR1 to RR4, S randomly redire t the request to another server R other than S; or, in BRmethod from line BR1 to BR5, S omputes equivalent lasses to �nd whether a better server exists.4

The equivalent lass of server S respe tive to a spe i� request is the set of servers whose urrentresponse time for this spe i� request is within a threshold ompared with that of the server S. Ifall the servers are in the same equivalent lass (i.e., there is no better server), S has to a ept therequest. Otherwise, S redire ts the request to R whi h ould provide \best" servi e.As shown in line 10 to 14, there is no information available about this lient. If S's load islight, it a epts this request. Otherwise, it redire ts this request to a light loaded server. In orderto avoid the request os illating among servers, S has to a ept this request if all other servers areheavily-loaded.1. if (a lient X is a regular lient)2. if (the server load is light) and (the path from S to X is not slow)3. a ept the request4. else===================================================RR1. if (all other servers are heavily loaded)RR2. a ept the requestRR3. elseRR4. randomly redire t the request to R other than S===================================================BR1. omputing equivalent lassesBR2. if (no better server)BR3. a ept the requestBR4. elseBR5. redire t the request to the ``best'' server R===================================================10. else11. if (server load is light) or (all other server are heavily loaded)12. a ept the request and generate metri s about the lient13. else14. redire t to a light-loaded serverFigure 2: Server sele tion algorithms RR and BR3 Simulation3.1 System ModelsIn our simulation, a system onsists of lients, servers and network paths. Figure 1 shows anexample system with 2 servers, 4 lients and 8 network paths.5

Client Model: A lient generates requests and re eives replies from servers. For generatingrequests, the size of a request is Pareto-distributed with an average request size of 10 KBytes[6℄ and a shape parameter 1.66 [9℄; the inter-arrival of requests is exponential distributed. Inaddition, assume a new request an be generated even though previous requests have not beenserved. For re eiving a reply, if the reply is a redire tion a knowledgment, the lient resendsthe request to the redire ted server; if the reply is data, the lient re ords the response timeof the request.Server Model: When a server re eives a request, it �rst he ks whether a epts this requestor redire ts it to another server using the server sele tion algorithms presented above. If aredire ting de ision is made, the server sends a redire ting a knowledgment ba k to the lient;otherwise, the server a epts this request into a FIFO queue whi h holds all waiting requestswhen the server is serving a previous request.A server always pro esses the request at the head of the waiting queue if the queue is notempty. The ost of pro essing a request on a server onsists of its queuing ost, its operatingsystem ost for starting and ending a servi e, and its storage a essing ost. Queuing ost isthe time period between its arriving and leaving the queue. Operating system ost is ountedusing an average value (e.g., 0.1ms). Storage a essing ost are omputed through a storagemodel [13℄ with the parameters, su h as an average seek time (e.g., 10ms), an average rotationdelay (e.g., 1ms), a ontroller overhead (e.g., 1ms), and the transferring ost from the storageto the memory with a 20 Mbytes/se ond I/O bus. The e�e t of disk a he and memory a hewill be onsidered in our future paper.Network Model: Ea h lient has a network path to ea h server, while the bottlene k bandwidthof the path an be approximated using the formula in [17, 25℄ through the Maximum TransferUnit (MTU), the Round-Trip Time (RTT) and the loss rate of the path.Bandwidth = 1:3 �MTURTT � pLossRate>From the results in [4, 6, 9, 17, 25℄, the typi al values of loss rates, MTUs, and RTTs ofnetwork paths are: a loss rate is between 0.01% and 5%, a MTU is between 576 bytes to 1500bytes, and a RTT is between 20 ms and 500 ms. In our simulation, the loss rate, MTU andRTT of a path are generated within the above ranges to simulate the dynami status of thenetwork paths. 6

3.2 Initial Simulation ResultsIn this se tion, we present our initial simulation results. Besides the proposed two server sele tionalgorithms, RR and BR, two other stati server assignment approa hes are also tested in our simu-lation to investigate the e�e t of the server sele tion algorithms. The �rst one is alled Fixed-Server(FS) assignment method: a lient keeps requesting servi es from a �xed server whi h is initiallyrandomly hosen by the lient, and a server has a �xed number of lients during a simulation. Thisis the most naive situation. The se ond method is alled Fixed-Group (FG) assignment approa h:all lients are divided into equal size groups, and ea h group of lients only onta ts with a singledefault server. In this method, ea h server has the same number of lients during a simulation. FSand FG are similar to the methods used in existing systems.Our initial simulation results are given as follows. In this group of simulations, 16 lients requireservi es from 4 servers during a period of 1 minute. In Table 1, the average response time of all lients under four approa hes are given, as are the maximum, minimum and standard deviation ofthe response time. The total number of requests served are listed in the last olumn of Table 1. Itis easy to see that RR and BR served more requests than FS and FG.Table 2 shows the queuing times on the simulated servers. Similarly, the average value, themaximum, the minimum and the standard deviation are given.Choosing the average results of BR as the base 1.00, Table 3 learly shows that RR and BRare signi� ant better than FS and FG both in the average response time among lients and theaverage queuing time among servers. An interesting �nding is that RR is only slightly worse thanBR BR requires mu h more omputing ost than RR, but RR did more redire tions than BR.In our simulation, 51664 times of redire tions took pla e in the RR approa h, while 47686 timesof redire tions happened in the BR approa h. A large s ale simulation will show even greaterdi�eren es between them. Table 1: Response times among all lientsMethod AVG MAX MIN STD # Requests ServedFS 4.048 14.16 0.007 3.953 6955FG 1.189 6.932 0.009 1.851 7949RR 0.142 3.489 0.003 0.196 8341BR 0.140 2.647 0.003 0.210 83437

Table 2: Queuing delays among all serversMethod AVG MAX MIN STDFS 4.028 14.17 0.0 3.956FG 1.166 6.749 0.0 1.850RR 0.049 0.342 0.0 0.028BR 0.044 0.327 0.0 0.028Table 3: Summary of simulation resultsFS FG RR BRAverage Response Time over all lients 28.91 8.49 1.01 1.00Average Queuing Time over all servers 85.70 24.80 1.04 1.00The dynami methods, su h as RR and BR, distributed the requests more evenly among serversthan the stati methods su h as FS and FG. The FG method is better than the FS method instati methods while the RR method is slightly worse than the BR method in dynami methods.Therefore, we hoose the FG and RR methods as representatives to ompare stati methods withdynami methods. Figure 3 shows the queuing times of requests on four server in the FG and RRmethods. The x axis is the number of requests whi h have been served on servers, and the y axis isthe queuing time. Figure 3(a) shows that one waiting queue keeps building up in the FG method;however, Figure 3(b) shows that all the queuing times of requests are distributed in a small rangein the RR method. Figure 4 shows response times of 4 lients under the FG and RR methods.The x axis is the number of requests whi h lients have re eived their replies, and the y axis is theresponse time. In Figure 4(a), the response time of one lient keeps in reasing in the FG method;on the other hand, Figure 4(b) learly shows that, the response time of all lient stay within a smallrange in the RR method.4 Con luding RemarksIn this paper, we study two server sele tion algorithms under a server-based framework that wehave developed in [14℄. Both algorithms employ server load and network performan e metri s olle ted through a shared passive measurement me hanism to determine the appropriate server8

0

1

2

3

4

5

6

7

0 500 1000 1500 2000 2500

Que

uein

g T

ime

The Number of Requests

Queueing Time for RR algorithm

Server 0Server 1Server 2Server 3

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 500 1000 1500 2000 2500

Que

uein

g T

ime

The Number of Requests

Queueing Time for random algorithm

Server 0Server 1Server 2Server 3

(a) FG (b) RRFigure 3: The queueing delays of all 4 servers under FG and RR algorithmsfor a lient request. Simulations show that both dynami algorithms are signi� antly better thanstati methods in terms of response times and queuing delays. Work is under way to implementthis me hanism and investigate its performan e over the Internet.Referen es[1℄ H. Balakrishnan, V. N. Padmanabhan, S. Seshan, M. Stemm, and R. H. Katz, \Analyzing stability inwide-area network performan e," in Pro . ACM SIGMETRICS'97, June 1997.[2℄ H. Balakrishnan, V. N. Padmanabhan, S. Seshan, M. Stemm, and R. H. Katz, \TCP behavior of abusy Internet server: analysis and improvements," in Pro . IEEE INFOCOM'98, San Fran is o, CA,Mar h 1998.[3℄ P. Barford and M. E. Crovella, \Generating representative web workloads for network and serverperforman e evaluation," Te hni al Report BU-CS-97-006, De . 1997.[4℄ J. Bolot, \Chara terizing end-to-end pa ket delay and loss in the I Internet," Journal of High SpeedNetworks, vol.2, no.3, pp.305{323, De . 1993. 9

0

1

2

3

4

5

6

7

0 200 400 600 800 1000 1200 1400

Res

pons

e T

ime

The Number of Requests

Response Time for RR algorithm

Client0Client1Client2Client3

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0 200 400 600 800 1000 1200 1400 1600

Res

pons

e T

ime

The Number of Requests

Response Time for random algorithm

Client0Client1Client2Client3

(a) FG (b) RRFigure 4: The response times of the �rst 4 lients under FG and RR algorithms[5℄ C. M. Bowman, et. al., \Harvest: a s alable, ustomizable, dis overy and a ess system," Te hni alReport CU-CS-732-94, Dept. of Computer S ien e, University of Colorado at Boulder, 1994.[6℄ N. Cardwell, S. Savage, and T. Anderson \Modeling the performan e of short TCP onne tions," O t.1998.[7℄ R. L. Carter and M. E. Crovella, \Measuring bottlene k link speed in pa ket-swit hed networks,"Performan e Evaluation, vol. 27 & 28, 1996.[8℄ R. L. Carter and M. E. Crovella, \Dynami server sele tion using bandwidth probing in wide-areanetworks," in Pro . IEEE INFOCOM'97, Kobe, Japan.[9℄ C. R. Cunha, A. Bestavros, and M. E. Crovella, \Chara teristi s of WWW lient-based tra es," inPro . ACM SIGMETRICS'96.[10℄ Y. Dong, Y. T. Hou, and Z.-L. Zhang, \A novel server-based measurement infrastru ture for EnterpriseNetworks ", Te hnique Report TR 98-031, Dept. of Computer S ien e, University of Minnesota, 1998.[11℄ Z. Fei, S. Bhatta harjee, E. W. Zegura, and M. Ammar, \A novel server sele tion te hnique forimproving the response time of a repli ated servi e," in Pro . IEEE INFOCOM'98, San Fran is o, CA,1998. 10

[12℄ J. Guyton and M. S hwartz, \Lo ating nearby opies of repli ated Internet servi es," in Pro . ACMSIGCOMM'95, Cambridge, MA, 1995.[13℄ J. Hennessy and D. Patterson, Computer Ar hite ture: A Quantitative Approa h, Se ond Edition,Morgan Kaufmann Publishers, In ., 1996.[14℄ Y. T. Hou, Y Dong, and Z.-L. Zhang, \Network performan e measurement and analysis - part 1: aserver-based measurement infrastru ture", Te hni al Memorandum FLA-NCRTM98-01, Fujitsu Labo-ratories of Ameri a, Sunnyvale, CA, July, 1998.[15℄ Internet Distan e Maps Proje t, http://idmaps.ee s.umi h.edu/.[16℄ V. Ja obson, path har - A Tool to Infer Chara teristi s of Internet Paths,http://ee.lbl.gov/nrg-talks.html, April 1997.[17℄ J. Mahdavi and Sally Floyd, \TCP-Friendly Uni ast Rate-Based Flow Control", The TCP-FriendlyWebsite, http://www.ps .edu/networking/t p friendly.html.[18℄ S. M Canne, C. Leres, and V. Ja obson, t pdump, ftp://ftp.ee.lbl.gov/t pdump.tar.Z, Lawren eBerkeley National Laboratory.[19℄ K. Moore, J. Cox, and S. Green, \SONAR - A Network Proximity Servi e," Internet-Draft,http://www.netlib.org/utk/proje ts/sonar/, Aug. 1996.[20℄ A. Myers, P. Dinda, and H. Zhang, \Performan e hara teristi s of mirror server on the Internet," InPro . IEEE INFOCOM'99, Mar h 1999.[21℄ LBNL Network Simulator Version 2, http://www-mash. s.berkeley.edu/ns/, 1998.[22℄ V. Paxson, \Measurements and analysis of end-to-end Internet dynami s," Ph.D. Dissertation, U.C.Berkeley, 1997.[23℄ V. Paxson, J. Mahdavi, A. Adams, and M. Mathis, \An ar hite ture for large-s ale Internet measure-ment," IEEE Commun. Magazine, pp. 48{54, Aug. 1998.[24℄ S. Seshan, M. Stemm, and R. Katz, \SPAND: shared passive network performan e dis overy," in Pro .USENIX'97, 1997.[25℄ M. Yajnik, J. Kurose, and D. Towsley \Pa ket loss orrelation in the MBone multi ast network,"Te hni al Report UMASS CMPSCI 96-32, Dept. of Computer S ien e, Univ. of Mass. at Amherst,1996. 11