Project 2 Presentation & Demo Course: Distributed Systems By Pooja Singhal 11/22/2011 1.

Project 2 Presentation & Demo

Course: Distributed Systems

By Pooja Singhal

11/22/2011

1

Outline Requirement Requirement Analysis Challenges Design

Data Structures 3-Tiered Model Algorithms

Implementation Learning and Experience Summary Acknowledgement Demo

2

Requirement

Design and Implement a 3-Tiered Client Server Model. Given a City and State, find 5 nearest Airports. Using RPC System: Linux machine, Sun RPC, Language: C++/C

3

3) 3-Tiered Model Requirement

Client

Places Server

Airport Server

4) Requirement of Algorithm

To search Nearest Neighbors

4

Requirement Analysis

1) Parsing of Places2k.txt – Fast and Efficient. Which DS?

2) Parsing of airport-locations.txt - Spatial Partitioning. Which DS?

Challenges

3-Tiered Client Server Model. Spatial Partition: Did not know much about it! How to Search 5 nearest Airports ? Again No idea!

1) Parsing of Places2k.txt

2) Design and Implement Client

3) Design and Implement Places Server

Test the code So far

4) Design and Implement 3-Tiered Model

5) Design and implement Spatial Partitioning of data in airport-locations.txt

6) Design and implement N nearest Airports Search5

Design

6

Design TacticsWeekly Submissions made the job easy!

First week Design: Parse both the files, Data Structures

Second Week Design: IDL Design : Structure Design, Function Design Client Design and Logic: Places Server Design: Server for Client

Final Week Design 3-Tiered Model Design Change of Data Structure for airport-locations.txt records Nearest Neighbor Search design

7

8

CLIENT PLACES SERVER AIRPORT SERVER

3-Tiered Model

Client Design

Client Gets the location from User in the form of CITY and STATE. Pass it to Places Server Display the results

9

Places Server Design

Phase 1As a Server to Client:

On start up, parse places2k.txt in hash table Hash Table key is combination of “CITY” and “STATE” Latitude and Longitude are stored as DATA along with the key Gets the inputs (CITY and STATE) from Client Make the Key: Apply Hash Function Search Hash Table If Found: Get Latitude and Longitude

Phase 2: As a Client to Airport Server:

Pass on Latitude and Longitude to Airport Server Get the result back from Airport Server Return the result to Client.

10

Airport Server Design

11

Act as a Server to Places Server On start up, parse airport-location.txt Creates a K-D Tree in memory Gets the latitude and location from places server. Search Nearest Neighbor Calculate the Distance Sort the results on Distance Return 5 neared neighbor back to places server

Design - Data Structures

12

Data Structures (1) Hash Table

Store places2k.txt records Key is combination of CITY and STATE Data: Latitude and Longitude Advantages: Fast

K-D Tree Spatial Partitioning of 2 Dimensional Points consist of Latitude and Longitude Node consists of 2 Dim points (Latitude, Longitude) Node Data: Airport Code, Airport Name, State

Linked List Store Results consisting of 5 nearest airports Since, pointers do not get passed over RPC, needed to store the address of the

next record

13

Data Structures (2)

K-D TreeSpace Partitioning Data Structure for storing a finite set of points in a k-dimensional spaceInvented by J Luis Bentley in 1975Is a Binary Tree: Special example of Binary Space Partitioning TreesApplications in wide areas: Neural Networks, searching multidimensional data

14Source: http://pointclouds.org/documentation/tutorials/kdtree_search.php

(2,3), (5,4), (9,6), (4,7), (8,1), (7,2)

1. X Plane Division

2. Y Plane Division

3. X Place Division

Design - Algorithms

15

AlgorithmsNearest Neighbor Search

Starting with the root node, the algorithm moves down the tree recursively Once the algorithm reaches a leaf node, it saves that node point as the "current

best“The algorithm unwinds the recursion of the tree, performing the following steps at each node: If the current node is closer than the current best, then it becomes the current best.The algorithm checks whether there could be any points on the other side of the splitting plane that are closer to the search point than the current best

done by intersecting the splitting hyperplane with a hypersphere around the search point that has a radius equal to the current nearest distance.

Since the hyperplanes are all axis-aligned this is implemented as a simple comparison to see whether the difference between the splitting coordinate of the search point and current node is less than the distance (overall coordinates) from the search point to the current best.

If the hypersphere crosses the plane, there could be nearer points on the other side of the plane, so the algorithm must move down the other branch of the tree from the current node looking for closer points, following the same recursive process as the entire search.

If the hypersphere doesn't intersect the splitting plane, then the algorithm continues walking up the tree, and the entire branch on the other side of that node is eliminated.

When the algorithm finishes this process for the root node, then the search is complete.16

17

Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

18

Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

19

Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

20

Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

21

Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

22

Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

Implementation

23

ImplementationWeekly Submissions made the job easy!First week Implementation:

Parsing of places2k.txt and airport-locations.txt, Storage in Hash Tables

Second Week Implementation: Make Client and Places Server work Properly IDL implementation: client.x Implementation of Client Logic : client.c Places Server Implementation: places_server.c , client_svc.c

Final Implementation Phase 1 : 3-Tiered Model Implementation Phase 2: Implementation of K-D Tree Phase 3: Implementation of Nearest Neighbor Search

24

client.xclient.c

places_server.c client_svc.c

Implementation Phase 1 : 3-Tiered Model Implementation

2nd IDL : placesclient.x created places_server.c modified to call airport server Ist IDL client.x modified to include “host” input

Phase 2 : K-D Tree Creation Airport server creates K-D tree and stores airport records. Libkdtree++ is used: kd_create(), kd_insert3()

Phase 3: Nearest Neighbor Search kd_nearest_range(tree, point, radius) Calculation of Distance of all the points inside the circle with the Point Top 5 Nearest Airports were selected and returned Change of Resultant Structure: Made as Linked List

25

placesclient.x

placesclient_svc.c

airport_server.c

Learning and ExperienceGREAT LEARNING EXPERIENCE !!

3-Tiered Client - Server ModelData Distribution on different ServersSpace Partitioning of multi dimensional dataSearch in Multi Dimensional data – Practical ApproachWorking with Hash Tables, K-D Trees, Linked List, Sort

26

Summary

Implemented 3-Tiered Client Server Model Use of Hash Table to store places2k.txt Use of K-D Tree to store airport-locations.txt Use of Nearest Neighbor search algorithm Use of Linked List to return Result containing nearest airports

27

Acknowledgements

Martin Krafft, Paul Harris, Sylvain Bougerel Library: libkdtree- Open Source STL Like implementation of K-D Trees

28

Demo

29

30