Milestone 1 and SVN Hans-Arno Jacobsen Take note of TA Schedule et al. (Tech Talks in Lab.) Note.

Milestone 1 and SVNHans-Arno Jacobsen

Take note of TA Schedule et al. (Tech Talks in Lab.)http://ece297.msrg.utoronto.ca/T2011s/TeachingAssistants

Note the team registration deadline, Friday, January 15th.

If we don’t hear from you, we’ll assign you a team.

I shall follow the submission instructions.

Labs Tue & Wed 3-6 PM:GB243, GB251, SF2102

Tech talks 3-4 PM:GB404 (Tue) & GB405

(Wed)

Agenda

• Storage server Milestone 1 overview

• Subversion (svn)– Code versioning– Concurrent development

Application

Storage server Milestone 1

4

Client

S

Table Table

Table Network

Config.

Storage server

Table

Cities table

key province temperature

Toronto Ontario -7.0

Winnipeg Manitoba -17.4

Vancouver British Columbia 8.1

get(…), set(…), …

Tables are memory

resident only

One client at a time

Limited API

“Toronto Ontario -7.0“ Data schema is

strings only

auth(…)

Tables and records

• Storage server manages tables

• A table is a collection of records

• Each record is uniquely identified by a key

• Records comprise one or more values

• A record is a string with

a maximum number of

characters (Milestone 1!)

5

Course Mark Letter

ECE297 95 Aplus

ECE344 92 Aplus

ECE451 87 A

“95 Aplus“

6

Keys are unique

6

Course Mark

ECE297 95

ECE344 92

ECE451 87

Course is the table’s key field. The key must be unique.

Course Mark

ECE297 95

ECE344 92

ECE297 87

ECE451 87

Storage server API

void *storage_connect(char *hostname, int port)

int storage_disconnect(void *conn)

int storage_auth(char *username, char *passwd, void *conn)

int storage_get(char *table, char *key, struct storage_record *record, void *conn)

int storage_set(char *table, char *key, struct storage_record *record, void *conn)

What do I need to implement?

8

What do I need to implement?

• Client-side library functions (get, set etc.)• Server-side functions (get, set etc.)• Communication between client and server

– How does the server know what function is called?

– How does the server know what the input to the function is?

• Table management– How can the server find a table, a record in a

table? Determine the table/record does not exist?

9

What do I need to implement?

Client

S

Table Table

Table

Config.

Storage server

Table

… storage_connect(…)… storage_auth(“admin”, “dog4sale”, …)… // gets & sets… storage_disconnect(…)

• Client-server authentication

server_host localhost server_port 1111server_username admin server_password_enc xxxnq.BMCifhUtable marks

> cd storage/src > make encrypt_passwd > ./encrypt_passwd dog4sale

xxxnq.BMCifhU

Authentication onthe server side &

associated error handling is missing

What don’t I need to implement?

• Read the skeleton code!

ECE 29711

ECE 297 12

get & set

Course Mark

ECE297 95

ECE344 92

ECE451 87

“MyCourses”

struct storage_record { char value[MAX_VALUE_LEN]; uintptr_t metadata[8];

}; (given, see handout)???

• int storage_get(const char *table, const char *key, struct storage_record *record, void *conn);

• int storage_set(const char *table, const char *key, struct storage_record *record, void *conn);

???

What else do we need?

• Besides retrieving (get) and inserting (set) a record?

Functionality

• Retrieve an existing record– storage_get(…) function

• Insert a new record– storage_set(…) with a key that does not already exist

in the table

• Update an existing record– storage_set(..) with a key that already exists in the

table

• Delete a record– storage_set() with a key that already exists in the

table and a NULL value as record

Error conditions• ERR_INVALID_PARAM (all 5 functions)

– parameters do not conform to the specification. • ERR_CONNECTION_FAIL (all 5 functions)

– connection problems to server• ERR_AUTHENTICATION_FAILED

– client-server authentication problems• ERR_TABLE_NOT_FOUND (storage_get()/_set)

– specified table does not exist• ERR_KEY_NOT_FOUND (storage_get())

– server indicates that the specified key does not exist in the specified table.

• ERR_UNKNOWN– flag any other errors (out of memory, file not found, …)

Questions?

BPMeduNet

• Exchange program with– Free U. Berlin, Germany– U. of Applied Sciences, Graz, Austria– U. of Coimbra, Portugal

• Spend a semester at one of these places

• Do a summer research internship there

• Scholarships available

Subversion (svn)

Note the quiz in the lab

later in January

ECE 297 26

Subversion (client: svn)• Main tasks

– Version control• “I just fixed something and nothing works anymore”

– Shared, concurrent development• Development in a team with 2 to 2000 developers• Also helpful, if you work alone

• Also applicable to concurrent development on text, web pages, …

• Concurrent development is the reality (cvs, svn, git & github, ClearCase, …)

ECE 297 27

Basic idea

• Like a file server, but tracks ALL modifications to files and to directory structure

• Where is yesterday’s version?• Where is Tuesday’s version from 1:03 AM?• Who modified yesterday’s version? What were the

changes? What’s the difference to the current copy?

svn repository

Harry’s working copy Sally’s working copy

How to proceed from here?

• All:– Go through the examples in our Assignment 1

handout– Read at least Chapter 2 of the svn-book (pp. 15 – 37)

• You should master a handful of commands– svn help <command> !!– svn add, svn delete, svn checkout, svn update, svn

commit– Helpful are also: svn info, svn help, …

• Also, see Tech talk in labs on svn !

svn’s copy-modify-merge model

• Each user connects to repository to retrieve working copy of source tree

• Users work simultaneously and independently and modify private copies (i.e., their working copy)

• Ultimately, working copies are merged– svn assists in merging– User may have to resolve conflicts

manually!

Copied from

http://svnbook.red-bean.com/

svn checkout

Both edit files

svn commit

svn commit

1 2

3 4

Copied from

http://svnbook.red-bean.com/

svn update

svn commitsvn update

1 2

3 4

Copied from

http://svnbook.red-bean.com/

svn update

svn commitsvn update

C – conflict(there may be a conflict)

Conflict example$ svn update Conflict discovered in

'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit, (h)elp

for more options : p

C sandwich.txt Updated to revision 2.

$ lssandwich.txt

sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2

E.g., Sally changes sandwich.txt,

but does not yet commit.

Meanwhile, Harry commits

changes to sandwich.txt.

Sally has to update her

working copy before she can commit,

and she gets a conflict.

Conflict example$ svn update Conflict discovered in

'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit,

(h)elp for more options : p

C sandwich.txt Updated to revision 2.

$ lssandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2

Sally’s file prior to svn update

Conflict example$ svn update Conflict discovered in

'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit,

(h)elp for more options : p

C sandwich.txt Updated to revision 2.

$ lssandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2

Sally’s file prior to any of her changes.

Her last svn checkout.

(a.k.a., BASE revision)

Conflict example$ svn update Conflict discovered in

'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit,

(h)elp for more options : p

C sandwich.txt Updated to revision 2.

$ lssandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2

File just received from repository due to svn

update.

(a.k.a., HEAD revision)

Top piece of bread Mayonnaise Lettuce Tomato Provolone <<<<<<< .mine Salami Mortadella Prosciutto ======= Sauerkraut Grilled Chicken >>>>>>> .r2 Creole Mustard Bottom piece of bread

$ cat sandwich.txt

Conflict markers

Changes by user to conflicting area in file

Changes received from repository within conflicting

area in file.

Conflict example

$ svn resolve --accept working sandwich.txt Resolved conflicted state of 'sandwich.txt'

$ svn commit -m “<message text>."

Others:$ svn resolve –accept base …$ svn resolve –accept mine-full …$ svn resolve –accept theirs-full …

Conflict example

Say, we had disregard the conflict created by

svn update and go ahead and commit

$ svn commit -m "Add a few more things"

svn: Commit failed (details follow): svn: Aborting commit: '/home/sally/svn-

work/sandwich.txt' remains in conflict

Questions

Effective use of svn

• Commit often, but don’t break the build

• Comment your commits

• Retrieve an older version

• See who modified what

• …

Pass/Fail quiz to bank points

• Optional quiz to bank a few points that can be traded in for failing tests later on (redemption would happen automatically)

• Close to the code submission deadline for Milestone 1, we’ll quiz you in the lab on – the effective use of svn and – the proper understanding of the submit script

• Detailed modus operandi see our web site

ECE 297 44

Recommended Development Cycle

• Update sources from repository• Develop the feature, fix the bug, …• Add new test cases to test new functionality• Test system with existing and new test cases

– Fix any problems and regressions

• Test again until all tests pass• Commit sources including the new test cases to

the repository

ECE 297 45

Questions

ECE 297 46

The storage server vis-à-vis a fully fledged DBMS

• In addition to the functionality provided by our storage server, a database management system (DBMS) offers among others:– A declarative query language, an expressive data

definition language (e.g., SQL)– Transaction support– Recovery from failure– Security and access control– …

• Examples of DBMSs are IBM DB2, Microsoft SQL Server, Oracle …

46

Why then a storage server at all?

• DBMSs are developed to be most flexible and general

– Broadest possible applicability

• Flexibility and generality comes at a price

– $$$$, maintenance, administration, performance

• Many applications do not require bells & whistles of a DBMS

• Each DBMS contains a storage server

• Storage server as basic building block

• Management of configuration data …

ECE 297 47

ECE 297 48

errno

• Functions return values to indicate failure– typically -1, null pointer, or constants, e.g.,EOF

• But, what exactly is the error?– Need to consult error code stored in the

variable errno declared in errno.h

• You can’t use errno to determine failure

• If a call failed, then examine errno

ECE 297 50

Design considerations

• How should the data managed by the storage server be stored on disk?

• What is the communication protocol between the client library and storage server?

Design considerations

• How is the connection state between client and server represented? – Is the connection reused for multiple get and set

requests from the client? – What are the implications of your design when

multiple clients interact with the server at the same time?

• How do the server and client library handle failures? – For example, what happens if there's an error in the

configuration file? – What if the server is not running? – What if an invalid table name is given?

ECE 297 53

ECE 297 54Copied from http://svnbook.red-bean.com

High-level SVN Architecture