Database!technology! Data!models!and!history5 New!database ...

TDDD37

!Database!technology!

TDDD37/TDDD46!5

[email protected]

hBp://www.ida.liu.se/~TDDD375

TDDD37

Todays!lecture5

•  Formalities!about!the!course5•  Database!introduction5•  Data!models!and!history5•  Course!content5•  New!database!research!topics!5

25

TDDD37

Course!organization5

•  12!lectures5•  WriBen!exam5•  Lab!exercises5

o  2!computer!labs5o  1!paper!lab!5

•  Project!work5•  2!Lektion!(guided!exercises)5

35 TDDD37

Teaching!staff5

•  Fang!Wei9Kleiner:!Examiner,!lecturer!and!lab!assistant.5•  Jose!M.!Peña:!Lecturer.5

45

TDDD37

Course!literature5

•  Elmasri,!R.,!Navathe,!S.,!Fundamentals!of!Database!Systems,!6th!edition,!Addison9Wesley,!20115

•  Lab!compendium!(available!from!the!course!website)5

55 TDDD37

Laboratory!schedule5

•  Lab!schedule!on!the!course!web9page.5o  One!group!has!two!rooms.5

•  Project!work:!Less!supervision.5o  One!mandatory!section!(Lektion)!discussing!your!data!model.5

65

TDDD37

Laboratory!information5

•  You!need!a!special!database!account5o  automatically!created!at!course!registration5o  [email protected]!if!any!problems5

•  Subscribe!for!labs!via!course!home!page5o  Deadline:!Nov!11.5o  Students!that!retake!the!labs:!discuss!with!Fang5

75 TDDD37

Handing!in!labs5

•  Each!lab!requires!a!wriBen!report5•  Labs!can!be!handed!in:5

o  At!IDAs!postboxes,!opposite!to!Café!Java!(take!a!brown!envelope!for!internal!mail!that!is!usually!available!there,!write!the!name!of!the!lab!assistant!and!drop!it!in!the!mail!room)5

•  Labs!can!be!fetched:5o  From!the!teacher!at!the!lab5o  At!the!OUT!box!outside!each!teachers!office5o  Remember!to!fetch!your!labs,!it!is!your!only!proof!that!you!have!

passed!the!labs.5

85

TDDD37

Labs!reports!deadlines5

•  Lab!reports!are!handed!in:5o  Lab1,!2,!3:!As!soon!as!possible5o  Project’s!ER!diagram:!No!later!than!Dec!25o  Project’s!report:!Before!Christmas5o  Project’s!demo:!last!project!lab5o  Send!e9mail!to!Urkund!with!your!project!code!(read!more!in!

course!website)5

•  Next!occasions:!Next!two!exams!(sometime!in!2014)5•  Later:!Redo!the!labs!!!5

95 TDDD37

Lab!policy5

•  You!are!expected!to!do!the!lab!assignments!by!yourself.!Merely!copying!others!solutions!will!not!be!tolerated,!even!if!you!make!cosmetic!changes!to!the!code/solution.!If!we!suspect!that!this,!or!any!other!form!of!cheating!has!happened!we!will!report!it!to!the!disciplinary!board!of!the!university.5

•  Be!prepared!to!be!asked!questions!by!your!lab!assistant!about!detailed!and!specific!code!and!also!inquiries!about!why!you!have!selected!a!specific!solution.!This!applies!to!all!lab!group!members.5

•  If!you!have!problems!meeting!a!deadline!it!is!much!beBer!to!talk!to!the!instructor!about!it!than!to!cheat.5

105

TDDD37

WriBen!exam5

•  Two!parts:5o  Theoretical!and!practical5o  Have!to!pass!both!parts5

•!Equipment5o  Dictionary!–!not!electronic5o  No!calculator5o  No!books!allowed5

•  Registration!to!the!exam5o  Studentportalen5o  Exam!Jan!17!20145o  Old!exams!available!at!the!course!website,!from!Expeditionen!in!

the!E!building5o  hBp://www.ida.liu.se/~jospe/LostaTentor.pdf5

115 TDDD37

Pre9knowledge!for!the!course5

•  Basic!data!structure:!trees,!graphs,!etc.5•  Algorithms:!sort,!search,!hash!functions5•  Hardware:!storage!structure,!harddisk,!memory,!cache,!

CPU5•  Operation!systems:!concurrency!control5

•  !!we!will!not!repeat!these!issues!in!the!class5

125

TDDD37

LiU:!Disciplinary!actions5

•  Any!kind!of!academic!dishonesty,!such!as!cheating,!plagiarism,!use!of!unauthorized!assistance,!fraud!and!failure!to!comply!with!University!examination!rules,!may!result!in!the!filing!of!a!complaint!to!the!University!Disciplinary!CommiBee.!The!potential!penalties!include!expulsion,!suspension,!and!revocation!of!previously!earned!grade!or!degree.5

•  LiU!Rules!and!regulations5

135 TDDD37

Organization!issues55

•  Question?5

145

TDDD37

Types!of!Database!Applications5

•  Traditional!Applications:5o  Numeric!and!Textual!Databases5

•  More!Recent!Applications:5o  Multimedia!Databases5o  Geographic!Information!Systems!(GIS)5o  Data!Warehouses5o  Real9time!and!Active!Databases5o  Internet!search!engines5o  Mobile!apps5o  Many!others5

155 TDDD37

What!is!a!database?!9!Definition5

•  Database:5o  an!integrated!collection!of!data!records,!files,!and!other!objects.!5

•  Database!Management!System!(DBMS):5o  A!software!package/!system!to!facilitate!the!creation!and!

maintenance!of!a!computerized!database.5

•  Database!System:5o  The!DBMS!software!together!with!the!data!itself.!Sometimes,!the!

applications!are!also!included.5

165

TDDD37

Pre9DBMS!(1960s)5

175 TDDD37

A!new!generation!of!DBMS5

185

TDDD37

The!relational!revolution!(1970’s)5

•  A!simple!data!model:!data!is!stored!in!relations!(tables)5•  A!declarative!query!language:!SQL5

SELECT Account.owner

FROM Account, Branch WHERE Account.balance = 0

•  Programmer!specifies!what!answers!a!query!should!return,!but!not!how!the!query!is!executed5

•  DBMS!picks!the!best!execution!strategy!based!on!availability!of!indexes,!data/workload!characteristics,!etc.5

•  Provides!physical/data/independence2

195 TDDD37

Physical!data!independence5

•  Applications!should!not!need!to!worry!about!how!data!is!physically!structured!and!stored5

•  Applications!should!work!with!a!logical/data/model/and!declarative/query/language2

•  Leave!the!implementation!details!and!optimization!to!DBMS5

•  The!single!most!important!reason!behind!the!success!of!DBMS!today5o  And!a!Turing!Award!for!E.!F.!Codd!in!19815

205

TDDD37

Course!goals5•  Design!a!database!for!a!given!application5

o  EER9modeling,!Normalization5•  Design!and!use!a!relational!database5

o  Concept!of!relations5o  Use!SQL,!MySQL5

•  Understand!how!the!database!is!stored!on!the!computer5o  Basic!technology,!file!structures,!indexing5o  Impact!on!database!performance5o  B9Trees,!Hashing5

•  Understand!how!databases!can!support!multiple!users5o  Views5o  Transactions,!serialization5

•  Understand!how!persistency!can!be!guaranteed5o  Recovery5

Application5

System5

215 TDDD37

Category!of!data!models5

•  Conceptual!(high9level,!semantic)!data!models:5o  Provide!concepts!that!are!close!to!the!way!many!users!perceive!

data.5o  (Also!called!entity9based!or!object9based!data!models.)5

•  Physical!(low9level,!internal)!data!models:5o  Provide!concepts!that!describe!details!of!how!data!is!stored!in!

the!computer.5

•  Implementation!(representational)!data!models:5o  Provide!concepts!that!fall!between!the!above!two,!used!by!many!

commercial5o  DBMS!implementations!(e.g.!relational!data!models!used!in!

many!commercial!systems).!5

225

TDDD37

Example!of!a!database5

•  Mini9world!for!the!example:5o  Part!of!a!UNIVERSITY!environment.5

•  Some!mini9world!entities:5o  STUDENTs5o  COURSEs5o  SECTIONs!(of!COURSEs)5o  (academic)!DEPARTMENTs5o  INSTRUCTORs5

235 TDDD37

Example!of!a!database5

•  Some!mini9world!relationships:5o  SECTIONs!are!of!specific!COURSEs5o  STUDENTs!take!SECTIONs5o  COURSEs!have!prerequisite!COURSEs5o  INSTRUCTORs!teach!SECTIONs5o  COURSEs!are!offered!by!DEPARTMENTs5o  STUDENTs!major!in!DEPARTMENTs5

•  Note:!During!the!course!you!will!learn!how!to!express!this!as!an!E/R!model!and!translate!the!model!into!a!data!model.5

245

TDDD37 255 TDDD37

Characteristics!of!DB!approach5

•  Self9describing!nature!of!a!database!system:5o  A!DBMS!catalog!stores!the!description!of!a!particular!database!

(e.g.!data!structures,!types,!and!constraints)5o  The!description!is!called!meta9data.5o  This!allows!the!DBMS!software!to!work!with!different!database!

applications.5o  Important!information!for!query!optimization.5

•  Support!of!multiple!views!of!the!data:5o  Each!user!may!see!a!different!view!of!the!database,!which!

describes!only!the!data!of!interest!to!that!user.5

265

TDDD37 275 TDDD37

Schema!vs.!instance5

•  Database!Schema:5o  The!description!of!a!database.5o  Includes!descriptions!of!the!database!structure,!data!types,!and!

the!constraints!on!the!database.5

•  Database!State:5o  The!actual!data!stored!in!a!database!at!a!particular!moment!in!

time.5o  This!includes!the!collection!of!all!the!data!in!the!database.5o  Also!called!database!instance!(or!occurrence!or!snapshot).5o  The!term!instance!is!also!applied!to!individual!database!

components,!e.g.!record!instance,!table!instance,!entity!instance5

285

TDDD37

An!Example!schema5

295 TDDD37

Example!of!instances5

305

TDDD37

DBMS!languages5

•  Data!Definition!Language!(DDL)5

•  Data!Manipulation!Language!(DML)5o  High9Level!or!Non9procedural!Languages:!These!include!the!

relational!language!SQL5•  May!be!used!in!a!standalone!way!or!may!be!embedded!in!a!programming!language5

o  Low!Level!or!Procedural!Languages:5•  These!must!be!embedded!in!a!programming!language5

315 TDDD37

Concurrency!control5

•  Sharing!of!data!and!multi9user!transaction!processing:5o  Allowing!a!set!of!concurrent!users!to!retrieve!from!and!to!update!

the!database.5o  Concurrency!control!within!the!DBMS!guarantees!that!each!

transaction!is!correctly!executed!or!aborted5o  Recovery!subsystem!ensures!each!completed!transaction!has!its!

effect!permanently!recorded!in!the!database5o  OLTP!(Online!Transaction!Processing)!is!a!major!part!of!database!

applications.!This!allows!hundreds!of!concurrent!transactions!to!execute!per!second.5

325

TDDD37

Handling!multiple!users5

335 TDDD37

Persistency!and!error5

345

TDDD37

History!of!data!models5

•  Network!Model!(introduced!in!the!60’s)5•  Hierarchical!Model!(introduced!in!the!60’s)5•  Relational!Model!(introduced!in!the!70’s)5•  Object9oriented!Data!Models!(introduced!in!the!80’s)5•  Object9Relational!Models!(introduced!in!the!90’s)5•  XML!databases!and!Hybrid!models!(2000!9!)5

355 TDDD37

Today5

•  Relational!databases!dominating.5•  Hierarchical!and!navigational!databases!still!in!use.

(Mainly!in!flight!industry.)5•  Object9oriented!databases!in!use,!especially!as!object9

relational!databases.5•  XML9databases!–!new,!first!commercial!products!

appearing.5

365

TDDD37

Major!DBMS!today5

•  Oracle5•  IBM!DB2!(from!System!R,!System!R*,!Starburst)5•  Microsoft!SQL!Server5•  Teradata5•  Sybase!(acquired!by!SAP)5•  Informix!(acquired!by!IBM)5•  PostgreSQL!(from!UC!Berkeley’s!Ingres,!Postgres)5•  Tandem!NonStop!(acquired!by!Compaq,!now!HP)5•  MySQL!(acquired!by!Sun,!then!Oracle)5•  ?!SQLite5•  ?!Microsoft!Access5•  ?!BerkeleyDB!(acquired!by!Oracle)5

375 TDDD37

XML5

•  Extensible/Markup/Language/(XML)/5o  Standard!for!data!representation!and!exchange!5o  Document!format!similar!to!HTML!5o  Tags!describe!content!instead!of!formaBing!5o  Semi9structured,!flexible!schema5o  Hierarchical,!tree!formed5

•  Query!language5o  XPath5o  XQuery5

385

TDDD37

GIS!and!Graph!databases5

•  Geographic!Information!Systems5o  Systems!storing,!displaying,!updating!spatial!data5o  Spatial!data:!describing!objects!that!have!a!distinct!position!!!

aBributes!and!relationships!to!each!other!within!a!defined!space5o  Storage!of!the!objects!and!indexing5o  Queries!such!as!Nearest!Neighbor!problems!(kNN)5

•  Graph!databases5o  Road!networks!(route!planning)5o  Social!networks5o  Bio!Data5o  !!how!to!store!and!query!graph!data?5

395 TDDD37

Knowledge!bases5

•  Knowledge!base!55o  Knowledge!base!=!Data!+!Rules5o  Sometimes!SQL!is!too!weak!to!express!certain!queries5o  Finding!shortest!paths!between!two!vertices!in!a!graph!is!not!

expressible!in!SQL5o  !!solution:!adding!recursion!!!Datalog5

•  Semantic!Web5o  Adding!semantics!to!the!web!data5o  Facilitate!the!semantic!search!on!the!internet5o  Challenges:!Store,!indexing!and!query!such!huge!body!of!data5

405

TDDD37

NoSQL5

•  NoSQL!stands!for!Not!only!SQL5•  Initiated!by!Google5

o  Hardware!to!store!the!indexed!data:!PCs,!but!many!of!them5o  Software:!Hadoop!–!in!charge!of!distributing!the!task!over!the!

network!and!collecting!the!results5o  Data!are!stored!with!redundancy5

•  Amazon,!Facebook!and!twiBer!etc.!have!their!own!system!for!data!management,!in!the!same!line5

•  New!trend!on!database!systems!dealing!with!huge!amount!of!dynamic!data!without!explicit!schema,!in!a!distributed!environment.5

415