Temporal Databases: Queries

Temporal Databases: Queries

Kristian Torp

Department of Computer ScienceAalborg University

people.cs.aau.dk/˜[email protected]

November 2, 2015

daisy.aau.dk

Kristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 1 / 122

http://people.cs.aau.dk/~torp

mailto:[email protected]

http://daisy.aau.dk

Outline

1 Overview and FoundationCurrent, Valid Time, Transaction Time, and BitemporalOverlap Between Intervals

2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators

6 TimesliceValid TimeBitemporal

7 Summary


Learning Goals

Learning GoalsUnderstand the differences between valid time and transaction time

Understand bitemporal support

Temporal queries: aggregation, join, and timeslicing

Temporal interval and overlap

NoteThe four-timestamp data model used is widely supported

n person will make at least n + 1 temporal data models

Words period and interval are centralAnd switched English versus SQL


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Bitemporal Graphs: Explicit Valid Time

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Bill, Prozac

Bill, Prozac

Bill, Prozac

Bill, Morphine


Temporal Data: Tabular Overview

pname medicine

Bill ProzacBill Morphine

pname medicine vts vte tts tte

Bill Prozac 1 6 1 5Bill Prozac 1 13 5 8Bill Prozac 1 9 8 UC

Bill Morphine 10 15 2 UC

Current Bitemporal

pname medicine vts vte

Bill Prozac 1 9Bill Morphine 10 15

pname medicine tts tte

Bill Prozac 1 5Bill Prozac 5 8Bill Prozac 8 UC

Bill Morphine 2 UC

Valid Time Transaction Time


Bitemporal Graphs: Implicit Valid Time

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Joe, Shoe Joe, Shoe

Joe, Toy Joe, Toy

Joe, Shoe


Overview: Tabular Form

ename dept

Joe Shoe

ename dept vts vte tts tte

Joe Shoe 1 now 1 5Joe Shoe 1 5 5 UC

Joe Toy 5 now 5 10Joe Toy 5 10 10 UC

Joe Shoe 10 now 10 UC

Current Bitemporal

ename dept vts vte

Joe Shoe 1 5Joe Toy 5 10Joe Shoe 10 now

ename dept tts tte

Joe Shoe 1 5Joe Toy 5 10Joe Shoe 10 UC

Valid Time Transaction Time


now in Real Life

Examples of now in dataMarriage certificate

Hiring period

Permanent salary benefits

How long you have owned your car/house/dog

Office room allocation

Sickness period

NoteWhen end-date is uncertain/unknown/unspecified use nowInterpretation of now different valid time and transaction time

Also called until changed (UC) for transaction time


Valid Time or Transaction Time?

What do you think?Reservation time this room this seminar?

The appointments in your calendar?

The timestamps in a GPS log from a vehicle?

The positions that you have held listed on your CV?

File modification timestamp?

TV commercial: “Offer only valid next week!”

Web log?

Mortgage loan period?

Diary?

Bank day/date (tricky)?

Birthday (tricky)?


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Allen’s Operators

0 1 2 3 4 5 6 7 8 9 10 11 12

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

X Note13 cases are complete

How intervals can overlap

The symmetriesHalf-open/closed intervals

Does not matter!

All cases have namesSome of them are weird

J.F.Allen CACM 26(11), Nov.1983


Simplified Overlap Operators

Example (Allen)

0 1 2 3 4 5 6 7 8 9 10 11 12

YY

YY

YY

YY

YY

YY

YX

Example (Simplified)

0 1 2 3 4 5 6 7 8 9 10 11 12

Y YY

YY

YX


Names Simplified Overlap OperatorsExample (Naming)

0 1 2 3 4 5 6 7 8 9 10 11 12

Before After

Left Overlaps

During

Right Overlaps

Total Overlaps

X

NoteThese names are not official just used for reference in thispresentation.


Five Cases Simplified Overlap Operators

Example (The Cases)

0 1 2 3 4 5 6 7 8 9 10 11 12

Y YY

YY

YX

The Boolean Expressions

Case Expression

No Y.vte ≤ X.vts or X.vte ≤ Y.vtsRight X.vts < Y.vts and Y.vts < X.vteLeft X.vts < Y.vte and Y.vte < X.vteFull Y.vts ≤ X.vts and X.vte ≤ X.vte

Do intervals overlap?X.vts < Y.vte AND Y.vts < X.vte

Get Overlap[greatest(X.vte, Y.vte), least(X.vts, Y.vts))

provided that the intervals X and Y overlap


Query Types: Current, Sequence, and Non-Sequenced

NoteCurrent state for backwards compatibility

Sequenced for built-in temporal semantic/support

Non-sequenced for accessing timestamps columns directlyKristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 17 / 122

Valid Time versus Transaction Time

ExerciseIn groups, use 3 minutes to compare valid time and transaction time.

Valid TimePast, present, and future

User supplied

Supports now

Physical delete

“History correction”

Transaction TimePast and present

System supplied

Supports now

Logical delete

History preserving (log)

Valid Time for sure if:Time is into the future

Time can be supplied by the user





User supplied

Supports now

Physical delete



System supplied

Supports now

Logical delete








User supplied

Supports now

Physical delete



System supplied

Supports now

Logical delete





Summary: Motivation I

Valid, Transaction, and User-Defined TimeValid time and transaction time are orthogonalValid time: Past, present, and future

Can be user-defined (default similar to transaction time)

Transaction time: Past and presentAlways system defined

User-defined time (if not valid time or transaction time)e.g., birthday

Query TypesCurrent

Sequenced

Non-sequenced


Summary: Motivation II

Database VariablesSpecial variable now

Different semantics valid time and transaction time

Databases do not support variablesnow = 9999-12-31 (maximum value of date domain)

IntervalsTimestamps are half-open intervals

Closed intervals +/- 1 therefore more complicated

Overlap between intervals is essential to understandNot difficult5 cases to consider


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


The Idea: One

Example (Emp)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HRKim, HR

Example (Dept)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

HR, 10

Example (Emp Temporal Natural Join Dept)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HR, 10Kim, HR, 10


The Idea: Two

Example (Emp)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HRKim, HR

Example (Dept)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

HR, 10HR, 20


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HR, 10Kim, HR, 10

Kim, HR, 20


The Idea: Three

Example (Emp)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HRKim, HR

Example (Dept)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

HR, 5


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15


The Idea: Four

Example (Emp)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HRKim, RD

Example (Dept)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

HR, 10RD, 20


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Joe, HR, 10Kim, RD, 10


Natural Join Informal Definition

Idea Temporal Natural JoinThe explicit attribute must be equal

The intersection of the timestamp intervals must be non-empty

The Overlap Cases to Consider

0 1 2 3 4 5 6 7 8 9 10 11 12

Before After

Left Overlaps

During

Right Overlaps

Total Overlaps

X


Temporal Natural-Join Example

s e l e c t cname , adr , ptype , p . vts , p . v te −− t o t a l over lapfrom c u s t v t as c , p r o p e r t y v t as pwhere c . c id = p . c idand c . v t s <= p . v ts and p . v te <= c . v te unions e l e c t cname , adr , ptype , c . vts , p . v te −− r i g h t over lapfrom c u s t v t as c , p r o p e r t y v t as pwhere c . c id = p . c idand c . v t s between p . v t s and p . v teand c . v te > p . v te unions e l e c t cname , adr , ptype , p . vts , c . v te −− l e f t over lapfrom c u s t v t as c , p r o p e r t y v t as pwhere c . c id = p . c idand c . v te between p . v t s and p . v teand c . v t s < p . v t s union −− dur ings e l e c t cname , adr , ptype , c . vts , c . v tefrom c u s t v t as c , p r o p e r t y v t as pwhere c . c id = p . c idand c . v t s between p . v t s and p . v teand c . v te between p . v t s and p . v te


Quiz: What is the Natural-Join Result

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Bill, supportPam, sale Pam, RD Pam, RD

Sam, RDJim, RD Jim, sale Jim, RD

Tim, web Tim, saleTom, sale Tom, test

Joe, RD Joe, sale Joe, RD

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

RD, 50 RD, 60 RD, 80sale, 10 sale, 20 sale, 25

web, 5test, 20 test, 15

support, 30


Summary: Temporal Join

Main PointsTemporal natural join most widely used

Like in non-temporal databases

Valid-time natural-join complicated because multiple overlapscenarios to considerIt is possible to join a valid-time table with a non-temporal table

On all rows in the non-temporal table the valid-time is [-forever, forever)

NoteEfficiency may be a problem

Can you see why?

Temporal versions of Cartesian product, outer joins, and equi join arewell-defined

See www.cs.arizona.edu/projects/stagg/papers/TR-71.pdf


www.cs.arizona.edu/projects/stagg/papers/TR-71.pdf

Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


The Data Foundation

Query in EnglishFind the number of persons employed over time and in which intervals.

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Joe, 10 Joe, 12 Joe, 13Jim, 12 Jim, 14

Jill, 10 Jill, 15Pat, 9 Pat, 9 Pat, 10

Bill, 9Boris, 9

Pam, 11Kurt, 11


Constant Regions

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25



Bill, 9Boris, 9

Pam, 11Kurt, 11

NoteVertical lines at all vts and vte values


The Result

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25



Bill, 9Boris, 9

Pam, 11Kurt, 11 cnt vts vte

3 1 35 3 55 5 66 6 75 7 114 11 135 13 145 14 166 16 175 17 186 18 195 19 215 21 234 23 24


Main Idea in Finding Constant Regions

Five cases to considerNo other interval overlaps me

Another interval has vts between my vts and vte

Another interval has vte between my vts and vte

Another interval totally overlaps me

Another interval is after me

NoteFor each case there can be no vts or vte between returned result

A single interval can qualify for more than one of the cases aboveSQL is declarative therefore ”after” could be ”before”

But just one of the cases can be considered


Main Query

Example (The Main Query)s e l e c t count (ename) as cnt ,

cons t reg ions . v t s as vts ,cons t reg ions . v te as vte

from emp vt , cons t reg ionswhere emp vt . v t s < cons t reg ions . v teand cons t reg ions . v t s < emp vt . v tegroup by cons t reg ions . vts , cons t reg ions . v teorder by vts , v te


Constant Regions Part 1: No Overlap

wi th cons t reg ions as (−− get t1 . v t s and t1 . v tes e l e c t t1 . v t s as vts , t1 . v te as vtefrom emp vt t1where not e x i s t s (

s e l e c t ∗from emp vt t3where ( t1 . v t s < t3 . v t s and t3 . v t s < t1 . v te ) or

( t1 . v t s < t3 . v te and t3 . v te < t1 . v te ) )

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

result = t1

not exists t3

not exists t3


Constant Regions Part 2:

union−− t1 . v t s and t2 . v t ss e l e c t t1 . v t s as vts , t2 . v t s as vtefrom emp vt t1 , emp vt t2where t1 . v t s < t2 . v t s and t2 . v t s < t1 . v teand not e x i s t s (

s e l e c t ∗from emp vt t3where ( t1 . v t s < t3 . v t s and t3 . v t s < t2 . v t s ) or

( t1 . v t s < t3 . v te and t3 . v te < t2 . v t s ) )

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

t1

t2

result



union−− t1 . v t s and t2 . v tes e l e c t t1 . v t s as vts , t2 . v te as vtefrom emp vt t1 , emp vt t2where t1 . v t s < t2 . v te and t2 . v te < t1 . v teand not e x i s t s (

s e l e c t ∗from emp vt t3where ( t1 . v t s < t3 . v t s and t3 . v t s < t2 . v te ) or

( t1 . v t s < t3 . v te and t3 . v te < t2 . v te ) )

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

t1

t2

result


Constant Regions Part 4: Before

union−− t1 . v te and t2 . v t ss e l e c t t1 . v te as vts , t2 . v t s as vtefrom emp vt t1 , emp vt t2where t1 . v te < t2 . v t sand not e x i s t s (

s e l e c t ∗from emp vt t3where ( t1 . v te < t3 . v t s and t3 . v t s < t2 . v t s ) or

( t1 . v te < t3 . v te and t3 . v te < t2 . v t s ) )

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

t1

t2

result



union−− t1 . v te and t2 . v tes e l e c t t1 . v te as vts , t2 . v te as vtefrom emp vt t1 , emp vt t2where t2 . v t s < t1 . v te and t1 . v te < t2 . v teand not e x i s t s (

s e l e c t ∗from emp vt t3where ( t1 . v te < t3 . v t s and t3 . v t s < t2 . v te ) or

( t1 . v te < t3 . v te and t3 . v te < t2 . v te ) ) )

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

t1

t2

result


Correctness of Constant Regions

Five = Original row + Four Combinations[t1.vts, t1.vte)

[t1.vts, t2.vts)

[t1.vts, t2.vte)

[t2.vte, t2.vts)

[t2.vte, t2.vte)

NoteAll cases are covered

Informal argument


Quiz

Query in EnglishFind the constant regions and average budget for all departments overtime.

Example (Data Foundation)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25



support, 30


Quiz: Solution, One

Example (Data Foundation with Dividers)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25



support, 30

Rest of SolutionNow we have the constant regions

Next step is to do the aggregation within each region


Quiz: Solution, Two

cnt vts vte

50.00 1 430.00 4 621.67 6 821.25 8 1026.67 10 1133.33 11 1533.75 15 1632.50 16 1737.50 17 2245.00 22 24

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

50.0030.00

21.6721.25

26.6733.33

33.7532.50

37.5045.00


Summary: Aggregation

Main PointsConstant regions must be identified

Can be done in multiple waysApproach used is consider efficient (allowed to prove me wrong :-))

Due to various ways intervals may overlap the queries are long

Same approach to average, count, minimum, and so on

NoteThe approach also works if grouping on non-temporal column

It is an exercise to figure-out how!


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Coalescing in Figures

Example (How long have Lars and Peter been with us?)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Lars, web Lars, web

Lars, web

Lars, test Lars, testPeter, web Peter, test Peter, test

Example (Result)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

LarsPeter

NoteSelect just the employee name

Coalescing resulted in fewer rows


Coalescing in Figures

Example (How long have Lars and Peter been with us?)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Lars, web Lars, web

Lars, web


Example (Result)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

LarsPeter

NoteSelect just the employee name

Coalescing resulted in fewer rows


Coalescing in Figures, cont

Example (How long have the ’web’ and ’test’ projects been active?)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Lars, web Lars, web

Lars, web


Example (Result)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

testweb web

NoteCoalescing gives better overview

Coalescing gives new insight


Coalescing in Figures, cont

Example (How long have the ’web’ and ’test’ projects been active?)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Lars, web Lars, web

Lars, web


Example (Result)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

testweb web

NoteCoalescing gives better overview

Coalescing gives new insight


Example: Valid-Time State Data Model

Example (State and Event)

pname vts vte

Bush 1989 1993Clinton 1993 1997Clinton 1997 2001Bush jr. 2001 2005Bush jr. 2005 2009Obama 2009 2013Obama 2013 2017

pname vts vte

Bush 1989 1993Clinton 1993 2001Bush jr. 2001 2009Obama 2009 2017

QuestionsDo the tables have the same content?

Does it have anything to do with duplicate elimination?

What can you say about primary keys (tricky)?


Example: Event Data Model

Example (Similar Content)

pname vts vte

Bush 1989 1993Clinton 1993 1997Clinton 1997 2001Bush jr. 2001 2005Bush jr. 2005 2009Obama 2009 2013Obama 2013 2017

pname vts

Bush 1989Clinton 1993Clinton 1997Bush jr. 2001Bush jr. 2005Obama 2009Obama 2013

pname vts

Bush 1989Clinton 1993Bush jr. 2001Obama 2009

QuestionsAny implicit assumptions for state or event table?

Future data any differences?

What are the pros/cons state versus event table?


Quiz: Coalescing

QuestionsHow much space can be saved coalescing temporal data?

Informal argument using examples!

Can data in valid-time event tables be un-coalesced?Informal argument using examples!

AnswersThe saving coalescing the data stored can be significant. Considermany overlaps of value equivalent rows.

Event-tables can also be un-coalesced.


Quiz: Coalescing

QuestionsHow much space can be saved coalescing temporal data?

Informal argument using examples!

Can data in valid-time event tables be un-coalesced?Informal argument using examples!

AnswersThe saving coalescing the data stored can be significant. Considermany overlaps of value equivalent rows.

Event-tables can also be un-coalesced.


Advantages and Disadvantages

CoalescedMinimal space

Row delete info. lost

Queries fasterLess data

Modifications slowerMany checks

Un-coalescedAdditional space

Possibly much larger

Queries slowerMore data

Modifications fasterLess checking


Coalescing in SQL (Proposed by Celko/Bohlen)s e l e c t d i s t i n c t f i . ename , f i . v ts , l a . v tefrom pro jemp vt f i , p ro jemp vt l awhere f i . v t s < l a . v teand f i . ename = l a . enameand not e x i s t s ( −− ensure no holes and extend towards f i . v t s

s e l e c t ∗from pro jemp vt miwhere mi . ename = f i . enameand f i . v t s < mi . v t s and mi . v t s < l a . v teand not e x i s t s (

s e l e c t ∗from pro jemp vt a1where a1 . ename = f i . enameand a1 . v t s < mi . v t s and mi . v t s <= a1 . v te ) )

and not e x i s t s ( −− ensure only maximal per iodss e l e c t ∗from pro jemp vt a2where a2 . ename = f i . enameand ( a2 . v t s < f i . v t s and f i . v t s <= a2 . v te

or a2 . v t s <= l a . v te and l a . v te < a2 . v te ) )order by f i . ename , f i . v t s


Quiz: Coalesced?

Example (Is the data coalesced?)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Shoe Joe, Toy

Tom, Shoe Tom, Shoe Tom, Toy


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


Tom, Shoe

Tom, Toy


Quiz: Coalesced?


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


Tom, Shoe Tom, Shoe Tom, Toy


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


Tom, Shoe

Tom, Toy


Questions Related to Coalesced

QuestionsCan coalescing increase the number of rows in table?

What happens if you coalesce a table/query twice?

If you make a projection on a coalesce table is the result coalesced?

If you make a selection on a coalesce table is the result coalesced?

Are ename and vts a primary key of a coalesced table?


Summary: Coalescing

Main PointsSimilar to duplicate elimination

also called fold/unfold

Coalesced = uncoalesed is domain specificFollow the data

Can data be coalesced without information loss?

NoteAvoid making coalescing into “politics” or “religion”

Some persons have strong feelings for/against coalescing

Coalescing is an expensive operation to performIt is related to duplicate elimination

Nothing to do with the coalesce functionDo you know what this function does?

Additional information on coalescingtimecenter.cs.aau.dk/TimeCenterPublications/TR-9.pdf


timecenter.cs.aau.dk/TimeCenterPublications/TR-9.pdf

Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


OverviewExample (One)

VT0 1 2 3 4 5 6 7 8 9 10

RSR ∪ SR ∩ S

VT0 1 2 3 4 5 6 7 8 9 10

RSR − SS − R

Example (Two)

VT0 1 2 3 4 5 6 7 8 9 10

RSR ∪ SR ∩ S

VT0 1 2 3 4 5 6 7 8 9 10

SR − SS − R

NoteThink Venn diagrams

No big surprises!



VT0 1 2 3 4 5 6 7 8 9 10

RSR ∪ SR ∩ S

VT0 1 2 3 4 5 6 7 8 9 10

RSR − SS − R

Example (Two)

VT0 1 2 3 4 5 6 7 8 9 10

RSR ∪ SR ∩ S

VT0 1 2 3 4 5 6 7 8 9 10

SR − SS − R


No big surprises!



VT0 1 2 3 4 5 6 7 8 9 10

RSR ∪ SR ∩ S

VT0 1 2 3 4 5 6 7 8 9 10

RSR − SS − R

Example (Two)

VT0 1 2 3 4 5 6 7 8 9 10

RSR ∪ SR ∩ S

VT0 1 2 3 4 5 6 7 8 9 10

SR − SS − R


No big surprises!Kristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 58 / 122

Set Intersection: As Figure

Example (Emp1)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Jens JensLene Lene

Example (Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

JensLene Lene

Example (Emp1 ∩ Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

JensLene Lene


Set Intersection: As Figure

Example (Emp1)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Jens JensLene Lene

Example (Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

JensLene Lene

Example (Emp1 ∩ Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

JensLene Lene


Set Intersection: As Table

Example (Emp)

ename dept vts vte

Jens RD 0 5Lene Cust 2 10Jens Sup 6 15Jens Cust 16 20

Example (Boss)

ename dept vts vte

Lene Cust 5 9Jens Cust 17 20

Example (Emp ∩ Boss)

ename dept vts vte



Set Intersection: The CasesExample (Before)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

empty

Example (After)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

empty

Example (Right Overlap)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result

Example (Left Overlap)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result

Example (Included)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empbossresult

Example (Totally Overlaps)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empbossresult


Set Intersection: Informally

DefinitionExplicit columns must be equalIf overlap between the intervals then take the

The largest of the vts valuesThe smallest of the vte values

Otherwise skip the row

NoteRecall that vts ≤ vte for interval to be valid


Set Intersection in SQL

s e l e c t emp. ename , emp. dept , −− over lap emp and bossg rea tes t (emp. vts , boss . v t s ) as vts ,l e a s t (emp. vte , boss . v te ) as v te

from emp dept vt as emp, bossemp dept vt as bosswhere emp. ename = boss . ename and emp. dept = boss . deptand emp. v ts < boss . v te and boss . v t s < emp. v teand grea tes t (emp. vts , boss . v t s ) <= l e a s t (emp. vte , boss . v te )


Set Union: As Figure

Example (Emp1)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Jens JensLene Lene

Example (Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

JensLene Lene

Example (Emp1 ∪ Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Jens JensLene Lene


Set Union: As Figure

Example (Emp1)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Jens JensLene Lene

Example (Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

JensLene Lene

Example (Emp1 ∪ Emp2)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Jens JensLene Lene


Set Union: As Table

Example (Emp)

ename dept vts vte


Example (Boss)

ename dept vts vte


Example (Emp ∪ Boss)

ename dept vts vte



Set Union: The CasesExample (Before)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empbossresult result

Example (After)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result result


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result

Example (Included)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empbossresult


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empbossresult


Set Union: Informally

DefinitionExplicit columns must be equalIf overlap between the intervals then take the

The smallest of the vts valuesThe largest of the vte values

Else take the intervals in the first table,

and take the intervals in the second table

NoteThree cases to consider

Mutual exclusive cases therefore union all between cases

The rows must be snapshot equal, i.e., have the same explicitcolumns


Set Union in SQLs e l e c t emp. ename , emp. dept , −− over lap emp and boss

l e a s t (emp. vts , boss . v t s ) as vts ,g rea tes t (emp. vte , boss . v te ) as v te

from emp dept vt as emp, bossemp dept vt as bosswhere emp. ename = boss . ename and emp. dept = boss . deptand emp. v ts < boss . v te and boss . v t s < emp. v te union a l ls e l e c t emp. ename , emp. dept , emp. vts , emp. v te −− emp no over lapfrom emp dept vt as empwhere not e x i s t s (

s e l e c t ∗from bossemp dept vt as bwhere emp. ename = b . ename and emp. dept = b . deptand emp. v ts < b . v te and b . v t s < emp. v te ) union a l l

s e l e c t boss . ename , boss . dept , boss . vts , boss . v te −− boss no over lapfrom bossemp dept vt as bosswhere not e x i s t s (

s e l e c t ∗from emp dept vt as bwhere boss . ename = b . ename and boss . dept = b . deptand boss . v t s < b . v te and b . v t s < boss . v te )

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−Kristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 68 / 122

Set Difference: As Figure

Example (Emp)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Jens, RD Jens, Sup Jens, CustLene, Cust

Example (Boss)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Jens, CustLene, Cust

Example (Emp - Boss)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Jens, RD Jens, Sup Jens, CustLene, Cust Lene, Cust


Set Difference: As Table

Example (Emp)

ename dept vts vte


Example (Boss)

ename dept vts vte


Example (Emp - Boss)

ename dept vts vte

Jens RD 0 5Lene Cust 2 5Lene Cust 9 10Jens Sup 6 15Jens Cust 16 17


Set Difference: The CasesExample (Before)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result

Example (After)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result

Example (Included)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

result result


VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

empboss

empty


Set Difference: Informally

DefinitionExplicit columns must be equalIf overlap between the intervals then

Row with interval [emp.vts, boss.vts) if emp.vts< boss.vtsRow with interval [boss.vte, emp.vte) if boss.vte< emp.vte

Otherwise skip the row

NoteBefore or after does not matter

Total overlap does not add rows to the result

During is handled like a combination of left and right overlap

One interval may be split into one or two intervals


Set Difference in SQL, Part 1

s e l e c t emp. ename , emp. dept , emp. vts , emp. v te −− no over lapfrom emp dept vt as empwhere not e x i s t s (

s e l e c t ∗from bossemp dept vt as bwhere emp. ename = b . ename and emp. dept = b . deptand emp. v ts < b . v te and b . v t s < emp. v te )

union


Set Difference in SQL, Part 2

s e l e c t emp. ename , emp. dept , emp. vts , boss . v t s −− r i g h t over lapfrom emp dept vt as emp, bossemp dept vt as bosswhere emp. ename = boss . ename and emp. dept = boss . deptand emp. v ts < boss . v t s and boss . v t s < emp. v te and

not e x i s t s (s e l e c t ∗from bossemp dept vt as bwhere emp. ename = b . ename and emp. dept = b . deptand emp. v ts < b . v te and b . v t s < boss . v t s )

unions e l e c t emp. ename , emp. dept , boss . vte , emp. v te −− l e f t over lapfrom emp dept vt as emp, bossemp dept vt as bosswhere emp. ename = boss . ename and emp. dept = boss . deptand boss . v te < emp. v te and emp. v ts < boss . v te and

not e x i s t s (s e l e c t ∗from bossemp dept vt as bwhere emp. ename = b . ename and emp. dept = b . deptand boss . v te < b . v te and b . v t s < emp. v te )


Quiz: Set Operations

QuestionsR ∪ S = S ∪ R ?

Yes!

R ∩ S = S ∩ R ? Yes!

R − S = S − R ? No!Can R ∪ S return fewer rows than there are in R? Yes!

In contrast to the non-temporal version

Can R − S return more rows than there are in R? Yes!In contrast with non-temporal version

Can R ∩ S return more rows than there are in R? Yes!In contrast with non-temporal version

Can R − S return an empty result? Yes!Like the non-temporal version!

Can R ∩ S return an empty result? Yes!Like the non-temporal version!



QuestionsR ∪ S = S ∪ R ? Yes!

R ∩ S = S ∩ R ?

Yes!










R ∩ S = S ∩ R ? Yes!

R − S = S − R ?

No!Can R ∪ S return fewer rows than there are in R? Yes!









R ∩ S = S ∩ R ? Yes!

R − S = S − R ? No!Can R ∪ S return fewer rows than there are in R?

Yes!In contrast to the non-temporal version








R ∩ S = S ∩ R ? Yes!



Can R − S return more rows than there are in R?

Yes!In contrast with non-temporal version







R ∩ S = S ∩ R ? Yes!




Can R ∩ S return more rows than there are in R?

Yes!In contrast with non-temporal version






R ∩ S = S ∩ R ? Yes!





Can R − S return an empty result?

Yes!Like the non-temporal version!





R ∩ S = S ∩ R ? Yes!






Can R ∩ S return an empty result?

Yes!Like the non-temporal version!




R ∩ S = S ∩ R ? Yes!








Summary: Set Operators

Main PointsNo big surprises in how union, intersection, and difference are defined

The size of the result sets may be a surprise!Set difference is surprisingly difficult

Challenge that single row can be split into two rows

NoteNotation of union compatibility still applies!Alternative interpretations of the set operators exists

However not in widespread usage


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Timeslicing: As a Figure

Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy

Jim, Toy

Jill, Shoe

Example (In Table Format)

ename dept vts vte

Joe Shoe 2 8Joe Toy 10 now

Jim Toy 4 14Jill Shoe 12 now



Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy

Jim, Toy

Jill, Shoe

Example (Timeslice at time 1)

ename dept



Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy

Jim, Toy

Jill, Shoe


ename dept

Joe Shoe



Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy

Jim, Toy

Jill, Shoe


ename dept

Jim Toy



Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy

Jim, Toy

Jill, Shoe


ename dept

Joe ToyJim ToyJill Shoe



Example

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy

Jim, Toy

Jill, Shoe


ename dept

Joe ToyJill Shoe

NoteTimeslicing can be both in valid time and transaction time


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Current TimeSlice (now = 20)

Example (Content)

ename dept vts vte



Example (Timeslice)

ename dept

Joe ToyJill Shoe

Example (SQL Statement)s e l e c t ename , deptfrom empwhere vte = 99 −− 99 i s the value used f o r now

NoteOnly explicit attributes

SQL statement is simple


Timeslice Back in Time 13 (now = 20)

Example (Content)

ename dept vts vte



Example (Timeslice)

ename dept

Joe ToyJim ToyJill Shoe

Example (SQL Statement)s e l e c t ename , deptfrom empwhere v ts < 13 and vte >= 13

Notebetween cannot be used due to half-open intervals

Query different from current timeslice

SQL statement again simpleKristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 81 / 122

Summmary: Valid-Time Timeslice

Main PointsCurrent state most often used

Timeslice back and forward in time same query

SQL statements are simpleMust pick a value for now

Typically 9999-12-30 or 9999-12-31

Transaction-time slicing basically the same as for valid-time

NoteOften current state separated from other data


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


First Example of Transaction-Time TimesliceExample (Bitemporal Graph)

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe Joe, Shoe

Joe, Toy Joe, Toy

Joe, Shoe

Table ContentContent as of time 15

Only one employee, to make graph readableKristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 84 / 122

Timeslice at 1

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe

ename dept vts vte

Joe Shoe 1 now


Timeslice at 2

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe

ename dept vts vte

Joe Shoe 1 now


Timeslice at 3

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe

ename dept vts vte

Joe Shoe 1 now


Timeslice at 4

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe

ename dept vts vte

Joe Shoe 1 now


Timeslice at 5

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, Toy

ename dept vts vte



Timeslice at 6

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, Toy

ename dept vts vte



Timeslice at 7

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, Toy

ename dept vts vte



Timeslice at 8

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, Toy

ename dept vts vte



Timeslice at 9

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, Toy

ename dept vts vte



Timeslice at 10

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, ToyJoe, Toy

Joe, Shoe ename dept vts vte



Timeslice at 11

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, ToyJoe, Toy




Timeslice at 12

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, ShoeJoe, Shoe

Joe, ToyJoe, Toy




Timeslice at 13

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe Joe, Shoe

Joe, ToyJoe, Toy

Joe, Shoeename dept vts vte



Timeslice at 14

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe Joe, Shoe

Joe, ToyJoe, Toy

Joe, Shoeename dept vts vte



Timeslice at 15

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Joe, Shoe Joe, Shoe

Joe, Toy Joe, Toy

Joe, Shoe

ename dept vts vte



Summary: First Example

Note thatAll previous states retainedBackward compatible (with “plain” SQL)

End-used has supplied no valid-time values

Transaction-time slice contains valid-time columns


Second Example of Transaction-Time TimesliceExample (Bitemporal Graph)

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516

Bill, Prozac, Pepper



Bill, Morphine, Jones

Table ContentContent as of time 15

Only one patientKristian Torp (Aalborg University) Temporal Databases: Queries November 2, 2015 101 / 122

Timeslice at 1

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516


pname medicin vts vte

Bill Prozac 1 6


Timeslice at 2

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516






Timeslice at 3

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516






Timeslice at 4

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516






Timeslice at 5

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516







Timeslice at 6

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516







Timeslice at 7

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516







Timeslice at 8

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 9

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 10

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 11

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 12

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 13

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 14

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Timeslice at 15

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516








Summary: Second Example

Note thatAll previous states retainedValid-time does not retain your errors!

However, bitemporal does!

Not-backwards compatible (with “plain” SQL)End-used has supplied all valid-time values


Quiz: Does Bill Die?

Example (Medication for Bill)

TT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

VT

0123456789

10111213141516





Alive or Dead?If prozac and morphine is a leathal combination does Bill die?


Summary: Timeslicing

Main PointsTimeslicing see the table as of a point in timeTimeslicing can be both in valid-time and transaction-time

For transaction-time timeslicing only into the pastFor valid-time timeslicing both into the past and the future

Timeslicing is a fairly simple SQL select statement

NoteCurrent timeslice = the database state stored by non-temporaldatabases

Physical database design: current timeslice stored separately fromother data


Outline


2 Temporal Join

3 Aggregation

4 Coalescing

5 Set Operators


7 Summary


Quiz

Example (Emp Table One)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy Joe, Shoe

Tom, Shoe

Tom, Toy

QuestionsWere Joe and Tom employed during the same period?

Did Joe and Tom work in the same department at the same time?

Is the data coalesced?


Quiz

Example (Emp Table Two)

VT0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Joe, Shoe Joe, Toy Joe, Food

Tom, Food

Tom, Shoe

Tom, Toy

Tom, Toy

QuestionsWere Joe and Tom employed during the same period?

Did Joe and Tom work in the same department at the same time?

Is the data coalesced?


Summary: Main Ideas

Query TypesTime dimensions: valid time, transaction time, and user-defined time

Bitemporal = valid time + transaction time

Query types: current, sequenced, non-sequencedExtensions of plain SQL: Join, aggregation, set operations

Also projection and selection

New possibilities: Timeslicing

New problems: CoalescingIntervals overlap, central to understand how works

Not difficult, just many cases

NoteA temporal data model implemented using plain SQL

IBM DB2 10, SQL Server 2015, and TeraData 13.10 have built-inbitemporal support


Temporal Databases: Queries

Technology