Adaptive Cleaning for RFID Data Streams. RFID: Radio Frequency IDentification.

Adaptive Cleaning for Adaptive Cleaning for RFID Data StreamsRFID Data Streams

RFID: Radio Frequency RFID: Radio Frequency IDentificationIDentification

RFID data is dirtyRFID data is dirtyShelf 0 Shelf 1

RFIDReaders

StaticTags

Mobile Tags

15ft1.5ft

3ft9ft

3ft

3ft

3ft

A simple experiment:•2 RFID-enabled shelves•10 static tags•5 mobile tags

RFID Data CleaningRFID Data Cleaning

Time

Raw readings

Smoothed output

• RFID data has many dropped readings• Typically, use a smoothing filter to

interpolateSELECT distinct tag_idFROM RFID_stream [RANGE ‘5 sec’]GROUP BY tag_id

Smoothing Filter

Smoothing filter Smoothing filter

Middleware

Clean RFID

Completeness

Tag dynamics

Read all tags in range

RFID Data CleaningRFID Data Cleaning

Time

Raw readings

Smoothed output

• RFID data has many dropped readings• Typically, use a smoothing filter to

interpolateSELECT distinct tag_idFROM RFID_stream [RANGE ‘5 sec’]GROUP BY tag_idBut, how to set the size

of the window?

Smoothing Filter

Window Size for RFID Window Size for RFID SmoothingSmoothing

Fido moving Fido resting

Small windowRealityRaw readings

Large window

Need to balance completeness vs. capturing tag movement

Truly Declarative Truly Declarative SmoothingSmoothing

• Problem: window size non-declarative Application wants a clean stream

of data Window size is how to get it

• Solution: adapt the window size in response to data

RFIDRFID

Epoch TagID ReadRate0 1 .90 2 .60 3 .3

Tag 1

Tag 2

Tag 3

Tag 4

Antenna & readerTags

E1 E2 E3 E4 E5 E6 E7 E8 E9E0

Read Cycle (Epoch)

(For Alien readers)

Tag List

1. Interrogation cycle2. Epoch

Controlled condition real condition

SMURFSMURF• Statistical Smoothing for Unreliable RFID

Data• Adapts window based on statistical

properties• Mechanisms for:

• Per-tag and multi-tag cleaning

Multi-tagCleaning

SMURFPer-tag

Cleaning

raw RFID streams

cleanedcount readings

cleanedper-tag readings

Application(s) Application(s)

Per-Tag Smoothing: Model and Per-Tag Smoothing: Model and BackgroundBackground

• Epoch t, Tag population Nt

• pi,t: Per epoch sampling prob.Response count of tag i per epoch

(total interrogation cycle)

Epoch TagID ReadRate0 1 .90 2 .60 3 .3

• Smoothing window size wi epoch • Per epoch sampling prob: pi

• Number of successful observations of tag i Binominal distribution B(wi,pi)

Per-Tag Smoothing: Model and Per-Tag Smoothing: Model and BackgroundBackground

Per-Tag Smoothing: Model and Per-Tag Smoothing: Model and BackgroundBackground

• Use a binomial sampling model

Time (epochs)

pi

1

0

Smoothing Window

wi Bernoulli trials

piavg

Si

(Read rate of tag i)

E1 E2 E3 E4 E5 E6 E7 E8 E9E0

Set of epochs where tag i can be seen

• We want to ensure that there are enough epochs in Wi such that tag i is observed (if it exists within the reader’s range) Completeness

Per-Tag Smoothing: CompletenessPer-Tag Smoothing: Completeness

Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness

• If the tag is there, read it with high probability

Want a large window

pi

1

0

Reading with a low pi

Expand the window

Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0

Per-Tag Smoothing: CompletenessPer-Tag Smoothing: Completeness

Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness

Expected epochs needed to read

With probability 1-

Desired window size for tag i

1ln*1

avgi

ip

w

Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions• Detect transitions as statistically

significant changes in the data

pi

1

0

Statistically significant difference Flag a transition and

shrink the window

The tag has likely left by this point

Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0

• Significant difference between mean observed sample size Si and expected size

• Find outlier (2)

Number of successful epochs in a window

Si

Mean

Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions

# expected readings Is the difference

“statistically significant”?# observed

readings

)1(**2|*||| avgi

avgii

avgiii ppwpwS

•Statistically significantStatistically significant

Algorithm Algorithm

SMURF in ActionSMURF in ActionFido moving Fido resting

SMURF

Experiments with real and simulated data show similar results

Normal sliding window Completeness Transition