Modeling Delta Encoding of Compressed Files S.T. Klein, T.C. Serebro, D. Shapira
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Modeling Delta Encoding of Compressed Files
S.T. Klein, T.C. Serebro, D. Shapira
Delta Encoding
Example:
S=The Prague Stringology ClubT=The Prague Stringology Conference 06
Δ=(1, 24)onferenc(3,2)06
Compressed Differencing
Goal- Create a delta file of S and T, without decompressing the compressed files.
S T
Δ(S,T)
E(S)Delta encoding:Semi Compressed Differencing:
E(T)SE(S)Full Compressed Differencing:
LZW compressionSTR = input character WHILE there are input characters {C = input character IF STR C is in T then
STR = STR C ELSE {
output the code for STR add STR C to T STR = C
}} output the code for STR
S =abccbaaabccba
Example
E(S) =1233219571
construct the trie of E(S)i 1while i ≤ u{ P Starting at the root,
traverse the trie using P When a leaf v is reached k depth of v in trie output the position in S
corresponding to v ii+ k}
uii TTT ...1
Semi Compressed Differencing Algorithm
E(S) =1233219571, T =ccbbabccbabccbba.
(3,2) b (5,2) (9,3)(5,2)(9,3) b (5,2)
Example
Δ(S,T)=
Full Compressed Differencing Algorithm1 construct the trie of E(S)2 flag 0 // output character k3 counter 1 // position in T4 input oldcw from E(T)5 while oldcwNULL // still processing E(T) {5.1 input cw from E(T)5.2 node Dictionary[oldcw]5.3 if (Dictionary[cw] NULL)5.3.1 k first character of string corresponding to Dictionary[cw]5.4 else5.4.1 k first character of string corresponding to node5.5 if ((node has a child k) and (cwNULL))5.5.1 output (pos+flag,len-flag) corresponding to child k of node5.5.2 flag 15.6 else5.6.1 output (pos+flag, len-flag) corresponding to node5.6.2 create a new child of node corresponding to k5.6.3 flag 05.7 pos of child k of node counter5.8 oldcw cw5.9 counter counter + len - flag }
E(S) =1233219571 E(T) =33221247957
Example
ExampleE(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=coldcw=3
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3k=c
3
Example
4(1,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3k=cΔ(S,T)=<3, 2>
3
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3k=c
Example
4(1,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3 flag=1k=cΔ(S,T)=<3, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=3 flag=1k=c
Example
4(1,2,c)
Δ(S,T)=<3, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=2 flag=1k=c
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=2 flag=1k=b
<5, 1>
5(2,2,c)
Example
4(1,2,c)
Δ(S,T)=<3, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=2 flag=1k=b
<5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=3cw=2 flag=1k=b
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=2cw=2 flag=1k=b
6(3,2,b)
b
<b, 0>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=2cw=2 flag=0k=b
Example
4(1,2,c)
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=2cw=2 flag=0k=b
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=2 flag=0k=b
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=1 flag=0k=a
6(3,2,b)
Δ(S,T)=<3, 2> <5, 1>
4(1,2,c)
5(2,2,c)
7(4,2,b)
<5, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=1 flag=1k=a
b
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=1 flag=1k=a
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaboldcw=2cw=1 flag=1k=a
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaboldcw=1cw=2 flag=1k=b
6(3,2,b)
4(1,2,c)
5(2,2,c)
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
b
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccoldcw=2cw=4 flag=1k=c
6(3,2,b)
b
4(1,2,c)
5(2,2,c)
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
9(6,2,b)
<3, 1>
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbaoldcw=4cw=7 flag=1k=b
6(3,2,b)
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
9(6,2,b)
<3, 1>
10(7,3,c)
b
(2, 1)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbaoldcw=4cw=7 flag=0k=b
b
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
6(3,2,b)
b
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
9(6,2,b)
<3, 1>
10(7,3,c)
b
(2, 1)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabcoldcw=7cw=9 flag=0k=b
11(9,3,b)
b
(4, 2)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccboldcw=9cw=5 flag=0k=c
<9, 3>
12(11,3,b)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccboldcw=9cw=5 flag=1k=c
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccbbaoldcw=5cw=7 flag=1k=b
13(13,3,c)
b
(3, 1)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccbbaoldcw=7cw=Null flag=0k=b
(4, 2)
Combination of Pairs
Δ(S,T)=<3, 2> <5, 1> <5, 2><2,1> <3, 1> (2, 1)(4, 2)<9, 3> (3, 1)(4, 2)
S =abccbaaabccbaS =abccbaaabccba
<3, 2> <5, 1>
S =abccbaaabccba
<3, 3>
If two consecutive ordered pairs are of the form and , we combine them into a single ordered pair
1, li 21, lli 21, lli
Combination of Pairs
If two consecutive ordered pairs are of the form and , we combine them into a single ordered pair
1, li 21, lli 21, lli
Δ(S,T)= <5, 2><2,1> <3, 1> (2, 1)(4, 2)<9, 3> (3, 1)(4, 2)
S =abccbaaabccbaS =abccbaaabccbaS =abccbaaabccba
<3, 3> <2,1><3, 1><2, 2>
Δ(S,T)= <5, 2> (4, 2) <9, 3> (4, 2)<3, 3> <2,2 > c b
Encoding the delta fileΔ(S,T)= <5, 2> (4, 2) <9, 3> (4, 2)<3, 3> <2,2 > c b
File consists of:
(pos, len) in S
(pos, len) in T
Characters
flags
Experiments:
S = xfig.3.2.1 T = xfig.3.2.2
|T| = 812K|Gzip(T)| = 325K|LZW(T)| = 497K|Δ(S,T)| 3K
Related Documents
