Deconstructing Columnar Transposition Ciphers

Deconstructing columnar transposition ciphers

Robert Talbert, PhD.

Department of MathematicsGrand Valley State University

[email protected]

Twitter: @RobertTalbert Google+: google.com/+RobertTalbert

11.07.2013

R. Talbert (GVSU) Deconstructing CTCs 11.07.2013 1 / 33

Prelude

LOOK IN THE REFRIGERATOR

LOOKINTHEREFRIGERATOR

L O O

K I N

T H E

R E F

R I G

E R A

T O R

LKTRRETOIHEIROONEFGAR

A columnar transposition cipher (using three columns)


Prelude

LOOK IN THE REFRIGERATORLOOKINTHEREFRIGERATOR

L O O

K I N

T H E

R E F

R I G

E R A

T O R




Prelude


L O O

K I N

T H E

R E F

R I G

E R A

T O R




Prelude


L O O

K I N

T H E

R E F

R I G

E R A

T O R




Prelude


L O O

K I N

T H E

R E F

R I G

E R A

T O R




LOOKINTHEREFRIGERATOR −→ LKTRRETOIHEIROONEFGAR

L K T

R R E

T O I

H E I

R O O

N E F

G A R

−→ LRTHRNGKROEOEATEIIOFR

L R T

H R N

G K R

O E O

E A T

E I I

O F R

−→ LHGOEEORRKEAIFTNROTIR



L K T

R R E

T O I

H E I

R O O

N E F

G A R


L R T

H R N

G K R

O E O

E A T

E I I

O F R




L K T

R R E

T O I

H E I

R O O

N E F

G A R


L R T

H R N

G K R

O E O

E A T

E I I

O F R



LHGOEEORRKEAIFTNROTIR

L H G

O E E

O R R

K E A

I F T

N R O

T I R


After four rounds of encryption, we have the original message.Also, three characters in the messages never moved.



L H G

O E E

O R R

K E A

I F T

N R O

T I R





L H G

O E E

O R R

K E A

I F T

N R O

T I R





L H G

O E E

O R R

K E A

I F T

N R O

T I R




Questions

1 Why do columnar transposition ciphers cycle back on themselves, andwhat’s the smallest number of encryption steps needed to make thishappen?

2 What characters in a message are fixed in place by a columnartransposition cipher, and is there an efficient way to predict wherethey will be?

3 What else can we say about the security of this cipher?


Questions





Questions





How encryption and ciphers work in general

Goal: Transform information into format readable only by sender andchosen recipients.

Readable messagePlaintext

Transformed messageCiphertext

Readable messagePlaintext

EK(M) DK(EK(M))

KEY KEY

Assume that the information is being sent over an open channel.

Ciphertext should yield little/no information about the originalcontents of the message.


Example: Shift cipher

Key: Positive integer s, decided upon in advance by sender andrecipient

Encryption process: Shift every letter in the message forward in thealphabet by s positions, wrapping around the end of the alphabet ifnecessary.

Example: Suppose s = 20 and encrypt MATH RULES.

M A T H R U L E S

G U N B L O F Y M

Ciphertext: GUNBLOFYM. Decrypt by shifting backwards by 20... orforwards by .


Interlude: Integer congruence modulo n

Definition

Let n be any positive integer and a, b integers. We say that a iscongruent to b modulo n if n divides b − a. Notation: a ≡ b (mod n).

Examples:

12 ≡ 5 (mod 7)

8675309 ≡ 9 (mod 10)

−20 ≡ 6 (mod 26)

780 ≡ 0 (mod 26)

The smallest natural number to which a is congruent modulo n =Remainder left over when dividing a by n.


Mathematizing the shift cipher

Number letters 0, 1, . . . , 25

Key: Positive integer s

Es(m) = (m + s) (mod 26)

Ds(m) = (m + (26− s)) (mod 26)

Original M A T H R U L E S

Number-fied 12 0 19 7 17 20 11 4 18

+key 32 20 39 27 37 40 31 24 38

mod 26 6 20 13 1 11 14 5 24 12

Letter-fied G U N B L O F Y M

Ds(Es(m)) = m + s + 26− s (mod 26) = m + 26 (mod 26) = m


Mathematizing the shift cipher

Number letters 0, 1, . . . , 25

Key: Positive integer s

Es(m) = (m + s) (mod 26)

Ds(m) = (m + (26− s)) (mod 26)

Original M A T H R U L E S

Number-fied 12 0 19 7 17 20 11 4 18

+key 32 20 39 27 37 40 31 24 38

mod 26 6 20 13 1 11 14 5 24 12

Letter-fied G U N B L O F Y M

Ds(Es(m)) = m + s + 26− s (mod 26) = m + 26 (mod 26) = m


Columnar transposition ciphers

Message length = L (remove spaces, punctuation, etc.)

Key: Positive integer C

Encryption: Feed characters of message into a rectangular grid, Ccolumns and dL/Ce rows one row at a time.

Decryption: Read off characters from the grid one column at a time.




















CTC Example: C = 3, L = 9

MATH RULES MHLARETUS MATH RULESEK(M) DK(EK(M))

C = 3 C = 3

M A T

H R U

L E S


Same message, different C

If C = 2:

MATHRULES −→

M A

T H

R U

L E

S

−→ MTRLSAHUE

The ciphertext depends on both the message length L and the number Cof columns.


CTCs are functions

The CTC with C = 3, L = 9 is a one-to-one, onto function{0, 1, 2, . . . , 8} → {0, 1, 2, . . . , 8}

M A T H R U L E S

0 1 2 3 4 5 6 7 8

M H L A R E T U S

0 1 2 3 4 5 6 7 8

n f(n)

0 0

1 3

2 6

3 1

4 4

5 7

6 2

7 5

8 8

n 0 1 2 3 4 5 6 7 8

f (n) 0 3 6 1 4 7 2 5 8

Let g be the CTC using 2 columns on 9 characters.

n 0 1 2 3 4 5 6 7 8

g(n) 0 5 1 6 2 7 3 8 4


CTCs are functions

The CTC with C = 3, L = 9 is a one-to-one, onto function{0, 1, 2, . . . , 8} → {0, 1, 2, . . . , 8}

M A T H R U L E S

0 1 2 3 4 5 6 7 8

M H L A R E T U S

0 1 2 3 4 5 6 7 8

n f(n)

0 0

1 3

2 6

3 1

4 4

5 7

6 2

7 5

8 8

n 0 1 2 3 4 5 6 7 8

f (n) 0 3 6 1 4 7 2 5 8

Let g be the CTC using 2 columns on 9 characters.

n 0 1 2 3 4 5 6 7 8

g(n) 0 5 1 6 2 7 3 8 4


Permutations

Definition

A permutation on a finite set X is a bijection X → X .

n 0 1 2 3 4 5 6 7 8

f (n) 0 3 6 1 4 7 2 5 8

Notation: f = (1, 3)(2, 6)(5, 7). Each group = cycle. 0, 4, 8 = fixedpoints. So this f is a product of disjoint 2-cycles.

n 0 1 2 3 4 5 6 7 8

g(n) 0 5 1 6 2 7 3 8 4

g = (1, 5, 7, 8, 4, 2)(3, 6). Disjoint product of a 6-cycle and a 2-cycle.

Theorem (Cayley)

Every permutation can be written as a product of disjoint cycles.


Permutations

Definition


n 0 1 2 3 4 5 6 7 8

f (n) 0 3 6 1 4 7 2 5 8


n 0 1 2 3 4 5 6 7 8

g(n) 0 5 1 6 2 7 3 8 4


Theorem (Cayley)



Permutations

Definition


n 0 1 2 3 4 5 6 7 8

f (n) 0 3 6 1 4 7 2 5 8


n 0 1 2 3 4 5 6 7 8

g(n) 0 5 1 6 2 7 3 8 4


Theorem (Cayley)



CTCs and permutations

Every CTC with C columns enciphering a message of length L is apermutation on {0, 1, . . . , L− 1}. Notation: πC ,L.

Example

π3,9 = (1, 3)(2, 6)(5, 7)

π2,9 = (1, 5, 7, 8, 4, 2)(3, 6)

π3,21 = (1, 7, 9, 3)(2, 14, 18, 6)(4, 8, 16, 12)(5, 15)(11, 17, 19, 13)

π4,77 = (1, 20, 5, 21, 25, 26, 45, 31, 65, 36, 9, 22, 44, 11, 60, 15, 61, 35, 66, 55,

71, 75, 76, 19, 62, 54, 52, 13, 23, 63, 73, 38, 48, 12, 3, 58, 53, 33, 28, 7, 59,

72, 18, 43, 68, 17, 24, 6, 40, 10, 41, 30, 46, 50, 51, 70, 56, 14, 42, 49, 32, 8,

2, 39, 67, 74, 57, 34, 47, 69, 37, 29, 27, 64, 16, 4)

Demo: Python function to generate the cycle breakdown of πC ,L.


Will repeated encryption using a CTC always eventually lead back to theplaintext?

YES because of the cyclical nature of permutations.Example: π3,9 = (1, 3)(2, 6)(5, 7)

n π3,9(n) (π3,9 ◦ π3,9)(n)

0 0 01 3 12 6 23 1 34 4 45 7 56 2 67 5 78 8 8

Example: π3,21 = (1, 7, 9, 3)(2, 14, 18, 6)(4, 8, 16, 12)(5, 15)(11, 17, 19, 13)repeats itself after four iterations.


Will repeated encryption using a CTC always eventually lead back to theplaintext?YES because of the cyclical nature of permutations.Example: π3,9 = (1, 3)(2, 6)(5, 7)

n π3,9(n) (π3,9 ◦ π3,9)(n)

0 0 01 3 12 6 23 1 34 4 45 7 56 2 67 5 78 8 8



Will repeated encryption using a CTC always eventually lead back to theplaintext?YES because of the cyclical nature of permutations.Example: π3,9 = (1, 3)(2, 6)(5, 7)

n π3,9(n) (π3,9 ◦ π3,9)(n)

0 0 01 3 12 6 23 1 34 4 45 7 56 2 67 5 78 8 8



Definition

The order of the columnar transposition πC ,L is the smallest positiveinteger such that

πC ,L ◦ πC ,L ◦ · · · ◦ πC ,L︸︷︷︸k times

= πkC ,L = identity function

Fact

Every permutation has a finite order, and that order is the least commonmultiple of the lengths of the cycles in its disjoint cycle factorization.

Can we determine the order of πC ,L using only the values of C and L?


Definition

The order of the columnar transposition πC ,L is the smallest positiveinteger such that

πC ,L ◦ πC ,L ◦ · · · ◦ πC ,L︸︷︷︸k times

= πkC ,L = identity function

Fact

Every permutation has a finite order, and that order is the least commonmultiple of the lengths of the cycles in its disjoint cycle factorization.

Can we determine the order of πC ,L using only the values of C and L?


Formula for πC ,L

Results following are from

R. Talbert, “The cycle structure and order of the rail fence cipher”,Cryptologia, 30(2):159—172, 2006

Theorem (The Big Formula)

Let πC ,L be the permutation underlying a columnar transposition cipherwith C columns and text length L. Let 0 ≤ n < L, n′ = nmodC , andL′ = LmodC . Then:

πC ,L(n) =

n − n′

C+ n′

(⌈L

C

⌉)− (n′ − L′) if L′ 6= 0 and n′ > L′

n − n′

C+ n′

(⌈L

C

⌉)otherwise


Illustration of The Big Formula: π4,26

0

11

13

25

C

�L/C�

π4,26(13) = 7 + 3 = 10

π4,26(11) = (3(7)− 1) + 2 = 22

n′ = number of columns preceding thecolumn containing n

n′dL/Ce = number of entries countedin columns preceding n

n − n′ = position of first character inn’s row

(n − n′)/C = number of rows precedingrow containing n

(n − n′)/C + n′dL/Ce = endingposition of n if no blanks encountered.

L′ = number of blanks in last row


The rail fence cipher

CTC with 2 columns = π2,L, the rail fence cipher.

Since C = 2, n′ and L′ are 0 or 1, so:

Theorem (RFC Formula)

Let π2,L be the permutation for a rail fence cipher on a plaintext of lengthL. Let n be an integer with 0 ≤ n < L. Then:

π2,L(n) =

n

2n even

n + L

2n odd, L odd

n + L− 1

2n odd, L even


The initial cycle

Definition

The initial cycle of πC ,L is the cycle in the decomposition of πC ,L thatcontains the number 1.

Example

π2,15 = (1, 8, 4, 2)(3, 9, 12, 6)(5, 10)(7, 11, 13, 14)

π2,21 = (1, 11, 16, 8, 4, 2)(3, 12, 6)(5, 13, 17, 19, 20, 10)(7, 14)(9, 15, 18)


Group structure of the initial cycle

Theorem

1 If L = 2k − 1 for some k > 1, then the initial cycle ofπ2,L = (1, 2k−1, 2k−2, · · · , 4, 2).

2 For any positive integers k and L, πk2,L(1) = 2l1−k mod L where l1 isthe length of the initial cycle.

Corollary

The initial cycle of π2,L is the cyclic subgroup generated by 2 in Z∗L.


Group structure of the initial cycle

Theorem

1 If L = 2k − 1 for some k > 1, then the initial cycle ofπ2,L = (1, 2k−1, 2k−2, · · · , 4, 2).

2 For any positive integers k and L, πk2,L(1) = 2l1−k mod L where l1 isthe length of the initial cycle.

Corollary

The initial cycle of π2,L is the cyclic subgroup generated by 2 in Z∗L.


Connecting initial cycle to other cycles

Example

π2,15 = (1, 8, 4, 2)(3, 9, 12, 6)(5, 10, 5, 10)(7, 11, 13, 14)

π2,21 = (1, 11, 16, 8, 4, 2)(3, 12, 6, 3, 12, 6)(5, 13, 17, 19, 20, 10)(7, 14, 7, 14, 7, 14)(9, 15, 18, 9, 15, 18)

Theorem

Suppose L is odd. For all x ∈ {0, 1, · · · L− 1},

π2,L(x) = (π2,L(1) · x) mod L.


Connecting initial cycle to other cycles

Example

π2,15 = (1, 8, 4, 2)(3, 9, 12, 6)(5, 10, 5, 10)(7, 11, 13, 14)

π2,21 = (1, 11, 16, 8, 4, 2)(3, 12, 6, 3, 12, 6)(5, 13, 17, 19, 20, 10)(7, 14, 7, 14, 7, 14)(9, 15, 18, 9, 15, 18)

Theorem

Suppose L is odd. For all x ∈ {0, 1, · · · L− 1},

π2,L(x) = (π2,L(1) · x) mod L.


Theorem

Suppose L is odd. If γ is a cycle of π2,L, then the length of γ divides thelength of the initial cycle, and hence the order of π2,L is the length of theinitial cycle.

Proof: Let orb(1) denote the initial cycle of π2,L. This is a group withtypical element πk2,L(1). Define a group action of orb(1) on a cycle γ ofπ2,L:

πk2,L(1) · x = πk2,L(x)

Exercise: This really is a group action, and if x ∈ γ, then

Fx = {g ∈ orb(1) : g · x = x mod L}

is a subgroup of orb(1). Classical group theory implies |orb(1)/Fx | = |γ|.


Things fall apart when C > 2

Example

π3,19 = (1, 7, 9, 3)(2, 13, 11, 16, 12, 4, 8, 15, 5, 14, 17, 18, 6)

Cycles of lengths 4 and 13 (order = 4× 13 = 52)

Nontrivial fixed point: 10

Initial cycle does not end in descending powers of 3 mod 19

Initial cycle does not act nicely on the long cycle

It’s not currently known exactly how the cycle structure of πC ,L isorganized if C > 2.


Question 2: Are there characters in the message that are fixed by a CTC?

YES:

The first character is always fixed.

The last character is fixed if and only if C divides L.

But what about “nontrivial” fixed points?

Research with Beth Bjorkman (GVSU mathematics undergrad): “Fixedpoints of columnar transposition ciphers”



YES:







YES:






Where nontrivial fixed points don’t appear

Each column of the enciphering grid for πC ,L can contain at most onefixed point.

Corollary: If n is a nonzero character position and C divides n, then nis not fixed.

Corollary: If n is a character position and n ≡ −1 (mod C ), then n isnot fixed if C divides L; and if C does not divide L, n is fixed if andonly if n = L− 1.

Several other formulas lead to a constant-time algorithm forlocating fixed points. (→ Demo)


What’s not known (yet)


The One Big Cycle problem

When does πC ,L consist of just one big cycle?

Example

π2,13 = (1, 7, 10, 5, 9, 11, 12, 6, 3, 8, 4, 2)

π3,29 = (1, 10, 13, 14, 24, 8, 22, 17, 25, 18, 6, 2, 20, 26, 28, 19, 16, 15, 5, 21,

7, 12, 4, 11, 23, 27, 9, 3)

π6,21 = (1, 4, 15, 14, 10, 16, 17, 20, 11, 19, 7, 5, 18, 3, 12, 2, 8, 9, 13, 6)


Data about OBC

L-values for which πC ,L is one big cycle (10 ≤ L ≤ 500)

C L ∈ [10, 500] yielding OBC Frequency

2 11, 13, 19, 29, 37, 53, 59, 61, 67, 83, 101, 107,131, 139, 149, 163, 173, 179, 181, 197, 211,227, 269, 293, 317, 347, 349, 373, 379, 389,419, 421, 443, 461, 467, 491

36

3 17, 29, 53, 89, 101, 113, 137, 149, 173, 197,233, 257, 269, 281, 293, 317, 353, 389, 401,449, 461

21

4 77 1

5 None∗ 0

6 11, 17, 21, 41, 59, 83, 89, 107, 113, 131, 137,179, 227, 233, 247, 251, 257, 347, 381, 401,419, 443, 449, 467, 491

25

∗First value for C = 5 that gives OBC is L = 5287R. Talbert (GVSU) Deconstructing CTCs 11.07.2013 30 / 33

Maximal order problem

When does πC ,L decompose into cycles, all of whose (distinct)lengths are mutually coprime?

Example

π3,19 = (1, 7, 9, 3)(2, 13, 11, 16, 12, 4, 8, 15, 5, 14, 17, 18, 6)

π5,27 = (1, 6, 7, 13, 19, 25, 5)(2, 12, 14, 24, 26, 11, 8, 18, 20, 4, 22, 16, 9,

23, 21, 10)(3, 17, 15)

For C = 2, this never happens.


Extension questions

Both are due to David Austin (GVSU).

Suppose C1 6= C2. Does the composition

πC2,L ◦ πC1,L

reduce to πC ,L for some C?Partial answer: Not always.

π3,9 ◦ π2,9 = (1, 6)(2, 3, 5, 8, 4) 6= πi ,9 ∀i

Extend the CTC idea to a 3-dimensional array. Is this cipherequivalent to a 2-dimensional columnar transposition? What if weused higher-dimensional arrays?


Thank you


Deconstructing Columnar Transposition Ciphers

Technology

t r r e

talbert gvsuh e r e

n th e r e f r ig e

r t h r n gk r o e o

h e i r oo n e f g

t o rdeconstructing

talbert gvsuo

talbert gvsut