1
Clock Synchronization in WSN:
from Traditional Estimation Theory
to Distributed Signal Processing
Yik-Chung WU
The University of Hong KongEmail: [email protected], Webpage: www.eee.hku.hk/~ycwu
2
• Applications require clock synchronization
– Event detection
– Data Fusion
– Sleep and wake-up cycle for power management
– TDD transmission schedule
3
Challenges?• Ideal case
• In reality delays exist
– The delay between radio chip
interpret and the CPU responding
– The time for radio chip to
transform the message to EM
wave
– The time for converting the
received EM wave into the
original message
– Finally signal to CPU reception is
completed
Node i Node j
My time is
4:28:12pm
4
System model• Clock model:
• Two-way message exchange is used
to establish the clock relationship
between two nodes:Real time
Ideal clockNode A clock
slope= α
q
slope=1
Node B clock( )i i ic t t q=
2 1
4 3
, ,
, ,
real_time real_time
real_time real_time
n n
n n
i j j nt t
j i i nt t
d w
d w
=
=
2 1
,
3 4
,
1 1[ ( ) ] [ ( ) ]
1 1
[ ( ) ] [ ( ) ]
j n j i n i j n
j i
j n j i n i i n
j i
c t c t d w
c t c t d w
q q
q q
=
=
5
Pairwise Synchronization: Gaussian case [1]
• Synchronize node j to node i (assume node i is the
reference)
• Approach 1: MLE
• Assume N rounds of time-stamp exchange, we have
2 1
,
3 4
,
1( ) ( )
1( ) ( )
j
j j
j
j j
j n i n j n
j n i n i n
c t c t w
c t c wdt
dq
q
=
=
211 ,11
21,
4 3,11 1
4 3,
( ) -1( )
( ) -1 1/( )
/( ) - ( ) 1
( ) - ( ) 1
j ji
j N j j Ni N
j j ii j
i Ni N j N
i
c t wc t
c t wc td
wc t c t
wc t c t
q
=
1 t jd = 1 T θ z
6
• Log-likelihood function:
• q is linear and can be easily estimated:
• Put it back to the log-likelihood function, d can be
obtained by maximizing
• Differentiating this function w.r.t. d and set it to zero:
• Finally, the clock parameters are recovered from
2
2 2, |
2 2ln ( , ) ln
i j
i j
Nf
dd
=
t T θ1θt T
1ˆ( ) ( ) ( )H H
j jj id d= θ T T T t 1
1 2
2( ) || ( ( ) )( ) ||H H
N j j j j id d
=
=
P
I T T T T t 1
1ˆ2
H H
i i
Hd
=
1 Pt t P1
1 P1
1 2 1ˆ ˆ ˆˆ ˆ ˆ ˆˆ 1/ [ ( )] , [ ( )] / [ ( )]j jd d d q= =θ θ θ
7
• Approach 2: Low complexity estimator
• d appears in both system model equations, so we can
eliminate it by adding two equations
• Putting the N rounds of message into a vector form:
• The MLE for this equation is
• This estimator is of lower complexity since there is no
need to compute d
• But since we are not estimating the unknowns from the
original equations, there may be some loss in
performance
2 3 1 4
, ,
1[ ( ) ( )] [ ( ) ( )] 2j n j n i n i
j
j j
n j n i nc t c t c t c t w wq
=
2 31 41 1 ,1 ,11 1
' '
1 4 2 3, ,
( ) ( ) -2( ) ( )1/
/
( ) ( ) ( ) ( ) -2
j
j j i ji i
i j
i N j
j
Ni N i N N
j
j N j
c t c t w wc t c t
w wc t c t c t c t
q
= =
t T θ z
' ' 1 ' 'ˆ ( )H H
j j j i
=θ T T T t
8
Comparison
• We have shown theoretically that the relative loss of the
performance bound of the low complexity estimator w.r.t.
to CRB is less than 1%
9
Pairwise synchronization under
exponential delays [2][3]
• Approach 1: MLE
• Revisit the message exchange equations:
• If wj,n and wi,n are i.i.d. exponential R.V.s, the likelihood function is (with j=1/j, j=qj/j)
2 1 3 4
, ,
1 1( ) ( ) , ( ) ( )
j j
j j j j
j n i n j n j n i n i nc t c t w c td c wdtq q
= =
1 2 3 4
1
2 2 3 4 1
1
2 1 3 4
1 1
({ ( ), ( ), ( ), ( )} | , , , )
exp [( ( ) ( )) 2 ( ) ( )]
[ ( ) ( ) 0] [ ( ) ( ) 0] [ 0]
N
i n j n j n i n n j j
NN
j n j n j i n i n
n
N N
j j n i n j j j n i n j
n n
f c t c t c t c t d
c t c t d c t c t
c t c t d c t c t d d
=
=
= =
=
10
• Closed-form estimate of can be obtained by
differentiating the above equation w.r.t. and set it to
zero
• Putting back into the likelihood function, we can show
that the MLE that maximizes the profile likelihood
function is
• This is a linear programming problem, and can be solved
using existing solver, but the worst case complexity is at least (N3)
• We have also proposed a low complexity algorithm for solving this problem, and the worst case only takes (N)
* * * 2 3
, ,1
2 1
1
3 4
1
[ , , ] arg max [( ( ) ( )) 2 ]
{ ( ) ( ) 0} ; 0subject to
{ ( ) ( ) 0} ; 0
j j
N
j j j n j n jd
n
N
j j n i n j n
N
j j n i n j n
d c t c t d
c t c t d d
c t c t d d
=
=
=
=
11
• Approach 2: Iterative weighted median
• Add the two message exchange equations:
• wj,n - wi,n becomes Laplacian R.V. with location parameter 0 and scale parameter 1/
• The log-likelihood function is
• An estimate of j and j can be obtained by minimizing
the second term
,,
2 3 1 4
, ,[ ( ) ( )] [ ( ) ( )] 2
s nr n
j j n j n i n i n j j n i n
TT
c t c t c t c t w w
==
=
, , 1 , ,
1
ln ({ , } | , ) ln | 2 |2
NN
s n r n n j j s n j r n j
n
f T T N T T
=
=
=
, ,,
1
min | 2 |j j
N
s n j r n j
n
T T
=
12
• Now, consider two sub-problems– When j is fixed, the problem is
The solution is the median value of the sequence
– When j is fixed, the problem is
This is a weighted median problem for the data set
Simple procedure exists to compute this
• Two steps are iteratively updated
• Since the objective function is convex, it will converge to
the global optimal solution
, ,
1
min 2 | 0.5( ) |j
N
j r n s n j
n
T T
=
, , 10.5( )
N
j r n s n nT T
=
, , ,
1
min | ( / 2 ) |j
N
r n s n r n j j
n
T T T
=
, , , 1, ( / 2 )
N
r n s n r n j nT T T
=
13
Comparison
Iterative weighted median
has a significant loss w.r.t.
optimal solution
• The main reason is that iterative weighted median method adds the two message exchange equations together before estimation
• This is in contrast to Gaussian setting where this operation does not lead to significant loss
14
Proposed low-complexity algorithm has
the same performance as LP solver
Proposed low-complexity
algorithm has
the lowest complexity
15
Network-wide synchronization• How to extend pairwise algorithm to work for network-
wide synchronization?
Tree based Clustered based
• Need overhead to build and maintain tree or cluster structure
• Error accumulation is quick as number of layers increases
• Vulnerable if gateway node dies
16
Fully Distributed Algorithms• Approach 1: Coordinate Descent [4]
• Add the two-way message exchange equations to first
eliminate d:
• Assume each node perform N times two-way message
exchanges with each of its direct neighbors
2 3 1 4
, ,
1 1[ ( ) ( ) 2 ] [ ( ) ( ) 2 ]j n j n j i n i n i j n i n
j i
c t c t c t c t w wq q
=
2
1 1 ( )
2
2 3 1 4
NLL({ , } , )
1 1[ ( ) ( ) 2 ] [ ( ) ( ) 2 ]
N MM
i i i
n i j i
j n j n j i n i n i
j i
c t c t c t c t
q
q q
=
= =
But this is non-convex w.r.t. unknowns
17
• With transformation
• This is convex w.r.t. i and i , and we can alternatively
minimize this NLL
• Differentiating NLL w.r.t. i and set it to zero (also w.r.t i ),
we get two coupled iterative equations
1/ and /j j j j j q = =
{ , }{ , },,
2
2 3 1 4
2
1 1 ( )
NLL({ , } , ) [ ( ) ( )] 2 [ ( ) ( )] 2
i ji js nr n
N MM
i i i j j n j n j i i n i n i
n i j iTT
c t c t c t c t =
= =
( ) ( ) { , } { , } ( ) { , } { , } { , } { , }
, , , , , ,
1 ( )( 1)
{ , } 2 { , } 2
, ,
1 ( )
( 1) ( ) ( ) { , } { , }
, ,
ˆˆ ˆ[2 2 ][ ] [ ]
ˆ
[( ) ( ) ]
1 ˆˆ ˆ4 [ ]4 | ( ) |
Nm m i j j i m i j i j j i j i
i j s n r n j r n s n s n r n
n j im
i Ni j j i
s n r n
n j i
m m m i j j i
i j i s n r n
T T T T T T
T T
T TN i
=
=
=
=
( ) { , } { , }
, ,
1 ( )
ˆ [ ]N
m i j j i
j r n s n
n j i
T T=
18
• Approach 2: Belief Propagation [5]
• The two-way time-stamp message exchange equation
(between node i and j) can be put into matrix form
• Marginalized posterior distribution at node i:
• Computational demanding, needs centralized processing
2 3 1 4
, ,
2 3 1 41 1 ,11 1
2 3 1 4
1 1[ ( ) ( ) 2 ] [ ( ) ( ) 2 ]
( ) ( ) - 2 ( ) ( ) -21/ 1/
/ /( ) ( ) - 2 ( ) ( ) -2
j n j n j i n i n i j n i n
j i
j j ji ij i
j j i i
j N j N i N i N
c t c t c t c t w w
c t c t wc t c t
c t c t c t c t
q q
q q
=
=
,1
, ,
, , ,
i
j N i N
j i j i j i j i
w
w w
=A β A β z
, , 1 1 1
1 { , }
( ) ( ) ( , | , ) ... ...M M
i i i j j i i j i i M
i i j E
g p p d d d d
=
β β A A β β β β β β
19
• Express the joint posterior distribution using factor graph
– Factor node: local likelihood function
– or prior distribution
– Variable node: i
• Marginal distribution at each node can be obtained by
message passing on factor graph:
– Message from variable node to factor node
, , ,
2
, , ,
( , | , )
( | , )
i j i j j i i j
j i j i j i i j N
f p
=
=
A A β β
A β A β I
( )i if p= β
,
,
( )
( )\
( )
( ( ))j
j
i
i
j
l
f j
f B f
l
j f j jm m
= β β
20
– Message from factor node to variable node:
– In each iteration, messages are updated
in parallel with the information from direct
neighbouring nodes
– Each node computes the belief locally:
• As the likelihood function is Gaussian,
the messages involved in this algorithm
keep the Gaussian form
, ,
( ) ( 1
,
)
(( ) )i j i j
l
j f
l
f i i jj i jf dm m
= β ββ
(
)
))
(
( () )(i
ll
i
f B
f i imb
= β
ββ
, , ,
( ) ( ) ( )( ) ( )| ,i j i j i j
l l l
i f i i i f i fm β β v C
, , ,
( ) ( ) ( )( ) ( | , )j i j i j i
l l l
f i i i f i f im β β v C
21
• In practice, each real node computes
both kinds of messages
• Information passes between real nodes
is set as message from factor-to-
variable, and inherits the properties
– Still Gaussian. Only mean and covariance
needed to be exchanged
– Updated in parallel by local computation
with received mean and covariance
messages from neighboring nodes
• The belief computed at node i, with the
received messages from all direct
neighbouring nodes, is still Gaussian:
where
( ) ( ) ( )|( ) ~ ,( )l l l
i i i ib β β μ P
,
( )
( )
( )
i j
l
i
j
l
f i
i
=
11
P C, ,
( ) ( )( )
( )
( )
i j i j
l l
f i
l
fi i
l
j i
i
=
1
vμ CP
22
• The estimate at node i is
• The qi and i can be recovered from after
convergence
• It is generally known that if the FG contains cycles,
messages can flow many times around the graph,
leading to the possibility of divergence of BP algorithm
• Two properties:
• BP in this application converges regardless of network
topology, even under asynchronous message update
• The converged solution can also be proved to be equal
to the centralized ML solution
( ( )( )) (ˆ )l
i
l l
i i ii db= = β β μββ
ˆiβ
23
Comparison• Simulation setting: 25 nodes, dij [8,12], qi[-5.5,5.5],
i[-0.955,1.055], , 1000 topologies,
initialization=[1, 0], N=10
2 0.1i =
reference node
24
BP algorithm under
asynchronous message
exchange
BP converges much faster
than CD as second order
information is included in the
messages
25
Distributed tracking with DKF [4]
• Clock parameters may stay constant within a short
period of time
• But it will change over time
• We can either redo synchronization (throwing away
previous estimates), or we can do tracking
• If the change is slow, tracking is preferred
• Re-representation of clock model:
• After sampling:
0
0
0
( ) ( )
( )
i i i i
t
i i
c t t p B t
d
q
q
=
=
This term is due
to phase noise
( ) '( )i i it p B t =
10
0 0 0
0
( 1)
( )
( ) [ ( ) 1] [ ( ) 1]
i
i
l
i i i i
m
l
l
c l l m l
q
=
=
26
• Writing the accumulated skew and offset in recursive
forms:
• Clock parameter evolution model:
• Measurement equations based on two-way message
exchange
• Gather all measurement equations for
' '
( )
0
( ) ( 1) [ ( ) ( 1)]
( ) ( 1) [ ( ) 1]
i
i i i
u l
i i i
l l p B l B l
l l l
=
=
Gaussian with zero
mean and variance 2pi
0 00
( )
( ) ( 1) ( )1 0 0
1( ) ( 1) ( )
iii
i i i
i i i
l
l l u l
l l u l
=
bAx
{ , } { , } { , }
, , 2 ( ) 2 ( )i j i j i j
r n s n j i lT T l l V =
( )j i
, , ( ) ( ) ( )i l i l i il l= z C x v
27
• If all information is gathered in a single place, the optimal
solution is centralized KF
• KF cannot be implemented in distributed way since the
Kalman gain matrix contains correlation among nodes
• Solution: Impose a block diagonal structure on the
Kalman gain matrix:
• We can solve for Ki(lk) in closed form
• Each round of tracking includes time-stamp exchange
and messages exchange for KF update
1 1 1
1 , , ( ) 1
ˆ ˆ( | ) ( | )
ˆ ˆ( | ) ( | ) ( )( ( | ))k
i k k i i k k i
i k k i k k i k i l i l i k k
l l l l
l l l l l l l
=
=
x A x b
x x K z C x
( )
2
( ) arg min Tr ( | )
s.t. ( ) ( )
k
k k kl
M T
k i i i k i
l l l
l l=
=
=
KK P
K U K Ω
28
• Initialization:
– but it takes a long time to converge
– CD + bootstrap for covariance estimation (5 rounds of initial
time-stamp exchange)
-1(0 | 0) [1 0] , (0|0)=T
i =x P I
29
• Start with 25 nodes (B=15)
• If a node fails during tracking, we simply remove it from the
equations
• If the node later resume working, we will use its previously stored
clock parameter estimates and covariance matrix
• If a new node suddenly join in, we will use-1(0 | 0) [1 0] , (0|0)=T
i i =x P I
30
Distributed algorithm under
exponential delays [6]
• The pairwise LP problem can be easily extended to
network-wide setting:
• This is a LP problem, but very large in size
• Centralized solution is computational expensive and has
large communication overhead
* * 2 3 4 1
,1 ( ) 1 1
2 1
3 4
[ , ] arg max ( ( ) ( )) ( ( ) ( )) 2
( ) ( ) 0
subject to ( ) ( ) 0 ( , ) , 1,...,
0
M N N
j j n j n i i n i n ij
i j i n n
j n j i n i j i ij
j n j i n i j i ij
ij
c t c t c t c t Nd
c t c t d
c t c t d i j E n
d
= = =
=
=
x d
x d
N
• Challenge of solving this problem in a distributed way:
constraints are coupled for parameters at different nodes
• By introducing slack variables w and auxiliary replica
variables z, we can transform the problem to
• This problem can be solved by ADMM (iteratively
minimizing the augmented Lagrangian function w.r.t. the
unknowns, x, d, w, z, and the Lagrange multipliers)
31
, , ,2 1 ( )
{ , } { , } { , } { , }
1 2
{ , } { , }
2 3 4
4 { , }
1
arg min ( 2 ) ( ) ( )
s.t. , ,
, ,
( , )
M MT
i i ij
i i j i
i j i j i j i j
i j
i j i j
ij N ij
i j
q q
N d
d
i j E
= =
=
= =
= =
=
x d w za x d 0 w 0
B x z E x z
1 z w z
z 0
• Properties:
– The resultant algorithm only involves local computation at each
node and communications with its direct neighbors
– Closed-form expressions are available for each update step
– It will converge to the centralized ML solution
• Contrast to existing applications of ADMM:
– Direct application of ADMM to the original LP would not result in
distributed algorithm
– Most existing applications that result in distributed algorithms are
for Gaussian likelihood
– Most existing works consider a single (or a small set of) common
parameter
32
33
• Simulation results on a 25
nodes network
• K=5• =1
• CD as initialization help to
speed up convergence
34
Conclusions• We discussed clock synchronization in wireless sensor
networks
• We started with pariwise synchronization: Gaussian case
and exponential case
• Then we discussed network-wide synchronization: CD,
BP, ADMM
• We also discussed tracking using distributed KF
• Future works:
– BP based distributed algorithm for exponential delays?
– How about arbitrary distributed delays, asymmetric delays?
References[1] Mei Leng and Yik-Chung Wu, ``On Clock Synchronization Algorithms for Wireless Sensor Networks
under Unknown Delay," IEEE Trans. on Vehicular Technology, vol. 59, no.1, pp. 182-190, Jan 2010.
[2] Mei Leng and Yik-Chung Wu, ``Low Complexity Maximum Likelihood Estimators for Clock
Synchronization of Wireless Sensor Nodes under Exponential Delays," IEEE Trans. on Signal
Processing, Vol. 59, no. 10, pp. 4860-4870, Oct 2011.
[3] Mei Leng and Yik-Chung Wu, ``On joint synchronization of clock offset and skew for Wireless Sensor
Networks under exponential delay," Proceedings of the IEEE ISCAS 2010, Paris, France, pp. 461-464,
May 2010.
[4] Bin Luo and Yik-Chung Wu, ``Distributed Clock Parameters Tracking in Wireless Sensor Network,"
IEEE Trans. on Wireless Communications, Vol. 12, no. 12, pp.6464-6475, Dec 2013.
[5] Jian Du and Yik-Chung Wu, ``Distributed Clock Skew and Offset Estimation in Wireless Sensor
Networks: Asynchronous Algorithm and Convergence Analysis," IEEE Trans. on Wireless
Communications, Vol. 12, no. 11, pp. 5908-5917, Nov. 2013.
[6] Bin Luo, Lei Cheng, and Yik-Chung Wu, ``Fully-distributed Clock Synchronization in Wireless Sensor
Networks Under Exponential Delays," Signal Processing, Vol. 125, pp. 261-273, Aug 2016.
http://www.sciencedirect.com/science/article/pii/S0165168416000578
Further related readings:
Yik-Chung Wu, Qasim M. Chaudhari and Erchin Serpedin, ``Clock Synchronization of Wireless
Sensor Networks," IEEE Signal Processing Magazine, Vol. 28, no. 1, pp.124-138, Jan. 2011. 35
Jun Zheng and Yik-Chung Wu, ``Joint Time Synchronization and Localization of an unknown node
in Wireless Sensor Networks," IEEE Trans. on Signal Processing, Vol. 58, no. 3, pp. 1309-1320,
Mar 2010.
Mei Leng and Yik-Chung Wu, ``Distributed Clock Synchronization for Wireless Sensor Networks
using Belief Propagation," IEEE Trans. on Signal Processing, Vol. 59, no. 11, pp. 5404-5414, Nov
2011.
36