Attack Analysis and Resilient Control Design for Discrete ... · Attack Analysis and Resilient Control Design for Discrete-time Distributed Multi-agent Systems Aquib Mustafa, Student

Attack Analysis and Resilient Control Design forDiscrete-Time Distributed Multi-Agent Systems

Aquib Mustafa, Student Member, IEEE and Hamidreza Modares, Member, IEEE

Abstract—This work presents a rigorous analysis of the adverseeffects of cyber-physical attacks on discrete-time distributedmulti-agent systems, and propose a mitigation approach forattacks on sensors and actuators. First, we show how an attackon a compromised agent can propagate and affect intact agentsthat are reachable from it. This is, an attack on a single nodesnowballs into a network-wide attack and can even destabilizethe entire system. Moreover, we show that the attacker canbypass the robust H∞ control protocol and make it entirelyineffective in attenuating the effect of the adversarial inputon the system performance. Finally, to overcome adversarialeffects of attacks on sensors and actuators, a distributed adap-tive attack compensator is designed by estimating the normalexpected behavior of agents. The adaptive attack compensator isaugmented with the controller and it is shown that the proposedcontroller achieves secure consensus in presence of the attacks onsensors and actuators. This controller does not require to makeany restrictive assumption on the number of agents or agent’sneighbors under direct effect of adversarial input. Moreover, itrecovers compromised agents under actuator attacks and avoidspropagation of attacks on sensors without removing compromisedagents. The effectiveness of the proposed controller and analysisis validated on a network of Sentry autonomous underwatervehicles subject to attacks under different scenarios.

Index Terms—Resilient control, Distributed multi-agent sys-tems, Adaptive control, Discrete-time systems.

I. INTRODUCTION

Distributed multi-agent systems (DMAS) [1]-[4] havegained remarkable attraction due to their potential applicationsin different growing fields such as robotics, power system,transportation, to name a few. Despite their numerous ad-vantages, DMAS are prone to cyber-physical attacks. Forinstance, in case of multi-vehicle systems, the GPS sensorcan be spoofed to corrupt the sensory data sent to the vehiclecontroller [5]. Corruption of sensory data or manipulation ofactuator input can severely and adversely affect the perfor-mance of the system. Therefore, design of robust, resilientand secure architectures is required to successfully achieve adesired coordinated goal in the presence of attack.

Considerable results have been presented for detection [6]-[12] and mitigation of attacks in DMAS. There are generallytwo approaches in designing mitigation techniques for DMAS.In the first approach, a monitor is designed to detect attackson neighbors and then remove compromised agents, onceidentified [13]-[21]. In these approaches, each normal agenteither uses an observer for each of its neighbors to detect

Aquib Mustafa and Hamidreza Modares are with the Department ofElectrical and Computer Engineering, Missouri University of Scienceand Technology, Rolla, MO 65401, USA (e-mails: [email protected];[email protected]).

abnormality [15] or discard neighbors information based onthe discrepancy between actual and malicious agents usingan iterative strategy [13]-[14]. The former approach requiresa model for each of its neighbors which makes it not scal-able. The latter requires meeting the F-total or the F-localcondition. That is, there should be an upper bound on Feither on the total number of adversarial agents, called as F-total or on the local number of compromised agents in theneighborhood of each intact agent, called as F-local. Althoughthese approaches can counteract variety of attacks, includingattacks on sensors, actuators and communication network, theymight harm the network connectivity by rejecting neighbor’sinformation even if there is no attack. This is because theymight not be able to distinguish between a change in neighborsbehavior due to attack and a legitimate change in the system.For example, in a leader-follower synchronization problem,a legitimate change in leader’s state can be detected as achange due to adversarial input by neighbors. Moreover, theseapproaches treat all types of attacks the same by discardingcompromised agents. However, as shown in this paper, attackson sensors and actuators can be recovered and compromisedagents can be brought back to the network without makingany restrictive assumption on the network connectivity. Thisavoids any unnecessary harm to the network connectivity.

In the second approach, local resilient control protocolsbased attack mitigation are designed to directly mitigate attackwithout identifying them. Reputation-based resilient controlprotocol is presented in [22] for leader-follower problemunder certain conditions. Game-theory based resilient controlarchitectures [23]-[26] are presented to minimize the effectsof adversarial input. With an assumption of having partialknowledge of attacker, a resilient receding horizon-based con-trol protocol is discussed in [27]-[29] for mitigation of thereplay attack. Secure state estimation and control under sensorattack is considered in [30]-[31]. A resilient control protocolis presented in [32] for single and double integrator systembased on local state emulator. In [33], an adaptive resilientcontrol protocol is presented for the attack on sensor andactuator of the system. Most of these results are presented forcontinuous-time systems. However, in real-time applications,the system communicates and broadcasts there information atdiscrete instants.

To design a resilient control protocol, one needs to identifythe adverse effects of the attack on the system performancefrom the attacker’s perspective. Despite tremendous progressin identifying the adverse effects of attacks on DMAS, there isstill a need to identify the vast effects of stealthy attacks andequipt the system with resilient control protocol to mitigate

arX

iv:1

801.

0087

0v3

[cs

.SY

] 1

1 M

ay 2

018

them. Toward this end, in this paper, first, we illustrate how anattack on a compromised agent spreads across the network andaffects intact agents that are reachable from a compromisedagent. Then, we show that the attacker can design a stealthyattack that has a common mode with the system dynamics andlaunch on a single root node to destabilize the entire system.We call this as the internal model principle for the attacker indiscrete-time DMAS. The attacker does not need to know thegraph topology or agents dynamics to design its attack signaland can eavesdrop on some sensory information to identifyone eigenvalue of the consensus dynamics. We also show thatthe attacker can entirely disable robust techniques such as H∞,used for attenuating the effects of adversarial inputs on theperformance of the system.

To mitigate the effect of the adversarial input, this workpresents a distributed adaptive resilient controller. First, theexpected normal behavior of agents are predicted using anobserver-like dynamics. Then, an adaptive attack compensatoris designed using predicted normal behavior of agents. Thedesigned adaptive attack compensator is augmented with thecontroller for the mitigation of the attack. Moreover, wehave shown the uniform boundedness for the system us-ing the proposed controller in the presence of the attack.The proposed adaptive resilient control protocol makes norestriction on graph topology as compared to the existingapproaches [13]-[21]. The proposed controller preserves thenetwork connectivity and mitigates the effect of adversarialinput on the actuator of the compromised agent. That is,not only the synchronization is achieved in the presence ofactuator attacks, but also compromised agents are recovered.On other hand, attacks on sensor affect only compromisedagents without being propagated in the network. Finally,simulation results validate the effectiveness of the proposedcontroller and theoretical analysis for a network of Sentryautonomous underwater vehicles under the influence of attacksfor different scenarios.

II. NOTATIONS AND PRELIMINARIES

In this section, the preliminaries of graph theory andstandard distributed consensus of multi-agent systems areprovided.

A. Graph TheoryA directed graph G consists of a pair (V , E ) in which set

of nodes and set of edges are represented by V = v1, . . . ,vNand E ⊂ V xV , respectively. The adjacency matrix is definedas A = [ai j], with ai j > 0 if (v j,vi) ∈ E . The set of nodes viwith edges incoming to node v j is called as neighbors of nodevi, namely Ni = v j : (v j,vi) ∈ E . The graph Laplacian matrixis defined as L = H−A, where H = diag(hi) is known as thein-degree matrix, with ∑ j∈Ni ai j as the weighted in-degree ofnode i. A directed tree is a connected digraph where everynode except one, known as root node, has the in-degree equalto one. A graph is said to have a spanning tree if a subset of theedges forms a directed tree. λ (.) represents the eigenvalues ofa matrix. (.)ad j refers to adjoint of a matrix. ker(.) denotesthe null space. Furthermore, λmax(.) and λmin(.) representmaximum and minimum eigenvalue of matrix, respectively.diag(.) denotes the diagonal matrix.

Assumption 1. The communication digraph G contains aspanning tree.

B. Standard Distributed Consensus in MAS

This subsection presents the standard distributed controlprotocol for consensus of discrete-time multi-agent systems.

Consider N agents with identical system dynamics repre-sented by

xi(k+1) = Axi(k)+Bui(k), i = 1, . . . ,N (1)

where xi(k)∈ Rn and ui(k)∈ Rm are the state and control inputof agent i, respectively. A and B are the system and and inputmatrices, respectively. (A,B) is assumed to be stabilizable.

Define the local neighborhood tracking error for the agenti as

εi(k) = (1+hi)−1

N

∑j=1

ai j(x j(k)− xi(k)) (2)

where ai j is the (i, j)-th value of the adjacency matrix.Consider the distributed control law for each node i as in

[34]

ui(k) = cKεi(k), i = 1, . . . ,N (3)

where c is a positive coupling constant, and K ∈ Rm×n is adesign feedback control gain matrix. Define the global statevector as x(k) = [xT

1 (k), xT2 (k), . . . , xT

N(k)]T ∈ RnN . Using (1)-

(3), the global dynamics of DMAS can be expressed as

x(k+1) = [IN⊗A− c(I +H)−1L⊗BK]x(k) (4)

The normalized graph Laplacian matrix L is defined as [34]

L = (I +H)−1L (5)

Let the eigenvalues of the normalized graph Laplacian matrixL be λi, ∀ i = 1, . . . ,N. Then, λi lies inside unit circle centeredat 1+ j0 for i = 2, . . . ,N and λ1 = 0 [35].

Using (4), the state of agent’s global dynamics is given by

x(k) = [IN⊗A− cL⊗BK]kx(0), Akcx(0) (6)

where Ac is the closed-loop matrix defined as

Ac = (IN⊗A− cL⊗BK) (7)

Lemma 1. [35] Let R ⊂ V be the set of root nodes and r =[p1, . . . , pN ]

T be the left eigenvector of the normalized graphLaplacian matrix L for λ1 = 0. Then, pi > 0 if i∈R and pi = 0if i /∈ R.

Theorem 1. [34]-[35] Let feedback gain K be designed suchthat A−cλiBK is Schur stable. Then, according to Lemma 1,the final consensus value for DMAS can written as

x(k) = (rT ⊗Ak)

x1(0)..

xN(0)

i = 1, . . . ,N as k→ ∞ (8)

III. ATTACK ANALYSIS FOR DISCRETE-TIMEDISTRIBUTED MAS

This section presents the attack modeling and analyzes itsadverse effects on the standard control protocol. The internalmodel principle for the attacker is presented to show that howa single compromised agent can destabilize the entire system.Then, the effect of the attack on the local neighborhoodtracking error is analyzed to show the ineffectiveness of thestandard robust H∞ control protocol (which is a well-knowndisturbance attenuation technique) in the presence of a stealthyattack.

Attacks on actuators of agent i can be modeled as

uci (k) = ui(k)+ γiua

i (k) (9)

where ui is the control law given in (3), uai represents the

attacker’s signal injected into the actuator of agent i, uci is the

distorted control law applied to (1) and the scalar γi is 1 whenthere is an attack on actuators of agent i and 0, otherwise.

Attacks on sensors of agent i can be modeled as

xci (k) = xi(k)+δixa

i (k) (10)

where xi represents the state of agent i, xai is the attacker’s

signal injected into the sensor of agent i, xci is the distorted

state and the scalar δi is 1 when there is an attack on sensorsof agent i and 0, otherwise.

Using the distributed control law (3) and (9) in (1), one canexpress the DMAS dynamics for an agent i as

xi(k+1) = Axi(k)+Bui(k)+B fi(k), i = 1, . . . ,N (11)

where fi(k) represents the overall attack signal injected intothe agent i, which is given by

fi(k) = c(1+hi)−1K(

N

∑j=1

ai j(δ jxaj(k)−δixa

i (k))

+γiuai (k)

(12)

Remark 1. An attacker can manipulate sensors or actuatorswithout physical tampering. Spoofing of global positioningsystem (GPS) of an unmanned vehicle or of phasor measure-ment unit’s (PMU’s) in power system are examples of attackswithout physical tampering.

A. Effects of Attack on Standard Distributed MAS

This subsection analyzes the effects of the attack on thestandard discrete-time DMAS (1). Theorem 2 investigates howan attack can propagate across the network.

Definition 1: In a graph, agent i is reachable from agent j ifthere is a directed path of any length from node j to node i.

Definition 2: An agent is said to be a compromised agent, ifit is directly affected by the attacker.

Theorem 2. Consider the discrete-time DMAS (11) under theattack fi(k). Let the control protocol be designed as (3) suchthat the closed loop matrix Ac in (7) is Schur. Then,

1) All agents reach consensus if fi(k) = 0,∀i = 1, . . . ,N.2) The intact agent deviates from the desired consensus

value if it is reachable from a compromised agent.

3) The deviation of the network from the desired behaviordepends on the number of compromised agents, theirattack signal magnitude and the number of agents reach-able from them.

Proof. It is shown in [35] that if c and K are designed sothat Ac in (7) is Schur, then all agents reach consensus. Thiscompletes the proof of part 1.

To prove part 2, define xa(k) =[(xa

1(k))T ,(xa

2(k))T , . . . ,(xa

N(k))T ]T and ua(k) =

[(ua1(k))

T ,(ua2(k))

T , . . . ,(uaN(k))

T ]T as a vector of signalsinjected to the sensors and actuators, respectively. The globaldynamics for DMAS (11) under the effect of attack can bewritten as

x(k+1) = Acx(k)+(IN⊗B) f (k), i = 1, . . . ,N (13)

where the injected global attack signal f (k) is

f (k) =−c(L⊗K)(δ ⊗ IN)xa +(γ⊗ IN)ua (14)

with γ = diag(γ1, . . . ,γN) and δ = diag(δ1, . . . ,δN). If f (k) 6= 0,then the solution of (13) is given by

x(k) = Akcx(0) +

k−1

∑p=0

(Ac)k−p−1(IN⊗B) f (p) (15)

with k ≥ p. At the steady state, one has

x(k)→k−1

∑p=0

(IN⊗A)k−p−1(IN⊗ In

−cL⊗A−1BK)k−p−1(IN⊗B) f (p)

(16)

For a positive integer n, binomial theorem for matrices canbe expressed as

(x+ y)n =n

∑k=0

Cnk xn−kyk (17)

with Cnk = n!

(n−k)!k! if x and y is commutative. Using (17), (16)becomes

x(k)→k−1

∑p=0

(IN⊗A)k−p−1

∗k−p−1

∑m=0

Ck−p−1m (−cL⊗A−1BK)m(IN⊗B) f (p)

(18)

Using (15) and (18), the state of the agent i can be writtenas

xi(k)→N

∑j=1

k−1

∑p=0

k−p−1

∑m=0

Ak−p−1Ck−p−1m (−1)mcmlm

i j(A−1BK)mB f j(p)

(19)where lm

i j , [(I +H)−1L]mi j and [ ]i j denotes the element (i, j)of a matrix. m represents the length of shortest directed pathfrom j to i [36]. Assume now that the agent j is under directattack, but agent i is intact, i.e. fi(k) = 0 and f j(k) 6= 0. Ifthe intact agent i is reachable from the compromised agent j,since lm

i j 6= 0 for some 0 < m < N−1, one can infer from (19)that the agent state xi(k) at steady state in (19) is non-zerowhich deduces that intact agent i is deviated from the desiredconsensus behavior. This completes the proof of part 2.

For the proof of part 3, taking the norm from both sides of(15), and using∥∥∥∥∥k−1

∑p=0

(Ac)k−p−1 f (p)

∥∥∥∥∥6 ‖ f (k)‖|λmin(Ac)|

(20)

yields

‖x(k)‖6 N f‖B‖b f

|λmin(Ac)|(21)

at steady state, where, N f is the number of agents for whichfi(k) is non-zero, b f is bound on adversarial input fi(k). Itwas shown in part 2 that if agent i is reachable from thecompromised agent j, then its deviation from the desiredbehavior is non-zero. That is, for the agent i which is reachablefrom a compromised agent, ‖xi(k)‖ is non zero and as can beseen from (21), its deviation bound depends on the numberof compromised agents N f and bound on adversarial input b f .This completes the proof. �

B. Internal Model Principle Approach for the Attacker

In the control systems, to reject a disturbance or followa reference trajectory, one needs to incorporate referencedynamics in the system. This is called the internal modelprinciple (IMP). We showed in the following Theorem 3 thatthe attacker can also leverage the IMP and incorporate someeigenvalues of the consensus dynamics in its attack design todestabilize the entire network.

We now take the role of the attacker and show that how itcan maximize the damage and cause a catastrophe. Conditionsunder which the attacker achieves this objective are provided.

Let the dynamics of the attacker on a node i be defined as

fi(k+1) =W fi(k) (22)

where W ∈ Rm×m. Consider the set of eigenvalues of W as

ΛW = [λW1 , . . . ,λWm ] (23)

and the set of eigenvalues of the system matrix A

ΛA = [λA1 , . . . ,λAn ], (24)

respectively.We assume that the system matrix A in (1) is marginally

stable, with eigenvalues on the unit circle centered at origin.This is a standard assumption in the literature for consensusand synchronization problems [37]. In fact, if A has stableeigenvalues, one can ignore them and reduce the dimension ofA. This is because, stable states of the agent have no effect onthe steady-state synchronization trajectory, and only contributeto the transient response.

Definition 3: (IMP-based attack) An attack signal is calledas IMP-based attack if at least one of the eigenvalues of theattacker dynamics in (22), belongs to the set of marginaleigenvalues of the system matrix A. That is, the attackerpartially incorporates the internal model of the system intoits attack dynamics.

Let define sum of product of the left eigenvector of thenormalized graph Laplacian matrix L and the attacker signalfi(k) as

Sh = ∑Nj=1 ph j fi(k) (25)

where ph j represents the element of left eigenvectors corre-sponding to eigenvalue λh of the normalized graph Laplacianmatrix L. Using Lemma 1, one can conclude that S1 6= 0 ifand only if j ∈ R for the eigenvalue λ1 of L [35].Theorem 3. Consider the discrete-time DMAS (11) under theattack fi(k) with the control protocol (3). Let fi(k) be designedas (22). Then,

1) The attacker destabilizes the complete network, if S1 6= 0and the sets defined in (23) and (24) have at least onecommon eigenvalue.

2) The attacker does not affect the stability of MAS (1),but cause deviation from consensus behavior, if S1 = 0or the sets defined in (23) and (24) have no commoneigenvalues.

Proof. The transfer function for the DMAS (1), from xi(z) toui(z) in z-domain can be written as

Gi(z) =xi(z)ui(z)

= (zI−A)−1B (26)

Using (3), the global control law under the influence of theattack can be expressed as

u(z) =−(cL⊗K)x(z)+ f (z) (27)

with u(z) = [uT1, . . . ,uT

N]T , x(z) = [xT

1, . . . ,xT

N]T and f (z) =

[ f T1, . . . , f T

N]T . Using (26) and (27), the system state in the

global form can be written as

x(z) = (IN⊗G(z))u(z) = (IN⊗G(z))(−(cL⊗K)x(z)+ f (z))(28)

where G(z) = diag(Gi(z)) with dimension RNxN . Let M be anon-singular matrix such that L = MΛM−1, with Λ be the Jor-dan canonical form of the normalized graph Laplacian matrixL. The left and the right eigenvectors of L corresponding tothe zero eigenvalue of the normalized graph Laplacian matrixare r and 1N , respectively [35]. Define

M = [1 M1], M−1 = [rT M2]T

where M1 ∈ RN×(N−1) and M2 ∈ R(N−1)×N . Using (28) withL = MΛM−1, one has

[INn + cMΛM−1⊗G(z)K]x(z) = (IN⊗G(z)) f (z) (29)

As MM−1 = IN , one can write (29) as

(M⊗ In)[InN + cΛ⊗G(z)K](M−1⊗ In)x(z) = G(z) f (z) (30)

Defining a state transformation as

x(z) = (M−1⊗ In)x(z) (31)

and premultiplying (30) with (M−1⊗ In) gives

x(z) = [INn + cΛ⊗G(z)K]−1(M−1⊗G(z)) f (z) (32)

Let assume for simplicity that all the Jordan blocks are simple,M−1 = [pi j] and M = [mi j], where pi j and mi j represent the

elements of matrices M−1 and M, formed by left eigenvectorsand right eigenvectors of the normalized graph Laplacianmatrix L, respectively. For the agent i, using (31) and (32),one has

xi(z) =N

∑h=1

mih[In + cKGi(z)λi]−1Gi(z)∑

Nj=1 pi j f j(z) (33)

The first eigenvalue of the normalized graph Laplacian matrixL is zero and its corresponding right eigenvector is 1N i.e.mi1 = 1. Using this fact with (33), one has

xi(z) = Gi(z)N

∑j=1

p1 j f j(z)+

N

∑h=2

mih[In + cλhGi(z)K]−1Gi(z)N

∑j=1

ph j f j(z)

(34)

Now, if we show that [In + cKGi(z)λh]−1 is Schur, then the

second term of (34) is bounded, even in the presence of attack.Since (A− cλhBK), ∀h = 2, . . . ,N is Schur, therefore if

we show that the roots of the characteristic polynomial(A− cλhBK) are identical to the poles of [In + cKGi(z)λh]

−1,then one can say [In+cKGi(z)λh]

−1 is also Schur. To this end,using (26), one has

∆|(zIn− (A− cλhBK))|= ∆|(zIn−A+ cλhBK)|= ∆|zIn−A|(In + cλh(zIn−A)−1BK)

=∆|zIn−A|[(∆|zIn−A|+ cλh(zIn−A)ad jBK)]

∆|zIn−A|

(35)

Hence, this proves that the roots of the characteristic poly-nomial (A − cλhBK) are identical to the poles of [In +cKGi(z)λh]

−1 using matrix properties from [38]. Therefore,[In+cKGi(z)λh]

−1 is Schur. Thus, it concludes that the secondterm of (34) is bounded and has no contribution in destabiliz-ing the system.

According to Lemma 1, P1 j in (34) is zero for non-rootnodes and ∑

Nj=1 p1 j fi(k) 6= 0 or S1 6= 0, if the attack is launched

on root nodes. Consider the IMP-based attack on a root node,which means that there is at least a common eigenvalue λAlbetween sets defined in (23) and (24). Using the transferfunction (26) and the attack signal defined in (22), one canwrite (34) as

xi(z) =N

∑j=1

p1 j(zIn−A)ad jB(zIn−W )ad j fi(0)

(z2 +λ 2Al)

2{n∏

i=1,i 6=l(z2 +λ 2

Ai)(z2 +λ 2

Wi)}

+

N

∑h=2

mih[1+ cKGi(z)λh]−1Gi(z)

N

∑j=1

ph j f j(z)

(36)

The first term of (36) shows that the pole λAl lies on theunit circle centered at the origin and has multiplicity greaterthan 1. Thus, the system states tend to infinity in the discrete-time domain as k→∞. Therefore, the attack on the root nodedestabilizes the entire network. This completes the proof ofpart 1.

If the attack is on a non-root node, then ∑Nj=1 p1 j fi(k) = 0.

So, (34) can be expressed as

xi(z) =N

∑h=2

mih[1+ cKGi(z)λh]−1Gi(z)

N

∑j=1

ph j f j(z) (37)

Then, according to (35), [In + cKGi(z)λh]−1 is Schur stable.

Therefore, the system states are bounded, even in the presenceof the attack. Moreover, the agents that are reachable fromthe attacker shows stable behavior, but deviation from thedesired consensus value. If ΛA∩ΛW 6= φ which implies that themultiplicity of poles lie on the unit is one. Therefore accordingto (34), the system states remain bounded and shows deviationfrom the desired consensus behavior due to the adverse effectof the attacker. This completes the proof. �

Remark 2. Note that the attacker does not need to knowthe system matrix A, and it can identify the eigenvalues ofdynamics through eavesdropping the sensory informations.Then, the attacker can identify root node and destabilize theentire system.

The following example presents the adverse effect of attackon the root node.

Example 1. Consider 4 agents having single-integrator dy-namics given by

xi(k+1) = xi(k)+ui(k) i = 1, . . . ,4 (38)

with the control protocol (3) and communicating to each otheraccording to graph structure in Fig.1.

In the absence of attack signal, agents achieve the desiredconsensus value, which is the average of the initial values ofAgents 1 and 2 in this example. If the attacker launches anattack on the Agent 1, then, the dynamics of the system at thesteady state can be written as

x1(k+1) = x1(k)+u1(k)+uac(k) = 0

⇒ x1(k)+(1+h1)−1(x2(k)− x1(k))+1 = 0

⇒ (x1(k)+ x2(k)) =−2

(39)

x2(k+1) = x2(k)+u2(k) = 0

⇒ x2(k)+(1+h2)−1(x1(k)− x2(k)) = 0

⇒ (x1(k)+ x2(k)) = 0

(40)

Fig. 1: Graph topology

x3(k+1) = x3(k)+u3(k) = 0

⇒ x3(k)+(1+h3)−1(x2(k)− x3(k)) = 0

⇒ (x2(k)+ x3(k)) = 0

(41)

x4(k+1) = x4(k)+u4(k) = 0

⇒ x4(k)+(1+h4)−1(x1(k)− x4(k)) = 0

⇒ (x1(k)+ x4(k)) = 0

(42)

However, the dynamics in (39) and (40) show that to reach asteady state, (x1(k)+ x2(k)) = −2 and (x1(k)+ x2(k)) = 0 atsame time. This is not possible for consensus on a boundedstate, and this can happen only, if they both go to infinity.Therefore, the system states never achieve the desired consen-sus behavior and they converge to infinity.

When Agent 1 is attacked with an IMP-based adversarialinput uac = 1, using Z-transform, one has

(z−1)x1(z) = u1(z)+uac(z) ⇒ x1(z) =u1(z)(z−1)

+z

(z−1)2

(43)It can be seen from (43) that an IMP-based attack can desta-bilize the entire system. This verifies the results of Theorem3.

Now, we present the analysis of the effects of the attack onthe local neighborhood tracking error (2). This analysis showsthat although attacks on sensors and actuators can be modeledas disturbances, existing disturbance attenuation techniques donot work for attack attenuation.

Disturbance attenuation approaches focus on minimizing theeffects of disturbance on the local neighborhood tracking error[39]-[40]. More specifically, the H∞ approach for DMAS (1)in presence of disturbance wi(k) designs a distributed controlprotocol as in (3), such that the desired consensus is achievedas in (8), if disturbance wi(k) = 0 and the bounded L2-gaincondition is fulfilled for any disturbance wi(k) ∈ L2[0,∞)

∞

∑k=0

εT (k)Mε(k)6 γ

2∞

∑k=0

wT (k)Nw(k) (44)

where γ > 0 is attenuation constant, M and N are positivedefinite weight matrices.

We present the following rigorous analysis for the effectsof the attack on the local neighborhood tracking error infollowing Theorem 4 and show that how an attacker canbypass existing H∞ disturbance attenuation approaches andmake them entirely ineffective.Lemma 2. Consider the normalized graph Laplacian matrix Ldefined in (5). Then, [LT L−2L] is negative semidefinite.Proof. Let λk be the eigenvalue of the normalized graphLaplacian matrix L. So, the eigenvalue of [LT L− 2L] can bewritten as

eig[LT L−2L] = λ2k −2λk

= (λk−1)2−1 (45)

Since all eigenvalues of matrix L lie inside unit circlecentered at 1+ j0, except λ1 = 0 [35], therefore (λk−1)2−1is less than or equal to zero for k = 1, . . . ,N. This shows that[LT L−2L] is negative semidefinite.

In the following theorem, for the sake of simplicity, weconsider the single integrator dynamics (38) and its globaldynamics is given by

x(k+1) = x(k)+u(k) (46)

Under the influence of attack, one can write the control inputu(k) in (46) as

u(k) = (−Lx(k)+ f (k)) (47)

Theorem 4. Consider the discrete-time DMAS with singleintegrator dynamics (46). Assume that the system is undera constant attack signal f (k). Then, the local neighborhoodtracking error for intact agents is zero while agents do notreach the desired consensus.

Proof. Consider the Lyapunov function for the discrete-timeDMAS as

V (x(k), f (k)) = (−Lx(k)+ f (k))T (−Lx(k)+ f (k)) (48)

The difference equation of the Lyapunov function (48) can bewritten as

∆V (x(k), f (k)) =V (x(k+1), f (k+1))−V (x(k), f (k))

= (−Lx(k+1)+ f (k+1))T (−Lx(k+1)+ f (k+1))

−(−Lx(k)+ f (k))T (−Lx(k)+ f (k)) (49)

For the constant attack signal f (k+1) = f (k), one can write(49) as

= (−Lx(k+1)+ f (k))T (−Lx(k+1)+ f (k))

−(−Lx(k)+ f (k))T (−Lx(k)+ f (k))

or equivalently,

= (−Lx(k+1))T (−Lx(k+1))− (−Lx(k))T (−Lx(k))

−2 f (k)T L(x(k+1)− x(k)) (50)

Using the scalar system dynamics (46) in (50), one has

= (−L[x(k)+u(k)])T (−L[x(k)+u(k)])

−(−Lx(k))T (−Lx(k))−2 f (k)T Lu(k) (51)

Using (47), equation (51) can be written as

= (−L[x(k)− Lx(k)+ f (k))])T (−L[x(k)− Lx(k)+ f (k))])

−(−Lx(k))T (−Lx(k))−2 f (k)T L(−Lx(k)+ f (k)))(52)

one can further simplify (52) as

= (−Lx(k)+ f (k))T [LT L−2L](−Lx(k)+ f (k)) (53)

Using Lemma 2, one has

∆V (x(k), f (k)) = (−Lx(k)+ f (k))T [LT L

−2L](−Lx(k)+ f (k))6 0 (54)

Then, using Lasalle’s invariance principle [41], the trajectories(x(k), f (k)) converge to a set that satisfy ∆V (x(k), f (k)) = 0.Based on (54), this yields

(−Lx(k)+ f (k)) ∈ ker(LT L−2L) (55)or

(−Lx(k)+ f (k)) = 0 (56)

From (55), one has (−Lx(k) + f (k)) = c1N . According tothis, the single integrator system dynamics becomes xi(k +1) = xi(k) + c, which shows that it destabilizes the system.Therefore, xi(k)→ ∞ as k→ ∞ ∀i = 1, . . . ,N with the localneighborhood tracking error goes to zero for all agent. Notethat, based on Theorem 3, (55) is the possible case when theattack is on a root node. On the other hand, for an attack on anon-root node agent, from (56), one has (−Lx(k)+ f (k)) = 0.Since for the intact agent i, fi(k) = 0, therefore, the localneighborhood tracking error for intact agents converge to zero,even in the presence of the attack.

We now show that intact agents do not reach the desiredconsensus, despite the fact the local neighborhood trackingerror is zero. From (56), one has

Lx(k) = f (k) (57)

which can be written for agent i as

(1+hi)−1

N

∑j=1

ai j(x j(k)− xi(k)) = fi(k) (58)

For a compromised agent i, since fi(k) 6= 0, then, one hasxi(k) 6= x j(k) for some i, j.

Now let assume that agent i is intact. Then, one has

(1+hi)−1

N

∑j=1

ai j(x j(k)− xi(k)) = 0 (59)

Consider the intact agent i as an immediate neighbor of thecompromised agent ic. Let assume by contradiction that onlythe compromised agent does not reach the desired consensusbut all the intact agents reach the desired consensus. Using(59), one can write

(1+hi)−1

∑j∈Ni

ai j(x j− xi)+aiic(xic − xi) = 0 (60)

Assuming that intact agents reach consensus, xi(k) =x j(k) ∀ j ∈ Ni. However, (60) cannot be satisfied if xi(k) =x j(k) ∀ j ∈ Ni because xic(k) 6= xi(k) and this contradict theassumption. Therefore, this shows that the intact agent i isdeviated from the desired consensus value. Similarly, one canuse the same argument to show that all reachable agents fromthe compromised agent will deviate from the desired consensusvalue. This completes the proof. �

Remark 3. If an intact agent i is an immediate neighbor of acompromised agent ic, then using (2), one can write the localneighborhood tracking error εi(k) with ai j = 1 as

εi(k) = (1+hi)−1

∑j∈Ni

(x j(k)− xi(k))

= |Ni| (1+hi)−1(

1|Ni| ∑

j∈Ni

x j(k) − xi)

= |Ni| (1+hi)−1(xavg − xi)

(61)

where xavg =∑

j∈Nix j(k)

|Ni| , which is not equal to xi(k) due toincoming information from a comprised agent xic(k). From(61), one can infer that the deviation of the intact agent fromthe desired consensus value depends on the number of the in-neighbors and deviation of the compromised agent ic from thedesired consensus value which depends on the magnitude ofthe injected attack signal. Moreover, the closer the agent is tothe source of the attack, the more its value will be deviatedfrom the desired consensus.

Corollary 1. Let the attacker design its attack signal usingthe internal model principle approach described in Theorem3. Then, it bypasses the H∞ control protocol.

Proof. In the absence of the attack, minimizing the local neigh-borhood tracking error results in minimizing the consensuserror. Therefore, the H∞ control in (44) is used to attenuate theeffect of adversarial input on the local neighborhood trackingerror. However, according to Theorem 4, in the presence ofIMP attack, by making the local neighborhood tracking errorgo to zero, agents do not reach consensus. This completes theproof. �

Theorem 4 and the following analysis highlight that whilethe local neighborhood tracking error is zero, agents might notreach consensus. Now, define a global performance functionΓ(k) as

Γ(k) = ∑i∈N

∑j∈Ni

∥∥xi(k)− x j(k)∥∥2 (62)

Define the set of intact agents as

Nint = N−Nc (63)

where N represents set of all agents and Nc represents set ofcompromised agents in the network.

Corollary 2. Consider the global performance function Γ(k)and the local neighborhood tracking error εi(k) defined in (62)and (3), respectively. Then,

1) Γ(k) and εi(k) ∀i = 1, . . . ,N converge to zero, if there isno attack. Moreover, agents achieve the desired consen-sus.

2) If the attacker designs an IMP-based attack on thenon-root node, then εi(k) ∀i ∈ Nint converges to zero,but Γ(k) does not converges to zero. That is, agentsdo not reach the desired consensus, while the localneighborhood tracking error is zero.

3) If the attacker designs an IMP-based attack on the rootnode, then εi(k) and Γ(k) ∀i = 1, . . . ,N go to zero,despite agents do not achieve the desired consensus andthe entire system get destabilized.

Proof. According to Theorem 1, the system achieves thedesired consensus if there is no adversarial input in the systemand this proves part 1 of corollary. If the attacker injectsan IMP-based attack signal into the non-root node of theDMAS, then based on Theorem, 3 and 4, one can inferεi(k)→ 0. However, as shown in Theorem 4, xi(k)−x j(k) 6→ 0,so Γ(k) 6→ 0 and this proves part 2. Based on Theorem 3, if theattacker injects an IMP-based attack signal into the root nodeof the DMAS, then xi(k)− x j(k)→ 0. However, the system

gets destabilized as xi(k)→∞ as k→∞, while εi(k) and Γ(k)∀i = 1, . . . ,N converge to zero. This completes the proof. �

Remark 4. The attacker can deceive the existing H∞ con-troller by using its IMP-based adversarial input. Although theglobal performance function Γ(k) reflects the adverse effectof the attacks, the local neighborhood tracking error does not.Therefore, the local performance measure does not ensuresthe global performance of the DMAS under the influence ofthe sophisticated attacks. This analysis reinforces the designof resilient control protocol to mitigate the adverse effects ofthe attack.

IV. RESILIENT DISTRIBUTED CONTROL PROTOCOL FORATTACKS ON SENSOR AND ACTUATOR : AN ADAPTIVE

APPROACH

This section presents the design of a resilient distributedcontrol protocol for the mitigation of the adverse effect ofattacks on sensors and actuators of an agent in the discrete-time DMAS. Regardless of the magnitude of attack f (k) onsensors and actuators of an agent and its reachability fromintact agents, the developed distributed adaptive compensatoris resilient against attacks and avoids catastrophic effects. Tothis end, first the expected normal behavior of agents arepredicted using an observer-like dynamics called as expectedstate predictor, and then an distributed adaptive compensatoris designed using predicted behavior of agents.

Consider the estimated state for agent i as xi(k). Thedistributed expected state predictor is designed as

xi(k+1) = Axi(k) + cBK(1+hi)−1

N

∑j=1

ai j(x j− xi) (64)

where the gain k and the coupling coefficient c are to bedesigned to ensure Ac in (7) is Schur. The global expectedstate predictor state vector for (64) can be written as x(k) =[xT

1 (k), xT2 (k), . . . , x

TN(k)]

T ∈ RnN .

Lemma 3. Consider the N expected state predictors given in(64). Let the feedback gain K and coupling coefficient c aredesigned to ensure Ac in (7) is Schur. Then, the expected statepredictor state x(k) converges to the desired consensus value.

Proof. The designed expected state predictor in (64) can beexpressed as

xi(k+1) = Axi(k)+Bui(k) (65)

whereui(k) = cKεi(k) (66)

with the local neighborhood tracking error ε(k) as

εi(k) = (1+hi)−1

N

∑j=1

ai j(x j− xi)) (67)

One can write the global expected state predictor state dynam-ics as

x(k+1) = Acx(k) ∈ RnN (68)

which yieldsx(k) = Ak

cx(0) ∈ RnN (69)

As A−cλiBK is Schur stable, with λi be the eigenvalues ofthe normalized graph Laplacian matrix L for i = 2, . . . ,N andλ1 = 0. Therefore, the expected state predictor states achievethe desired consensus value and written as

x(k) = (rT ⊗Ak)

x1(0)..

xN(0)

i = 1, . . . ,N as k→ ∞ (70)

�Remark 5. Note that a broad class of the DMAS includes

the leader-follower or the containment control problem (i.e.MAS with multiple-leader) for which even if the xi(0) 6= xi(0),Lemma 3 is valid. This is because, the reference trajectory tobe followed by agents is determined by the leaders, which areassumed to be trusted by using more advanced sensors andinvesting more security. The system (64) acts as a referencemodel for the agents and if xi(0) 6= xi(0), even for theintact DMAS, di in (74) will be nonzero until the differencebetween the initial conditions is gone. Agents converge to thedesired behavior irrespective of initial values. In the leaderlessconsensus problem, generally, initial values can be measuredwith a high precision. For example, the voltage and frequencysynchronization problem of the distributed electrical grid isa well-known leaderless consensus problem in which initialconditions can be measured using PMU’s.

Although the attacks on actuators and/or sensors can ad-versely affect the agents dynamics, they cannot affect thedynamics of the distributed expected state predictor (64),unless they entirely compromise the agent which is extremelyharder to do for the attacker.

The deviation of the agent’s behavior from the normalbehavior is estimated by distributed expected state predictor.Then, an adaptive attack compensator is developed using aexpected state predictor. The designed adaptive compensatoris augmented with the controller for the mitigation of theadversarial input.

Fig. 2: Architecture of the proposed adaptive resilient controller. Si represents the sensorof agent i∀i = 1, . . . ,N.

In contrast to existing detection-removing approaches [13]-[21], which require a strong network connectivity, the devel-oped resilient distributed controller preserves network topol-ogy and achieves the desired consensus without any restric-tions on the number of agents under sensor and actuatorattacks. Attacks on communication links i.e. denial of service(DoS) attack can be mitigated by integrating existing attackdetection/identification methodologies [13]-[21] with the pro-posed approach. Therefore, agents under the influence of theadversarial input on sensors and actuators can be recoveredusing the proposed resilient controller and be brought back tothe network in intact mode without being isolated.

We now design a distributed resilient control protocol as

ui,r(k) = ui(k)+ui,comp(k) (71)

where, ui(k) represents standard control protocol defined in (3)and ui,comp(k) represents the distributed adaptive compensatorprotocol responsible for rejection of the adversarial input.

Consider the feedback gain K in the control protocol (3)given as

K = (R1 +BT P1B)−1BT P1A = R−11 BT P1A (72)

where R1 is a positive definite design matrix, and P1 is solutionof

AT P1A−P1−AT P1B(R1 +BT P1B)−1BT P1A = Q1 (73)

with a positive definite matrix Q1.The designed distributed control protocol is given by

ui,r(k) = cKεi(k)−di(k) (74)

where di(k) is the estimated response of the adaptive com-pensator and K is the gain given by (72) and (73). The localneighborhood tracking error εi(k) in (74) is given by

εi(k) = (1+hi)−1

N

∑j=1

ai j(xcj(k)− xc

i (k)) (75)

The update for the distributed adaptive compensator is de-signed as

di(k+1) = θcK(εi(k)− εi(k))+θdi(k) (76)

where θ > 0 is a design parameter, and εi(k) and εi(k) aredefined in (75) and (67). Define Q2 = QT

2 > 0 as

Q2 = cR2(I +H)−1L = cR2L (77)

with some positive definite R2. Let the minimum eigenvalueof graph matrix L be λm.

Remark 6. The information exchanged among agents in (75)is the corrupted state measurement xc

i (k) defined in (10) whichis different from xi(k) defined in (11). However, if the agenti is intact or only under actuator attack, then xc

i (k) = xi(k). Ifattacker corrupt the sensor data, then xc

i (k) 6= xi(k). In mostof the existing DMAS work, it is assumed the states of theagents are measurable and also we consider the same in ourwork. Therefore, the attack on the sensor corrupts the statemeasurements.

The design of adaptive compensator using expected statepredictor is provided in the following theorem.

Theorem 5. Consider the effect of the attack on sensor andactuator of an agent in DMAS (11). Let the control protocolbe developed as (74)-(77). Then, the agent’s consensus errorare bounded, and the bound can be made arbitrarily small,despite the attack.

Proof. According to Lemma 4, the expected state predictorconverges to the desired consensus value. Therefore, consensusof discrete-time DMAS can be achieved by showing theconvergence of the agent state xi(k) to the predicted state xi(k).Define

x(k) = x(k)− x(k) (78)

Then, with (11) and (65), one can write x(k+1) as

x(k+1) = (IN⊗A− cL⊗BK)x(k)− (IN⊗B)d(k) (79)

whered(k) = d(k)− f (k) (80)

with d(k) = [dT1 (k),d

T2 (k), . . . ,d

TN(k)]

T ∈ RnN as the globaladaptive compensator vector and the dynamics of the attackf (k) is defined in (22).

Using (76), the global dynamics of the adaptive compen-sator can be written as

d(k+1) = θcL⊗ R−11 BT P1Ax(k)+θ d(k)+θ f (k) (81)

where R1 = R1 +BT P1B and f (k) = 2 f (k)− (γ⊗ IN)ua. Notethat f (k) = f (k) only if the actuator of the agent is compro-mised.

Define the Lyapunov candidate function function as

V (k) = xT (k)(Q2⊗P1)x(k)+θ−2dT (k)(R2⊗ R1)d(k) (82)

The difference equation of the Lyapunov candidate functioncan be written as

∆V (k) =V (k+1)−V (k)

= xT (k+1)(Q2⊗P1)x(k+1)− xT (k)(Q2⊗P1)x(k)︸ ︷︷ ︸part 1

+θ−1dT (k+1)(R2⊗R1)d(k+1)−θ

−1dT (k)(R2⊗R1)d(k)︸ ︷︷ ︸part 2

(83)

Using (79), part 1 of the difference equation of the Lyapunovcandidate function (83) can be expressed as

= xT (k)(Q2⊗AT P1A−2cQ2L⊗AT P1BK

+c2LT Q2L⊗ (BK)T P1BK− (Q2⊗P1))x(k)

−2xT (k)[Q2⊗AT P1B− cLT Q2⊗ (BK)T P1B]d(k)

+dT (k)(Q2⊗BT P1B)d(k)

(84)

Using the Young’s inequality, one can further simplify andexpress (84) as

6−xT (k)(Q2⊗Q1)x(k)− xT (k)(−Q2 +2cQ2L)⊗AT P1BK)x(k)

+2c2λmin(c2LT Lλmin(T Q−1

1 ))xT (k)(Q2⊗Q1)x(k)

−2xT (k)(Q2⊗AT P1B)d(k)+2dT (k)(Q2⊗BT P1B)d(k)(85)

where T = KT BT P1BK.We now consider the part 2 of the difference equation of

the Lyapunov candidate function in (83) as

θ−2dT (k+1)(R2⊗ R1)d(k+1)−θ

−2dT (k)(R2⊗ R1)d(k)(86)

where R1 = (R1 +BT P1B) is a positive definite matrix. Using(80), one can express (86) as

1θ 2 [d

T (k+1)(R2⊗ R1)d(k+1)−2dT (k+1)(R2⊗ R1) f (k+1)

+ f T (k+1)(R2⊗ R1)( f (k+1)− dT (k)(R2⊗ R1)d(k)](87)

Using the dynamics of the distributed adaptive compensatorin (81) with (87), one has

= xT (k)(cLT Q2⊗KT BT P1A)x(k)+2dT (k)(Q2⊗BT P1A)x(k)

+2 f T (k)(Q2⊗BT P1A)x(k)−2θ−1 f T (k+1)(Q2⊗BT P1A)x(k)

+(1−θ−2)dT (k)(R2⊗ R1)d(k)+2 f T (k)(R2⊗ R1)d(k)

−2θ−1 f T (k+1)(R2⊗ R1)d(k)+ f T (k)(R2⊗ R1) f (k)

−2θ−1 f T (k+1)(R2⊗ R1) f (k)

+θ−2 f T (k+1)(R2⊗ R1)( f (k+1)

(88)

On further simplification, one can write (88) as

= xT (k)(cLT Q2⊗KT BT P1A)x(k)+2dT (k)(Q2⊗BT P1A)x(k)

+2[ f (k)−θ−1 f (k+1)]T (Q2⊗BT P1A)x(k)

+(1−θ−2)dT (k)(R2⊗ R1)d(k)

+[ f (k)−θ−1 f (k+1)]T (R2⊗ R1)d(k)

+[ f (k)−θ−1 f (k+1)]T (R2⊗ R1)[ f (k)−θ

−1 f (k+1)](89)

Using the Young’s inequality, one can simplify (89) as

≤ 32

xT (k)(cQ2L⊗AT P1BK)x(k)+2dT (k)(Q2⊗BT P1A)x(k)

+(2−θ−2)dT (k)(R2⊗ R1)d(k)

+4[ f (k)−θ−1 f (k+1)]T (R2⊗ R1)[ f (k)−θ

−1 f (k+1)](90)

Integrating equation (85) and (90), one can express the differ-ence equation of the Lyapunov candidate function as

∆V (k) =V (k+1)−V (k)6−xT (k)(Q2⊗Q1)x(k)

−xT (k)(−Q2 +12

Q2L)⊗AT P1BK)x(k)

+2c2λmin(LT L)λmin(T Q−1

1 ))xT (k)(Q2⊗Q1)x(k)

+2dT (k)(Q2⊗BT P1B)d(k)+(2−θ−2)dT (k)(R2⊗ R1)d(k)

+4[ f (k)−θ−1 f (k+1)]T (R2⊗ R1)[ f (k)−θ

−1 f (k+1)](91)

Further, one can simplify and write (91) as

∆V 6−xT (k)(Q2⊗Q1)x(k)

−xT (k)(−Q2 +12

cQ2L)⊗AT P1BK)x(k)

+2c2λmin(LT L)λmin(T Q−1

1 ))xT (k)(Q2⊗Q1)x(k)

−(θ−2−2−2λmin(cLBT P1BR−11 )dT (k)(R2⊗ R1)d(k)

+4[ f (k)−θ−1 f (k+1)]T (R2⊗ R1)[ f (k)−θ

−1 f (k+1)](92)

One can infer that ∆V ≤ 0, if the fol-lowing conditions are satisfied:

∥∥d(k)∥∥ >

4θ−2−2−2λmin(cLBT P1BR−1

1 )

∥∥( f (k)−θ−1 f (k+1))∥∥ and

2λm

< c < 1

λm

√2λmin(T Q−1

1 ). This shows that agent’s consensus

error is bounded. Therefore, the actual agent’s state x(k)achieve the desired consensus behavior with a bounded errorthat can be made arbitrarily small. This completes the proof.�

Remark 7. As presented in Theorem 4 and Corollary 2, exist-ing H∞ approaches minimize the local neighborhood trackingerror of the system εi(k) and are not capable of attenuatingsophisticated attacks. In contrast, the designed distributedresilient control can successfully attenuate the adverse effectsof attacks using the distributed adaptive compensator. Thedeveloped compensator di(k) in (81) minimizes the deviationof the local neighborhood tracking error of the system εi(k)from the local neighborhood tracking error of the expectedstate predictor εi(k). We can also infer that, although theproposed controller is designed for leaderless multi-agentsystems, it can be used for the leader-follower systems andthe containment control systems.Remark 8. Compromised agents under the effect of the sensorattack might not be recovered completely and result a non-zero bound in (92). The proposed distributed adaptive lawcompensates the difference between the incoming neighboringsensor measurement (xc

i (k)) and the desired state xi(k) andxc

i (k) 6= xi(k) in the case of sensor attack. Under the actuatorattack xc

i (k) = xi(k) and the error bound for (92) can be madearbitrarily small.

V. SIMULATION RESULTS

This section presents simulation results to validate the ef-fectiveness of the presented work. We consider both leaderlessas well as leader-follower network for the simulation. First, aleader-follower network of autonomous underwater vehicle’s(AUV’s) in Fig. 3 is considered for the evaluation of thepresented results. Then, the leaderless network is considered.A. Leader-follower Network

The communication network is shown in Fig.3, consists ofSentry AUVs. Sentry AUV, manufactured by the Woods HoleOceanographic Institution [42]. The linearized model of theSentry is of 6 DOF, but it is generally decomposed into fournon-interacting subsystems which are speed subsystem (u),the roll subsystem (φ), the steering subsystem (ν ,r,ψ), thediving subsystem (ω,q,z,θ).

Here, we focus on the diving subsystem of Sentry AUV forthe desired depth maneuvering in the leader-follower network.

Fig. 3: Distributed network of AUVs under the influence of attack

Consider a graph topology shown in Fig. 4 for the teamof Sentry AUVs communicating with each other with thefollowing diving subsystem dynamics

xi(k+1) = Axi(k)+Bui(k)

where

A =

0.65 0.54 0.0 -0.00190.21 1.48 0.0 -0.010.83 0.84 1.0 0.990.11 1.21 0.0 0.99

and

B =

0.08 0.13- 0.13 0.200.02 0.09

- 0.07 0.09

with xi(k) = [(ωi(k), qi(k), zi(k), θi(k))]T , andui(k) = [δ b

i (k), δ bi (k)]

T , where (ωi(k), qi(k), zi(k), θi(k))and δ b

i (k), δ bi (k) represent the heave speed, pitch rate, depth

and pitch, and bone and stern plane deflections, respectively.

Fig. 4: Graph topology

In the network communication graph, we assumed agent0 represents a active non-autonomous leader which aim tofollow a desired sinusoidal depth trajectory and agents 1 to5 designate the followers. The leader has the control inputu0(k) = K0x0(k) + r(k), where K0 is state feedback gain,x0 denotes the leader state and r(k) represents the desiredsinusoidal trajectory, respectively. Since the leader input isnon-zero, slightly different discrete-time control protocol fromthat the one proposed in the paper is used for which the leaderexchanges its input signal u0 with its neighbors and agentsreach consensus by exchanging states and leader’s input. This,

however, does not change our attack analysis and mitigation.The state feedback gain K0 is given by

K0 =

[- 0.18 -2.25 0.13 -0.211.56 5.39 0.49 1.59

]In the absence of the attack, agents follow the desired depth

trajectory and illustrate the healthy behavior of the network asshown in Fig. 5.

Now, the effect of the attack on a non-root node is an-alyzed. First, we consider the attack on actuators of Agent3 (non-root node) with time-varying attack signal ua

3(k) =[10sin(k) 10sin(k)]′ at t = 61 sec. Fig. 6(a) and (b) showthat agents which are reachable from the compromised Agent3 are deviated from the desired behavior, despite the localneighborhood tracking error goes to zero for intact agents.These results follow the Theorem 2 and Theorem 4.

20 40 60 80 100 120 140Time (s)

5

10

15

20

Dep

th (

m)

LeaderAgent 1Agent 2Agent 3Agent 4Agent 5

Fig. 5: Agents depth trajectory in healthy mode

20 40 60 80 100 120 140Time (s)

5

10

15

20

Dep

th (

m)

LeaderAgent 1Agent 2Agent 3Agent 4Agent 5

65 70 75 8014

16

18

(a)

20 40 60 80 100 120 140Time (s)

-1.5

-1

-0.5

0

0.5

1

1.5

Err

or

(m)

Agent 1Agent 2Agent 3Agent 4Agent 5

(b)

Fig. 6: The Agents depth trajectory under the influence of the attack on AUV 3. (a)Depth without adaptive compensator (b) The local neighborhood tracking error withoutadaptive compensator

Let Q1 and R1 be identity matrix in (72) and (73), respec-tively. Then, the proposed resilient control protocol in (74)with adaptive attack compensator in (76) is applied in thenetwork for the mitigation of adversarial input. Then, Fig. 7(a)

20 40 60 80 100 120 140Time (s)

5

10

15

20D

epth

(m

)

LeaderAgent 1Agent 2Agent 3Agent 4Agent 5

(a)

20 40 60 80 100 120 140Time (s)

-1.5

-1

-0.5

0

0.5

1

1.5

Err

or

(m)

Agent 1Agent 2Agent 3Agent 4Agent 5

(b)

Fig. 7: The Agents depth trajectory under the influence of the attack on AUV 3. (a)Depth with adaptive compensator (b) The local neighborhood tracking error with adaptivecompensator

20 40 60 80 100 120 140Time (s)

5

10

15

20

Dep

th (

m)

LeaderAgent 1Agent 2Agent 3Agent 4Agent 5

72 74 76 78 80 82Time (s)

16

18

(a)

20 40 60 80 100 120 140Time (s)

5

10

15

20

Dep

th (

m)

LeaderAgent 1Agent 2Agent 3Agent 4Agent 5

(b)

Fig. 8: The Agents depth trajectory under the influence of the attack on AUV 2. (a)Depth without adaptive compensator (b) The local neighborhood tracking error withoutadaptive compensator

and (b) illustrate the response of the system under the influenceof the attacks using the proposed controller from t=61 sec. Thesystem states achieve the desired consensus behavior and the

local neighborhood tracking error goes to almost zero, evenin the presence of the attack. These results demonstrate theeffectiveness of the proposed resilient controller.

Now, consider the effect of constant attack signal on actu-ators of the Agent 2 (root-node) given by ua

2(k) = [5 5]′ att = 61 sec. Fig. 8(a) shows that the compromised agent affectthe reachable agents after attack. Now, the proposed resilientcontrol protocol in (74) with adaptive attack compensator in(76) is applied at t=61sec in the distributed network and Fig.8(b) shows that the system states achieve the desired consensusbehavior, even in the presence of the attack.B. Leaderless Network

Now, consider the same graph topology in Fig. 4 forleaderless DMAS for 5 agents without leader Agent 0. Forleaderless system the dynamics of agent i is considered as

xi(k+1) =[

0 −11 0

]xi(k)+

[01

]ui(k)

f or i = 1, . . . ,5(93)

First, the effect of the attack on a root node is analyzedwith the IMP-based attack signal.

Consider the effect of attack on actuator of Agent 2 by IMP-based attack signal i.e. ua

2(k) = sin(k). Fig. 9(a) shows that thecompromised agent destabilizes the entire network. All agentsof the DMAS deviate from the desired consensus behavior.The simulation results verify Theorem 2 and Theorem 3. LetQ1 and R1 be identity matrix in (72) and (73), respectively.Now, the proposed resilient control protocol in (74) with

10 20 30 40 50 60 70 80Time (s)

-100

-50

0

50

100Agent 1Agent 2Agent 3Agent 4Agent 5

(a)

10 20 30 40 50 60 70 80Time (s)

-2

-1

0

1

2Agent 1Agent 2Agent 3Agent 4Agent 5

(b)

Fig. 9: The DMAS response under the effect of IMP-based attack on agent 2 (root node)with adaptive compensator. (a) The agent’s state without adaptive compensator (b) Theagent’s state with adaptive compensator

adaptive attack compensator in (76) is incorporated and Fig.9(b) shows the response of the system. The system states

10 20 30 40 50 60 70 80Time (s)

-4

-2

0

2

4Agent 1Agent 2Agent 3Agent 4Agent 5

(a)

10 20 30 40 50 60 70 80Time (s)

-2

-1

0

1

2Agent 1Agent 2Agent 3Agent 4Agent 5

(b)

Fig. 10: The DMAS response under the effect of IMP-based attack on agent 3 (non-rootnode) without adaptive compensator. (a) The agent’s state (b) The local neighborhoodtracking error

10 20 30 40 50 60 70 80Time (s)

-2

-1

0

1

2Agent 1Agent 2Agent 3Agent 4Agent 5

(a)

10 20 30 40 50 60 70 80Time (s)

-1

-0.5

0

0.5

1Agent 1Agent 2Agent 3Agent 4Agent 5

(b)

Fig. 11: The DMAS response under the effect of IMP-based attack on agent 3 (non-root node) with adaptive compensator. (a) The agent’s state (b) The local neighborhoodtracking error

achieve the desired consensus behavior, even in the presenceof the attack on root node. This illustrates the mitigation of

sophisticated attack using the designed resilient controller.Now, we present the results for the effect of the attack

on a non-root node. Consider an IMP-based attack signalis launched on actuator of Agent 3 (non-root node) i.e.ua

3(k) = sin(k). Fig. 10(a) and (b) show that Agents 4 and5 which are reachable from the compromised Agent 3 donot converge to the desired consensus value and the localneighborhood tracking error goes to zero for intact agents.These results comply with Theorem 3 and Theorem 4. Then,the resilient control protocol in (74) with adaptive attackcompensator in (76) is used. Fig. 11(a) and (b) illustrate thatthe system states achieve the desired consensus behavior andthe local neighborhood tracking error goes to zero, even in thepresence of the attack on non-root node 3. This demonstratesthe mitigation of attack using the developed resilient controller.

VI. CONCLUSION

This paper presents a rigorous analysis of the effects ofattacks for leaderless discrete-time DMAS and designs aresilient distributed control protocol their mitigation. It isshown that the attack on a compromised agent can propagatethrough the entire network and affects intact agents those arereachable from it. Then, the IMP for the attacker shows thatan attack on a single root node can destabilize the entirenetwork. The attacker does not require to know about thecommunication graph and the system dynamics. Furthermore,the ineffectiveness of existing robust approach is discussed forsophisticated attacks. To overcome the effect of the attackson sensor and actuators of the agent in discrete-time DMAS,a resilient controller is developed based on a expected statepredictor. The presented controller shows that the attack onsensor and actuator can be mitigated without compromisingthe connectivity of the network and achieves the desiredconsensus. Although we have considered a general leaderlessconsensus problem for the proposed controller, it can be usedfor the other DMAS problems such as leader-follower andcontainment control problem. The analysis and effectivenessof the presented work have shown in simulation results.

REFERENCES

[1] R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and cooper-ation in networked multi-agent systems”, Proceedings of the IEEE, vol.95, pp. 215-233, Jan 2007.

[2] J. A. Fax, and R. M. Murray, “Information flow and cooperative controlof vehicle formations”, IEEE Transactions on Automatic Control, vol.49, no. 9, pp. 1465-1476, 2004.

[3] F. Bullo, J. Cortes, and S. Martinez, Distributed control of roboticnetworks: a mathematical approach to motion coordination algorithms.Princeton University Press, 2009.

[4] A. Jadbabaie, J. Lin, and A. Morse, “Coordination of groups of mobileautonomous agents using nearest neighbor rules”, IEEE Transactions onAutomatic Control, vol. 48, no. 6, pp. 988-1001, 2003.

[5] A.J. Kerns, D. P. Shepard, J. A. Bhatti, and T. E. Humphreys, “Un-manned aircraft capture and control via GPS spoofing”, Journal of FieldRobotics, vol. 31, pp. 617-636, 2014.

[6] F. Pasqualetti, F. Drfler, and F. Bullo, “Attack detection and identificationin cyber-physical systems”, IEEE Transactions on Automatic Control,vol. 58, pp. 2715-2729, 2013.

[7] K. G. Vamvoudakis, J. P. Hespanha, B. Sinopoli, and Y. Mo, “Detectionin adversarial environments,” IEEE Transactions on Automatic Control,vol. 59, no. 12, pp. 3209-3223, 2014.

[8] A. Teixeira, H. Sandberg, and K. H. Johansson, “ Strategic stealthyattacks: the output-to-output l2-gain,” In Proceedings of IEEE 54thAnnual Conference on Decision and Control, pp. 2582-2587, 2015.

[9] Z. Guo, D. Shi, K. H. Johansson, and L. Shi, “Optimal linear cyber-attack on remote state estimation,” IEEE Transactions on Control ofNetwork Systems, vol. 4, no. 1, pp. 4-13, 2017.

[10] R. Mitchell, and I. Chen, “Adaptive intrusion detection of maliciousunmanned air vehicles using behavior rule specifications”, IEEE Trans-actions on SMC systems, vol. 44, no. 5, pp. 593-604, May 2014.

[11] N. Bezzo, J. Weimer, M. Pajic, O. Sokolsky, G. J. Pappas, and I. Lee,“Attack resilient state estimation for autonomous robotic systems,” Inproceedings of IEEE/RSJ Conference on Intelligent Robots and Systems,pp. 3692-3698, 2014.

[12] C. Persis, and P. Tesi, “Resilient Control under Denial-of-Service”, Inproceedings of the 19th IFAC World Congress, South Africa, 2014.

[13] S. Sundaram, and C.N. Hadjicostis, “Distributed function calculationvia linear iterative strategies in the presence of malicious agents,” IEEETransactions on Automatic Control, vol. 56, no. 7, pp. 1495-1508, 2011.

[14] S. Sundaram, and B. Gharesifard, “Consensus-based distributed op-timization with malicious nodes,” In Proceedings of Conference onCommunication, Control, and Computing (Allerton), pp. 244-249, 2015.

[15] F. Pasqualetti, A. Bicchi, and F. Bullo, “Consensus computation inunreliable networks: A system theoretic approach”, IEEE Transactionson Automatic Control, vol. 57, no. 1, pp. 90-104, 2012.

[16] H.J. LeBlanc, H. Zhang, X. Koutsoukos, and S. Sundaram, “Resilientasymptotic consensus in robust networks,” IEEE Journal on SelectedAreas in Communications, vol. 31, no. 4, pp. 766-781, 2013.

[17] K. Saulnier, D. Saldana, A. Prorok, G. Pappas, and V. Kumar, “ResilientFlocking for Mobile Robot Teams,” IEEE Robotics and AutomationLetters, vol. 2, no. 2, pp. 1039-1046, 2017.

[18] S. M. Dibaji, H. Ishii, and R. Tempo, “Resilient Randomized QuantizedConsensus,” IEEE Transactions on Automatic Control, 2017.

[19] A. Teixeira, I. Shames, H. Sandberg, and K. H. Johansson, “A securecontrol framework for resource-limited adversaries,” Automatica, vol.51, pp. 135-148, 2015.

[20] H.J. LeBlanc, H. Zhang, S. Sundaram, and X. Koutsoukos, “Consensusof multi-agent networks in the presence of adversaries using only localinformation,” In Proceedings of the 1st international conference on HighConfidence Networked Systems, pp. 1-10, 2012.

[21] H.J. LeBlanc, and X. Koutsoukos, “Resilient synchronization in robustnetworked multi-agent systems,” In Proceedings of the 16th interna-tional conference on Hybrid systems: computation and control, pp. 21-30, 2013.

[22] W. Zeng, and M. Chow, “Resilient distributed control in the presence ofmisbehaving agents in networked control systems,” IEEE transactionson cybernetics, vol. 44, no. 11, pp. 2038-2049, 2014.

[23] T. Alpcan, and T. Basar, “A game theoretic analysis of intrusion detectionin access control systems,” In Proceedings of IEEE Conference onDecision and Control, pp. 1568-1573, 2004.

[24] Y. Mo, and B. Sinopoli, “Secure estimation in the presence of integrityattacks”, IEEE Transactions on Automatic Control, vol. 60, no. 4, pp.1145-1151, 2015.

[25] Q. Zhu, and T. Basar, “Robust and resilient control design for cyber-physical systems with an application to power systems,” In Proceedingsof IEEE Conference on Decision and Control and European ControlConference (CDC-ECC), pp. 4066-4071, 2011.

[26] A. Hota, and S. Sundaram, “Interdependent security games on networksunder behavioral probability weighting”, IEEE Transactions on Controlof Network Systems, 2016.

[27] M. Zhu, and S. Martnez, “On distributed constrained formation controlin operatorvehicle adversarial networks,” Automatica, vol. 49, no. 12,pp. 3571-3582, 2013.

[28] M. Zhu, and S. Martnez, “On the performance analysis of resilientnetworked control systems under replay attacks,” IEEE Transactions onAutomatic Control, vol. 59, no. 3, pp.804-808, 2014.

[29] M. Zhu, and S. Martnez, “Consensus-based distributed optimizationwith malicious nodes,” In Proceedings of Communication, Control, andComputing (Allerton), pp. 244-249, 2015.

[30] H. Fawzi, P. Tabuada, and S. Diggavi, “Secure estimation and control forcyber-physical systems under adversarial attacks,” IEEE Transactions onAutomatic Control, vol. 59, no. 6, pp. 1454-1467, 2014.

[31] Y. Shoukry, P. Nuzzo, A. Puggelli, A.L. Sangiovanni-Vincentelli, S.A.Seshia, and P. Tabuada, “Secure state estimation for cyber physicalsystems under sensor attacks: a satisfiability modulo theory approach,”IEEE Transactions on Automatic Control, 2017.

[32] G. D. Torre and T. Yucelen, “Adaptive architectures for resilient controlof networked multi-agent systems in the presence of misbehavingagents,” International Journal of Control, pp. 1-13, 2017.

[33] X. Jin, W.M. Haddad, and T. Yucelen, “An adaptive control architecturefor mitigating sensor and actuator attacks in cyber-physical systems,”IEEE Transactions on Automatic Control, 2017.

[34] F. Lewis, H. Zhang, K. Hengster-Movric, and A. Das, Cooperative Con-trol of Multi-Agent Systems: Optimal and Adaptive Design Approaches,Communications and Control Engineering, Springer, London, 2013.

[35] Z. Li, and Z. Duan, Cooperative Control of Multi-Agent Systems:A Consensus Region Approach. Automation and Control Engineering,Taylor and Francis, 2014.

[36] K.G. Vamvoudakis, F. L. Lewis, and G. R. Hudas, “Multi-agent differ-ential graphical games: Online adaptive learning solution for synchro-nization with optimality”, Automatica, vol. 48, no. 8, pp. 1598-1611,2012.

[37] Y. Su, and J. Huang, “Stability of a Class of Linear Switching Systemswith Applications to Two Consensus Problems”,” IEEE Transactions onAutomatic Control, vol. 57, no. 6, pp. 1420-1430, June 2012.

[38] D. Harville, Matrix algebra from a statistician’s perspective, Springer-Verlag New York, 1997.

[39] Q. Jiao, H. Modares, F. L. Lewis, S. Xu, and L. Xie, “Distributed gainoutput-feedback control of homogeneous and heterogeneous systems”,Automatica, vol. 71, pp. 361-368, 2016.

[40] Q. Liu, Z. Wang, X. He, and D. Zhou, “Event-Based H∞ ConsensusControl of Multi-Agent Systems With Relative Output Feedback: TheFinite-Horizon Case”, IEEE Transactions on Automatic Control, vol. 60,no. 9, pp. 2553-2558, 2015.

[41] A. Isidori, Nonlinear control systems, Springer Science and BusinessMedia, 2013.

[42] M. V. Jakuba, “Modeling and control of an autonomous underwatervehicle with combined foil/thruster actuators,” M.S. thesis, Dept. Mech.Eng., MIT Woods Hole Oceanographic Inst., Cambridge, MA, USA,2003.