Distributed genetic evolution in WSN

Distributed Genetic Evolution in WSN

Philip Valencia ∗†

[email protected] Lindsay †

[email protected] Jurdak ∗†

[email protected]∗CSIRO ICT Centre. Brisbane, Queensland. Australia

†The University of Queensland, School of ITEE. Brisbane, Queensland. Australia

ABSTRACTWireless Sensor Actuator Networks (WSANs) extend wire-less sensor networks through actuation capability. Design-ing robust logic for WSANs however is challenging sincenodes can affect their environment which is already inher-ently complex and dynamic. Fixed (offline) logic does nothave the ability to adapt to significant environmental changesand can fail under changed conditions. To address this chal-lenge, we present In situ Distributed Genetic Programming(IDGP) as a framework for evolving logic post-deployment(online) and implement this framework on a physically de-ployed WSAN. To demonstrate the features of the frame-work including individual, cooperative and heterogeneousevolution, we apply it to two simple optimisation problemsrequiring sensing, communications and actuation. The ex-periments confirm that IDGP can evolve code to achieve asystem wide objective function and is resilient to unexpectedenvironmental changes.

Categories and Subject DescriptorsI.2.2 [Computing Methodologies]: Artificial Intelligence—Automatic Programming [Program synthesis]

General TermsAlgorithms, Design

KeywordsDistributed, Online, Learning, Genetic Program

1. INTRODUCTIONWireless Sensor Networks (WSN) are typically passive

sensing systems comprised of resource-constrained devices.Each mote samples and sends sensor data to a base via wire-less multi-hop communications. Achieving reliable, robustdeployments of this seemingly simple configuration has beenchallenging. An increasing number of deployments not only

monitor the environment, but also perform actuation in re-sponse to sensed data. These Wireless Sensor and Actu-ator Networks (WSANs)[1] and their interaction with theenvironment can make the system state unpredictable. Of-ten, actuation is used to steer the system towards a de-sired collective behaviour. However the distributed natureof WSANs means motes have only partial knowledge of thesystem which makes determining the appropriate mote logica challenging task. This “inverse problem” of engineeringa collective behaviour is an increasingly common challengebeing also faced across disciplines such as Complex Sys-tems, Multi-Agent Systems (MAS), swarm and cooperativerobotics.

WSAN logic is currently designed prior to deployment,mostly by humans, and subject to their experiences and do-main knowledge. Explicit and implicit idealisations aboutthe real world when designing logic consequently limit ex-pected behaviour to specific operating and environmentalconditions. This is commonly referred to as “brittle” logic[6] and is largely due to the inability to predict the out-come of complex interactions between motes and a dynamicphysical environment.

Various efforts in automating the offline design of logic toreduce the human involvement ([17], [19]) in the design pro-cess have been attempted but can still fail post-deploymentdue to assumptions and idealisations of the offline simulatedenvironment not adequately reflecting the real world [12], [3].Since it is not possible to obtain complete a priori knowl-edge of the environment, unanticipated situations can arisewhich cause predetermined logic to fail. Online learning canaddress this issue by adapting logic post-deployment.

Building on the learning method of Genetic Program-ming (GP) [7], [9] that typically runs offline, we pro-pose the In situ Distributed Genetic Programming (IDGP)framework as a method to achieve WSAN logic which isrobust to changing complex environments by performingonline GP on each mote. This paper empirically demon-strates the feasibility of our approach on trivial objectiveswith well-understood characteristics and known optimal so-lutions. These objectives were deliberately crafted to enablethe framework to be studied in detail. However, we arguethat IDGP is also applicable to more complex WSAN prob-lems.

One such complex problem is that of dissuading a herdof cattle from grazing a designated area through a processknown as “Virtual fencing” [18]. Virtual fencing collars usehigh power GPS modules to maintain accurate position in-formation, which limits the battery lifetime and precludes

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.IPSN’10, April 12–16, 2010, Stockholm, Sweden.Copyright 2010 ACM 978-1-60558-988-6/10/04 ...$10.00.

13

long-term deployment. Many factors impact on the energy-accuracy tradeoff, such as individual animal and herd be-haviour, distance to the exclusion zone, time of day, weatherand potentially others. Modelling these factors typically re-quires extensive data gathering through numerous field tri-als and substantial offline processing to design appropriateprogram logic. As an online learning mechanism, IDGP en-ables complex WSAN applications such as virtual fencingto locally evolve their logic in response to behavioural andenvironmental changes.

The remainder of the paper is structured as follows: Sec-tion 2 surveys related work, Section 3 presents the IDGPframework, Section 4 describes our empirical experimentsand results for various sensing, actuation and communica-tion problems, Section 5 discusses the challenges and lessonslearnt from the experiments, before concluding in Section 6.

2. RELATED WORKEvolutionary algorithms (EAs) have been proposed as a

potential solution to the growing problem of designing logicfor WSNs [8], [19]. Despite the seemingly obvious fit of EAswith the intrinsic parallel nature of WSNs, few implementa-tions exist where evolution is performed within the networkof physical nodes [11].

The Genetically Programmed Network (GPN) [17] is anoffline approach which demonstrates system-wide coordinatedlogic can be obtained by evolving the communication con-nections between nodes whilst simultaneously evolving thelogic of each node. It also highlights the importance of evolu-tion of communications between nodes and is fairly agnosticabout the logic implementation. Another offline approach,the Distributed Genetic Programming Framework (DGPF)[19], simulates and evolves WSN logic offline through ge-netic programming across a number of computers. As withall offline approaches, the fidelity of the simulation plays acrucial role in how well the logic will perform once placedinto the“real-world” [3]. However, even when the simulationof physics is precise, brittle logic can still occur due to thefailure of capturing unexpected environmental changes.

The majority of online EA research is concerned with op-timising the behaviour of a single entity for a specific tasksuch as obstacle avoidance [3] or logistics optimisation [2].These frameworks perform the in situ evolution of logic ona single machine and do not address the distributed coordi-nation issue relevant to sensor networks. GP in WSNs wasproposed by [8] and [15] and while not physically demon-strated, served to highlight the importance of EA parametervalues for online systems.

Distributed Agent Evolution with Dynamic Adaptation toLocal Unexpected Scenarios (DAEDALUS) [5] is an exam-ple of an online evolving system which has been physicallyimplemented. It is demonstrated on a number of robots [4]where each robot is an entity of the evolutionary population.Robots execute their program and can share their programand fitness with other robots. Interestingly the robots areseeded with logic that has been evolved offline and then thelogic is slightly mutated. This pragmatic step is understand-able given that initial evolution would have yielded randombehaviour that could have damaged the robots. Furthermorethe time required to evolve any sort of useful behaviour mayhave exceeded the timeframe available to the authors. Thishowever introduces significant biases from the offline learn-ing phase that may preclude better solutions being found

during the online phase. However, the most significant dis-advantage is that this approach was designed for a “swarm”of robots and the evolution of logic is largely dependant onthe other robots being present and that all robots are homo-geneous. If an environmental change is significant or highlylocalised, this approach may not reconverge to an acceptablesolution in an acceptable time since this approach relies onmutation and other nodes being present.

Our vision is to be able to provide high level objectivesto WSANs of resource constrained devices and have themcontinuously evolve their logic in order to optimise the ob-jective. We are unaware of this having been achieved andso we draw upon the aforementioned research to provide aframework and implementation.

3. IDGP FRAMEWORKThis section presents the details of our In situ Distributed

Genetic Programming (IDGP), as shown in Figure 1. At thecore of the framework is the IDGP engine which runs locallyon each mote and performs the in situ evolution of programs.IDGP programs support the typical syntactic richness of thestandard C language, such as functions with multiple argu-ments, nesting and multi-line programs(see Program 2a).

The user policy, which includes the system fitness functionand other initialisation parameters, serves as a key input intoIDGP. With the framework running on every mote, neigh-bouring motes can exchange locally generated programs ina form of cooperative evolution which may improve the con-vergence time to an acceptable solution.

3.1 GP on MotesGenetic programming (GP) uses a genetic algorithm on a

population of variable-length computer programs to evolvelogic that satisfies a specified fitness function over a num-ber of generations. Implementations of genetic program-ming engines typically reside on high-end computers, suchas super-computers and clusters, with significant computa-tional power and memory. This is largely due to three in-herent properties of genetic programming: (1) large memoryrequirement for storing a program population; (2) large sim-ulation computation that grows for more complex problems;and (3) requirement for quick convergence time.

In contrast to high-end computers, motes are significantlyresource-constrained when compared to even modest mod-ern computers. Motes typically have low end processors,limited memory, limited energy storage and short range,low bandwidth communications capability. These limita-tions render GP engines for motes a challenge, as they placetight constraints on the representation of programs, the in-struction set and the code space footprint of the GP engine.

Alongside this challenge, the benefit of in situ logic evolu-tion is the elimination of the simulation processing require-ment, since evolution occurs at runtime and is tightly cou-pled to the dynamic physical environment. While simulationrelies on models of the environment, in situ evolution relieson real-time data from the environment, rendering it moreexpressive and adaptive. However, adaptive logic in WSANscarries the risk of changing the environment in such a waythat the system could reach a “dead end” state, where it isimpossible to evolve to an acceptable solution. As an initialdemonstration of the IDGP framework, we assume the fol-lowing: (1) neither logic evolution nor actuation will causean evolutionary dead end; and (2) the IDGP engine can ob-

14

Wireless Sensor Node

AB

...

User Policies

FitnessFunction

GP EngineParameters LMAX, NP

Generate Initial Programs

Execute and Evaluate Programs

Execution Thread

Evaluation Thread

Rank Programs ...

CD

EF

GH

IIJ

KL

KL

OP

WX

AB

QR

GZ

VP

Generate W Highly Ranked programs via clone with mutation

Broadcast a random selection of programs

Receive Programs

GH

V W

GH

KL

NP

LMAX

XR

OP

Wireless Sensor Node

IDGP Engine

.

.

.

Copy V Elite programs to new population

1

2

4

5

6

Biased crossover to breed X Child programs

7

V

GH

KL

GZ

VP

OB

AP

V W X

GH

KL

GZ

VP

OB

AP

V W X

US

Y

GH

KL

GZ

VP

OB

AP

V W X

US

TG

Y Z

GH

KL

Generate Y Random Programs

8

Incorporate Z Other Programs

9

3

SW

TG

XN

...

XN

TG

SN

...

OP

AB

... ... ...

FD

AB

Wireless Sensor Node

IDGP Engine

OP

DQ

FV

...

OA

AB

QB

...

Other Wireless Sensor Nodes

Figure 1: An overview of the In situ Genetic Programming (IDGP) Framework.

tain feedback about how the system is performing. Section5.5 explores the implications of relaxing assumption 1, whileassumption 2 is a strict requirement for IDGP.

3.1.1 Compact program representationIDGP employs prefix notation and represents programs

as parse strings of single byte instructions rather than themore traditional tree-based style using node pointers. This isan efficient method for achieving compact representation ofprograms in GP [8], however IDGP relaxes the constraint ofa fixed number of function arguments. IDGP can generatecomplex nested statements similar to standard tree-basedprogram representations. However, Linear GP (LGP) canavoid the computationally expensive machinery typical totree-based representations [14] since the crossover point isbetween any line and connecting fragments is always valid.To gain this benefit, we extend tree-based representationsuch that nested statements form a single line of a multi-lineprogram much like conventional C programs. Effectively,the interpreter parses one code tree per line of code until anEND OF PROGRAM instruction byte is reached.

3.1.2 Instruction setThe instruction set varies by platform, due to varying ca-

pabilities of different motes, and by the capability neededto address the system objective. The prefix notation imple-mented uses 1-byte instructions and an optional 2-byte da-tum field if required by the instruction. Constants (signed16-bit integers) are currently the only instruction which usethis datum field, and are therefore represented by 3 bytes.

3.1.3 Program GenerationWe use prefix notation representation to guarantee syn-

Table 1: Instruction Representation# Bits 2 6 16 (Optional)

Use VAR index Op Code Datum Field

tactically correct programs are generated. This is possiblesince the instruction set and number and argument types areknown for each instruction. The framework currently sup-ports the int16 type and functions with up to 3 arguments.It is also extendible to additional types and functions withmore arguments. The population of programs accounts forthe most significant portion of memory for in situ evolution.

For simplicity, and without loss of generality, IDGP cur-rently implements a user-defined maximum program lengthand fixed population size to address this design decision. Asthe IDGP engine is assembling the program, it uses the num-ber of remaining instructions to select instructions whosenumber of required arguments does not cause the programto exceed the maximum program length. This approachinduces a small syntactic bias towards functions with lessparameters and the bias is obviously greater when the max-imum program size is nearer to the maximum number ofarguments for any function. However, as the maximum pro-gram size increases with respect to the maximum number ofarguments, the syntactic bias diminishes.

A function selection bias, represented as a discrete prob-ability distribution function, is used to probabilistically se-lect instructions. This is currently user-supplied and used tobias the generation of programs away from extremely nestedprograms. If all functions have equal probability of being se-lected, the likelihood of generating deeply nested programsbecomes high while the number of generated program lines

15

Table 2: IDGP Program structureName Bytes Purpose

ProgramID

4 Unique program identifier which em-beds information on the distribution offunctions within the program

MutationRate

2 Specifies the mutation rate during pro-gram generation

ProgramBytes

1 Specifies the length of the program inbytes

Program 1 -LMax

The instructions ordered in prefix no-tation format

becomes low.Finally, a user-defined maximum nesting level constraint

is also imposed. During program generation, the IDGP en-gine passes terminals into all arguments of the current func-tion if it reaches the maximum nesting. The combinationof these constraints and biases generate programs with anumber of nested statements that is comparable to humangenerated code. Section 3.1.6 elaborates on the motivationfor multi-line programs.

The memory footprint required for the gene pool is Np

multiplied by the maximum program length, LMax. How-ever two identically sized populations of programs are allo-cated so that one population represents the current gener-ation being executed, while the other is used to store theprograms for the next generation. Depending on their pref-erence, users may choose to have many small programs orfewer large programs in a population. The total memoryrequired for all programs is thus 2 × Np × (LMax + LMeta),where LMeta is the length of any program metadata for aparticular program.

3.1.4 Program Meta-dataProgram metadata is also stored in addition to program

byte code. The program metadata block (see Table 2) in-cludes a program ID that is generated from the histogram ofinstructions within the program. This field is for identifyingprograms to quickly discern whether 2 programs have sim-ilar distributions of terminals and functions. An epigeneticmetadata field is reserved for the mutation rate of programswhen they act as parents. The final metadata field storesthe number of bytes in the program (which typically differsfrom the number of instructions) and is used by the frame-work for transmitting programs across multiple packets. Intotal, 7 bytes (LMeta) are used for metadata.

3.1.5 Program Execution and EvaluationThe genetic program engine initially generates a pool of

programs in a random fashion, biased by the function se-lection probabilities, as shown in step 1 of Figure 1. Eachprogram in the current generation is evaluated in turn (step2) where the GP engine launches a virtual machine inter-preter thread to run the evolved code and another to eval-uate program fitness. The fitness evaluation thread runs inparallel with the execution thread since fitness may need tobe calculated over the execution period of the program. Af-ter a specified evaluation period, the GP engine stops andkills the evaluation thread, to avoid potential deadlocks inexecution.

3.1.6 Genetic OperatorsAfter it executes all (Np) programs in the current genera-

tion, the IDGP engine ranks these programs (step 3) accord-ing to their achieved fitness. It then copies the NE programswith the highest scores into the next generation (elitist se-lection). Often simple mutations can assist the convergenceto an acceptable solution, so we also generate NH slightlymutated programs with high rankings. Next, NC childrenprograms are generated from a selection of NC parents as thesingle-point crossover and recombination yields 2 childrenprograms for each pair of parents chosen via roulette wheelselection without removal. Fitter programs may be selectedmultiple times, and even selected to breed with themselves.This may not necessarily generate a clone of itself since thecrossover point does not need to be symmetric. This abilityto create two new solutions from the same solution repre-sents a significant added capability of GP relative to typicalGenetic Algorithms (GA) [9].

When using prefix notation representation, crossover pointsneed to be carefully chosen to ensure the resulting programsare syntactically correct. The current implementation uses asimple approach of cutting programs at a zero nesting levelto ensure the generation of syntactically correct offsprings,whilst guaranteeing that the genetic information from bothparents is completely passed into the next generation.

During the evaluation of programs, other programs maybe transmitted from neighbouring motes. These programshave the potential to be worse than random if their objectiveis anticorrelated with the local mote’s. Communications isalso not guaranteed and so it is unknown how many otherprograms will migrate to this mote within one generation.We set an upper bound on how many others can be receivedwithin one generation and simply drop additional other pro-grams if the allocation has been reached. We term the num-ber of other programs that do arrive as Nothers. Finally, theremaining (Np − Nelite − Nhighrank − Nchildren − Nothers)programs are randomly generated programs. The newly cre-ated population is then ready to be evaluated.

The genetic operations and generation of the new popula-tion scales linearly with the number of programs in the pop-ulation. The IDGP overhead for the current population sizeof Np = 21 is less than 100ms per generation. This overheadis typically relatively small compared to the population eval-uation time. For example, with a program evaluation timeof 1 second per program, the overhead is <0.5%.

3.2 Distributed EvolutionSo far we have discussed logic evolution on a single mote,

however we wish to perform evolution across a distributednetwork of motes. The IDGP framework utilises the Is-land model [10] similar to the Broadcast-Distributed Par-allel (BDP) GP model developed by [8] to share logic be-tween motes and incorporate neighbouring information intheir evolution process. This coarse grained parallel geneticalgorithm approach is achieved by decomposing a programinto a number of radio packets and then reconstructing theprogram on the receiving mote. Other epigenetic featurescan also be shared via broadcasting over the radio, how-ever we limit the information transferred to programs only.Received programs may replace programs in the local pool(currently only random programs are replaced) and cannotbe selected as a parent program until after it has been eval-uated (see step 4 in Figure 1).

Distributed evolution requires fitness information to beavailable to motes independently of the state of other motes.

16

This decentralised approach offers scalability over a cen-tralised approach and robustness to network disconnections.Local feedback is an obvious way to achieve distributed evo-lution; however in many situations one may want a global so-lution which may not be locally optimal. The IDGP frame-work uses multihop radio packets (typically multicast) tosupply all motes with fitness information. This can then beused to overcome local optima in order to achieve desirablenetwork wide behaviour (see section 4.6).

4. EXPERIMENTS AND RESULTSTo illustrate various aspects of the IDGP framework, we

have devised a problem with a known optimal solution andimplemented the IDGP framework on a mote-class devicein order to solve the problem. However, unlike many GPsystems which perform evolution offline and are validatedthrough simulation, we perform all evolution in situ (i.e onthe physical motes). Thus experimental results are based ononline performance values rather than simulation.

4.1 The Mote PlatformThe FleckTM3b [16], shown in Figure 2, is a low power

wireless sensor network device designed by CSIRO specifi-cally for outdoor applications such as environmental moni-toring. The Fleck can measure many external and onboardsensors including the power used by the mote and features3 onboard LEDs - red, green and yellow which are softwarecontrolled.

We use the Fleck Operating System (FOS) which is alight-weight cooperative threading system designed specif-ically for the family of Fleck devices. FOS applications arewritten in C and support numerous high level functions suchas routed radio messages, sensing of onboard sensors and ex-ternal sensors. This allows highly functional programs to bewritten in a compact and elegant manner which are easilyunderstood.

4.2 The LED - Photodiode ProblemThis experiment aims to evaluate IDGP on a non-trivial

WSAN application that requires the sensing and actuationto be performed on a time-varying basis and demonstrateresilience of logic under changed conditions.

4.2.1 Existing Benchmark ProblemsThere are a number of well known “benchmark” optimi-

sation problems within the GP community [9] and many inthe broader optimisation community [13]. These problemstypically have well defined solution spaces and are useful forcomparing differences in optimisation techniques. However,none of these would demonstrate the capacity of the IDGPframework to generate WSAN solutions: in isolation, basedon local conditions; in a neighbourhood of motes with thesame objective but each with their own local characteristics;in a neighbourhood of motes with differing objectives, anddespite environmental changes.

4.2.2 Optimization ProblemTo demonstrate evolution of WSAN logic, we designed

a sensing-actuating objective function that uses the 3 on-board LEDs as the actuation capability and a light sensor(photodiode) as the sensing capability. The objective is tomaximise the light intensity received by the photodiode atthree time checkpoints while penalising energy usage at 6

Figure 2: FleckTM3b motes with a photodiode con-nected to an ADC input (shown left) which is facingthe onboard LEDs (shown right)

time checkpoints. The objective was weighted such that theoptimal logic (shown in Figure 3) would result in the LEDsblinking 3 times during the evaluation period of 500ms.

The light sensor (photodiode) is oriented roughly facingthe onboard LED (Fig 2) and is measured by the onboard10-bit ADC. The photodiodes have a reasonably narrow passband filterand are therefore more responsive to some LEDsthan others.

Program 1 The “Blink-3” objective function

fos_leds_set(0)idleOffset = fos_power_battery_current()delay(50)isOdd = TRUEWHILE program_is_executing

IF (isOdd)fitness = fitness + fos_adc_read()

ENDLEDmA = fos_power_battery_current() - idleOffsetfitness = fitness - 6 * LEDmAisOdd = not(isOdd)delay(100)

END

4.2.3 Experimental SetupThe LEDs affect the local environment by providing illu-

mination but are not necessarily the sole influence of light inthe environment. Sunlight, room lights and reflections arejust some of the “external” light sources that can contributeto the ambient light, and in turn affect the value read bythe light sensor. These “external influences” are often un-predictable, however are useful for demonstrating responsesto unanticipated environmental changes. At first instance,we avoid exposing the motes to external light by placing themotes in a box, lined with a light absorbing material andspaced such that the effect of the LEDs from any mote wouldhave minimal effect on other motes’ light sensors. This en-ables us to focus on the learning and performance of theIDGP framework. The light absorbing material surround-ing each mote was depressed to further reduce the influenceof light generated from neighbouring motes. Because theposition and orientation of the mote relative to the light ab-sorbing material can affect the sensor readings, no contactwith the motes was made during the main experiments. Op-timal solution fitnesses for each mote were measured beforeeach experiment and used to normalise fitnesses for subse-quent experiments.

17

4.2.4 Energy, Sensing, and Actuation

0 100 200 300 400 500

0

20

40

60

80

100

A

B

C

D

E

Milliseconds since program start

% o

f max

imum

AD

C r

eadi

ng

Figure 3: Sample points shown for the Blink3 ob-jective on an optimal solution (Program 2c)

The photodiodes are passive sensors and do not draw anypower from the Fleck. The onboard LEDs consume approx-imately 3mA each and can be measured by the onboardcurrent sensors. Note that LEDs are turned off before anyprogram is executed and evaluated. The penalty of the LEDcurrent draw is weighted such that a maximum fitness isachieved by toggling only the LED that most closely matchesthe light sensor filter band.

ADC readings at 50ms, 250ms and 450ms, and subtractthe weighted current draw at 50, 150, 250, 350 and 450ms.This produces an objective function described in pseudocodeby Program 1. An optimal solution to this objective functionis to have only the LED corresponding to the filter on at 50,250 and 450ms and all LEDs off at 150 and 350. As there are3 off-to-on LED transitions in the solution, we coin the nameBlink-3 for this objective function. A visual representationof this optimal solution is shown in Figure 3.

4.2.5 Optimal ProgramThe instruction set required by a program to achieve this

optimal fitness requires only constants and the delay(x) andset leds(x) functions. The IDGP instruction set was re-duced to these 3 instructions in order to improve convergencetimes. To ensure programs never exceed the maximum pro-gram size (in bytes, rather than in number of instructions),a “NOP” (no operation) instruction was introduced. Theeffect of this “NOP” instruction is typically negligible sinceit merely introduces a delay in the order of microseconds.

Using this instruction set, we design an optimal solution(Program 2a) to the Blink-3 objective and note that it is10 lines and an average nesting level of 1. Programs 2band 2c are shown for interest since retrospective inspectionof evolved code commonly revealed various forms of thesesuboptimal programs occurring.

4.3 Random Search BaselineUniform random selection of instructions will generate

programs with a calculable distribution of number of lines

Program 2 (a) is an optimal solution to the“Blink-3”objec-tive where X=1 and X=4 for motes with 550nm and 650nmpass filters respectively. Programs (b) and (c) are commonlocal maxima.

1: set_leds(X) 1: set_leds(X) 1: set_leds(X)2: delay(100) 2: delay(125) 2: delay(250)3: set_leds(0) 3: set_leds(0) 3: delay(250)4: delay(100) 4: delay(250) 4: END5: set_leds(X) 5: set_leds(X)6: delay(100) 6: delay(125)7: set_leds(0) 7: END8: delay(100)9: set_leds(X)

10: delay(100)11: END

Program 2a Program 2b Program 2c

per program and average nesting level. Typically, the opti-mal program length and average nesting is not known a pri-ori. To construct a best case benchmark of random searchwe set the instruction selection probabilities to maximise theprobability of randomly generating the known optimal so-lution Program 2(a). The selection probabilities of 0.43 forfunctions and 0.57 for terminals yields programs with an av-erage of 10.0098 lines and an average nesting level of 1.2850.Based on these probabilities we calculate the probability ofexactly generating Program 2(a) to be equal to 6.57 x 10−34.While this is practically zero, the probability of generatinga functionally equivalent program is much higher. Due tothe syntactic richness of the GP engine (multiple lines ofcode, nesting and a selection of functions), calculating theexact probability of generating a random program function-ally equivalent to Program 2(a) becomes intractable.

We therefore establish a baseline empirically by allowingmotes to generate populations of random programs (equiva-lent to the population used by the IDGP engine each gener-ation) and record the maximum fitness, pool average andstandard deviation for each population. Scores are nor-malised against the respective optimal fitness for each mote.In total, over 3.5 million randomly generated programs wereevaluated by the 8 motes using over 21 000 populations with21 programs per population. The maximum fitness foundwas 82.22% of the optimal solution fitness and the averagefitness of a random population (pool fitness) was 4.15% ofthe optimal solution fitness. This confirmed that findingan equivalent optimal solution to the Blink-3 objective is anon-trivial problem. We generate a curve which reflects thetypical maximum (elite) program fitness after i generations(evaluated populations) by randomly sampling the 21 000elite scores i times and taking the maximum fitness of thosesamples. The process is duplicated 10 000 times and the re-sults averaged for each generation value. This curve and theaverage pool fitness provide a comparison baseline for theexperiments in Section 4.4 and we refer to it as “Random”.

4.4 In situ EvolutionTo demonstrate that in situ evolution of logic can be per-

formed in isolation, we allow 8 motes to evolve indepen-dently and thus without sharing programs. The mutationrate was set to 5% and the subpopulation distribution de-fined as 1 elite, 3 highly ranked, 12 children, and 5 randoms.The size of the population (21 programs) and the distri-bution of subpopulations (elites, children etc) were chosen

18

heuristically and do not necessarily represent the optimallearning configuration. Figure 4 shows a typical run with 8individually evolving motes. Note that optimal scores havebeen normalised to each motes’ respective optimal solutionas measured before the experiment. There is an initial rapidlearning phase which can be largely attributed to learningto use the delay instruction in order to reach an evaluationpoint, followed by learning to use the LEDs.

Experiments were repeated to gain statistical confidenceand normalised before combining them by taking the aver-age of fitnesses at each generation. On average, motes will

0 500 1000 1500 2000 25000

0.2

0.4

0.6

0.8

1

Generation

Nor

mal

ised

Fitn

ess

Sco

re

Elite Scores and Average Pool Scores

81 Elite Scores81 Pool Average85 Elite Scores85 Pool Average88 Elite Scores88 Pool Average90 Elite Scores90 Pool Average92 Elite Scores92 Pool Average203 Elite Scores203 Pool Average204 Elite Scores204 Pool Average207 Elite Scores207 Pool Average

Figure 4: Individual evolution

typically find a solution that achieves 70% of the optimalsolution within 50 generations as compared to a randomsearch which yields an average fitness of less than 55% ofthe optimal solution within an equivalent number of pro-gram evaluations. This indicates that the framework’s GPimplementation performs better than random search, how-ever the subpopulation distribution is probably not an opti-mal learning strategy. Despite this, 3 out of the 8 motes inthis run have converged to the locally optimal solution bygeneration 1622. Due to noise in the system such as readingfluctuations and timing differences caused by other threads,it is possible for a mote to diverge from an optimal solution.

To demonstrate that cooperative evolution achieved viathe Island Model provides faster convergence than that ofevolution in isolation, we perform three experiments: (1)motes evolve their logic in isolation; (2) motes with a com-mon objective function share programs periodically via theirwireless radios, referred to as distributed homogeneous evo-lution; and (3) motes with different objective functions shareprograms periodically via their wireless radios, referred to asdistributed heterogeneous evolution. Although motes in thehomogeneous experiment share a common objective func-tion, their perception of the environment may be differentdue to spatial differences in light. In the heterogeneous ex-periments, both the objective functions and the environmen-tal perception of the motes can be different.

This experiment aims to explore whether motes with dif-ferent objectives can still benefit from using the Island model.

The importance of this arises from the likelihood that 2motes will neither be in exactly the same environment norhave exactly the same sensing configuration, and as suchmay require slightly different solutions to obtain acceptableperformance.

We test this by constructing groups of motes which arecomprised of 2 motes with 650nm pass filters (respondingmainly to red light) and 2 motes with 550nm (respondingmainly to green-yellow light) pass filters. The 4 motes arepermitted to share programs amongst each other. The mu-tation rate for all three experiments is 4%, and the sub-population distribution includes 1 elite, 3 highly ranked, 12children and 2 random programs and 3 “other” programs,which are obtained from other motes each generation.

0 50 100 150 200 250 300 350 400 450 50030

40

50

60

70

80

90

100

Generation

Nor

mal

ised

Fitn

ess

Sco

re (

%)

singlehomogeneousheterogeneous

Figure 5: Comparison of convergence: evolution inisolation, Island Model (homogeneous) and IslandModel (heterogeneous).

In all 3 experiments, the motes evolve for a minimum of50 generations. We then computed the aggregate normal-ized fitness score of each experiment for every generationin the learning process. Individual evolution was calculatedon 16 individual motes, homogeneous evolution on 4 runs of4 motes and heterogeneous evolution of 6 runs of 4 motes.Some runs were left to evolve longer than 50 generations toprovide some insight as to what may happen longer term.Figure 5 plots the normalized fitness scores of the 3 experi-ments versus generation.

The results shown in Figure 5 support the literature inthat sharing genetic material via the Island Model facilitatesfaster convergence than evolution in isolation. Both the ho-mogeneous and heterogeneous experiments reach a fitnessscore of 80% within 20 generations, while the single moteevolution takes 80 generation to reach the same fitness score.This results shows that groups of motes with similar objec-tives can help each other expedite convergence to reasonableperformance and still further specialise to their local fitnesslandscape.

Note that the fitness for motes in distributed heteroge-neous networks is similar to the fitness of the isolated motesby generation 100. By generation 400, the heterogeneous

19

evolution approach improves its fitness further thanks to itsgenetic diversity of programs, whereas the fitness of isolatedmotes remains the same.

4.5 Adaptation to Dynamic EnvironmentsThe motes in the previous experiments were placed in a

box and shielded from external light sources. Program 2awas designed with the assumption that there were no otherexternal light sources present. To simulate an unexpectedenvironmental change we place some of the motes outside thebox which are then subject to other light sources, namelysunlight through room windows and the fluorescent roomlighting. The optimal solution is dependant on the intensityof the ambient light into the photodiode and the energy con-sumed by the LEDs. As the ambient light increases, turningall LEDs off all the time becomes the optimal strategy.

Motes 85 and 207 were removed from the box after genera-tions 51 and 52 respectively. The elite scores reveal that thefitness exceeds that of the “known solution” for that whenthe mote is in the box. It can be seen in Figure 6 thatinitially the improvement is due to the additional ambientlight, however as the fitness increases after this it is conjec-tured that the mote discovers that it does not need to turnthe LEDs on at all to maintain a high fitness, and in doingso saves power and hence generates a higher fitness. Man-ual analysis of various programs within the period wheremotes were out of the box reinforces this conjecture withmany programs simply delaying more than the evaluationperiod. Interestingly, after the motes were placed back intothe box, the motes almost instantly returned to a fitness ofthat achieved before removal from the box, indicating thatthe genetic material maintained some history.

50 100 150 200

0.75

0.8

0.85

0.9

0.95

1

1.05

Generation

Fitn

ess

Sco

re

Elite Scores for Nodes Taken Out of the Bag

207 Elite Scores

Quick reconvergencedue to genetic memory

Adaptation to unexpected environmental change

Node takenout of bag

Node placedback in bag

Figure 6: Adaptation to the “unexpected event”

4.6 Evolution of CommunicationsIn the previous experiments, fitness was calculated en-

tirely locally on the motes and was based on mapping sensedvalues to actuation. In many scenarios however, it is likelythat fitness may come from neighbouring motes or a system-wide objective function.

In this experiment we wish to demonstrate the evolutionof communication and actuation based on communication.Again we desire to keep the problem well defined and assimple as possible. We therefore restrict the problem to

the function set {radio tx int(x), set leds(x), radio rx int()}and only use 2 motes which we will call Mote A and MoteB. We now generate IntToRfmM packets on a base mote(connected to a PC) at a rate of 1Hz and send these packetsto Mote A only by specifically addressing the MAC desti-nation address. All combinations of LEDs are cycled every8 seconds which in turn makes a sensible evaluation periodfor programs and was used for this experiment. The LEDstatus of Mote B is measured via a remote procedure call(RPC) and used to calculate a fitness based on the countervalue sent to Mote A and the LED status of Mote B. Hencethe system would ideally learn to receive the counter packetat Mote A, forward this to Mote B which in turn would re-ceive it and set its LEDs accordingly. Importantly, we donot explicitly reward communications, rather communica-tions must evolve in order to achieve the desired objective(LED state of Mote B). An optimal pair of programs to thissimple objective are programs 3a and 3b.

Base Node

Fitness Objective

Broadcast and receive programs

Wireless Sensor Node A

Execution Thread

Fitness

OP

...

Evaluation Thread

while(1)

Fitness--

Delay 100ms

while(1)

count++

FosMacSend(NodeA, count)

if NodeB.getleds() == (count & 7)

NodeA.addScore(80)

NodeB.addScore(80)

Delay 1s

Unicast MAC Msg

−

+

Routing RPC Msg

+

Routing RPC Msgs

radio_rx_int

Wireless Sensor Node B

Execution Thread

Fitness

OP

Evaluation Thread

while(1)

Fitness--

Delay 100ms

−

radio_rx_int (Broadcast MAC)

radio_tx_int (Broadcast MAC)

GH

...

IDGP Framework Communications

Evolved Logic Communications

Figure 7: Communications topology for the“PacketForwarder-RfmToLeds” experiment.

For a distributed system we assume motes are execut-ing and evaluating programs asynchronously with respect toeach other and to the calculation of the system-wide objec-tive (i.e. there is no global synchronisation). Frequent peri-odic fitness scores are supplied to the motes and summed lo-cally over the execution time of 8 seconds per program. Ini-tially, negative fitness feedback was supplied to both moteswhen Mote B’s LED status did not match the desired pat-tern and positive feedback when it did match. Interestingly,motes evolved a “denial of service” strategy (i.e congestingthe channel and maximising packet collisions) to reduce thenumber of received negative fitness packets. This convergedto a fitness of zero where no positive or negative feedbackwas received by the mote. Several approaches were testedto address this issue. We determined that a combinationof frequent, small negative feedback generated locally withpositive larger global feedback allowed the motes to convergeto the optimal solution. This combination does not“reward”motes that congest the channel as only positive fitness valuesare transmitted wirelessly.

20

Program 3 (a) and Program 3 (b) are the optimalprograms for Mote A and Mote B respectively to the“PacketForwarder-RfmToLeds” objective.

radio_tx_int(radio_rx_int()) set_leds(radio_rx_int())end end

Program 3a “PacketForwarder” Program 3b “RfmToLeds”

The selection of the fitness function for a given objectiveis critical to achieving an acceptable solution. We balancethe local negative and global positive scores such that theinert behaviour of Mote B leaving its LEDs unchanged forthe entire evaluation period, would yield an overall fitness ofzero. The evaluation function can be expressed as follows:

• subtract 80 from the mote fitness for each evaluation(over 8 seconds)

• add 80 to the mote fitness for each correct LED se-quence on Mote B matching the value sent from thebase to Mote A

The calculated optimal score is (8×80−80×1) = 560. Thepopulation structure was set to 5 elites, 3 highly ranked, 10children, 2 randoms and 1 other with a mutation rate of5%. The motes evolve and the resulting evolution of eliteand pool fitness shown in Figure 8.

Interestingly, the elite fitness achieved was 640 which isbetter than the expected maximum fitness. This “cheating”appears to result from extra instructions during the evalua-tion prolonging the mote execution just long enough to occa-sionally receive 9 positive fitness values. For reasons not yetfully understood, this strategy has an average fitness lessthan Program 3(a), so the pool fitness declines each timethis strategy becomes elite and subsequently dominates thepopulation. This causes the perpetual oscillation of bothelite and pool fitnesses (lagging in phase) as the pool andelite toggle between the 2 solutions.

0 5 10 15 20 25 30 35 400

200

400

600

800

Generation

Fitn

ess

Sco

re

170 Elite Scores 170 Pool Average 187 Elite Scores 187 Pool Average

Figure 8: “PacketForwarder-RfmToLeds”fitness evo-lution

5. DISCUSSIONEvolving logic post deployment (in situ) has both oppor-

tunities and disadvantages. In this section we discuss someof the lessons learnt from the challenges faced when evolvinglogic in situ across a WSAN.

5.1 The EnvironmentThe evaluation of logic in situ avoids reliance on syn-

thetic (simulated) models of the environment, and there isno better representation of the environment than the envi-ronment itself. Not only does this prevent the evolution of

logic being made brittle by exploiting artefacts in the simu-lation, it also means that the mote does not need to performany simulation at all which is a great benefit for resource-constrained devices. However, a significant disadvantage ofthis approach is that there is only one single shared environ-ment for all programs across all motes. It is impossible toreset the environment back to exactly the same initial condi-tions prior to the evaluation of every program. Hence, eachprogram may leave a “footprint” which may help or hinderother programs and this generates a credit assignment prob-lem which can slow or even prevent convergence. Even moreimportant is whether the environment could be changed ina way that permanently prevents the desired objective to befulfilled. Unlike offline evolution, one cannot go back in timeand trial another optimisation trajectory if the current onehas failed.

5.2 The Performing-Learning ParadoxOffline learnt logic can deliver immediate high perfor-

mance upon deployment, however due to unexpected changesin the environment, may not perform well over the longerterm. Online learning, on the other hand, allows the systemto adapt to unexpected changes, and so it should, in theory,be able to provide better performance than that of offline de-veloped logic under unanticipated environmental conditions.Unfortunately, it requires time to learn how to achieve thedesired performance (i.e. convergence and re-convergencetimes) and there is no guarantee that an acceptable solutionwill be found within an acceptable timeframe. Furthermore,learning implies that the system is not performing optimally.

Paradoxically, to achieve better performance, logic otherthan the current best performing must be executed whichgenerally performs worse. To an external observer, onewould see this as variation of performance over time. Im-plicitly, performance is dependant on the fitness as mea-sured within the observer’s critiquing period. Ideally then,the best current performing logic would be executed withinthis critiquing period, and learning (evaluating and execut-ing other programs) would occur outside of this period. Ifperformance is calculated continuously however, the aver-age observed performance will match the pool fitness. Here,optimising the average (pool) performance is desirable.

5.3 IDGP ConfigurationThe configuration of IDGP framework parameters impacts

convergence times and performance. Design considerations,such as using homogeneous or heterogeneous nodes, are im-portant to how the system will achieve its purpose. The sizeand composition of the population directly affects both con-vergence times and performance, which require configura-tions that maximise performance within the size constraintsimposed by the memory limitations of the device. For someobjective functions, all acceptable solutions may be largerthan the maximum program length LMax, in which case weeither increase LMax by reducing the number of programsNp, we increase the physical memory size, or we adopt a hu-man coded solution which can afford to be up to Np timesthe size of an IDGP program.

IDGP programs should be as syntactically rich as pos-sible in order to maximise the efficiency and performanceof the solution programs, while maximising the number ofprograms within the memory constraints in order to achievetimely convergence. Determining the optimal instruction set

21

and program size for a particular objective remains an openissue for future investigation.

In this paper we used a small instruction set of 3 instruc-tions to demonstrate IDGP for simple well-defined objec-tives. We have recently conducted experiments with an in-struction set of 13 instructions to solve the“PacketForwarder-RfmToLeds” objective which demonstrated similar conver-gence times.

The “No Free Lunch Theorem” [20] states that no solutionfinding mechanism is better than others over all problems.IDGP is well suited to complex WSAN problems with nowell known solution, or where adaptivity of logic is likely tobe necessary. IDGP is not a solution for every WSAN prob-lem, and we recommend the designer to estimate whetheran alternative approach, such as human-crafted logic, couldproduce acceptable logic more efficiently.

5.4 Challenges of Distributed EvolutionThere are many challenges with evolving logic on devices

simultaneously across a network. Perhaps the most impor-tant aspect is setting the fitness objective appropriately andsupplying fitness information to the motes. Section 4.6 high-lights the importance of this by premature convergence tothe“denial of service” strategy. We modified the fitness feed-back to a combination of strong positive reinforcement withsmall, but frequent, negative reinforcement. We recommendthis approach in systems where the framework and evolv-ing code share the same communications medium. Unfortu-nately identifying such causes of premature convergence isnon-trivial.

In systems where motes maximise a purely local fitness,evolution is reasonably straight forward since it is essentiallymany motes evolving independently of each other. Motescan still benefit from sharing optional information (such asprograms), and will often converge on the same solution fora given objective. With a high selective pressure acting onlocal environmental differences, speciation of logic will occurand motes will essentially revert to individual evolution asdemonstrated in section 4.4. However, when the fitness ob-jective requires cooperation or feedback from other motes,the potential for information to be lost or misleading thenarises. For example, in section 4.6, motes evaluate their pro-grams asynchronously which can reward “leech programs”which rely on the prior program to perform well.

The random order of programs (synchronised or not) causesa “macro-crossover” of programs, where various combina-tions of programs from different subpopulations are executedsimultaneously. For example, Mote A may be executing itselite program while Mote B is executing a random program.For distributed evolution the probability of well performingprograms to execute simultaneously needs to be significantlyhigh. Otherwise in the absence of a combination that per-forms better than the elites on each mote, the motes would“forget” the elite and devolve. Thus the biasing of the popu-lation composition is important to convergence and the sub-ject of future research.

5.5 Constrained EvolutionIn our solutions we were able to ensure that the system

does not enter a state where it is impossible to evolve toan acceptable solution. Since the actuation, LEDs and ra-dio, cannot permanently alter the environment (includingitself), an acceptable solution for these problems always ex-

ists and therefore the GA can always potentially find oneof these solutions. In more general applications it will beimportant to design node hardware and software architec-tures such that IDGP programs are executed and evaluatedin isolation from core functionality to ensure that evolvedprograms cannot hog critical resources or cause undesirableactuations. This raises the issue of which problems are suit-able targets for IDGP implementation as a topic for furtherresearch. In some cases it may be possible to impose con-straints during evolution to ensure solutions stay within safebehaviour. However setting constraints inherently reducesthe solution space that can be searched meaning that bettersolutions may exist that cannot be evaluated.

6. CONCLUSIONWe have presented In situ Distributed Genetic Program-

ming (IDGP) as a framework for the automated online cre-ation of logic on WSAN nodes by using a light weight geneticprogramming engine suitable for resource constrained sen-sor node devices. The in situ evolution of logic was demon-strated on a network of physical sensor nodes which wereable to evolve their logic in order to adapt to local conditionsas well as unanticipated environmental changes. Empiricalresults indicate that sharing evolved logic with neighbouringnodes typically provided faster convergence initially, how-ever potentially converging prematurely.

In conclusion, the IDGP framework provides a method fortasking WSANs with a high level system objective. Throughcontinuous online evolution, nodes learn to sense, communi-cate and actuate in order to optimise for a specified objec-tive.

7. REFERENCES[1] I. Akyildiz and I. Kasimoglu. Wireless sensor and

actor networks: Research challenges. In Ad HocNetworks Journal, volume 2, no. 4, pages 351–367.2004.

[2] P. A. N. Bosman and H. L. Poutre. Learning andanticipation in online dynamic optimization withevolutionary algorithms: The stochastic case. In Proc9th Ann Conf on Genetic and EvolutionaryComputation, pages 1165–1172, 2007.

[3] D. Floreano and F. Mondada. Automatic creation ofan autonomous agent: Genetic evolution of aneural-network driven robot. In Proc 3rd Int Conf onSimulation of Adaptive Behavior, 1994.

[4] S. Hettiarachchi. Distributed online evolution forswarm robotics. Presented at Autonomous Agents andMulti Agent Systems, Doctoral Mentoring Program,2006.

[5] S. Hettiarachchi, W. Spears, W. Kerr, D. Zarzhitsky,and D. Green. Distributed agent evolution withdynamic adaptation to local unexpected scenarios. InProc. Second GSFC/IEEE Workshop on RadicalAgent Concepts. Springer-Verlag, 2006.

[6] J. H. Holland. Escaping brittleness: The possibilitiesof general-purpose learning algorithms applied toparallel rule-based systems. In T. M. M. RyszardS. Michalski, Jaime G. Carbonell, editor, Machinelearning: An artificial intelligence approach,volume II, pages 593–623. Morgan Kaufmann, 1986.

22

[7] J. H. Holland. Adaptation in Natural and ArtificialSystems. MIT Press, Cambridge, MA, USA, 1992.

[8] D. M. Johnson, A. Teredesai, and R. T. Saltarelli.Genetic programming in wireless sensor networks. In8th European Conf onGenetic Programming, LNCS3447, pages 96–107, 2005.

[9] J. Koza. Genetic Programming: On the Programmingof Computers by Means of Natural Selection. MITPress, 1992.

[10] W. Martin, J. Lienig, and J. Cohoon. Populationstructures: island (migration) models: evolutionaryalgorithms based on punctuated equilibria. In Back,Fogel, and Michalewicz, editors, EvolutionaryComputation, chapter 6.3:1-6.3:16. Institute of PhysicsPublishing, Oxford University Press, New York, 1997.

[11] G. Nan and M. Li. Evolutionary based approaches inwireless sensor networks: A survey. In FourthInternational Conference on Natural Computation,volume 5, pages 217–222, 2008.

[12] H. N. Pham, D. Pediaditakis, and A. Boulis. Fromsimulation to real deployments in wsn and back. InProc. IEEE Int. Symposium on a World of Wireless,Mobile and Multimedia Networks, pages 1–6, June2007.

[13] J. Pinter. Global optimization: Software, testproblems, and applications. In P. Pardalos andH. Romeijn, editors, Handbook of Global Optimization,volume 2, chapter 15, pages 515–569. KluwerAcademic Publishers, Dordrecht Boston London, 2002.

[14] R. Poli, W. B. Langdon, and N. F. McPhee. A fieldguide to genetic programming. Published viahttp://lulu.com and freely available athttp://www.gp-field-guide.org.uk, 2008.

[15] Q. Qiu, Q. Wu, D. Burns, and D. Holzhauer. Lifetimeaware resource management for sensor network usingdistributed genetic algorithm. In Proc. of the Int.symposium on Low power electronics and design,pages 191–196. New York, NY, USA, 2006.

[16] P. Sikka, P. Corke, L. Overs, P. Valencia, andT. Wark. Fleck - a platform for real-world outdoorsensor networks. In 3rd International Conference onIntelligent Sensors, Sensor Networks and Information,pages 709–714. 2007.

[17] A. Silva, A. Neves, and E. Costa. Evolving controllersfor autonomous agents using genetically programmednetworks. In Proc. 2nd European Workshop onGenetic Programming, LNCS 1598, pages 255–269.Springer-Verlag, 1999.

[18] T. Wark, P. Corke, P. Sikka, L. Klingbeil, Y. Guo,C. Crossman, P. Valencia, D. Swain, andG. Bishop-Hurley. Transforming agriculture throughpervasive wireless sensor networks. PervasiveComputing, IEEE, 6(2):50–57, April-June 2007.

[19] T. Weise and K. Geihs. Genetic programmingtechniques for sensor networks. In P. J. Marron,editor, Proceedings of 5. GI/ITG KuVS FachgesprachDrahtlose Sensornetze, pages 21–25. University ofStuttgart, Stuttgart, Germany, 2006.

[20] D. H. Wolpert and W. G. Macready. No free lunchtheorems for search. In Technical report TR-95-02-010.Santa Fe Institute, Sante Fe, NM, USA, 1995.

23