An NK-like model for complexity - UFPR

J Evol EconDOI 10.1007/s00191-013-0334-4

REGULAR ARTICLE

An NK-like model for complexity

Marco Valente

© Springer-Verlag Berlin Heidelberg 2013

Abstract The level and nature of complexity is widely regarded as an importantdeterminant of a number of economic, technological and organizational phenomena.A popular modeling tool for the representation of complexity in economics and orga-nizational sciences is the NK model that represents the complexity stemming fromthe interactions among the elements of a system. This paper proposes an enhancedmodel for complexity that, though maintaining the core design (and properties) ofthe NK model, provides a more intuitive and richer representation of complexity,extending its applications and deepening the understanding of its effects on economicsystems. The proposed pseudo-NK (pNK) model is defined on real-valued variables,as opposed to the binary variables required by NK, so as to allow for richer and moreintuitive definitions of distance and search strategies. It also admits as a source ofcomplexity not only the number of interactions, as in NK, but also their intensity,opening a novel way to express and measure the level of complexity. Finally, insteadof relying on statistical properties of a large dataset of random values, pNK is definedas a deterministic function, far simpler to implement, to interpret and to calibrate forspecific requirements. The paper replicates known results and presents original ones;in both cases, the proposed model proves a powerful tool for the investigation of therole of complexity, particularly in agent-based models.

This paper was presented to the International Schumpeter Society Conference, 2010,Aalborg (DK). I wish to thank Tommaso Ciarli for valuable comments and suggestionsduring the development of the model. Luigi Marengo, Lee Altenberg and an anonymousreferee provided encouraging comments and corrections to earlier versions. Anyremaining error or imprecision is, obviously, my sole responsibility.

M. Valente (�)Universita dell’Aquila, L’Aquila, Italye-mail: [email protected]

M. ValenteLEM, Pisa, Italy

M. Valente

Keywords NK model · Complexity · Simulation models

JEL Classifications C63 · D83 · O32 · O33

1 Introduction

Borrowing ideas developed for a given purpose and applying them to a completelydifferent domain is a widely used method to advance knowledge and solving prob-lems, up to the point that, one may suggest, copying and mixing different ideasis the main driver of human creativity and of scientific advancements. The use ofmetaphor and concepts developed in different domains, however, runs the risk of mis-adaptation. Though the core of the original idea can be useful in the new domain,it is frequently necessary to make some adaptations, removing and adding elementsas necessary for its novel application. Witness, for example, the pioneering attemptsto fly by means of heavier than air flying machines. For centuries inventors attackedthe problem following the strategy of imitating the solution observed in most birds,i.e. moving wings. As long as the imitation was too close (using wings for both sup-porting and propelling), the degree of success was very limited. Only when the coreidea was radically revised, separating the function of propelling from the function ofsupporting, did the imitation actually succeed.

Concerning the study of complexity, biologists have developed the NK model(Kauffman and Levin 1987; Kauffman 1993) representing, in very stylized andelegant way, the effects of fitness improving mutations in a context in which inter-dependency makes rough landscapes. NK has been particularly successful becauseit provides a simple tool for generating an abstract representation of a problem (theprobability of survival of a species) that, contrary to other modelization techniques,could be easily tuned to make the search for a solution harder or simpler. The rep-resentation of the problem relies on the core of complexity, that is, interdependency,reproduced by means of a large number of uniformly distributed random numbers.The result is that NK fitness landscapes show statistical properties depending on theinteraction levels only, which are very robust to the choice of the set of random num-bers used.1 From a biological perspective, NK is very useful because it is well knownthat genes’ effects on phenotypes are strongly influenced by their interactions. NKposes the interaction explicitly at the center of the analysis, and carefully avoids mak-ing any further assumptions, being able to derive interesting results concerning theexpected characteristics of a species, its pattern of evolution, and even the reasons forthe spontaneous emergence of modularity.2

However, biologists are not the only scientists interested in the modeling of com-plexity by means of interactions. Economists and scholars in organization scienceshave had a long-time interest in the study of the effects of interactions (Simon 1969).Therefore, many researchers from these fields have adopted NK as their instrument

1See, e.g., Weinberger (1991), Durret and Limic (2003), Skellett et al. (2005), Kaul and Jacobson (2006).2See, e.g., Wagner and Altenberg (1996), Altenberg (1995).

An NK-like model for complexity

of choice to represent and to study properties of organizational structures, techno-logical innovation, etc. Originally devised as a metaphor for how nature deals withcomplex problems, NK-inspired models have been used to study artificial systems,organizations, technological developments, industrial dynamics, and much more. Insuch applications, NK typically represents a complex problem, where the perfor-mance depends on the interactions among several components. Simulated agents (saya firm or a generic problem solver) are engaged in exploring the problem space onthe base of local and myopic information. The researchers are interested in assess-ing the results as a function of the complexity of the task and the limited capacitiesof the agents. Simulation modelers in economics and management have been eageradopters of variations of NK models producing a sizable number works.3

This work is based on the conviction that, however successful, the results fromcomplexity studies in economics and organizational sciences have been limited bya mis-adaptation (or, better, insufficient adaptation) of NK in its passage from biol-ogy to social sciences. As an example of the limitation imposed by NK, consider theproblems concerning the search strategy. The original NK model relies on randommutations of one single dimension replicating the equivalent of random mutationin natural systems. Human individuals or organizations, conversely, are likely toinclude at least a bit of intentional, purpose-driven behavior, however limited byinformational constraints. For this reason, many scholars proposed modifications ofthe baseline NK model to include different search strategies. However, in order todesign appropriate intentional strategies for search it would be useful, to the modeler,to know where the optimum is located, and possibly also to have the opportunity tocontrol some properties of the problem space, such as the specific location and natureof local optima. However, NK has been designed as a metaphor for natural systems,which are passively observed and not controlled, and therefore these possibilities,useless from a biological perspective, are precluded.

The increasing popularity of NK made ever more evident its limitations, andseveral contributions propose variations altering some aspects of its mechanisms(Li et al. 2006). Along these lines, the goal of the present paper is to propose a radi-cally alternative implementation of the core features of NK in such a way as to makethe model more flexible and adaptive for the novel applications to which it is increas-ingly put at work. The major changes of our proposal consist in the replacement ofthe binary variables required in NK with more general (and familiar) real-valued vari-ables, and the shifting from a stochastic definition of the landscape to a deterministicfunctional one. Besides being able to inherit the same properties of the original NKmodel, our proposal allows us also to distinguish the structure of interactions (whichcomponent interacts with other components) from the intensity of interactions (howstrong is a given interaction).

In the following, we present the new model by hinting at the improvements itcan bring to the study of complexity, particularly when purpose-driven agents areinvolved. During the presentation, we alternate examples of applications of the new

3See, for example, Levinthal (1997), Frenken et al. (1999), Kauffman et al. (2000), Rivkin (2000), Rivkinand Siggelkow (2002), Lenox et al. (2006), Frenken (2006).

M. Valente

model to replicate known results generated with NK with original exercises permittedby the new model. The next section discusses the core elements and properties ofNK, highlighting its shortcomings, particularly for application in fields different frombiology. After that, we describe the new model, called pseudo-NK model (pNK),showing how it is able to replicate all the relevant properties of the original NK andremoving, or relaxing, its major shortcomings. The third section presents a seriesof results produced by the simplest pNK version, based on two dimensions only,and exploiting only the intensity of interdependency, the novel feature introduced bypNK. The last section discusses the properties of a generalised pNK defined over Ndimensions. A final section draws the conclusions.

2 Pseudo-NK model for complexity

An NK model can be considered as composed by two distinct components: a problemspecification represented by a space of potential solutions to an implicit problem, anda search algorithm scanning this space hunting for better and better solutions. Theproblem space is represented by all the binary strings of length N, each associatedwith a fitness value, that is, the pay-off for that solution. The search algorithm con-sists in a routine generating a pattern, that is, a sequence of solutions starting froma randomly chosen initial solution, or point in the N-dimensional binary space. Thesearch routine defines the way in which we move from any one point to the next,that is, how to generate a new string from an existing one. For example, the typicalroutine, originally proposed, consists in choosing randomly one of the N elementsof the current string and flipping (“mutate”) its value, from 0 to 1 or vice-versa, sothat the new string will differ from the old one for only one bit. If the fitness of thenew string is higher than that of the previous one, the new string is adopted, mak-ing a “step” in the pattern of exploration of the space. Otherwise, if the fitness ofthe mutated fitness is lower than that of the present one, the mutation is rejected andthe search will continue from the same string. The repeated application of the searchalgorithm generates a path across the space of solutions. The path will be made byfitness-increasing steps, because of its construction, and terminates when it reachesa string from which all accessible strings have lower fitness, and hence there are nofurther possible steps. These strings are referred to as “peaks”, trapping any searchbased on hill-climbing strategies.

Two aspects make NK particularly attractive. First, it is possible to determine howcomplex should be the space of solutions, or fitness landscape. The effect of com-plexity is represented by the probability to encountering the highest fitness pointduring a search pattern. Hence “simple” means that it is very likely to find this point,while “difficult” means that many searches will stop at lower fitness peaks. In NK,it is possible to tune the degree of complexity (i.e. the number of sub-optimal localpeaks) by setting the number of interactions among the variables of the space, indi-cated by K. Landscapes with no or few interactions represent simple problems, i.e.contain few local peaks. In the simplest possible case (K = 0), there exists onlyone peak, so that all searches starting from any initial point will lead to this globaloptimum. Increasing K generates “harder” problems, represented by landscapes

An NK-like model for complexity

with a larger number of local peaks and decreasing chances of reaching the globaloptimum.

The second aspect of NK is the representation of the search algorithm. The orig-inal proposal, coming from a biological metaphor, assumes the search strategy to belocal and myopic. Local because the search implies the impossibility to observe thespace beyond the narrow neighborhood of the current string; myopic because it pre-vents collecting past information or predict future events, focusing on the immediategoal of just improving the current condition. In other applications, NK models areendowed with search strategies on longer distances, for example admitting the possi-bility for long jumps (Levinthal 1997), search within modules of a given dimension(Frenken et al. 1999) and other methods (Auerswald et al. 2000; Kauffman et al.2000).

These two elements, complexity due to interactions and algorithmic search strat-egy, provide a stylized representation of a complex task faced by a would-be solverwith limited capacity and information. Hence, the model allows us to study theeffects of varying degrees of complexity upon actors with different problem solvingapproaches.

In its original context, NK has been used as a pure representation of complex-ity, studying, for example, the expected number, location and average fitness of localpeaks as a function of K, playing down the role of the search strategy, since thisrepresents merely the blind action of nature. However, in other fields, such as man-agement, organization theory, economics of technical change, etc., the popularity ofNK stems from the large number of extensions to its basic implementation, particu-larly concerning the opportunities of search performed by purpose-driven intelligentactors. Hence, in these fields, NK has become a sort of sand-box where it is possi-ble to test different problem solving approaches facing tasks of varying complexity.For example, Levinthal (1997) introduces the possibility for agents (representingfirms) to perform “long jumps” besides local search and imitation of successfulcompetitors. Rivkin and Siggelkow (2002) propose a model where decisions are dis-tributed between a “CEO” concerned with overall fitness, and managers interestedin “parochial” interests whose success depends on a portion of the landscape undertheir control. Lenox et al. (2006) use a version of NK to represent a technologicalspace explored by competing firms.

In general, NK provides a working implementation for the intuition that sim-ple problems can be successfully solved, while increasingly complex ones generatesub-optimal solutions with degrading expected payoff’s. Using NK, researchers areable to investigate likely consequences expected under controlled conditions for theenvironmental complexity of the problem and solving strategy adopted. The verysimplicity of the model invites users of NK to devise variations of the baseline versionto implement customized search strategies for specific types of problem solvers.

In this work, we claim that NK, however effective in offering a representationof complexity and its consequences, contains a number of unnecessary biases andlimitations when used to represent complexity in an economic context.

A first problem arises from the necessity to use of binary variables, representingpresence/absence of a given feature. This aspect generates confusion because the onlypossible measure of the distance between two points is the Hamming distance, that

M. Valente

is, the number of variables with different values between two points. This conceptof distance is very different from the familiar Euclidean distance applicable to real-valued spaces. In a binary space, a “step” in a given direction can be only one unitlong, though it can be directed in N different dimensions. Besides making impossiblethe graphical representation of the landscape, the use of binary variables generatesconfusion and misinterpretations. For example, one is entitled to speak of “valleys”and “peaks”, but it would be wrong to assume that they have the three dimensionalshape evoked by those names.

A second problem arises from the use of randomness to generate the landscape, theproperties of which are actually represented only as statistics over a reasonable largenumber of replications. In practice, each landscape will have a different (random)distribution of the relevant points, such as the global maximum or the local peaks,and the distribution cannot be pre-determined nor observed without testing each andevery point of the landscape. This means that the modeler is as ignorant of the fitnesslandscape as the virtual agents exploring it, making their assessment very difficult incase of large landscapes.

A last issue hindering the use of NK models consists in the relative difficulty inits implementation and use. Though the programming of a NK is relatively straight-forward, its structure relies on a combinatorial number of random values, the sheernumber of which is simply impossible to manage even for modest number of dimen-sions. The reason is that a fitness landscape requires the storage of a database madeof N*2K+1 random values, which can easily exceed the capacity of any computereven for low levels of K, in effect making it impossible to build all the points ofa landscape. It is possible to find technical solutions to generate portions of largelandscapes so as to collect sample statistics of their properties,4 but the necessity torely on samples further diminishes the capacity of the modeler to investigate and toexploit the landscape to assess the results.

2.1 Improving on NK

The limitations listed above make NK difficult to use and its results complicatedto interpret when it is applied to investigate complexity in artificial systems. Wepropose here an alternative model replicating (and extending) the features of NK thatare relevant to economic and organizational scientists, without the limitations of theoriginal NK highlighted above. The new model, which we call pseudo-NK (pNK),enjoys the following properties:

• Functional representation of fitness: the fitness function is defined as a deter-ministic function of a multidimensional vector. Therefore, it can be implementedas a routine, without requiring large data sets of random numbers, simplifyingthe coding of very fast implementation for even large landscapes.

• Multidimensional real-valued landscape: the landscape of pNK is composedof a subset of the n-dimensional real-valued space: �x = {x1, x2, ..., xN} ∈ �N ,

4Altenberg (1997) is credited as the first to propose a solution to this problem.

An NK-like model for complexity

with fitness represented by a real-valued function f (�x). Both domain and co-domain can be freely determined by the modeller.

• User determined maximum and landscape’s overall shape: f (�x) has a max-imum5 in a point �x∗ (such that f (�x∗) ≥ f (�x), �x ∈ �) determined by the user.The shape of the landscape is a well-behaved function, so that the modeller canevaluate analytically the properties of the landscape.

• User determined interdependency: for any given couple of dimensions i andj, the user can set a varying degree of interdependence ai,j , ranging from fullindependence to maximum interdependence. Intermediate and varied levels ofinterdependency allow us, for example, to define landscapes where a dimensiondepends strongly on some dimensions and weakly on others.6

Besides these improvements, the definition of interdependency used in pNKremains the same as that used in the NK model: dimension xi is dependent on dimen-sion xj if the maximization of the fitness function requires a different value for xi fordifferent values of xj . In a different terminology, we can say that there exists depen-dency of xi on xj if, for at least some value of the other N-2 variables, the derivativeof f (�x) in respect to xi changes signs for different values of xj . It is worth notingthat, in NK (and pNK, too), interdependence does not simply imply that modifica-tions xj affects how xi impact on the fitness function f (�x). In fact, it may be possiblethat such influence is strictly monotonic, that is, for different xj we observe that xi

has different impacts on the overall fitness value, but they always have the same sign.Though in this case we do have interdependency, in a sense, this is not affecting thepossibility of identifying the optimal value for xi independently from xj . Only when∂f (�x)∂xi

changes sign for different values of xj , is the fitness-optimizing value of xi afunction of xj . In this case, a hill-climbing strategy based on varying xi only is liableto be stuck in a local peak determined by the specific value of xj .

In the following, we describe a possible implementation of a fitness function forpNK providing the properties listed above.

2.2 pNK fitness function

pNK, as NK, consists of a fitness function defined on a set of N variables anda search algorithm. The fitness function proposed here7 for pNK borrows heavilyfrom the NK implementation, with three notable differences. First, it considers real-valued variables (instead of binary); second, it is a deterministic function, ratherthan stochastic; third, it allows for different degrees of interdependence, instead ofpresence/absence.

5Actually, it is possible to extend the model to admit multiple global maxima, though we will ignore thispossibility.6A different version of the standard NK models, the so-called generalized NK, also allows user determinedinterdependency (Altenberg 1995, 1997).7Here we adopt a specific functional form providing the properties required to pNK; however, there is awhole class of functions that may be used, depending on specific requirements.

M. Valente

The overall fitness value of a point of the landscape domain (i.e. a point of �N ) is,as in NK, the average of N fitness contributions for each of the variables:

f (�x) =∑N

i=1 φi (�x)

N(1)

where φi(...) is the fitness contribution function for dimension i. While in NK φi is arandom value, in pNK this is a deterministic function defined as:

φi(�x) = f Max

(1 + |xi − μi(�x)|) (2)

where f Max is a user-determined parameter indicating the maximum of the function(typically 1). Equation 2 essentially states that φi is a decreasing function of thedistance between the variable’s value and another function μi(�x), defined as:

μi(�x) = ci +N∑

j=1

ai,j xj (3)

The value μi defines a sort of “target” for dimension i that, when hit by xi , gener-ates the maximum level of contribution of the variable to the overall fitness function.However, the value of xi maximizing φi may not be the most desirable, concerningthe overall fitness value. In fact, xi influences also all contributions φj for the vari-ables where aj,i �= 0. Therefore, it is well possible that moving xi to maximize φi

actually decreases the overall fitness value because of the deterioration generated inother fitness contributions φj ’s where aj,i �= 0.

The fitness function here8 proposed allows for ample flexibility. In particular, it ispossible to determine features of the landscape that are not available in NK models:

• Set maximum fitness. Simply setting f Max determines the maximum value ofthe fitness function.

• Set the global optimum. For any dimension i, it is possible to compute ci suchthat the maximum fitness is obtained at a desired point �x∗ = {

x∗1 , x∗

2 , ..., x∗N

}.

To ensure that this point is the global maximum, we need to set ci = x∗i −∑

j �=i ai,j x∗j .

• Set interdependencies. Varying the values of ai,j it is possible to make more orless relevant the interdependency between two dimensions.

The first two properties are useful to exploit pNK in a context where the mod-eller is interested in determining a specific maximum fitness value at a specific pointof the landscape. However, the last property is far more interesting, since it allowspNK to model complexity in a more detailed way in respect of NK. In fact, NKimplements interdependency as a presence/absence property: if dimension j affects

8The proposed model can easily be modified changing the functional form for Eqs. 2 and 3. The sameproperties of the model discussed here are maintained provided that μi(�x) is a positive function of ai,j xj

and that φi(�x) is an inverse function of |xi − μi(�x)|. Using different functional forms for these elementswill change the density of local peaks and the overall shape of the landscape, though maintaining the sameproperties discussed here.

An NK-like model for complexity

i then the contribution of i changes when j changes, and the two contributions aretotally unrelated. Conversely, pNK allows to tune the level of interdependency: theeffect of j on i can be null, weak or maximum, depending on a single parameter.Furthermore, since pNK is deterministic and expressed in a functional form, it is pos-sible to represent and even visualize (when N = 2) the effect of sliding levels ofinterdependency.

The explicit distinction between complexity stemming from interactions’ struc-tures and from interactions’ intensities is theoretically relevant, as noted bySimon (1969) in his analysis of nearly-decomposable systems, suggesting that,though every element of a system is influenced by any other, some of the interac-tions are stronger than others. This distinction makes it possible to manage apparentlyintractable problems, trading off optimality against the reduced costs of dealing withsimplified systems.

In NK, since interactions have only one level, this distinction is blurred, and theintensity of interactions can only be crudely regulated by increasing or decreas-ing the number of connections, that is, K. However, in many cases, researchers areinterested in representing complexity explicitly, and exclusively, as stemming frominteraction intensities. The next section shows how many of the applications of NKto the organization literature can be reproduced by using a space made of N = 2dimensions only, and using instead the level of the only existing interaction. Theintuitiveness of the bi-dimensional representation, due to the possibility of visualizethe landscape, allows a far greater insight than the original exercises implementedin NK.

Before reporting on these exercises, the next paragraph defines the algorithm ofsearch strategies in a pNK context, as required in dealing with landscapes made ofreal-valued variables.

2.3 Search strategy on pNK

The baseline research strategy in the NK model lies in the so called one-bit mutation.This consists in generating a new point starting from an existing one by modifyinga single bit of the binary string representing the point. The new point is accepted asa step if the fitness associated with the new point is higher than that of the existingone; otherwise the step is rejected.

On a real-valued fitness landscape, the one-bit mutation strategy can be expressedby the following algorithm applied to a generic point in order to generate asubsequent point of a pattern.

1. Choose randomly one dimension among the N available.2. Choose randomly one direction: + or −.3. Change by � the value of the chosen dimension in the chosen direction.4. If the change leads to a point with higher fitness, move to the new point.5. If the change leads to a point with lower fitness, remain at the same point.6. Continue from 1.

A path of a landscape is therefore made up of a sequence of steps generatedby repeating applications of the above algorithm. The routine is repeated until no

M. Valente

change of � on any of the variables of the landscape is able to produce a fitnessincrement. The points on which this happens constitute the (local or global) peaks ofthe landscape.

In the economic and organizational literature, there are many alternative proposalsfor search algorithms designed to represent specific behaviors. Each can easily beadapted to work on a real-valued landscape, actually providing far more flexibility.This is because in pNK it is possible to define three distinct features of a step: a)its “angle” (the dimension(s) chosen); b) its direction (increase or decrease in thecurrent value); c) its length, set by �. As a consequence, it is possible, for example,to distinguish two different types of “long jumps” (Levinthal 1997): one made ofsmall modifications of many dimensions, and another made of a large change in asingle-dimension.

For our purposes, we will consider � as a constant parameter represented by asmall value. This value is of only minor importance since its only role is to discretizethe continuous real-valued space. The discretization permits us to ignore the smalldifferences of fitness between two very close points. This choice is due to the neces-sity, in this first version, to maintain the myopic nature of search strategies in complexlandscapes as in the original version of NK. Limiting � to be small and constant, infact, implies forcing the search strategy to consider only a small patch of the land-scape surrounding the agent’s position. However, it is easy to devise extensions ofpNK in which agents searching a landscape may attempt both steps in various dimen-sions but also modify the length of steps �, for example, extending � when trappedin a local optimum.

3 Complexity from intensity of interdependency

The interdependency among components of a system generates complexity by meansof its structure (the network of interactions) and their intensities. In this section,we study the properties of pNK in relation to the intensity of interdependency only,neglecting the role of the structure. For this purpose, we consider two dimensionsonly (N = 2), so that the structure of interdependency, trivially made up of the onlypossible connection between the two existing dimensions, cannot play any role. Wewill show that, even using this minimal landscape, pNK is able to replicate (and,actually, enrich) a large number of results originally produced with NK. Limiting toa space with small dimensionality, we will be able to produce the same results in afar simpler and more intuitive manner.

We will consider the implementation of pNK as a landscape containing a singleglobal peak located at a pre-determined point. We will consider only symmetric land-scapes, where the two interdependency relations between the variables are identical inabsolute levels and opposite in values. This is obtained by considering a1,2 = −a2,1and 0 ≤ |ai,j | ≤ 1. Firstly, we show the topology of the space represented by pNK,exploiting the possibility to visualize the 3D graph made of two dimensions for theplane and the third for the fitness corresponding to each point. We will then moveto replicate, to extend and to comment on various results obtained by consideringmyopic explorations of this space.

An NK-like model for complexity

Example of fitness landscape

98 98.5

99 99.5

100 100.5

101 101.5

102X1 98

98.5 99

99.5 100

100.5 101

101.5 102

X2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 1 Fitness landscape for N = 2 with symmetric and relatively strong interdependency. Global opti-mum set at (100,100) with maximum fitness set to 1, and |ai,j | = 0.7. A hill-climbing strategy based onone-dimensional mutation corresponds to moves parallel to one of the axes. If, as in the figure, the ridgesof the landscape are diagonal to the axes, they represent areas of local optima for one-dimensional searchstrategies. In fact, any vertical or diagonal step would force a “step down” from the ridge resulting in afall of fitness. Only if the ridges are parallel to the axes (ai,j | = 0) can a one-dimensional search strategywalk on the ridges, and it will always reach the global maximum

3.1 Visualisation of 2-D pNK fitness landscapes

Figure 1 shows a fitness landscape built on two dimensions. The graph reports alandscape with the global optimum set at x∗ = (100, 100), maximum fitness tof Max = 1, and a high, but not maximum, level of symmetric interdependency|ai,j | = 0.7.

The landscape formed by the chosen fitness function9 is composed of a singlepeaked surface with monotonically decreasing fitness for points increasingly distantfrom the optimum, though points with the same distance from the optimum mayhave different fitness values. The surface is composed by four “ridges”, separated by“valleys”, leading to the global optimum. The two ridges are orthogonal because ofthe assumption of symmetry of interdependency, i.e. ai,j = −aj,i . The central aspectof a bi-dimensional pNK is, obviously, its degree of interdependency. In graphicalterms, the interdependency is represented by the angles of the ridges in respect of theaxes, which are controlled by the absolute values of the parameters |ai,j |.

To make a more complete analysis of the effects of interdependency the graphsreported, in Fig. 2 describe five different landscapes generated by pNK for |ai,j | =0, 0.25, 0.5, 0.75, 1. For |ai,j | = 0, the two ridges leading to the global maximum

9Different functional forms can be generated by altering the expression of the fitness contributions φi . Theonly requirement is that the φi is inverted related to the difference |μi − xi |.

M. Valente

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fitness |a|=0

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fitness |a|=0.25

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fitness |a|=0.50

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fitness |a|=0.75

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fitness |a|=1.00

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

Fig. 2 Fitness landscapes for N = 2 and |ai,j | = 0.0; 0.25; 0.50; 0.75; 1.00

are parallel to the axes. For |ai,j | = 1, the ridges run on the diagonals, at 45 degreesin respect of the axes. For intermediate values, they have increasing angles.10

The next paragraph explores the areas of local peaks trapping one-dimensionalsearch strategies for each of the above landscapes.

3.2 Local peaks and interdependency

In contrast to NK, pNK is not only able to replicate the statistical relation betweencomplexity and extension of local peaks, but it is also possible to locate them and toexplain the mechanism trapping the search strategy and hence causing the peak.

The angle of the ridges in respect of the axes is crucial to determine the propertiesof pNK. In fact, any point other than the ridges (the “valleys”) cannot have localpeaks, since there is always a whole area of neighbors closer to the optimum and with

10All the experiments reported in the paper are produced with simulation programs implemented withLaboratory for Simulation Development-LSD (Valente 2008). The LSD platform is available for downloadat www.labsimdev.org. The code for the models and the specific exercises, together for the instructions ontheir use, is available upon request.

An NK-like model for complexity

higher fitness. Hence, either a pattern remains always in a valley, and hence ends upnecessarily in the global optimum, or it reaches one of the ridges, which are thereforethe only areas that needs to be studied.

The one-dimensional search strategy means that the only admissible steps arethose made maintaining constant one variable, that is, moving “horizontally” or “ver-tically”, parallel to one of the axes. If the ridges of the fitness surface run parallelto the axes, than such a strategy will surely bring us to the global optimum. This isbecause the ridges “dominate” the valleys, and therefore are likely to be reached bythe pattern.11 Once a point on a ridge is reached, the strategy is able to walk on thetop of it, eventually reaching the global optimum.

Conversely, for landscapes with diagonal ridges (i.e. forming large angles with theaxes), search patterns are likely to get trapped in local peaks. In effect, most of thepoints on the ridges constitute, for these landscapes, the set of local optima. This isbecause, when the ridges are diagonal, moving parallel to the axes (in the directionpointing towards the global optimum) generates two opposite variations of fitness.First, since the step will bring away from the ridge, the fitness of the new point willtend to be lower than the starting point on top of the ridge. Second, since one directionof the step brings us closer to the optimal point (because of the angle of the ridges),the fitness will tend to increase. The net effect is uncertain in general, and depends onwhich segment of the ridge the pattern has reached, besides their angle in respect tothe axes. Given the functional form chosen, the slope of the ridges is gentler the far-ther away from the global optimum, and therefore, in these areas, the positive effecton fitness by getting closer to the maximum will be weaker. Conversely, segments ofthe ridges near the global optimum have steeper fitness and shallower valleys, andtherefore it is more likely that the fitness loss caused by stepping off the ridge issmaller than the gain in getting closer to the maximum. As a result, patterns reachingthe ridges far away from the optimum are likely to get trapped, while those managingto stay in valleys until close to the maximum will avoid the trap of local peaks.

To visualize the properties of the intensity of interdependency, we consider the(average) final fitness reached by search strategies. For each landscape, we ran about30,000 independent searches, starting from randomly chosen initial points. Eachsearch consists in steps performed by adding or subtracting � to a randomly chosenvariable, and moving to the new point in case the change generates higher fitness.12

For each search, we associate the coordinates of the starting point to the fitnessreached when the search terminates, averaging the values of different searches fromthe same starting point. Points from which searches lead to higher fitness final pointsare shown with brighter colors, while those producing patterns trapped in low fitnessvalues are indicated with darker colors.

11Valleys, trivially, lead always to the global optimum, so we need not consider these areas.12Tests with different values of � showed the irrelevance of the value chosen for this parameter, but forthe level of detail of the graphs. The value used for the simulations is 0.05, so that the portion of the graphsshown (from 98 to 102 on both dimensions) is effectively considered as a square lattice made of 80 unitson each side and composed of 6,400 points. Therefore, the 30,000 runs generate, on average, about fiverandom searches started from each point of the landscape.

M. Valente

The graphs reported in Fig. 3 show these values for different complexity levels, i.e.values of |ai,j |. The graph for |ai,j | = 0 is not reported, since it consists of a uniformplane of value 1, meaning that all searches starting from every point of the landscapeconsistently manage to reach the global maximum. The remaining cases show that,while |ai,j |’s increase, the areas of starting points bringing to the global optimumshrinks. Under the most challenging case (|ai,j | = 1), we obtain a Fuji-like figure:only the searches starting close to the global optimum are able to reach the top spot.In this setting, any initial point far from the global peak generates a pattern leading tofinal fitness values orderly distributed according to the distance from the optimum.The reason is that, on this maximum-complexity landscape, movements parallel tothe axes lead to a point on the ridges with a probability increasing with the distancefrom the global optimum. The fitness values of the ridges also have decreasing fitnessfarther from the global peak, and this is why the final fitness of a search started farfrom the optimum is generally lower.

For intermediate values of complexity, we observe “propeller”-like figures. Forthese cases, the farther from the global peak the search starts, the lower the prob-ability of reaching the global optimum. However, the probability is not distributedsymmetrically on both sides of the ridges. In fact, given that the angle of the ridges ispositive but lower than 45 degrees, points on different sides from the ridges have dif-ferent probabilities of being stuck in local optima. A pattern leading on the “lucky”side of a ridge are more likely to get closer to the global optimum, while points onthe “wrong” side are more likely to get stuck earlier on.

This result confirms the capacity of pNK to replicate the core property of NK:increasing the complexity of the landscape generates lower and lower expected fit-ness resulting from the same myopic search strategy. pNK obtains this result in a far

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.951

Random search |a|=0.25

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.951

Random search |a|=0.50

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.951

Random search |a|=0.75

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.951

Random search |a|=1.00

98 98.5 99 99.5 100 100.5 101 101.5 102 98

98.5

99

99.5

100

100.5

101

101.5

102

Fig. 3 Final fitness produced by random one-dimensional search strategy for landscapes with N = 2 anddifferent values of |ai,j |. The graphs report the average final fitness in correspondence with each startingpoint. The average is computed from an expected number of about five searches from each point

An NK-like model for complexity

simpler setting than that required in NK and provides a more intuitive mechanism.We have used a landscape with the minimal number of dimensions to express inter-dependency and generated complexity by means of the intensity of the interaction.Furthermore, pNK defines precisely the locations leading to global or local max-ima, providing deterministic and logical reasons for the eventual result. Once havingestablished these basic properties, we can use pNK to investigate more elaboratedones.

3.3 Emergence of order

The most attractive feature of NK is its representation of lock-ins of myopic explo-rations on complex landscapes. Smooth, simple landscapes allow all or most ofthe explorations to reach the highest fitness point, while rough, complex land-scapes are likely to trap the exploration in local peaks. Levinthal (1997) interpretsthis feature in terms of expected diversity of organizations engaged in the searchfor efficient strategies. He notes that, after starting with a population of firmsscattered randomly on different points of the landscape (i.e. with different initialconditions), the local (“adaptive”) search rapidly brings a steep reduction of thefirms’ population variety. This is due to the convergence of firms in high-fitnessareas, that is, adopting successful strategies that can easily be reached by “adapta-tion”. Conversely, complex landscapes present a large number of local peaks withrelatively small basins of attraction. Consequently, firms starting from differentareas will end up in different local peaks and we will observe a larger variety ofstrategies.

In the original paper, this result was supported by an experiment showing thenumber of different points reached, on average, by 100 firms over 100 independentlandscapes setting N = 10 and repeating the experiments for K = 0, 1, 5 (Levinthal1997, fig.1, p.940). The graph shows that, through time, the number of points reachedby the population of firms in the three cases considered drops sensibly, with the mostcomplex landscape maintaining the highest diversity (i.e. larger number of differentpoints).

As a test to assess the capacity of the new model, we used pNK to replicate thesame exercise. As expected, we obtain the same results, though with a far richer detailand greater ease of interpretation. Moreover, the more intuitive structure of pNKmanaged to produce greater insight extending the original result. Figure 4 reportson the results produced by pNK using N = 2 dimensions only, and expresses thecomplexity of the landscape by means of the degree of interdependency |ai,j |. Thedata are generated defining 100 different landscapes, each with a different value ofinterdependency between the two variables |ai,j | ranging from null to maximum. Oneach landscape, we performed 30,000 searches starting from randomly chosen pointsand registering the coordinates of the initial and final point of the search, and thefitness level of the latter. The graph reports the variance of one dimension’s finallocation, as a proxy for the variety of the results produced in each search, and theaverage fitness of these final points. The data show that, for landscapes with lowcomplexity, all searches manage to reach the global optimum (fitness 1.0), generatingthe highest level of homogeneity (variance of locations is null). For increasing levels

M. Valente

Fig. 4 Variance of locations and average fitness by search strategies on N = 2 pNK fitness landscapes.Data produced by 30,000 searches for each of 100 landscapes with |ai,j | ranging from 0.0 to 1.0

of interdependency, we find that the final locations are increasingly dispersed, asindicated by the higher and higher levels of variance, confirming the original result.

We can also observe another result deriving from increasing complexity: decreas-ing levels of local peak fitness. That is, sub-optimal points trapping search patternsin rougher landscapes show lower fitness than in the case of smoother ones. Thisis due to the obvious consideration that higher complexity stops searches earlieron “bad” local peaks, while gentler landscapes are more likely to allow search pat-terns either at the top, or closer to it. Although intuitively obvious, such resultscannot be observed in NK because of its statistical properties. It is well knownthat NK landscapes generate higher levels of fitness of global peaks for higher K’s(Kauffman 1989). Therefore, one will observe that a low ranking local peak in a land-scape with high complexity shows a higher fitness value than the global peak of alandscape with low complexity. This property is due to mere statistical reasons moti-vated by the way NK generates fitness values.13 Though innocuous when comparingresults from searches on the landscapes with the same complexity, this aspect of NKcreates serious problems when comparing results from landscapes defined with dif-ferent complexity levels. For example, presenting a model where innovative firms’

13In NK, the fitness is computed out of a sample of random numbers the dimension of which is propor-tional to 2K . Hence, rougher landscapes use a larger sample than smoother ones, increasing the probabilityof finding a higher value.

An NK-like model for complexity

qualities are obtained by fitness levels produced with NK, Lenox et al. (2006) write“One could make a compelling argument [for] this trait of NK [...]. We decided, how-ever, to eliminate this feature of the NK specification by expressing individual firmmarginal cost or quality as a percentage of the global optimal” (pag. 763).

As in the cited work, it is frequently the case that researchers are interested in com-paring the relative fitness produced by search strategies in different landscapes. WhileNK requires an elaborated transformation, necessarily arbitrary, to compare fitnessvalues from landscapes with different complexities, pNK maintains the same over-all distribution of fitness values, allowing an objective, direct comparison. The nextexperiment exploits this feature, comparing two different search strategies acrosslandscapes with different intensities of interdependency.

3.4 Greedy vs. random strategies

In this experiment, we investigate the performance of two different strategies fordealing with complexity. We will show that each of the two strategies is preferable(i.e. provides higher fitness) within a certain level of complexity, providing an insightthat may prove useful to actors dealing with complex tasks.

We continue to use the simplest pNK problem space, made of two dimensionsonly, and consider different scenarios with increasing levels of complexity as repre-sented by the intensity of the interdependency. For each scenario, we compare theaverage result produced by two research strategies, both myopic (only surroundingpoints can be explored) and allowing changes on one single dimension at each step.The first is the standard strategy, testing random points in the immediate neighbor-hood of the currently held point. The second strategy is slightly more elaborate;instead of testing a point chosen randomly (within the permitted range), it tests allpossible one-dimensional changes (four, for the 2 dimensional space), and opts forthe one providing the highest possible fitness increase (if any). Obviously, the twostrategies provide the same step in case only one change is able to provide a fitnessincrement. However, when both dimensions could deliver a fitness increment, thesecond, “greedy”, strategy will consistently choose the step providing the largest fit-ness increment, while the random strategy is expected to do so only half the time. Thequestion we explore is whether the supposedly “smarter” strategy (the one able tochoose always the best step) is actually superior to the random one. Figure 5 answersthis question, showing the fitness provided, on average, by the two strategies for dif-ferent levels of intensity of interdependency |ai,j |, ranging from null to maximumcomplexity.

As expected, for low values of |ai,j |’s (simple landscapes), both strategies managesystematically to reach the global maximum, as indicated by the value of 1 for bothstrategies provided for low values of the interdependency parameter.14

14We do not consider the speed of research of the two research strategies, that is, the number of stepsrequired on average to reach the global optimum. In fact, we may expect that the greedy strategy is fasterthan the random one, requiring a smaller number of steps to reach the eventual peak. In the followingsection, we will discuss the issue of speed of research showing how pNK is able, contrary to NK, toprovide an intuitively appropriate representation of the length of research patterns.

M. Valente

0.7

0.75

0.8

0.85

0.9

0.95

1

0 0.2 0.4 0.6 0.8 1

Av.

fitn

ess

|a|

RandomGreedy

Fig. 5 Average final fitness over 30,000 searches starting from randomly chosen points with random andgreedy strategies on 11 landscapes, with ai,j ’s ranging from 0.0 to 1.0

For the opposite cases, with high levels of complexity (|ai,j | approaching 1), thegreedy strategy manages to obtain higher fitness than the random one. This may beeasily explained since, in these contexts, the best that can be obtained is a relativelyhigh local peak, given that there are very few chances of reaching the global peak bymeans of any hill-climbing strategy based on the adaptation of a single dimensiononly. Consequently, the patterns will generally stop after a few steps, and thereforethe best that can be obtained is reaching the highest among the local peaks aroundthe starting point. Clearly, in these environments, the greedy strategy is the mostappropriate.

The dominance of the greedy strategy, however, is limited to the cases with highcomplexity. Conversely, for moderate levels of complexity, when |ai,j | takes values inthe range from about 0.2 to 0.4, the humbler, random strategy dominates the greedy,supposedly smarter one. The explanation for such an apparent counterintuitive resultcan be found comparing the patterns generated by the two strategies. Landscapeswith moderate complexity have areas of local peaks, but smaller than those with highcomplexity. The results show that rushing to find the best local peak in the immediateneighborhood of the starting point is not the best strategy when the local peaks arefew and distant. Rather, avoiding them altogether may be the best way to carry on,ascending slowly (i.e. with smaller fitness increments per step), but ensuring a longerpath (i.e. with more steps).

An NK-like model for complexity

This phenomenon offers interesting insights. Consider an organization where theactivities of its members are only moderately interdependent, so that a given changeby one member will be positive or negative for the organization, depending on thecurrent activities by one or more members, but these links are not very strong. Imag-ine that each member loyally proposes to the top management, say the CEO, thecourse of actions within his department that ensures the highest improvement to thefirm’s performance. If the CEO is forced to implement only one of the received pro-posals, an apparently rational behavior would be to choose on the basis of the highestexpected performance gain. Our result says that there are cases in which the organi-zation would be better off by choosing an alternative proposal providing a dominatedexpected payoff. These cases are those in which higher performance is obtained atthe (hidden) cost of restricting subsequent exploration. Hence, the correct criterionto choose should be more elaborate than mere expected performance measured inthe near future, but should consider also the effect of today’s choice on tomorrow’soptions.

This may be considered as an example of the phenomenon called “premature con-vergence” by researchers in problem-solving algorithms (Michalewicz 1996). Theseare the cases in which a problem solver focuses on good solutions discovered inearlier steps, preventing the exploration of a larger area potentially containing evenbetter solutions. In organization studies, this phenomenon has already be noted:“adaptive processes, by refining exploitation more rapidly than exploration, are likelyto become effective in the short run but self-destructive in the long run” (March1991). This exercise shows clearly the cost of short-term “greed”, providing a formalcontext for observing the effects of exploitation versus exploration. The prematureconvergence is due to the constraints posed by interdependency starting to bite earlyon during a research strategy, preventing the reaching of high fitness portions ofthe search space. In cases of moderate interdependency, it would be better to notpursue the search for optimality, but to leave some room for (locally) sub-optimaldecisions that, increasing the variety of exploration, enlarge the areas open to futureexplorations. In short, there are cases in which the good is preferable to the best.

Concluding this section, we have shown that it is possible to replicate many resultsoriginally produced with NK by using the simplest pNK implementation, based ontwo dimensions only. We have seen that many (all?) results concerning on the effectsof complexity can be reproduced by neglecting the role of the structure of complexityand focusing only on the role of the intensity of interdependency. Furthermore, thesesame results, besides being produced more efficiently, are more coherent and pro-vide additional insights than was the case with NK. The following section continuesthe tests of pNK, extending the analysis to include discussions on the complex-ity stemming from different interaction structures defined over a high number ofdimensions.

4 Complexity from interdependencies’ structure

In this section, we analyze the role of the structure of interdependencies in generatingcomplexity. The first paragraph defines a basic representation of complexity structure

M. Valente

and discusses the (partly) interchangeable role of complexity structure and intensity.The following paragraph presents some results stemming from the exploitation ofcomplexity structure using pNK, highlighting the greater insights available in respectof the original analysis based on the NK representation.

4.1 Complexity: structure and intensity of interactions

Consider an N dimensional problem space. As in a standard NK model, we knowthat varying the number of interactions (increasing K) we can vary the roughness ofthe resulting landscape, increasing the probability that hill-climbing myopic searchwill end in a local peak. The standard NK model assumes no structure of the inter-dependency among variables, limiting their number and assigning randomly theinterdependency links.15 It is possible, however, to construct an NK model in sucha way as to impose a desired interdependency structure on the problem space. Tocreate a very basic structure, we assign each variable to a group so that each groupcontains the same number of variables. We then set interdependency connectionsamong all the variables in the same group, and no connection between variables fromdifferent groups. Hence, the larger the groups, the larger the number of interactionsand, consequently, the landscape will be rougher. Notice that, in this setting, it ismore convenient to indicate with K the number of variables within groups, ratherthan the number of interactions affecting a variables as in the original NK model.Consequently, in this case, K varies from 1 (groups composed by only one variable,therefore having no interactions) to N (all variables placed within a single group, andall variables having connections with every other variable). Conversely, the originalK, referring to the number of interactions, varies between 0 and N-1.

We can test this setting in a pNK model by evaluating the performance of a one-dimensional search strategy for various levels of K. However, contrary to the originalNK model, we can vary also the strength of interaction setting |ai,j | to values in[0, 1]. Figure 6 reports the results generated with these settings.

The simulations are performed on different landscapes made up of N = 24 vari-ables and eight different values of K: 1,2,3,4,6,8,12,24. For each K, we generate11 landscapes using as many values of |ai,j | ranging from 0 to 1. On the resulting88 landscapes, we ran 300 independent explorations based on the random strategydescribed above. The data show the average value for each combination, using thefitness of the local peak if the search got trapped into one, or the fitness value ofthe point reached after 30,000 steps otherwise. As before, we find that these resultsreplicate and extend prior results generated with NK.

First, they show that increasing K decreases the average fitness we can expectto reach, confirming that the choice of a “structured” K, instead of randomly cho-sen connections, does not affect the role of K in producing a tougher environment.Moreover, as noted in the previous section, pNK allows us to compare directly the

15Such a random structure actually leads quickly to the production of fully uncorrelated structures, evenfor relatively low K values (Frenken et al. 1999).

An NK-like model for complexity

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 5 10 15 20 25

K

Average Fitness

|a|=0.0|a|=0.1|a|=0.2|a|=0.3|a|=0.4|a|=0.5|a|=0.6|a|=0.7|a|=0.8|a|=0.9|a|=1.0

Fig. 6 Average fitness produced at the end of 300 searches for each of the eight landscapes with K ={1, 2, 3, 4, 6, 8, 12, 24} and for various value of ai,j . Interdependent dimensions have |ai,j | from 0.0 to 1.0while independent ones have ai,j = 0

results from different landscapes, without the bias produced by different distributionsof fitness values for local peaks across landscapes with different complexity.

The novel results stem from the effect of the intensity of interaction. The graphshows clearly that weak interactions (|ai,j | < 1) allow the exploration to reach onaverage higher local peaks than the case of maximum interaction, and the weaker theintensity, the higher the expected fitness of the final points of an exploration.

This latter result provides support for the intuition that, for a given strategy ofsearch, the number of interactions and their intensity are two distinct and comple-mentary mechanisms to generate complexity. Once we define a given measure ofcomplexity, such as, for example, the expected fitness reached by an adaptive, one-dimensional mutation search strategy, it is possible to obtain different landscapesexhibiting the same level of complexity. They will differ in having a higher numberof interactions and lower intensity, or, vice versa, a lower number of interactions andhigher intensity. Generating similar results for one measure of complexity, however,does not imply necessarily that these landscapes will share identical properties. Infact, they will likely differ in respect of other measures of complexity, or in respect ofother applications, such as, for example, the population dynamic properties of agentsexploring the landscapes. We will not further explore this aspect, leaving to futurework the analysis of the properties of landscapes built with different combinations

M. Valente

of number and intensity of interactions. Rather, we concentrate on the effects of thecomplexity structure in respect of research strategies.

4.2 Modular strategies and decomposability

In the previous exercise, we adopted the standard research strategy based on aone-dimensional step. However, contrary to natural systems, artificial ones, such asindividuals or firms, can be expected to recognize the limitations of explorationsbased on one-dimensional steps in the presence of complexity. Therefore, problemsolvers facing highly complex tasks are likely to engage in more sophisticated explo-ration strategies to avoid the traps of local optima. An interesting issue is, therefore,what are the costs and advantages of different exploration strategies in respect ofdifferent problems?

To analyze this issue, we need to remind ourselves that the very definition of alocal peak depends on both the properties of the landscape and the search strategyadopted to explore it. In general, the shape of a landscape (e.g. being “smooth” or“rough”) depends not only on its own structure but also on the the rule defining theadmissible steps over the landscape. For example, a point may be a local peak if weallow the exploration to consider only points within a given radius from the currentone, but it may not be a local peak if we extend the radius of feasible steps. Con-sequently, we may intuitively conjecture that for each type of complexity structurethere will be a matching exploration strategy able to “smooth away” any local peak,and therefore systematically lead to the global optimum. As we will see, however, itis possible to show that, while this is true, at least within a given sub-set of structures,other considerations may suggest that sub-optimal search strategies leading to localpeaks are sometimes more desirable than “optimal” search strategies that, though notadmitting any local peak, yet show other problems advising against their adoption.

Since its inception (Simon 1969) the concept of complexity has been strictlyconnected with discussions concerning specific structures of interactions, particu-larly those exhibiting (or lacking) modularity. Modularity is the property of systemsmade of several components; in modular systems, groups of components have strongreciprocal interactions with members of the same group, while showing little orno interaction with components outside their groups. Non-modular (or integral)systems, conversely, have no visible structure, with interactions spanning differentcomponents.

Many disciplines use modularity as a central concept to explain a variety ofphenomena in natural (Callebout and Rasskin-Gutman 2005) and artificial systems(Brusoni et al. 2007). For decades, organizational scholars have represented the prob-lem of organizational design from a problem solving perspective (Miles and Snow1978), comparing the structure of an organization to the structure of its tasks, suchas production processes or development of new products. According to this perspec-tive, the performance of an organization can be evaluated on the basis the matchingbetween its internal structure and the external requirements. The concept of modular-ity can thus be applied to both the problem task and to the problem solver. To avoidconfusion we propose to use different terms, frequently used as synonyms, to express,on the one hand, the interdependency structure of the problem space and, on the

An NK-like model for complexity

other hand, the property of the search strategy adopted by the problem solver. So, wewill call decomposability the exogenous property of the problem space, such as thedefinitions and the technical interactions among components of a product. The termmodularity will be used to indicate the property of the searching strategy adopted.

As an experiment, we match the “block-based” complexity structure defined bygrouping the variables within subsets by defining search strategies also based on thegrouping of variables. We call C the size of the block of variables as considered bythe problem solver strategy. Such generalized search strategies can be represented bythe following algorithm:16

1. Choose randomly one block among the N/C blocks in the problem solverstrategy.

2. Choose randomly an integer in the range [1;C] (call this number c).3. Choose randomly c dimensions within the chosen block.4. Change by � the value of the chosen dimensions in a random direction.5. If the change leads to a point with higher fitness, move to the new point.6. If the change leads to a point with lower fitness, remain at the same point.7. Continue from 1.

In practice, a step of the research strategy consists in choosing randomly a newpoint differing from the old one by between 1 and C dimensions. Hence, the single-dimension mutation can be considered as a block-based search strategy when C = 1.For larger C values, the strategy is able to explore a wider area proportional to C.

Figures 7 and 8 report on an experiment presented in Frenken et al. (1999), but usepNK instead of the standard NK. Each of the eight graphs considers landscapes builtwith different K values, where the interdependent dimensions are chosen in “blocks”,so that each dimension within a block is linked to exactly K-1 other dimensions in thesame block, and have no connection to the dimensions in other blocks.17 Each graphconcerns the results from landscapes with different K values, and reports the averagefitness of the populations of agents adopting different modular research strategies ofdimension C.

The results generated with the pNK model coincide perfectly with those generatedwith the standard NK. It can be shown that only search strategies having C being amultiple of K can reach the global optimum, and this is what we actually observefor small K values. However, for more and more complex landscapes (higher K val-ues), other search strategies seem to dominate. The reason lies in the fact that searchstrategies with large modules are very slow, since they essentially test randomly allpossible combinations of the dimensions in each group. In this case, “optimal” searchstrategies take far too long to show their superiority, and therefore “quick & dirty”strategies, though doomed to eventually get stuck in a local peak, actually dominate

16For simplicity, we consider only C values that are integer divisors of N, though the results would notchange otherwise. It would only mean that the last class would contain a smaller number of variables thanthe other classes.17For obvious reasons of comparability, we assume that each interdependency in pNK is maximum, i.e.|ai,j | = 1 for each i and j dimensions within the same block.

M. Valente

1 7500 15000 22500 30000

0.446249

0.584687

0.723124

0.861562

1

Cl. 1 Cl. 2 Cl. 3 Cl. 4 Cl. 6 Cl. 8 Cl. 12

1 7500 15000 22500 30000

0.586186

0.689639

0.793093

0.896547

1

Cl. 1 Cl. 2 Cl. 3 Cl. 4 Cl. 6 Cl. 8 Cl. 12

1 7500 15000 22500 30000

0.49223

0.619173

0.746115

0.873058

1

Cl. 1 Cl. 2 Cl. 3 Cl. 4 Cl. 6 Cl. 8 Cl. 12

1 7500 15000 22500 30000

0.50476

0.628531

0.752302

0.876073

0.999844

Cl. 1 Cl. 2 Cl. 3 Cl. 4 Cl. 6 Cl. 8 Cl. 12

K=1 K=2

K=3 K=4

Fig. 7 Average fitness across time for classes of agents mutating blocks made of C = {1, 2, 3, 4, 6, 8, 12}dimensions. Simulations reported for landscapes with K = {1, 2, 3, 4}

slow ones for a long period before ending up trapped into a local peak. Hence, if therelevant period for competition is sufficiently short, it is well possible that “domi-nated” strategies (in the long term) are actually superior to “optimal” ones for anyreasonable length of time.

4.3 pNK and agent-based models

In our review of the limitations of NK we mentioned the difficulty in using NK asa model for complexity to plug into a wider model of, say, organizations, markets,etc. This is because the NK properties are statistical, made evident only by collectinga sufficiently large number of repetitions. However, each single run of a landscapeexploration is ill adapted to represent an actual exploration process. The reason is thatthe pattern actually generated by a simulated agent on a NK landscape is composedof very few fitness increasing steps in between a long series of failed attempts. Thisfeature prevents the direct use of NK to represent agents performing activities suchas, e.g., R&D, where the modeller is likely to expect innovative firms to generate astream of innovations, though at random rates of arrival. On the contrary, a patternin NK consists in one or a few jumps in between long periods of failed attemptedmutations. Such limitations makes hard, for example, to assume that the rate of arrivalof innovations, or their relative innovativeness, is proportional to the amount of R&Dinvestment.

Such limitations do not affect pNK. The proposed model produces search patternscloser to the intuitive sequence of fitness increasing steps, each composed, as in NK,by twists and turns (looking for the right direction), but also as actual steps in a

An NK-like model for complexity

1 7500 15000 22500 300000.38741

0.539009

0.690607

0.842206

0.993804

Cl. 1 Cl. 2 Cl. 3 Cl. 4Cl. 6 Cl. 8 Cl. 12

1 7500 15000 22500 300000.421594

0.543429

0.665264

0.787098

0.908933

Cl. 1 Cl. 2 Cl. 3 Cl. 4Cl. 6 Cl. 8 Cl. 12

1 7500 15000 22500 300000.300552

0.442172

0.583792

0.725412

0.867032

Cl. 1 Cl. 2 Cl. 3 Cl. 4Cl. 6 Cl. 8 Cl. 12

1 7500 15000 22500 300000.178852

0.34511

0.511368

0.677626

0.843884

Cl. 1 Cl. 2 Cl. 3 Cl. 4Cl. 6 Cl. 8 Cl. 12

K=6 K=8

K=12 K=24

Fig. 8 Average fitness across time for classes of agents mutating blocks made of C = {1, 2, 3, 4, 6, 8, 12}dimensions. Simulations reported for landscapes with K = {6, 8, 12, 24}

familiar Euclidean space. To highlight the difference between NK and pNK in thisrespect, Fig. 9 shows a comparison between 1,000 searches (one-bit mutations) on anNK landscape composed of N = 24 dimensions and defined as moderately complex(K = 3). The graphs represent the scatter plots between the fitness value of the localpeak eventually reached by the search patterns and the number of fitness increasingmutations produced by the patterns, that is, the successful “steps”. The left graph(NK) shows a random cloud of points, indicating that there is no relation between the

0.61

0.62

0.63

0.64

0.65

0.66

0.67

0.68

0 2 4 6 8 10 12 14 16 18Num. of successfull mutation

Fitness

0.52

0.54

0.56

0.58

0.6

0.62

0.64

0.66

160 180 200 220 240 260 280 300Num. of successfull mutation

Fitness

NK pNK

Fig. 9 Values of fitness and the number of successful mutations for NK and pNK. In both cases, the graphsreport the values for N = 24, K = 3 and agents adopting a one-bit mutation strategy. The data are obtainedby 1,000 independent runs on the same landscapes. Values were collected at the end of exploration, whena local peak is reached

M. Valente

number of steps and the final fitness reached by the search strategy. Moreover, theaverage number of steps is about nine, meaning that the patterns generated by NK arepretty short. Conversely, the right graph shows that patterns generated with pNK havea more plausible positive correlation between the number of positive mutations andthe final fitness of a search path, representing the obvious fact that to reach higherfitness areas you need a longer walk. Besides the correlation, moreover, patterns inpNK are overall much longer (between about 160 and 300 steps), providing a farbetter metaphor for results from innovating activities.

As an example of the possibilities offered by pNK, Ciarli et al. (2007, 2008) usea model derived from pNK to express the technological race of innovative firms. Inthese models, firms undergo several economic activities (e.g. sell final and/or inter-mediate products, collect revenues, set prices, etc.) as well as research activities(searching for better technologies on a complex technological landscape). In theseworks, the relative timing between the output of research (i.e. better technologies)and their economic impact (i.e. higher competitiveness) is crucial. An NK model,implying a slow and erratic pattern of “discoveries”, would not be able to representthe intended functions. Conversely, pNK provides the flexibility to represent sen-sible search patterns made of gradual improvements resulting from the appropriateresearch strategy. Moreover, the functional form of the complexity model and thepossibility to control every aspect of the landscape gives the opportunity to extend therepresentation of the complexity space. In these works, for example, the whole tech-nological landscape is represented as shifting in time under the pressure of exogenousmovements of the technological frontier. Therefore, the very same point of the land-scape (i.e. technology) provides different fitness values (relative competitiveness)through time, as the whole landscape drifts away. Consequently, firms must con-tinuously invest in research even just to maintain the current relative technologicalposition of their product, as represented by the fitness. Such a feature is easily rep-resented in pNK, allowing the explorations of how the overall industry performancediffers under, say, fast or slow changes of the technological frontier.

5 Conclusions and further research

This work proposes a new model to represent complexity by implementing the coreelements of NK in ways that are more suited for applications in economics ratherthan the original biological metaphor. The proposed model, pNK, generates a com-plex landscape by means of the (controlled) interaction among the dimensions of thelandscape as in NK. However, pNK considers real-valued dimensions (instead of thebinary ones of NK), allows for grades of interaction (instead of presence/absence),and is implemented by means of a deterministic function, greatly simplifying theimplementation of the model and the interpretation of the results. These differencesmake it possible to enhance significantly the study of complexity, in general, and itsapplications to specific domains, such as economics and organization theory.

The paper describes several experiments using pNK to represent complex land-scapes, showing how the core properties of NK are maintained in the new model,adding the benefit of dealing with the more intuitive context of a real-valued space. In

An NK-like model for complexity

particular, we show that intensity of interdependency is equivalent to the number ofinterdependencies, at least for many results presented in the literature. This permitsus to replicate the same properties using a familiar bi-dimensional plane and pro-vide intuitive graphical representations. Moreover, we present a number of originalresults made possible by pNK. Extensive experiments support the claim that pNK isfar better suited to implement complexity within models entailing the representationof simulated agents facing complexity.

Several lines of research may be devised starting from the present work. First,there are many properties of the proposed model that still need to be explored. Forexample, it may be interesting to study the properties of non symmetric landscapes,produced when |ai,j | �= |aj,i |. More in general, there are many functional shapesthat may be used as a fitness function; since this determines the relative distribu-tion of local peaks, it may be useful to generate a library of functions for differentuses.

A related line of research is to study the possibility to generate landscapes on thebasis of empirical data. Many theoretical and applied researchers would be interestedin the possibility of interpolating a data set of available evidence to be considered asa sample of scattered points, filling the gaps with econometric techniques and even-tually producing an estimated fitness landscape of a specific complex problem. Thismay contribute to a promising literature on the empirical applications of complexitytheory in economics (Frenken and Nuvolari 2004; Alkemade et al. 2009), and wouldobviously find many uses for both theoretical as well as applied issues.

Finally, pNK is particularly suited to be used in conjunction with agent-basedsimulation models. That is, using pNK as “complexity module” within modelsrepresenting systems dealing with the exploration of complex spaces, such as the eco-nomics of technical change or organizational theory. Its features make it very simpleto devise search strategies modeled on the behaviors of actual agents, either individu-als or organizations, facing complex tasks. The comparison of varied complexity andbehaviors adopted to deal with it has provided in the past many insights using NK,and more can be expected by using the more powerful and flexible pNK model.

References

Alkemade F, Frenken K, Hekkert MP, Schwoon M (2009) A complex systems methodology to transitionmanagement. J Evol Econ 19(4):527–543

Altenberg L (1995) Genome growth and the evolution of the genotype-phenotype map. In: BanzhafW, Eeckman FH (eds) Evolution and biocomputation: computational models of evolution, vol 899.Springer, Berlin, pp 205–259

Altenberg L (1997) NK fitness landscapes. In: Baeck T, Fogel D, Michalewicz Z (eds) Handbook ofevolutionary computation. Taylor & Francis, New York

Auerswald P, Kauffman SA, Loboc J, Shell K (2000) The production recipes approach to modeling tech-nological innovation: an application to learning by doing. J Econ Dyn Control 24(3):389–450. ISSN0165-1889

Brusoni S, Marengo L, Prencipe A, Valente M (2007) The value and costs of modularity: a problem-solving perspective. Eur Manag Rev 4:121–132

Callebout W, Rasskin-Gutman D (eds) (2005) Modularity: understanding the development and evolutionof natural complex systems, Vienna series in theoretical biology. MIT Press, Cambridge

M. Valente

Ciarli T, Leoncini R, Montresor S, Valente M (2007) Innovation and competition in complex environ-ments. Innov Manag Policy Pract 9(3–4):292–310

Ciarli T, Leoncini R, Montresor S, Valente M (2008) Technological change and the vertical organizationof industries. J Evol Econ 18(3):367–387

Durret R, Limic V (2003) Rigorous results for the NK model. Ann Probab 31(4):1713–1753Frenken K (2006) A fitness landscape approach to technological complexity, modularity, and vertical

disintegration. Struct Chang Econ Dyn 17(3):288–305Frenken K, Nuvolari A (2004) The early development of the steam engine: an evolutionary interpretation

using complexity theory. Ind Corp Chang 13(2):419–450Frenken K, Marengo L, Valente M (1999) Interdependencies, nearly-decomposability and adaptation. In:

Brenner T (ed) Computational techniques for modelling learning in economics. Springer, Berlin,pp 145–165

Kauffman SA (1989) Adaptation on rugged fitness landscapes. In: Stein D (ed) Lectures in the sciencesof complexity. The santa ire institute series. Addison-Wesley, New York, pp 527–618

Kauffman SA (1993) The origins of order: self-organization and selection in evolution. Oxford UniversityPress, Oxford

Kauffman SA, Levin S (1987) Towards a general theory of adaptive walks on rugged landscapes. J TheorBiol 128:11–45

Kauffman SA, Lobo J, Macready WG (2000) Optimal search on a technology landscape. J Econ BehavOrgan 43:141–166

Kaul H, Jacobson SH (2006) Global optima results for the Kauffman NK model. Math Program106(2):319–338. ISSN 0025-5610

Lenox MJ, Rockart SF, Lewin AY (2006) Interdependency, competition, and the distribution of firm andindustry profits. Manag Sci 52(5):757–772

Levinthal DA (1997) Adaptation on rugged landscapes. Manag Sci 43(7):934–950Li R, Emmerich MTM, Eggermont J, Bovenkamp EGP, Back T, Dijkstra J, Reiber JHC (2006) Mixed-

integer NK landscapes. In: Runarsson TP, Beyer H, Burke E, Merelo-Guervos J, Whitley LD, Yao X(eds) Parallel problem solving from nature—PPSN IX. Lecture Notes in Computer Science. Springer,Berlin, pp 42-51

March JG (1991) Exploration and exploitation in organizational learning. Organ Sci 2(1):71–87Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs, 3rd edn. Springer, BerlinMiles RE, Snow CC (1978) Organizational strategy, structure, and process. MacGraw-Hill, New YorkRivkin JW (2000) Imitation of complex strategies. Manag Sci 46(6):824–844Rivkin JW, Siggelkow N (2002) Organizational sticking points on NK landscapes. Complexity 7:31–43Simon HA (1969) The sciences of the artificial. MIT Press, CambridgeSkellett B, Cairns B, Geard N, Tonkes B, Wiles J (2005) Maximally rugged NK landscapes contain the

highest peaks. In: GECCO ’05: proceedings of the 2005 conference on genetic and evolutionarycomputation. ACM Press, New York, pp 579–584. ISBN 1-59593-010-8

Valente M (2008) Laboratory for simulation development—LSD, working paper 2008/63. Sant’AnnaSchool for Advanced Studies, LEM, Pisa

Wagner GP, Altenberg L (1996) Complex adaptations and the evolution of evolvability. Evolution50(3):967–976

Weinberger E (1991) Kauffman, S.A. 1993. The origins of order: self-organization and local properties ofKauffman’s N-k model: a tunably rugged energy landscape. Phys Rev A 44(10):6399–6413