A Theory Of Attending And Reinforcement In Conditional Discriminations

A THEORY OF ATTENDING AND REINFORCEMENT IN CONDITIONAL DISCRIMINATIONS

JOHN A. NEVIN, MICHAEL DAVISON, AND TIMOTHY A. SHAHAN

UNIVERSITY OF NEW HAMPSHIRE, UNIVERSITY OF AUCKLAND AND NATIONAL RESEARCH CENTREFOR GROWTH AND DEVELOPMENT, NEW ZEALAND,

AND UTAH STATE UNIVERSITY

A model of conditional discrimination performance (Davison & Nevin, 1999) is combined with thenotion that unmeasured attending to the sample and comparison stimuli, in the steady state and duringdisruption, depends on reinforcement in the same way as predicted for overt free-operant respondingby behavioral momentum theory (Nevin & Grace, 2000). The rate of observing behavior, a measurableaccompaniment of attending, is well described by an equation for steady-state responding derived frommomentum theory, and the resistance to change of observing conforms to predictions of momentumtheory, supporting a key assumption of the model. When probabilities of attending are less than 1.0, themodel accounts for some aspects of conditional-discrimination performance that posed problems forthe Davison-Nevin model: (a) the effects of differential reinforcement on the allocation of responses tothe comparison stimuli and on accuracy in several matching-to-sample and signal-detection tasks wherethe differences between the stimuli or responses were varied across conditions, (b) the effects of overallreinforcer rate on the asymptotic level and resistance to change of both response rate and accuracy ofmatching to sample in multiple schedules, and (c) the effects of fixed-ratio reinforcement on accuracy.Some tests and extensions of the model are suggested, and the role of unmeasured events in behaviortheory is considered.

Key words: attending, behavioral momentum, conditional discrimination, matching to sample, signaldetection, observing behavior

_______________________________________________________________________________

In order for stimuli to control behavior,organisms must attend to them. Dinsmoor(1985) suggested that effective stimulus con-trol depends on contact with the relevantstimuli via overt observing behavior, and thatfor a complete understanding ‘‘. . . we areobliged to consider analogous [to observing]processes . . . commonly known as attention.The processes involved in attention are not asreadily accessible to observation as the moreperipheral adjustments, but it is my hope andmy working hypothesis that they obey similarprinciples.’’ (p. 365). Although attention isusually construed as a cognitive process, weview attending as unmeasured (possibly co-

vert) operant behavior that accompanies mea-surable observing. Attending, we suggest, isselected and strengthened by the reinforcingconsequences of overt discriminated operantbehavior that would be less frequently rein-forced in the absence of attending. FollowingDinsmoor, we assume that the unmeasuredbehavior of attending to discriminative stimuliis related to the rates of reinforcementcorrelated with those stimuli in the same wayas measured free-operant response rate.

In this paper, we develop a model ofattending that parallels a version of behavioralmomentum theory for free-operant respond-ing and incorporate it into a general accountof discriminated operant behavior (Davison &Nevin, 1999). We begin by reviewing behav-ioral momentum theory as it applies to re-sistance to change of overt responding, extendit to account for steady-state response rate, andpropose a model of attending based on itsprinciples. Next, we review the Davison-Nevinmodel and indicate some of its shortcomings.We then show that when the momentum-based model of attending is combined with theDavison-Nevin model, the combination canaccount for some data that posed problems forthe original model: the effects of differentialreinforcement for the two correct responses in

Preparation of this article was supported by NIMHGrant MH65949 to the University of New Hampshire. Apreliminary version was presented at the meeting of theSociety for the Quantitative Analyses of Behavior, May2004. We thank Stephen Lea for his thoughtful commentsand suggestions on an earlier version of the manuscript.

doi: 10.1901/jeab.2005.97-04

Correspondence should be addressed to John A. Nevin,RR2, Box 162, Vineyard Haven, Massachusetts, 02568 (e-mail: [email protected]); Michael Davison, Depart-ment of Psychology, University of Auckland, Private Bag92019, Auckland, New Zealand (e-mail: [email protected]); or Timothy Shahan, Department ofPsychology, Utah State University, Logan, Utah, 84322(e-mail: [email protected]).

JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 281–303 NUMBER 2 (SEPTEMBER)

281

conditional discriminations; the effects ofoverall reinforcer rate on conditional discrim-ination accuracy in the steady state and duringdisruption by briefly imposed experimentalvariables; and the effects of fixed-ratio re-inforcement for conditional discrimination.

The development of our model depends onthree fundamental assumptions. First, mea-sured operant behavior in the steady state aswell as during disruption depends on thereinforcer rate correlated with a distinctivestimulus, relative to the overall reinforcer ratein the experimental context, according toa function derived from behavioral momen-tum theory (Nevin & Grace, 2000). Second,attending to discriminative stimuli dependson the reinforcer rate that accompaniesthe unmeasured behavior of attending inthe same way as measured operant respond-ing, both in the steady state and duringdisruption, with reinforcer rate expressedrelative to the context in which those stimuliappear. And third, given that a subject attendsto the relevant stimuli, its behavior is describedby the Davison-Nevin (1999) model of condi-tional-discrimination performance.

Readers may question the utility of a modelthat invokes unmeasured, possibly covert,attending behavior (see, for example, thevigorous discussions of the varieties of theoryin behavior analysis edited by Marr [2004]).There are several reasons for pursuing thisapproach. First, we show that the value ofa single variable in the model—the probabilityof attending—determines the form of rela-tions between measured discrimination per-formance and empirical variables. Thus themodel provides a basis for organizing diverseresults that have been reported in the litera-ture. Second, the model identifies a behavioralprocess with properties like those of overtbehavior that leads to testable predictions andchallenges researchers to investigate directlymeasurable counterparts of the terms of themodel in relation to reinforcement variables.Third, even if overt counterparts of itsterms prove to be elusive, the model providesa way to infer the effects of reinforcementon covert activities that are involved instimulus control, consistent with the radicalbehaviorist view that ‘‘. . . private events arenatural and in all important respects likepublic events’’ (Baum, 1994, p. 41; see alsoSkinner, 1974).

BEHAVIORAL MOMENTUM THEORY

Behavioral momentum theory (Nevin &Grace, 2000) has been concerned with re-sistance to change during relatively short-termdisruption. It is related metaphorically toNewton’s second law in classical mechanics,which states that the change in the velocity ofa body is directly proportional to an externalforce and inversely proportional to the body’sinertial mass. Nevin, Mandell, and Atak (1983)modeled response rates during disruption as

logBx

Bo~

{x

m, ð1Þ

where Bo is baseline response rate, Bx isresponse rate during disruption, x is the valueof the disrupter with its decremental effectsindicated by the minus sign, and m isbehavioral mass. Virtually all of the relevantresearch has employed multiple variable-in-terval (VI) VI schedules to control obtainedreinforcer rates and to permit within-sessioncomparisons of resistance to change. In a re-view of all his data, Nevin (1992b) suggestedthat behavioral mass (m) in a schedule com-ponent was a power function of reinforcer ratein that component (rs), relative to the overallaverage reinforcer rate in a session (ra), whichis based on time in both components and inintercomponent intervals. Thus m 5 (rs /ra)b,where b measures the sensitivity of relativeresistance to relative reinforcement. For ex-ample, if b 5 1.0, the ratio of log proportionsof baseline in two multiple-schedule compo-nents is equal to the ratio of reinforcer rates.The value of b has been found to beapproximately 0.5 in a number of studies thatarranged multiple VI VI schedules (Nevin,2002) and will be used throughout this paper.Equation 2, with reinforcement terms insertedin lieu of behavioral mass m, describes thehighly reliable finding that resistance tochange relative to baseline is greater ina multiple-schedule component with morefrequent reinforcement:1

1 The role of context in determining resistance tochange is not ideally clear. Nevin (1992a) found a strongeffect when context varied between conditions, andcontext played a major role in several other studiesreviewed by Nevin (1992b). However, Nevin and Grace(1999) found no effect when context varied withinsessions. A systematic replication of Nevin (1992a) byGrace, McLean, and Nevin (2003) obtained mixed results.

282 JOHN A. NEVIN et al.

logBx

Bo~

{x

rs=rað Þb: ð2Þ

By expressing responding during disruptionrelative to baseline, Equation 2 ignores thedeterminers of steady-state responding. Buteven in the steady state, responding is mea-sured against a background that includespotential disrupters such as competition fromunspecified activities that entail their ownunmeasured reinforcers. Herrnstein (1970)formalized this idea in his well-known hyper-bola relating steady-state response rate B andreinforcer rate r :

B~kr

rzre, ð3Þ

where re represents unmeasured ‘‘extraneous’’reinforcers, and k is the asymptotic responserate as r goes to infinity.

Equation 2 can be rewritten to describeresponse rate in the steady state as well asduring disruption, thus bringing behavioralmomentum theory to bear on baseline re-sponse rates as well as their resistance tochange (see Nevin & Grace, 2005). Addinga scale constant k9 to express predictions inresponses per minute, converting to naturallogarithms, and exponentiating, Equation 2becomes:

B~k0 exp{x

rs=rað Þb: ð4Þ

The disrupter x in Equation 4 plays the samerole as extraneous reinforcers re in Equation 3for steady-state response rate. Other param-eters must be added to the numerator ofthe exponent to characterize the effects ofshort-term disrupters in tests of resistance tochange. Like k in Equation 3, k9 is the asymp-totic response rate as rs /ra goes to infinity.

The steady-state predictions of Equations 3and 4 for single VI schedules arranged oversuccessive conditions are strikingly similar. Togenerate representative predictions fromEquation 3, we set k 5 100 and re 5 40. Wethen estimated the parameters of Equation 4that gave similar predictions, with ra setarbitrarily at 1.0 because when reinforcer rateis varied over successive conditions, the ‘‘com-ponent’’ becomes the experimental session,and the overall average reinforcer rate in-

cludes that during the subject’s extraexperi-mental life, which presumably has a low andconstant value. With b 5 0.5, x 5 5.0, and k9 5115, the predictions of Equation 4 are virtuallyindistinguishable from those of Equation 3, asshown in Figure 1 (if another value werechosen for ra, the value of x would differ).More generally, the extensive data that areadequately fit by Equation 3 also will beadequately fit by Equation 4.

A reliable finding of research on resistanceto change is that adding experimentally de-fined extraneous reinforcers to one compo-nent of a multiple schedule both decreasesresponse rate and increases resistance tochange in that component even if theadded reinforcers are qualitatively differentfrom those produced by responding (e.g.,Grimes & Shull, 2001; Shahan & Burke,2004). This general result is not well explainedby Equation 3 (see Nevin, Tota, Torquato, &Shull, 1990), but follows from Equation 4.Moreover, Herrnstein’s (1970) extension ofEquation 3 to describe multiple-scheduleperformance made some predictions that haveproven erroneous. Because Equation 4 char-acterizes resistance to change as well as steady-state performance, we will employ it through-out this paper.

In addition to describing rates of food-reinforced operants, Equation 4 also describesrates of observing responses that producecontact with discriminative stimuli. Shahan(2002) examined the effects of variations inrate of primary reinforcement on observing-

Fig. 1. The relations between steady-state response rateand reinforcer rate on VI schedules according toHerrnstein (1970; Equation 3 here) and according toa modified version of behavioral momentum theory(Equation 4, with parameter values in the legend).

THEORY OF ATTENDING 283

response rates of rats. Observing responsesproduced exposure to stimuli differentiallycorrelated with otherwise unsignaled periodsof response-independent sucrose deliveries orextinction. The rate of sucrose deliveries wasvaried across conditions. As shown in Figure 2,his average data conform closely to thepredictions of Equation 4 with b equal to 0.5.

Relatedly, Shahan, Magee, & Dobberstein(2003) examined the resistance to change ofobserving behavior in pigeons. In their experi-ments, unsignaled periods of food reinforce-ment on a random-interval (RI) schedulealternated with extinction in both componentsof the multiple schedule. Observing responsesin both components produced stimuli corre-lated with the RI and extinction periods. TheRI schedule in one component arranged fooddeliveries at a rate four times higher than inthe other component. Observing occurred ata higher rate and was more resistant toprefeeding and intercomponent food deliver-ies in the component in which it produceddiscriminative stimuli associated with a higherrate of primary reinforcement. A structural-relation analysis of their data found that theexponent b in Equation 4 was close to 0.5 forboth observing and food-key responding. Thusobserving—an overt analog of unmeasuredattending to discriminative stimuli—is func-tionally similar to response rates and resistanceto change in single and multiple schedules offood-maintained responding. For these rea-sons, we use Equation 4 to predict theprobability of attending as a function of

reinforcer rate in the model of conditionaldiscrimination performance developed below.

THE DAVISON-NEVIN MODELOF CONDITIONAL

DISCRIMINATION PERFORMANCE

In a typical observing-response procedure,responses produce stimuli that signal theconditions of reinforcement for a single re-sponse. In a conditional discrimination pro-cedure such as matching to sample (MTS), bycontrast, reinforcers are given for one or theother of two responses depending on the valueof a preceding stimulus. Specifically, one oftwo sample stimuli is presented at the start ofeach trial. After a fixed period of exposure, orafter completion of a response requirement,two comparison stimuli are presented, one ofwhich is the same as the sample. A response tothe comparison that is the same as the sampleis deemed correct and may be reinforced. Inarbitrary or symbolic matching, the compar-isons are physically different from the samples,and reinforcement availability is determinedby a rule specifying the correct comparison foreach sample. In experiments characterized assignal detection or recognition, responses areusually defined topographically (e.g., pecks atleft or right key with pigeons; saying ‘‘Yes’’ or‘‘No’’ with humans). The conditional-discrim-ination paradigm encompasses both matching-to-sample and signal detection, and our modelapplies to both tasks.

For simplicity and consistency with previousanalyses, we will designate the samples as S1

and S2, with responses B1 and B2 defined bythe comparisons C1 and C2. Thus, in a standardMTS procedure with pigeons, red or greenillumination of a center key will be designatedS1 or S2. Illumination of the side keys with redand green, alternating irregularly between leftand right, will be designated C1 and C2, andpecks on the keys displaying C1 and C2 will bedesignated B1 and B2 regardless of their left orright position.

In discrete-trial conditional discriminations,it is convenient to array the stimuli (S1 and S2)and responses (B1 and B2) in a 2 3 2 matrix asshown in the upper panel of Figure 3. Cellentries are subscripted by row–column nota-tion and represent numbers of events. Thus,for example, B11 and B22 are the numbers ofcorrect responses, and B12 and B21 are the

Fig. 2. The relation between the rate of observing byrats (from Shahan, 2002) and the predictions of Equation4. Parameter values are given in the legend, and VACindicates the proportion of variance explained.


numbers of incorrect responses, each talliedover a specified period of experimentation.Likewise, R11 and R22 are the numbers ofreinforcers for correct responses, and R12 andR21 are 0 (no experimentally arranged con-sequences) in all cases considered here.Following Alsop (1991) and Davison (1991),Davison and Nevin (1999) assumed that theeffects of R11 and R22 generalized to the othercells as a result of confusion between thestimuli and contingencies within the matrix.They identified confusability as the inverse ofdistance in a two-dimensional psychometricspace, with its axes defined by stimulus-behavior and behavior-reinforcer contingen-cies. The distance between two discriminatedoperants with different stimuli is given by dsb ,which depends on the physical differencebetween S1 and S2 and the sensory capacitiesof the subject. The distance between operantswith different response definitions or contin-gencies is given by dbr , which depends on thephysical difference between the definitions ofB1 and B2 and on variables such as unsignaleddelays to reinforcement that would alter thediscriminability of the behavior-reinforcer

contingency. Confusabilities are expressed asthe inverse of distances, 1/dsb and 1/dbr ; ifeither parameter equals 1.0, discriminationperformance is at chance. The resulting matrixof direct and generalized reinforcers is shownin the bottom panel of Figure 3.

Davison and Nevin (1999) assumed thatresponses were allocated to the cells of thematrix in the top panel of Figure 3 so as tomatch the ratios of the sums of direct andgeneralized reinforcers, as shown in the lowerpanel. The resulting expressions are cumber-some and can be found in Davison and Nevin(pp. 447–450), together with a more extensiverationale for their approach. From theseexpressions, Davison and Nevin calculatedthe expected numbers of responses in eachcell of the matrix for various values of dsb , dbr ,and R11/R22, and predicted the value ofdiscrimination accuracy defined as log D 50.5*log(B11/B21*B22/B12). Note that log D isthe log of the geometric mean of the predictedratios of correct to incorrect responses on S1

and S2 trials. It is defined identically toa frequently used empirical measure of dis-crimination accuracy, log d, which is calculatedfrom experimental data (see Davison & Tustin,1978).

Although the Davison-Nevin (1999) modelwas quite successful in accounting for a widerange of results for conditional discrimina-tions in discrete trials, it had three short-comings—all of which were acknowledged—that are addressed in this paper.

First, the model predicts that when theR11/R22 ratio is varied with dsb constant,the relations between log(B1/B2) andlog(R11/R22), plotted separately for S1 andS2 trials, are curvilinear and converge aslog(R11/R22) becomes extreme. As a result,the predicted relation between log D andlog(R11/R22) is concave down with a maximumat log(R11/R22) 5 0. This function form hasrarely been reported in any of the manyrelevant studies (e.g., McCarthy & Davison,1980); indeed, a review by Johnstone andAlsop (1999) found that many reported func-tions were concave up, exactly the opposite ofthe Davison-Nevin predictions.

Second, the model predicts that log D is thesame for all conditions with the same dsb , dbr ,and R11/R22 ratio, regardless of the absoluterates or values of R11 and R22. Thus it couldnot explain the positive relation between

Fig. 3. The basic conditional-discrimination matrix fortwo stimuli and two responses; cells are designated by row-column notation as shown at the top, and cell entriesrepresent numbers of events. The lower panel presents thematrix of effective reinforcers with R11 contingent on B11

and with R22 contingent on B22, generalizing to the othercells according to the Davison-Nevin (1999) model.


signaled reinforcer probability and accuracy ofsignal detection reported by Nevin, Jenkins,Whittaker, and Yarensky (1982, Experiment 2)and systematically replicated with MTS inmultiple schedules by Nevin, Milo, Odum,and Shahan (2003).

Third, although the Davison-Nevin (1999)model could be extended to free-operantmultiple-schedule performance when S1 andS2 durations are lengthened so that B1 and B2

can occur repeatedly, with B1 and B2 identical,their model was cumbersome and laden withfree parameters (pp. 467–469). Moreover,they did not attempt to model the well-established finding that response rates inmultiple schedules are more resistant tochange in the richer component.

The mispredictions of steady-state discrimi-nation performance result from an implicitassumption in the Davison-Nevin (1999)model—namely, that the subject always at-tends to the stimuli, no matter how infrequentthe reinforcers. To address these shortcomingsin the Davison-Nevin model, we will useEquation 4, derived from behavioral momen-tum theory, to predict the probability ofattending to the stimuli in discrete-trial condi-tional discriminations as well as response ratesand their resistance to change, which are welldescribed by Equation 4.

A MODEL OF ATTENDING

In this section, we outline a model ofattending to the sample and comparisonstimuli in a conditional discrimination. Themodel has two components: a structure that isindependent of reinforcement effects, anda momentum-based model of attending inrelation to reinforcement.

Model Structure

In a standard MTS trial with pigeons assubjects, the sample S1 or S2 is presented ona center key followed by comparisons C1 andC2, which define choice responses B1 or B2, onthe side keys. We assume that on each trial thesubject attends to the sample with probabilityp(As), and then attends to the comparisonswith probability p(Ac). We assume further thatp(As) is the same for S1 and S2, whichever ispresented on a given trial, and that p(Ac) is thesame for C1 and C2. The process may berepresented as a Markov chain as shown in

Figure 4. If the subject attends to the sampleand comparisons, it emits B1 or B2 as predictedby Davison and Nevin (1999; see bottom panelof Figure 3). The formulas for probabilities ofB1|S1 and B1|S2 are given in Figure 4, State 1. Ifthe subject does not attend to the samples, S1

and S2 are ignored (or completely confused),so that dsb in the model is 1.0. If it then attendsto the comparisons, the probabilities of B1|S1

and B1|S2 are determined by R11/R22 asmodulated by dbr only (State 3 in Figure 4).Note that if R11 5 R22, the expressions forState 3 reduce to 0.5. If the subject does notattend to the comparisons, C1 and C2 areignored, dbr is 1.0, and the expressions inFigure 4, States 2 and 4, reduce to 0.5.Consequently, responses are directed random-ly to the left or right keys with probability 0.5regardless of whether the subject attended tothe sample or not (for present purposes, wewill neglect inherent side-key biases).

The overall performance resulting froma mixture of trials with and without attendingto sample and comparison stimuli is predictedby pooling the response probabilities for trialsin each of the four states summarized inFigure 4 weighted by p(As), p(Ac), 1-p(As),and 1-p(Ac). The basic idea is the same asHeinemann and Avin’s (1973) quantificationof attending during the acquisition of a condi-tional discrimination (see also Blough, 1996).Our analysis of attending in relation to re-inforcer probabilities in multiple schedules(see below) implies that attending, like overtoperant behavior, can be controlled by envi-ronmental stimuli such as the key colorssignaling schedule components. As such,our account is consistent with the work ofHeinemann, Chase, and Mandell (1968), whodemonstrated control of attending to one orthe other of two stimulus dimensions bydifferential reinforcement with respect tothose dimensions.

The discriminability parameters dsb and dbr

and the attending probabilities p(As) andp(Ac) have somewhat similar functions in theproposed model. For example, setting dsb 51.0 has the same effect as setting p(As) 5 0.Likewise, setting dbr 5 1.0 has the same effectas setting p(Ac) 5 0. However, there are someimportant differences between them. As notedabove, Davison and Nevin (1999) conceptual-ized dsb and dbr as distances in a psychometricspace within which various discriminated


Fig. 4. Schematic model of attending in a conditional discrimination. The subject attends to the sample with p(As). Ifit attends, the discriminability of stimulus–behavior relations is dsb in the Davison-Nevin (1999) model; if it does notattend, dsb 5 1. It then attends to the comparisons with p(Ac). If it attends, the discriminability of behavior–reinforcerrelations is dbr in the Davison-Nevin model; if it does not attend, dbr 5 1. Combining these possibilities leads to four states;for each state, the Davison-Nevin expressions for the probabilities of B1|S1 and B1|S2 (see Figure 3) are given in the lowerportion of the figure.


operants could be arrayed. Thus their valuesreflect long-term structural features of theexperiment such as the sensory capacities ofthe subject, the physical differences between S1

and S2 and between C1 and C2, and thedistinctiveness of the contingency between B1

and B2 (which are defined by C1 and C2) andthe reinforcers R11 and R22. By contrast,attending occurs probabilistically from trialto trial, where p(As) and p(Ac) depend onreinforcer rates in the same way as free-operant response rates. The independence ofattending and discriminability as model param-eters suggests that a subject could attend withhigh probability (because of frequent re-inforcement) to stimuli that were difficult todistinguish (low dsb or dbr), or conversely, at-tend with low probability (because of infre-quent reinforcement) to stimuli that werehighly discriminable.

Effects of Overall Reinforcer Rate

We assume that unmeasured attending isrelated to the rate of reinforcement correlatedwith a stimulus relative to its context in thesame way as food-reinforced free-operantresponding or overt observing behavior thatproduces discriminative stimuli. Thus attend-ing to the samples and comparisons would bemore probable and more resistant to changein the presence of stimuli correlated withhigher rates of reinforcement.

We propose that attending to S1 and S2 ina conditional discrimination is given by thefollowing version of Equation 4:

p Asð Þ~ exp{x

rs=rað Þb, ð5Þ

where x is background disruption or distrac-tion that interferes with attending to thesample. The reinforcer rate in a schedulecomponent rs is here identified with thereinforcer rate for observing or attendingbehavior preceding and during sample pre-sentation. Thus rs is given by reinforcers persession divided by total time from onset ofintertrial intervals or multiple-schedule com-ponents to offset of the samples. The sessionaverage reinforcer rate ra is defined asabove for Equation 4. No scalar analogous tok9 in Equation 4 is needed because theasymptote of p(As) is 1.0. Equation 5 statesthat attending in a schedule component is

positively related to component reinforcer raters relative to the overall average session re-inforcer rate ra. Therefore, p(As) is predictedto be higher and more resistant to change inthe richer of two multiple-schedule compo-nents with a given increase in disruption, justlike response rate. We also assume thatattending to C1 and C2 is similarly dependenton reinforcer rate relative to its context.However, for p(Ac), the relevant reinforcerrate is that obtained within the MTS trial aftersample offset, designated rc , and the context isthe schedule component within which the trialoccurs. Thus

p Acð Þ~ exp{z

rc=rsð Þb, ð6Þ

where z represents background disruption ofattending to C1 and C2, which may or may notbe the same as x, the background disruption ofattending to the samples. In standard MTStrials, the reinforcer rate rc is given by thereciprocal of the mean latencies of B1 and B2.When a retention interval intervenes betweensamples and comparisons, rc is given by thereciprocal of the sum of the retention intervaland the mean latencies of B1 and B2. Becauselatencies of responding to the comparisonstimuli are rarely reported, we assume 1-slatencies throughout. The model parametersand related terms are summarized in Appen-dix A.

Predicted Effects of Differential Reinforcement

The predictions that follow from variationsin p(As) within the Markov structure of Figure 4are illustrated in the following section. Theseillustrative predictions assume that the ratio ofreinforcers for the two sorts of correctresponses, R11/R22, is varied systematically withtotal reinforcement, R11 + R22, held constant.The constancy of R11 + R22 implies constancyof p(As) and p(Ac) while the ratio of re-inforcers, R11/R22, is varied. Therefore, Equa-tions 5 and 6 are irrelevant and we can ignorethe parameters b, x, and z. Because p(Ac) islikely to be close to 1.0 in detection or MTSexperiments with frequent reinforcementand no retention interval separating samplesand comparisons, we concentrate on thepredicted effects of p(As) , 1.0, correspond-ing to values of x . 0 in Equation 5, with p(Ac)5 1.0.


Davison and Tustin (1978) suggested thatwhen the difference between S1 and S2 isconstant, with the reinforcer ratio R11/R22

varied, the ratios of B1 to B2 on S1 and S2 trialscould be described by the generalized match-ing law. In logarithmic form, neglecting in-herent bias:

On S1 trials, logB11

B12~a log

R11

R22zlog d, and ð7aÞ

on S2 trials, logB21

B22~a log

R11

R22{ log d, ð7bÞ

where a represents sensitivity to reinforcementratios and logd provides an empirical measureof discrimination between S1 and S2 (seeabove). Many data sets conform reasonablywell to the predicted linear relation betweenlog response and reinforcer ratios. However,the equations of Davison and Nevin (1999)predict functions for S1 and S2 that are curved,becoming horizontal at high and low reinforc-er ratios, respectively.

The empirical measure of discriminationaccuracy, log d, is obtained by subtractingEquation 7b from Equation 7a (we uselogarithms to the base 10 here and for theapplications below):

log d~0:5 logB11

B12{ log

B21

B22

� �: ð8Þ

Equation 8 implies that measured discrimina-tion, log d, is independent of the rein-forcer ratio, and a number of studies havereported rough constancy of log d when thereinforcer ratio is varied. By contrast, theDavison-Nevin equations predict that log D isan inverted-U function of the log reinforcerratio. As noted above, such functions are rarelyreported.

Interestingly, when the ratio of reinforcersfor the two correct responses, R11/R22, isvaried with total reinforcement, R11 + R22,constant, the model summarized in Figure 4can generate a range of function forms forresponse ratios and log D, as shown in Figure 5.The filled symbols in the top left panel showthe Davison-Nevin (1999) predictions withboth p(As) and p(Ac) set at 1.0, with R11/R22

varied over a wide range. If p(As) is reduced to.7 (unfilled symbols), the functions becomemore nearly linear, with some wiggles thatwould be difficult to detect in real data. In the

top right panel, the corresponding functionsfor log D show the strong inverted-U formpredicted by Davison and Nevin when p(As) 51.0 (filled symbols), and a more nearlyhorizontal gull-wing form when p(As) 5 .7(unfilled symbols). The bottom left panelshows the effects of different values of dsb withdbr fixed at 100 and p(As) 5 .7. If dsb 5 1000(an extremely easy discrimination, filled sym-bols), the functions are nonlinear with twoclear inflections. If dsb 5 10 (a moderatelydifficult discrimination, unfilled symbols) thefunctions are essentially linear over the rangefrom 21.5 to +1.5 log units on R11/R22. (Thefunction for dsb 5 100, p(As) 5 .7 in the toppanel is an intermediate version.) The corre-sponding functions for log D in the bottomright panel are U-shaped for dsb 5 1000 (filledsymbols) and roughly horizontal for dsb 5 10(unfilled symbols) over the same range. Thusthe relation between log D and log R11/R22 cantake on a variety of forms, depending on thevalues of dsb and p(As). Because most signal-detection research has arranged moderatelydifficult discriminations and a restricted rangeof R11/R22, the absence of clear curvilinearityin the response-ratio functions and the appar-ent independence of measured discrimina-tion, log d, from the reinforcer ratio are notsurprising.

It is important to note that althoughvariations in dsb and p(As) lead to clearlydistinguishable predictions, the same is nottrue for dbr and p(Ac). For example, with p(As)5 1.0, predictions for dbr 5 100, p(Ac) 5 .8, areindistinguishable from those for dbr 5 8, p(Ac)5 1.0. However, it is possible to distinguishtheir effects empirically. Because we haveidentified dbr with the discriminability of theresponse-reinforcer contingency, the modelpredicts that the estimated value of dbr

should be positively related to the physicaldifference between C1 and C2. By contrast,p(Ac) depends on the within-trial reinforcerrate, so its estimated value should be constantwith respect to the difference between C1 andC2. We show below that when Jones (2003)varied the difference between C1 and C2 inMTS, with R11/R22 varied over a wide rangeand R11 + R22 constant, his data accordwith these expectations. Conversely, in delayedmatching to sample, dbr should be constantand p(Ac) should decrease as rc decreaseswith the length of the retention interval. A


model and analysis of delayed discrimina-tion by Nevin, Davison, Odum, and Shahan(in preparation) will address these expecta-tions.2

APPLICATIONS: DIFFERENTIALREINFORCEMENT

We now apply the model of attendingsummarized above to three studies that ar-ranged differential reinforcement for the twocorrect responses, B11 and B22, in conditionaldiscriminations while holding reinforcer totalsconstant. First, Jones (2003) varied the re-inforcer ratio over a wide range with two levelsof discriminability between the comparisons

2 A preliminary version entitled ‘‘Reinforcement, attend-ing, and remembering’’ was presented at the meeting ofthe California Association for Behavior Analysis, February2005.

Fig. 5. The left column displays predicted functions relating log ratios of responses, B1/B2, to the log ratio ofreinforcers for correct responses, R11/R22, separately for S1 and S2 trials. The upper panel shows the effects of two valuesof p(As) with dsb 5 dbr 5 100. The lower panel shows the effects of two values of dsb with dbr and p(As) constant. The rightcolumn displays predicted relations between log D, which is given by the difference between the corresponding functionsin the left column, and the log reinforcer ratio. p(Ac) is set at 1.0 for all predictions. See text for further description.


C1 and C2. Second, Alsop (1988) varied thereinforcer ratio over three values with fivelevels of discriminability between samples S1

and S2. Third, Nevin, Cate, and Alsop (1993)varied the reinforcer ratio over five values withtwo levels of discriminability between samplesS1 and S2 and between responses, B11 and B22.Taken together, these studies test the ability ofthe present model to account for data that arenot entirely consistent with Davison andNevin’s (1999) model, which in effect assumedthat p(As) and p(Ac) were always 1.0.

Jones (2003) reported a comprehensive setof MTS data with R11 + R22 constant andR11/R22 varied over an unusually wide rangeand with unusually extensive exposure to eachcondition. The sample and comparison stimulidiffered in brightness. In Part 1 of hisexperiment, the differences between both S1

and S2 and C1 and C2 were large; in Part 2,the difference between C1 and C2 was re-duced. The functions relating log(B1/B2) tolog(R11/R22) for S1 and S2 trials were curvilin-ear and were not well described by the basicDavison-Nevin (1999) model (see Jones, 2003,Figure 5). However, predictions of the presentmodel fitted the data quite well, as shown inFigure 6. Because the overall reinforcer ratewas the same in all conditions of both parts,Equations 5 and 6 are not relevant, so we fittedp(As) and p(Ac) directly for the entire data set.We estimated dbr separately for Parts 1 and 2(designated dbr1 and dbr2) because the compar-ison stimuli differed between parts, with dsb thesame for both parts because the S1-S2 differ-ence was the same. There were 40 indepen-dent data points fitted by five parameters: dsb ,dbr1, dbr2, p(As), and p(Ac); their values aregiven in the panels of Figure 6. The overallproportion of variance explained by the model(VAC) is .98. The best-fitting value of dsb was500, but varying dsb over the range from 100 to1000 decreased VAC by less than .02. Thereason for the relatively poor estimation of dsb

is that high values correspond to very low errorrates. For example, dsb 5 100 corresponds toone error in 100 trials, whereas dsb 5 1000corresponds to one error in 1000 trials. Forthis reason, the value of dsb accounts for ratherlittle of the data variance in easy discrimina-tions.

The best-fitting values of dbr1 and dbr2 were200 and 13. The lower value of dbr for the dataof Part 2 reflects the reduced discriminability

of the comparisons when the differencebetween C1 and C2 was decreased. When dbr1

and dbr2 were varied independently, dbr1 couldvary from 50 to 1000, and dbr2 could vary from5 to 20, with no more than .02 loss in VAC.Thus, although dbr1 and dbr2 were not tightlyestimated, the data were best fitted with dbr1

greater than dbr2, whereas p(Ac) was con-strained to take the same value for both partsbecause the conditions of reinforcement werethe same. These results are consistent with the

Fig. 6. Fits of predictions by the model to the data ofJones (2003), Part 1 (upper panel) and Part 2 (lowerpanel). Parameter values are shown in the legend for eachpanel; overall VAC 5 .98. See text for explanation.


separate determination of dbr and p(Ac) by theresponse-defining stimuli and by reinforce-ment variables.

In one condition of Part 2, Jones (2003)arranged extinction for B1 on S1 trials andcontinuous reinforcement for B2 on S2 trials.Thus the ratio of R11 to R22 was zero, andthe data cannot be plotted in Figure 6. Never-theless, the data are interesting and challengingas an extreme case. According to the model ofattending shown in Figure 4, p(B1|S1) andp(B1|S2), and therefore log(B1|S1/B2|S1) andlog(B1|S2/B2|S2), must be the same wheneverR11 or R22 is zero. That is, discriminationbetween S1 and S2 is predicted to be zero. Tothe contrary, Jones obtained average values of20.04 and 20.80 for log(B11/B21) andlog(B12/B22), respectively, implying fairlygood discrimination (log d 5 0.38).

To interpret his results, Jones (2003) con-strued trials with C1 on the left and C2 on theright, and trials with C1 on the right and C2

on the left, as different configurations, thusdefining eight discriminated operants main-tained by different relative frequencies ofreinforcement. His data for the condition with

extinction versus continuous reinforcement, aswell as his other data, were explained by thisapproach. As shown in Figure 6, our attention-based model accounts for all his other dataquite well without invoking different config-urations of C1 and C2. Furthermore, our modelcan accommodate the effects of extinction onS1 trials by adding a small value to all cells ofthe reinforcement matrix of Figure 1, basedon the fact that all responses have the effect ofadvancing the trial sequence and thereby leadto delayed reinforcement. Doing so wouldintroduce another free parameter into themodel, and we forego this added complexityfor the present.

A second data set was provided by Alsop(1988), who varied the physical differencebetween S1 and S2 (rather than between C1 andC2, as in Jones, 2003) over five conditions ina signal-detection procedure where B1 and B2

were defined by their location. Differencesbetween S1 and S2 were defined ordinally,including a condition with no difference, withreinforcer ratios R11/R22 arranged at 9:1, 1:1,and 1:9 in each condition. The left panel ofFigure 7 presents Alsop’s (1988) mean data,

Fig. 7. The left panel displays the relation between log d and log(R11/R22) with the difference between S1 and S2

varied over five conditions, including zero difference, from Alsop (1988). The functions are identified by the ordinaldifference between S1 and S2. The right panel displays predicted relations between log D and log (R11/R22) for differentvalues of dsb , shown with each function; other parameter values are given in the legend. Overall VAC 5 .96. See text forexplanation.


showing that the shape of the function relatinglog d to R11/R22 depended on the level ofdiscrimination between S1 and S2. The rightpanel of Figure 7 shows our model’s predic-tions with different values of dsb for eachfunction, but with dbr , p(As), and p(Ac) thesame for all five functions. Thus predicted logD values were derived from eight free param-eters estimated from 30 response ratios, withoverall VAC 5 .96. The predicted functions areconcave up at high levels of discrimination andnearly horizontal at intermediate and lowlevels, corresponding to the trends in Alsop’sdata. With p(As) , 1.0, our model captures themain effects in Alsop’s data with the smallestpossible number of free parameters. However,the asymmetries in the data cannot beexplained by our model without additional(and ad hoc) parameters.

A third data set was provided by Nevin et al.(1993), who varied the differences betweensamples S1 and S2 and between responses B1

and B2 in a factorial design. S1 and S2 weredefined as brighter or dimmer lights on onekey, and B1 and B2 were defined as shorter orlonger response latencies on a second key.Over four sets of conditions, the differencebetween S1 and S2 luminances was either large(0.066 log units) or small (0.032 log units),and the difference between B1 and B2 latencieswas either large (0 to 1 s vs . 2 s) or small (1 to2 s vs. 2 to 3 s). Within each set of conditions,the reinforcer ratio R11/R22 was varied system-atically over five values with two replicationsfor R11 5 R22, giving 56 data points in all. Totalreinforcement, R11 + R22, was constant acrossall conditions. The data are shown in Figure 8together with model predictions and param-eter values. Measured discrimination (log d;the separation between response-ratio func-tions for S1 and S2) was directly related to thedifferences between S1 and S2 and between B1

and B2. The sensitivity of response ratios toreinforcer ratios (the slope of the response-ratio functions; a in Equations 7a and 7b) wasdirectly related to the difference between B1

and B2, but inversely related to the differencebetween S1 and S2.

This complex pattern of results is predictedby our model with two values of dsb , two valuesof dbr , p(As) 5 .97, and p(Ac) 5 .99. The overallVAC is 0.92; if dsb and dbr were allowed to takedifferent values for each set of conditions, VACimproves by less than 0.01. The data generally

fall above the predictions, suggesting a biastoward the shorter-latency response that wasmost pronounced when the difference be-tween B1 and B2 was small. The model cannotaccount for inherent biases of this sort withoutintroducing an additional parameter thatwould take different values across sets ofconditions. Also, the predicted functions forsmall differences between S1 and S2 and B1 andB2 (lower right panel) are curved, whereas thedata are linear. The major result, though, isthat the main effects of the differencesbetween S1 and S2 and between B1 and B2

were well predicted with dsb independent ofthe difference between B1 and B2 and with dbr

independent of the difference between S1 andS2. When Nevin et al. (1993) fitted their datawith an earlier version of the Davison-Nevin(1999) model proposed by Alsop (1991) andDavison (1991), they found that dbr dependedon the difference between S1 and S2, andthat the differences between S1 and S2 andbetween B1 and B2 interacted in determiningdsb . The present model, with probabilities ofattending slightly less than 1.0, remedies thesedifficulties.

In summary, when reinforcer totals areconstant and reinforcer ratios are varied, themodel correctly predicts the forms of func-tions relating log response ratios or log d to logreinforcer ratios when the differences betweenthe sample stimuli and the comparison stimulior responses are varied separately in condi-tional discriminations. Moreover, it does sowith a minimum of parameters, and the valuesof discriminability parameters dsb and dbr

correspond at least ordinally with empiricalvariables. More important, dsb remains con-stant when the comparison stimuli or re-sponses are varied, and dbr remains constantwhen the samples are varied. We turn now tothe effects of varying total reinforcement onthe probabilities of attending to the samplesand comparisons, which were constant withinthe experiments analyzed above.

APPLICATION: MULTIPLE SCHEDULESOF REINFORCEMENT FOR

CORRECT RESPONSES

As noted above, the original Davison-Nevin(1999) model predicted that variations in totalreinforcement, R11 + R22, would have no effecton accuracy of conditional discrimination


performance, and did not consider the effectsof total reinforcement on its resistance tochange. However, Nevin et al. (2003) foundthat both steady-state accuracy and its resis-tance to change depended on total reinforce-ment in much the same way as response rate inmultiple schedules where R11 + R22 variedbetween components.

In a variation of a paradigm introduced bySchaal, Odum, and Shahan (2000), Nevinet al. (2003) arranged equal VI 30-s schedulesin two multiple-schedule components, whereresponding produced MTS trials with verticaland slanted lines as the samples (S1, S2) andcomparisons (C1, C2). Reinforcer probabilitiesfor correct matches were .8 in one component

Fig. 8. Fits of predictions of the model to the data of Nevin et al. (1993). The panels display data and predictions forsets of conditions with large or small differences between S1 and S2 and between B1 and B2. Parameter values are given ineach panel; overall VAC 5 .92. See text for explanation.


(designated rich) and .2 in the other (desig-nated lean). After stable response rates andaccuracies were established, performance wasdisrupted by prefeeding, by presenting foodduring intercomponent (ICI) intervals, or byinserting a 3-s retention interval before onsetof C1 and C2, each for five sessions. Baselineperformances were reestablished after eachdisruption. Finally, food reinforcement wasdiscontinued for 10 sessions of extinction.Nevin et al. found that response rates anddiscrimination accuracies (measured as log d)usually were higher in baseline and were moreresistant to all four disrupters in the richercomponent. They concluded that the strengthof discriminating, like the strength of free-

operant responding, was positively related toreinforcer rate. Here, we show that thesereinforcement effects on discrimination accu-racy follow directly from our model.

A summary of the average data from Nevinet al. (2003) is shown in Figure 9. Theresponse-rate data are consistent with standardmultiple-schedule results in that baseline re-sponse rates were higher and decreased lessunder disruption, relative to baseline, in therich component. The discrimination datasuggest that accuracy was similarly related tocomponent reinforcer rates. Specifically, logd was higher and decreased less, relative tobaseline, in the rich component, and thedecreases were similar to those observed for

Fig. 9. A summary of the average data of Nevin et al. (2003). The upper left panel shows response rates and the lowerleft panel shows discrimination accuracies, measured as log d, in baseline and during disruption by prefeeding (PF), ICIfood presentations at random times (RT), the abrupt introduction of a 3-s retention interval between sample andcomparison stimuli (Del), and the termination of reinforcement (Ext, in two 5-session blocks), separately for rich(shaded bars) and lean (open bars) components. The corresponding right panels show average levels of performanceduring disruption expressed as proportions of the immediately preceding baseline.


response rate during prefeeding, intercompo-nent food, and extinction. However, whenperformance was disrupted by inserting a 3-sretention interval, response rates were relative-ly unaffected whereas log d decreased to near-chance levels.

We begin by showing that Equation 4, withthe numerator of the exponent modified byadding terms corresponding to the addition ofthe various disrupters, describes the averagemultiple-schedule VI response rates of Nevinet al. (2003). Equation 4 is repeated here asEquation 9 with added terms representing thefour disruptive operations:

B~k 0 exp{ xzf zvzqzczdrð Þ

rs=rað Þb, ð9Þ

where k9, b, x, rs, and ra are defined as above; frepresents the additional disruptive effect ofprefeeding; v represents the additional effectof ICI food; q represents the additional effectof the retention interval; and c and d represent

the additional effects of discontinuing thecontingency and changing the reinforcer ratefrom rs to zero during extinction (see Nevin,McLean, & Grace, 2001; Nevin & Grace, 2005).We used a nonlinear curve-fitting program(Microsoft Excel Solver) to estimate values ofthe parameters, with rs and ra based onprogrammed reinforcer rates. The results areshown in Figure 10, left panel, and the fittedparameter values are given in Table 1, with theexponent b fixed at 0.50. Because there wereseven free parameters and 18 data points, anexcellent fit is hardly surprising.

As described above, we assume that theeffects of reinforcement on attending to S1

and S2 are quantitatively similar to effects onresponse rate in the data of Nevin et al. (2003),and rewrite Equation 4 for p(As) with termsadded as in Equation 5:

p Asð Þ~ exp{ xzf zvzqzczdrð Þ

rs=rað Þb: ð10Þ

Fig. 10. The left panel shows the agreement between obtained response rates displayed in Figure 9 and thosepredicted by Equation 9 with parameter values given in Table 1. The right panel shows the agreement between logd values displayed in Figure 9 and those predicted by Equations 10 and 11 with parameter values given in Table 1 (filleddiamonds). The unfilled diamonds represent log d during introduction of a 3-s retention interval.

Table 1

Parameter values for model fits to the data of Nevin et al. (2003). Response rates were fitted byEquation 4, and log d was fitted by Equations 10 and 11 with dsb 5 dbr 5 150, b 5 0.5.

k9 x z f v q c d VAC

Responses perminute

109 0.33 — 0.40 0.34 0.12 0.15 0.001 .96

Log d — 0.08 0.00 0.18 0.05 1.41 0.05 0.000 .81


Likewise, for attending to C1 and C2, we rewriteEquation 6 with the same added terms:

p Acð Þ~ exp{ zzf zvzqzczdrð Þ

rc=rsð Þb: ð11Þ

In the procedure of Nevin et al. (2003), rc/rs

is the same in both rich and lean componentsbecause reinforcers occur four times morefrequently within MTS trials, just as withinthe components themselves, so p(Ac) must bethe same. Moreover, because within-trial laten-cies are short relative to the average timebetween trial presentations, rc/rs is high andp(Ac) should approximate 1.0. Therefore,States 2 and 4 of Figure 4 will rarely ifever be entered, and variations in the datawill depend primarily on variations in p(As)except when retention intervals are used asdisrupters.

We fitted Equations 10 and 11 to theaccuracy data of Nevin et al. (2003) bydetermining the parameter values that mini-mized the sum of squared differences betweenpredicted log D and obtained log d. Thecalculations for predicting log D are summa-rized in Appendix B; the full worksheet forestimating parameter values is available on theJEAB website. As noted above, it is necessary toassume or to fit dsb and dbr in order to calculatepredicted log D. In a study of symbolicmatching to sample, Godfrey and Davison(1998) found that varying the differencebetween S1 and S2 did not affect dbr , and thatvarying the difference between C1 and C2 didnot affect dsb (see also the analysis of Nevinet al., 1993, above). Moreover, they found thatwhen the difference between S1 and S2 was thesame as the difference between C1 and C2, dsb

was equal to dbr . Accordingly, we will assumethat dsb 5 dbr here and in another MTS studybelow. With dsb 5 dbr 5 150, predicted log Dapproximated the maximum average value oflog d in the baseline data, so this value wasused in fitting the full data set.

The results are shown in the right panel ofFigure 10. Note that the unfilled diamonds,which represent the disruptive effects ofinserting a 3-s retention interval betweensample and comparison stimuli, are wellexplained by the decrease in rc that necessarilyfollows when a nonzero retention interval isintroduced; a fully developed model of de-layed discriminations is in preparation.

The best-fitting parameters are given inTable 1; the value of z was 0.0, so that p(Ac)in baseline was 1.0 as suggested above. Thevalue of dsb 5 dbr has relatively little impact onthe quality of the fit: Values ranging from 100to 1000 altered VAC by less than .02 forreasons noted above. Clearly, the fit is lesssatisfactory than for response rate, but exam-ination of Figures 9 and 10 suggests that muchof the data variance arises from variations inlog d between successive baseline determina-tions, displayed in the roughly vertical clustersof data points in Figure 10. Most importantly,our model of attending captures the majorordinal results of Nevin et al. that were prob-lematic for the Davison-Nevin (1999) model:Accuracy of discrimination, like response rate,is higher and more resistant to change in thericher component.

APPLICATION: FIXED-RATIOREINFORCEMENT FOR

CONDITIONAL DISCRIMINATIONS

Another finding that raises problems for theDavison-Nevin (1999) model is the progressiveincrease in accuracy within a series of trialsbetween reinforcers that are contingent ona fixed number of correct responses insuccessive (but not necessarily consecutive)trials. The result holds for MTS (Mintz,Mourer, & Weinberg, 1966; Nevin, Cumming,& Berryman, 1963) and for signal detection(Nevin & MacWilliams, 1983). It is problematicbecause, as noted above, the Davison-Nevinmodel predicts that if dsb and dbr are constant,then variations in overall reinforcer rate orprobability between conditions, between com-ponents, or within trial sequences have noeffect on accuracy. Although it might beargued that the delay to reinforcement forresponses early in the fixed ratio woulddegrade dbr , the discriminability of response-reinforcer relations, it would then be necessaryto assume that dbr itself comes under thecontrol of the number of trials elapsing sincereinforcement. It is at least equally reasonableto assume that attending, like overt respond-ing, depends on delay to reinforcement. Herewe show that if successive trials are treatedanalogously to multiple-schedule componentsin the paradigm of Nevin et al. (2003), withreinforcer rate in each successive trial given bythe reciprocal of delay to reinforcement, the


progressive increase in accuracy within afixed ratio follows directly from our model.The assumption of correspondence betweenschedule components and successive trials issupported by the work of Mintz et al., whoprovided exteroceptive cues correspondingto postreinforcer trial number, demonstratedthat the cues controlled accuracy, and ob-tained results similar to those of Nevin et al.(1963).

Nevin et al. (1963) required five key pecks atthe center-key sample, following which thecomparisons were presented on the side keyswhile the sample remained lighted. A 1-sintertrial interval separated successive trials.Reinforcement followed 10 (not necessarilyconsecutive) trials with correct responses. Ina separate condition, all correct responseswere reinforced (FR 1). To determine p(As)and p(Ac), we used Equations 5 and 6 with ra asthe overall average reinforcer rate in a sessionand rsi as the reinforcer rate (reciprocal of thesum of intertrial time plus cumulative time tocomplete five sample-key pecks from the ithtrial to reinforcement, as reported by theauthors) in the ith trial after a previousreinforcer. The within-trial reinforcer rate rci

was taken as the reciprocal of cumulativelatencies from the ith trial to reinforcement,assuming 1-s latencies to the comparisons. Asa result, both rs/ra and rc/rs increase system-atically as the ratio advances. Then, with dsb 5dbr 5 400 (chosen to approximate the datawith FR 1) and with b 5 0.5, the modelaccounted for 91% of the data variance. Again,the values of dsb and dbr had little effect on thequality of the fit: With dsb 5 dbr ranging from100 to 4000, VAC was altered by less than .01for the reasons described above. The predictedfunction agrees reasonably well with theaverage data, as shown in Figure 11. Themodel parameters are also shown in Figure 11.According to the model, the fact that x 50 implies that p(As) 5 1.0 (i.e., the subjectsattended to the samples on every trial) so theentire effect arises from variations in attendingto the comparisons, p(Ac).

In summary, the Davison-Nevin (1999)model in conjunction with a model ofattending to sample and comparison stimuli(Figure 4) explains the effects of differentialreinforcement for the two correct responseson response-ratio functions and on measureddiscrimination when total reinforcement is

constant across conditions. It also accountsfor the effects of varying the discriminability ofthe samples or comparisons. When the prob-ability of attending is assumed to depend onvariations in reinforcement in accordance withEquations 5 and 6, derived from behavioralmomentum theory, the model also explainsthe positive relation between baseline accuracyand resistance to change and total reinforce-ment in multiple schedules. It also accountsfor the progressive increase in accuracy underfixed-ratio reinforcement of conditional-dis-crimination performance.

GENERAL DISCUSSION

Our model has at least two levels: its coreassumptions, and their instantiation in a modelthat generates predictions for comparison withempirical data sets. The core assumptions thatwere set forth in the Introduction are repeatedhere.

Core Assumptions

First, the measured rate of an overt freeoperant, both in the steady state and duringdisruption, depends on reinforcer rate relativeto the context according to a function(Equation 4) derived from Nevin’s (1992b)formulation of behavioral momentum. Thisextension of behavioral momentum theory to

Fig. 11. Proportion of correct responses over nineconsecutive unreinforced trials when reinforcement isavailable for the 10th trial with a correct response. Data areaverages from Nevin et al. (1963); the predicted function isgiven by Equations 5 and 6 with parameter values shown inthe figure. See text for explanation.


steady-state response rate is supported by thesimilarity of the predictions of Equation 4 tothose of Herrnstein’s (1970) widely acceptedformulation of steady-state response rate(Equation 3). In addition, Equation 4 wasderived from Equation 2, which describesa great deal of the data on resistance tochange, and several studies of resistance tochange support Equation 4 over Equation 3.

Second, in a conditional discrimination,unmeasured probabilities of attending to thesample and comparison stimuli depend onreinforcement correlated with those stimulirelative to the context within which theyappear according to the same function as forovert responding, both in the steady state andduring disruption, with independent param-eters characterizing disrupters of attendingto the samples and comparisons. This assump-tion cannot be supported directly becauseattending is measurable only by inferencefrom a model. However, indirect supportcomes from research on observing behavior,which is widely construed as an overt, measur-able expression of attending and which wasrelated to reinforcement and disruption in thesame way as suggested by Equation 4 (Shahan,2002; Shahan et al., 2003).

Third, given that a subject attends to therelevant stimuli, its behavior is described bythe Davison-Nevin (1999) model of conditional-discrimination performance. This assumptionis supported by the various lines of evidencemarshaled by Davison and Nevin (1999).

From Assumptions to Predictions

The probabilities of attending to the sam-ples and comparisons, p(As) and p(Ac), aredetermined by Equations 5 and 6 with param-eters x and z representing the disruptive effectsof unspecified but constant background fac-tors such as competition from extraneousactivities and their reinforcers. Additionalparameters are needed to represent the effectsof added, experimentally defined disrupters.All disrupter values are free parameters withvalues constrained to be greater than or equalto 0.

The relevant reinforcer rate for attending tothe samples, rs , is calculated over the timebefore sample presentation, when a subjectmay engage in observing behavior or unmea-sured attending that is reinforced by sampleonset, plus the time when the sample is

present. The session average reinforcer rate,ra , is calculated over an entire session exclud-ing reinforcer durations.

To estimate the probability of attending tothe comparisons, we have used the within-trialreinforcer rate, rc , based on time from offset ofthe sample to reinforcement (or time out ifthe response is not reinforced), with thereinforcer rate for the samples, rs , as thecontext. In calculating within-trial reinforcerrates, we assumed 1-s latencies to the compar-isons in order to avoid infinite reinforcer ratesin trials with zero retention intervals. We alsoassumed 1-s latencies in application to the dataof Nevin et al. (2003) when a retention intervalwas introduced as a disrupter.

We have assumed that the parametersrepresenting the effects of background dis-rupters can take different values for attendingto the samples (x) and comparisons (z). Whenoverall reinforcement was held constant andno explicit external disrupters were arranged,as in Jones (2003), Alsop (1988), and Nevinet al. (1993), p(Ac) was equal or close to 1.0. Asa result, p(Ac) did not contribute to the datafits and could be omitted from the model forthose studies. However, the effects of explicitdisrupters in Nevin et al. (2003) and of fixed-ratio reinforcement in Nevin et al. (1963)appear in the model as values of p(Ac), as wellas p(As), less than 1.0. Thus for generality ofmodel application, and for conceptual sym-metry, both p(As) and p(Ac) are needed.

In view of their close temporal proximity inzero-retention-interval procedures, it mayseem unreasonable to use different reinforcerrates, contexts, and background disrupters forattending to the samples and comparisons asspecified in Equations 5 and 6. For the pres-ent, this approach appears to be successful infitting a number of findings in the literature,but future research may suggest the need formodifications.

The exponent b in Equations 2, 4, 5, 6, 9, 10,and 11 represents the extent to which re-inforcement determines resistance to change.For a given rate of reinforcement relative tothe context, larger values of b correspond togreater resistance to background or experi-mentally arranged disrupters. Experimentswith many different reinforcer rates and twolevels of disruption, or with two reinforcerrates and many disrupter values, are needed toestimate b reliably. The experiments consid-


ered here were not designed to give reliableestimates of b ; accordingly, we set b 5 0.5because that value is approximated in a num-ber of multiple VI VI schedule studies design-ed specifically to evaluate it (Nevin, 2002).

Overview of Model Parameters and Fits

Ideally, a model parameter would take thesame value, within error, across different sortsof determinations. For example, timing ballsrolling down inclined planes, or timing theswings of a pendulum, give the same value ofacceleration due to gravity in classical mechan-ics. Such consistency is a rarity in the quanti-tative analysis of behavior. However, thepresent modeling efforts provide several ex-amples of parametric consistency between setsof conditions within an experiment. Forexample, the data of Jones (2003) were wellfitted by holding dsb , p(As), and p(Ac) constantacross conditions with differences in C1 andC2, and the data of Alsop (1988) wereadequately fitted by holding dbr , p(As), andp(Ac) constant across conditions with differ-ences in S1 and S2. Moreover, these constan-cies were dictated by the definitions of theterms of the model: dsb and dbr should dependon the sample and comparison stimuli and beindependent of reinforcer rates (Davison &Nevin, 1999), whereas p(As) and p(Ac) shoulddepend on reinforcer rates, which wereessentially constant across conditions in bothstudies. When reinforcer rates varied betweencomponents within an experiment, as in Nevinet al. (2003), the background disrupter valueswere held constant across components be-cause the evaluation of differential resistanceto change requires that the same disrupter beapplied to both components (Nevin, 1992b),and dsb and dbr were held constant acrosscomponents because the stimuli were un-changed. Overall, we obtained good to excel-lent fits by holding constant those parametersthat were identified with consistent aspects ofthe experiments, and allowing variation inparameters that were identified with experi-mental variables. Although the number ofparameters in our model is large by compar-ison with, say, the generalized matching law,all of them are necessary to represent exper-imentally defined features of the studiesexamined here, and the number of data pointsis substantially greater than the number ofparameters. The constancy of parameters

across independent studies may be evaluatedas systematic, parametric research on condi-tional discriminations continues.

Testing the Model

The model predicts several effects that havenot, to our knowledge, been explored in theresearch literature. For example, because weassume, following Nevin (1992b), that attend-ing is controlled by reinforcement relative toa context (i.e., rs/ra) variations in reinforcerrate in one multiple-schedule componentshould produce contrast effects in accuracy,as well as response rate, in a second, constantcomponent, with no changes in model param-eters. Also, the progressive increase in accuracywithin fixed-ratio trial sequences was modeledas resulting from increases in p(Ac). Becausep(Ac) determines the extent to which differ-ential reinforcement affects responding to C1

or C2, sensitivity to differential reinforcementshould increase as the ratio advances. Re-search along these lines could provide somedata of interest in their own right as well astests of the present model.

As the model is structured, there is no way tobe sure, a priori, whether a given experimentaldisrupter affects p(As), p(Ac), or both. In ouranalysis of the resistance-to-change data ofNevin et al. (2003), we assumed that alldisrupters operated identically on both p(As)and p(Ac) (see Equations 10 and 11). It wouldbe useful to devise methods for disruptingp(As) or p(Ac) separately—for example, pre-senting distracters only during presentation ofthe samples or the comparisons—to evaluatethe independence of these attentional terms.In particular, as shown by example inAppendix B, decreasing p(As) by increasingbackground disrupter x is predicted to reducemeasured accuracy more in the lean compo-nent of a multiple schedule, whereas decreas-ing p(Ac) by increasing background disrupter zis predicted to reduce measured accuracymore in the rich component. The effects oftargeted disrupters could provide strong testsof the model.

It would be of special interest to examinemeasurable aspects of behavior that mightaccompany or correspond to attending, suchas requiring the subject to respond differen-tially to the samples (e.g., Urcuioli, 1985) or toadopt different positions within the chamberto view the comparisons (Wright & Sands,


1981). Analyses of this sort could evaluate theeffects of disrupters directly for comparisonwith model estimates of p(As) and p(Ac).

FINAL REMARKS

In the larger scheme of behavior theory,there are at least three ways to conceptualizeour model. First, attending may be construedas a mental way station between discriminativestimuli and behavior, and as such our modelmay be accused of explaining behavioral databy ‘‘appeals to events taking place at someother level of observation, described in differ-ent terms, and measured, if at all, in differentdimensions’’ (Skinner, 1950, p. 193). Second,the probabilities of attending may be con-strued as intervening variables whose namesand attributes are irrelevant because they haveno significance beyond their role in organizingand summarizing data. Third, attending maybe construed as a hypothetical construct thatrefers to physically real but unmeasuredactivities that have properties similar to mea-sured overt responding, and that must beevaluated by inference via a mathematicalmodel. It is this final perspective that informsour theoretical efforts and provides challengesfor future research.

REFERENCES

Alsop, B. (1988). Detection and choice. Unpublished doctoraldissertation, University of Auckland, New Zealand.

Alsop, B. (1991). Behavioral models of signal detectionand detection models of choice. In M. L. Commons, J.A. Nevin, & M. C. Davison (Eds.). Signal detection:Mechanisms, models, and applications (pp. 39–55).Hillsdale, NJ: Erlbaum.

Baum, W. M. (1994). Understanding behaviorism: Science,behavior, and culture. New York: HarperCollins.

Blough, D. S. (1996). Error factors in pigeon discrimina-tion and delayed matching. Journal of ExperimentalPsychology: Animal Behavior Processes, 22, 118–131.

Davison, M. C. (1991). Stimulus discriminability, contin-gency discriminability, and complex stimulus control.In M. L. Commons, J. A. Nevin, & M. C. Davison(Eds.). Signal detection: Mechanisms, models, and applica-tions (pp. 57–78). Hillsdale, NJ: Erlbaum.

Davison, M., & Nevin, J. A. (1999). Stimuli, reinforcers,and behavior: An integration. Journal of the Experi-mental Analysis of Behavior, 71, 439–482.

Davison, M. C., & Tustin, R. D. (1978). The relationbetween the generalized matching law and signal-detection theory. Journal of the Experimental Analysis ofBehavior, 29, 331–336.

Dinsmoor, J. A. (1985). The role of observing andattention in establishing stimulus control. Journal ofthe Experimental Analysis of Behavior, 43, 365–381.

Godfrey, R., & Davison, M. (1998). Effects of varyingsample- and choice-stimulus disparity on symbolicmatching-to-sample performance. Journal of theExperimental Analysis of Behavior, 69, 311–326.

Grace, R. C., McLean, A. P., & Nevin, J. A. (2003).Reinforcement context and resistance to change.Behavioural Processes, 64, 91–101.

Grimes, J. A., & Shull, R. L. (2001). Response-independentmilk delivery enhanced persistence of pellet-rein-forced lever pressing by rats. Journal of theExperimental Analysis of Behavior, 76, 179–194.

Heinemann, E. G., & Avin, E. (1973). On the developmentof stimulus control. Journal of the Experimental Analysisof Behavior, 20, 183–195.

Heinemann, E. G., Chase, S., & Mandell, C. (1968, May 3).Discriminative control of ‘‘attention.’’ Science, 160,553–554.

Herrnstein, R. J. (1970). On the law of effect. Journal of theExperimental Analysis of Behavior, 13, 243–266.

Johnstone, V., & Alsop, B. (1999). Stimulus presentationratios and the outcomes for correct responses insignal-detection procedures. Journal of the ExperimentalAnalysis of Behavior, 72, 1–20.

Jones, B. M. (2003). Quantitative analyses of matching tosample performance. Journal of the ExperimentalAnalysis of Behavior, 79, 323–350.

Marr, M. J. (2004). The interpretation of themes: Openingthe Baum-Staddon inkbattle. Journal of the ExperimentalAnalysis of Behavior, 82, 71.

McCarthy, D. C., & Davison, M. (1980). Independence ofsensitivity to relative reinforcement and discrimina-bility in signal detection. Journal of the ExperimentalAnalysis of Behavior, 34, 273–284.

Mintz, D. E., Mourer, D. J., & Weinberg, L. S. (1966).Stimulus control in fixed-ratio matching to sample.Journal of the Experimental Analysis of Behavior, 9, 627–630.

Nevin, J. A. (1992a). Behavioral contrast and behavioralmomentum. Journal of Experimental Psychology: AnimalBehavior Processes, 18, 126–133.

Nevin, J. A. (1992b). An integrative model for the study ofbehavioral momentum. Journal of the ExperimentalAnalysis of Behavior, 57, 301–316.

Nevin, J. A. (2002). Measuring behavioral momentum.Behavioural Processes, 57, 187–198.

Nevin, J. A., Cate, H., & Alsop, B. (1993). Effects ofdifferences between stimuli, responses, and reinforcerrates on conditional discrimination performance.Journal of the Experimental Analysis of Behavior, 59,147–161.

Nevin, J. A., Cumming, W. W., & Berryman, R. (1963).Ratio reinforcement of matching behavior. Journal ofthe Experimental Analysis of Behavior, 6, 149–154.

Nevin, J. A., & Grace, R. C. (1999). Does the context ofreinforcement affect resistance to change? Journal ofExperimental Psychology: Animal Behavior Processes, 25,256–268.

Nevin, J. A., & Grace, R. C. (2000). Behavioral momentumand the law of effect. Behavioral and Brain Sciences, 23,73–130.

Nevin, J. A., & Grace, R. C. (2005). Resistance to extinction inthe steady state and in transition. Journal of ExperimentalPsychology: Animal Behavior Processes, 31, 199–212.

Nevin, J. A., Jenkins, P., Whittaker, S., & Yarensky, P.(1982). Reinforcement contingencies and signal de-tection. Journal of the Experimental Analysis of Behavior,37, 65–79.


Nevin, J. A., & MacWilliams, S. (1983). Ratio reinforce-ment of signal detection. Behaviour Analysis Letters, 3,317–324.

Nevin, J. A., Mandell, C., & Atak, J. R. (1983). The analysisof behavioral momentum. Journal of the ExperimentalAnalysis of Behavior, 39, 49–59.

Nevin, J. A., McLean, A. P., & Grace, R. C. (2001).Resistance to extinction: Contingency terminationand generalization decrement. Animal Learning andBehavior, 29, 176–191.

Nevin, J. A., Milo, J., Odum, A. L., & Shahan, T. A. (2003).Accuracy of discrimination, rate of responding, andresistance to change. Journal of the Experimental Analysisof Behavior, 79, 307–321.

Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L.(1990). Alternative reinforcement increases resistanceto change: Pavlovian or operant contingencies? Journalof the Experimental Analysis of Behavior, 53, 359–379.

Schaal., D. W., Odum, A. L., & Shahan, T. A. (2000).Pigeons may not remember the stimuli that rein-forced their behavior. Journal of the ExperimentalAnalysis of Behavior, 73, 125–139.

Shahan, T. A. (2002). Observing behavior: Effects of rateand magnitude of primary reinforcement. Journal ofthe Experimental Analysis of Behavior, 78, 161–178.

Shahan, T. A., & Burke, K. A. (2004). Ethanol-maintainedresponding of rats is more resistant to change ina context with added non-drug reinforcement. Be-havioural Pharmacology, 15, 279–285.

Shahan, T. A., Magee, A., & Dobberstein, A. (2003). Theresistance to change of observing. Journal of theExperimental Analysis of Behavior, 80, 273–293.

Skinner, B. F. (1950). Are theories of learning necessary?Psychological Review, 57, 193–216.

Skinner, B. F. (1974). About behaviorism. New York: Knopf.Urcuioli, P. J. (1985). On the role of differential

sample behaviors in matching-to-sample. Journal ofExperimental Psychology: Animal Behavior Processses, 11,502–519.

Wright, A. A., & Sands, S. F. (1981). A model of detectionand decision processes during matching to sample bypigeons: Performance with 88 different wavelengthsin delayed and simultaneous matching tasks. Journal ofExperimental Psychology: Animal Behavior Processses, 7,191–216.

Received September 5, 2004Final acceptance March 29, 2005

APPENDIX A

MODEL PARAMETERS AND TERMS

Components of Conditional DiscriminationsS1, S2 Sample stimuli in matching to sample or signal detectionC1, C2 Comparison stimuli in matching to sampleB1, B2 Responses defined by comparison stimuli or topography, represented as counts in the

conditional-discrimination matrix of Figure 1.R11, R22 Numbers of reinforcers for B11, B22

Model Structuredsb Discriminability of stimulus-behavior relation. Depends on S1-S2 difference, sensory

capacity; does not depend on reinforcer rate or allocationdbr Discriminability of behavior-reinforcer contingency. Depends on B1-B2 or C1-C2

difference, sensory or motor capacity; does not depend on reinforcer rate or allocationp(As) Probability of attending to S1 and S2. Depends on reinforcer rate relative to session

context; does not depend on reinforcer allocation, dsb , or dbr

p(Ac) Probability of attending to C1 and C2. Depends on within-trial reinforcer rate relative tocontext; does not depend on reinforcer allocation, dsb , or dbr

Momentum EquationsB Measured response rate (B/min)rs Component reinforcer rate in multiple free-operant schedules or reinforcer rate for

attending to S1 and S2

rc Within-trial reinforcer rate after offset of sample stimuli for attending to C1 and C2

ra Overall average session reinforcer rate; ra 5 1 for single free-operant schedulesx Background disruption or competition for responding and for attending to sample

stimuliz Background disruption or competition for attending to comparison stimulif, v, q, Parameters representing the effects of experimentally arranged disrupters:c, d prefeeding, ICI food, delay, contingency termination, and generalization decrement.b Sensitivity of changes in responding or in attending to values of rs/ra or rc/rs


APPENDIX B

Method for calculating predicted log D for the rich and lean components in the study byNevin et al. (2003), assuming dsb 5 dbr 5 150 (see Figure 4).

Rich LeanR11 R12 R21 R22 R11 R12 R21 R22

Scheduled reinforcers 96 0 0 96 24 0 0 24Effective reinforcers 96 1.28 1.28 96 24 0.32 0.32 24

Response probabilities p(B1|S1) p(B1|S2) p(B1|S1) p(B1|S2)

State 1 (attend to S1/S2,attend to C1/C2) .987 .013 .987 .013

State 2 (attend to S1/S2,no attend to C1/C2) .500 .500 .500 .500

State 3 (no attend to S1/S2, attendto C1/C2) .500 .500 .500 .500

State 4 (no attend to S1/S2,no attend to C1/C2) .500 .500 .500 .500

Reinforcement rates required to calculate p(As) and p(Ac) from Equations 5 and 6 for rich andlean components with VI 30-s schedules, 2-s samples, 1-s latencies to comparisons, four trials percomponent separated by 30-s intercomponent intervals:

Rich LeanRft/hr for attending to S1/S2 (rs) 90 22.5Rft/hr for attending to C1/C2 (rc) 2,880 720Session average rft/hr (ra) 44.4 44.4

Parameters: x 5 0.1, z 5 0.0, b 5 0.5

Rich LeanCalculated probabilities p(As) p(Ac) p(As) p(Ac)

of attending .932 1.00 .869 1.00

Rich LeanWeighted probabilities p(B1|S1) p(B1|S2) p(B1|S1) p(B1|S2)

of responding .954 .046 .923 .077

Predicted log D Rich Leanwith x 5 0.1, z 5 0.0 1.315 1.079with x 5 0.3, z 5 0.0 .927 .657with x 5 0.1, z 5 1.0 .866 .769

a Note that relative to log D with x 5 0.1, z 5 0, increasing x reduces log D rich less than log Dlean, whereas increasing z reduces log D rich more than log D lean.


A Theory Of Attending And Reinforcement In Conditional Discriminations

Documents