Canadian Journal of School Psychology 2000 Stage 67 84

Canadian Journal of School Psychology

The Validity ofFunctionalBehavioral Assessment withStudents ofAverage IntellectualAbilit

Scott A. StageUniversity of Washington

In the United States, functional behavioral assessment (FBA) is a new requirementofthe 1997Amendments to Individuals with Disabilities Education Actfor studentswho exhibit challenging behavior. FBA has been extensively studied with personswith severe andprofound intellectual disabilities within clinical settings. However,there is limited research with students with average intellectual ability in publiceducation settings. This article reviews seven pertinent validity indices associatedwith FBA: external validity, content validity, convergent validity, criterion validity,treatment validity, social validity, and habilitative validity. In short, much researchis needed to inform practitioners about conducting valid FBAwith students withaverage intellectual ability.Aux Etats-Unis, I'evaluation du comportement fonctionnel (ECP) (FunctionalBehavioral Assessment FBA) est une nouvelle exigence de la Loi constitutionnellede 1997 sur les individus ayant une deficience et qui demontrent un comportementa risque. Le ECP a ete, etudie dans un environnement clinique et d'unefa,on tresapprofondie avec des personnes d6montrant des troubles de deficienceintellectuelle. Cependant dans le milieu de l'education publique, il n'y a que trWspeu de recherches et surtout elles sont tres limitees avec des etudiants ayant uneintelligence moyenne. Cet article passe en revue sept indices valides, associesavec le ECP: validite externe, contenu, convergence, criteres, validite (r6sultat ?)du traitment, validite sociale et validite de 1'habilete (?). En resume, il y a unbesoin pour plus de recherche dans le domaine du ECP afin que les praticienspuissentfaire un suivi tres approfondi du ECP avec des etudiants ayant une habileteintellectuelle moyenne.

In the United States, the 1997 amendments to the Individuals with DisabilitiesEducation Act (IDEA '97) requires that students with disabilities who exhibitchallenging behavior have a proactive behavior management plan included intheir individual educational plan (IEP) based on a functional behavioralassessment (Yell, 1998; Yell & Shriner, 1997). There is exceptional empiricalsupport for the use of functional behavioral assessment (FBA) with personswith severe and profound intellectual disabilities in clinical settings (Blakeslee,Sugai, & Gruba, 1994; Derby et al., 1992; Gresham, Quinn, & Restori, 1999;Iwata et al., 1994; National Institute of Health, 1989). However, the validityof FBA with students with average intellectual ability in public education

Volume 15, Number 2, 2000, 67 84

at Alexandru Ioan Cuza on June 3, 2015cjs.sagepub.comDownloaded from

settings has not been established (Fox, Conroy, & Heckaman, 1998; Gable,1996, 1999; Gresham, Quinn, & Restori, 1999; Nelson, Roberts, Mathur, &Rutherford, 1999; Scott & Nelson, 1999).

Research on the validity of FBA is necessary to inform the field, soappropriate procedures and methods can be uniformly used with studentswith average intellectual ability within public education settings (Gable,1996, 1999; Gresham, Quinn, & Restori, 1999). In the United States, schoolpsychologists, special educators, and general educators are required toprovide valid assessments in keeping with the standards set forth by theAmerican Psychological Association (1992), the National Association ofSchool Psychologists (1992), and the American Educational ResearchAssociation's National Council on Measurement in Education (1985).Establishing valid FBA procedures protects consumers (i.e., parents, students,educators, and school psychologists) by providing procedures that are likelyto result in efficacious treatment.

Functional Behavioral Assessment ProceduresCone (1997) defines FBA as activities involved in formulating hypothesesabout potentially controlling environmental variables that influence behavior.The environmental variables that influence the behavior are determined bythe assessment of setting events, antecedents, and consequences that reliablypredict the occurrence and maintenance of the behavior (Bijou, Peterson, &Ault, 1968). Setting events are factors that precede the behavior in time buthave a strong influence on the occurrence of the behavior (Wahler & Fox,1981). For instance, the biological state of sleep deprivation can influence astudent's alertness and subsequent performance in school (O'Reilly, 1995).Antecedents are discriminative stimuli that signal a high probability thatthe behavior will be rewarded or punished. For instance, a teacher's proximityto a student can act as a stimulus to reduce the student's disruptive behavior(Dunlap et al., 1993). Consequences are events that follow a behavior thatare likely to maintain it through reinforcement. For instance, for somestudents, teacher approval for appropriate classroom behavior will increasethe likelihood that the students will comply with classroom rules (Becker,Madsen, Arnold, & Thomas, 1967).

The process of conducting a FBA is described in three separate phases(e.g., Carr, Levin, McConnachie, Carlson, Kemp, & Smith, 1994; Cone, 1997;Kern, Childs, Dunlap, Clarke, & Falk, 1994; Horner & Carr, 1997; O'Neill,Horner, Albin, Sprague, Storey, & Newton, 1997). (a) The descriptive phaserequires gathering information about the problem behavior within theenvironmental context that it occurs. (b) The interpretative phase requires

68 S. A. Stage


Functional Behavioral Assessment

generating hypotheses about the setting events and antecedents that predictthe occurrence of the behavior as well as the consequences that maintain thebehavior in the setting that the behavior occurs. (c) The verification phaserequires determining the causal relationship between these variables andtheir influence on the behavior using an experimental single-subject design.The following describes pertinent validity measures associated with eachphase of FBA. However, prior to this discussion, the limited researchconducted with FBA and students with average intellectual ability isdiscussed.

External ValidityValidity is the overall evaluation of the degree to which empirical evidencesupports the use of the assessment procedures used (Messick, 1995). Externalvalidity is the degree to which results derived from certain settings andpopulations generalize to other settings and populations (Campbell & Cook,1976). Almost all of empirical evidence for the use of FBA comes fromstudies involving individuals with severe or profound intellectual disabilitieswho exhibit self-injurious and/or disruptive behavior in clinic settings(Blakeslee, Sugai, & Gruba, 1994; Derby et al., 1992; Iwata et al., 1994;Nelson, Roberts, Mathur, & Rutherford, 1999). Of interest is whether FBAcan be successfully applied to students with other disabilities in publiceducation settings (Gresham, Quinn, & Restori, 1999; Nelson et al., 1999).Another consideration in regard to external validity is the response class ortarget behavior of the intervention (Gresham, Quinn, & Restori, 1999).Because IDEA '97 describes the use of FBA, in part, for school disciplinarypractices, external validity needs to be established with disruptive behaviorin public education settings. In the most extensive meta-analysis to date ofintervention studies directed at disruptive behavior in public educationsettings, Stage and Quiroz (1997) found only 6 studies that included 11students investigating the use of FBA-based interventions. They found thatall of the subjects in these studies were students in elementary school, fivewere general education students, three were students with mild mentalretardation, and three were students with emotional disturbance. Because29% of the students served with a mild disability in the United States arefrom 12 to 17 years of age (U.S. Department of Education, 1994), researchwith this age group is needed in order to determine what FBA procedureswork best with these students. None of the students in the meta-analysiswere reported to be of color or ethnically diverse. The lack of research withpersons of color is particularly problematic because 24% of the studentsserved with mild disabilities in the United States are African-American and

69


8% are Hispanic students (U.S. Department of Education, 1994). Researchis needed to devise culturally sensitive FBA procedures. Overall, there islimited support for the external validity of FBA procedures with studentswith average intellectual ability in public education settings. Inclusion ofethnic diversity is missing, and there is no representation of students inmiddle or high school settings.

Validity Measures for the Descriptive PhaseThe descriptive phase requires gathering information about the environmentalvariables that purportedly control the problematic behavior. This requirescollecting a representative sample of behavior from the domains of interestthat purportedly affect the behavior. In traditional psychological test theory,this requires that the instruments used have content validity (Anastasi, 1976;Cone, 1997; Linehan, 1980; Messick, 1988, 1995). FBA procedures useinterviews, rating scales, and descriptive observations to derive hypothesesabout the setting events, antecedents, and consequences that predict theoccurrence and maintenance of the behavior (Tobin, 1994). Therefore, theseinstruments need to sample behavior and conditions that act as setting events,antecedents, and consequences for the problem behavior of concern. Theimportance of determining adequate representation of the environmentalvariables or establishing content validity is particularly important for thedevelopment of proactive behavioral support plans for students apt toencounter disciplinary actions (e.g., Clarke, Dunlap, Foster-Johnson, Childs,Wilson, White, & Vera, 1995; Dunlap et al., 1993; Dunlap & Kern, 1996;Umbreit, 1995).

The content validity of interviewsInterviewing parents and teachers provides valuable information about thevariables that predict or maintain problem behaviors in classroom settings(Kern, Dunlap, Clarke, & Childs, 1994; O'Neill, Homer, Albin, Sprague,Storey, & Newton, 1990, 1997). The most widely cited and researchedinterview is the Functional Assessment Interview Form (e.g., Anderson,Freeman, & Scotti, 1999; Kern, Childs, Dunlap, Clarke, & Falk, 1994;Sturmey, 1994; Vollmer, Marcus, Ringdahl, & Roane, 1995) which wasdeveloped by O'Neill, Horner, Albin, Sprague, Storey, and Newton (1990,1997). The thoroughness of the Functional Assessment Interview Form(1997) and the open-ended nature of questions suggest that this interviewcovers the possible setting events, antecedents, and consequences to insurecontent validity.

70 S. A. Stage



Another interview used with students' demonstrating behavioral problemsis the Student-Directed Functional Assessment Interview Form (O'Neill,Horner, Albin, Sprague, Storey, & Newton, 1997). The thoroughness of thisinterview and design are similar to the format used with adult informantsand suggests good content validity.

The content validity of rating scalesRating scales are also used in FBA (Durand & Crimmins, 1988a; Lewis &Sugai, 1994). The most widely used and researched rating scale is theMotivational Assessment Scale (e.g., Crawford, Brockel, Schauss, &Miltenberger, 1992; Derby et al., 1992; Durand & Crimmins, 1988b, 1989;Sturmey, 1994; Zarcone, Rodgers, Iwata, Rourke, & Dorsey, 1991) whichwas developed by Durand and Crimmins (1988a). The_MotivationalAssessment Scale (1988) is a 16-item scale administered to teachers. Theitems address four maintaining consequences of the problem behavior,sensory stimulation, escape, attention, and tangible reinforcement. Therefore,this scale does not assess the setting events or antecedents that predict theoccurrence of the problem behavior so the content validity is lacking inregards to the assessment of setting events and antecedents.

The Problem Behavior Questionnaire (Lewis & Sugai, 1994) wasdeveloped to use with students in general education. There are 15 items onthis scale that are rated by the student's teacher. The teacher rates each itemon a seven-point Likert scale that represents the proportion of the time thatthe behavior is problematic. The items indicate whether the problem behavioris maintained by peer attention, teacher attention, escape from peer attention,or escape from teacher attention. In addition, some items address settingevents. Although this rating scale does not address antecedents, therepresentation of the environmental variables that influence the problembehavior are more thorough than the Motivation Assessment Scale (1988).

The content validity of descriptive behavioral observationsIn addition to interviews and rating scales, there are different types ofbehavioral observational methods that describe the behavior in the naturalenvironment that it occurs. For example, scatter plots (Touchette,MacDonald, & Langer, 1985) are used to identify settings and times in whichproblematic behavior occurs. However, this technique does not sufficientlyrepresent antecedents or consequences (Axelrod, 1987; Lerman & Iwata,1993). Abbreviated forms that chart the antecedents, behavior, andconsequences over time intervals are also used (Bijou, Peterson, & Ault,1968; O'Neill, Horner, Albin, Sprague, Storey, & Newton, 1997). The

71


Functional Assessment Observation Form (O'Neill, Horner, Albin, Sprague,Storey, & Newton, 1997) is a form that the teacher uses to record thefrequency of antecedents, behaviors of concern, and consequences across aweek using predetermined time intervals. This format of FBA isrepresentative of the environmental variables of interest and suggests goodcontent validity.

Another form of descriptive analysis uses time lag analysis. This techniquedetermines the conditional probability of the behavior occurring given theantecedents that precede the behavior and consequences that maintain it(Emerson, Reeves, Henderson, Robertson, & Howard, 1996; Gunter, Jack,Shores, Carrell, & Flowers, 1993; Lerman & Iwata, 1993; Shores, Jack,Gunter, Ellis, DeBriere, & Wehby, 1993). Time lag analysis has beenconducted with laptop computers in real time (e.g., Emerson, Reeves,Henderson, Robertson, & Howard, 1996; Gunter, Jack, Shores, Carrell, &Flowers, 1993) and with partial interval continuous behavior observationprocedures (Lerman & Iwata, 1993; Mace & Lalli, 1991). Time lag analysissystematically evaluates antecedents, problem behaviors, and consequencesalthough setting events are not systematically evaluated in this procedure.Yet, the setting event would be determined prior to conducting theobservation.

In summary of the content validity of the FBA instruments described,the Functional Assessment Interview Form (1997) and the Student-DirectedFunctional Assessment Interview Form (1997) systematically coverinformation about setting events, antecedents, and consequences that predictand maintain problem behaviors suggesting adequate content validity. Bothof the rating scales, the Motivational Assessment Scale (1988) and theProblem Behavior Questionnaire (1994) are limited in the representation ofthe environmental variables covered. The Motivational Assessment Scale(1988) covers consequences that maintain problem behavior but itemsregarding setting events and antecedents are not represented. The ProblemBehavior Questionnaire (1994) covers both setting events and consequencesalthough the relation between antecedents and the problem behavior is notaddressed. The descriptive behavioral observation methods, the FunctionalAssessment Observation Form (1997) and time lag analysis, coverantecedents and consequences that maintain problem behavior. TheFunctional Assessment Observation Form (1997) also establishes settingevents with the coverage of multiple situations that the student encounters.Taken together, although there are FBA instruments that appear to haveadequate content validity, further research is needed to determine whichinstruments offer the best coverage of setting events, antecedents, problem

72 S. A. Stage



behaviors, and consequent events for students with average intellectualability.

Validity Measures for the Interpretative PhaseIn the interpretative phase of FBA, the information gathered during thedescriptive phase is evaluated to formulate hypotheses about the relationshipof the environmental variables with the problem behavior (Cone, 1997). Theimportant measurement dimension at this phase is the agreement betweenthe different methods and informants to ensure validity. Convergent validityis established by determining the level of agreement between the differentassessments and informants used to generate the functional behavioralhypotheses (Cone, 1997; Gresham & Noell, 1998). In the case where theFBA yields disparate data about the setting events, antecedents, andconsequences, generation of a functional hypothesis based on thisinformation would lack convergent validity due to lack of agreement betweenthe assessment methods used. The studies reviewed below represent the bodyof work applicable to convergent validity and school-aged children.

The convergent validity of interviewsReed, Thomas, Sprague, and Horner (1997) investigated the agreement

between 10 middle-school students and their teachers on the Student-DirectedFunctional Assessment Interview Form (1997). Results of this investigationshowed the agreement between student and teacher was 26% for settingevents, 77% for antecedent events, 85% for problem behaviors, and 77% formaintaining consequences, with an overall agreement of 60%. This resultsuggests an overall agreement that is low although the agreement with settingevents influenced the overall agreement to a large extent. In addition, theinterview also yielded a hypothesis statement pertaining to the setting event,antecedent, and consequence identified in the interviews. Results showed22% agreement between the students and teachers for the full functionalbehavioral hypothesis statement. When only the antecedent and maintainingconsequence of the behavior were considered, students and teachers agreedon 65% of the statements. Sixty-five percent agreement on the functionalbehavioral hypotheses is considered too low to ensure convergent validity.The interview also provides a section that determines a behavioral supportplan to reduce the occurrence of the problem behavior. Overall agreementfor the behavioral support plan was 38%. The authors suggested that theoverall lack of agreement for the behavioral support plans were due tostudents' biases in regard to consequent management as the primary strategy

73


to alter the occurrence of problem behavior. The convergent validity betweenteachers' and students' rating of the environmental variables, functionalhypotheses, and behavioral support plans are too low to consider theinformation derived from this instrument valid.

The convergent validity of interviews, rating scales, and descriptivebehavioral analyses

The convergent validity of the Functional Analysis Interview Form (1990),the Motivation Assessment Scale (1988), and the Functional AnalysisObservation Form (1990) was reported by Arndorfer, Miltenberger, Woster,Rortvedt, and Gaffaney (1994). This study included five children, four ofwhich were five years of age or less, and a 13 years old with Down'ssyndrome. Parents participated in the functional interview, collected dataon the antecedent, behavior, and consequences. The percentage of agreementas to the maintaining function of the problem behavior was 80%. The onlyinstrument that reliably differed from the other instruments was theMotivation Assessment Scale (1988). When it was removed from thecalculation of agreement, the percentage rose to 93%. This result indicatesadequate convergent validity in regard to the maintaining function of thebehavior although the instruments were not used to assess the setting eventsor antecedents so interventions based on manipulation of setting events orantecedents would not be established.

In summary, two studies investigating the convergent validity of FBAwere reviewed. Reed, Thomas, Sprague, and Homer (1997) reported lowconvergent validity with teacher and student agreement with the Student-Directed Functional Assessment Interview Form (1997) on the antecedents,problem behavior, and maintaining consequences. The agreement betweenthe teachers and students in regard to the positive behavioral support wasquite low. An 80% agreement in the identification of the maintainingconsequence was found when the Functional Analysis Interview Form (1990),the Motivation Assessment Scale (1988), and the Functional AnalysisObservation Form (1990) were used to define the maintaining consequences(Arndorfer et al., 1994). The agreement in the identification of themaintaining consequences rose to 93% when the Motivation Assessment Scale(1988) was eliminated from the analysis. However, Arndorfer et al. (1994)only assessed consequent events. These studies suggest that convergentvalidity has not been adequately demonstrated and further research is neededto determine the convergent validity of the different types ofFBA procedures(i.e., interviews, rating scales, and descriptive analyses) in regard to

74 S. A. Stage



identifying setting events, antecedents, problem behaviors, andconsequences.

Validity Measures for the Verification PhaseVerifying the causal relationship between the environmental variablesidentified in the FBA procedures and the problem behavior is referred to asthe verification phase (Cone, 1997). There are two validity issues in regardto the verification phase of FBA: criterion validity and treatment validity.

Criterion validity andfunctional behavioral assessmentCriterion validity refers to how well the performance on one assessmentcorresponds to the performance on another assessment of interest (Ghiselli,Campbell, & Zedeck, 1981). In FBA, criterion validity is achieved when theexperimental manipulation of the identified environmental variables resultsin a change in the behavior of concern. Clear differences in the pattern ofbehavior response to the experimental manipulations suggest functionalcontrol of the behavior (Cook & Campbell, 1979; Kazdin, 1982). However,because single-subject data is most often analyzed via visual inspection ofdata, subjective judgements can affect the purported significance of thetreatment effect (e.g., DeProspero, & Cohen, 1979; Jones, Weinrott, &Vaught, 1978). Therefore, some researchers have derived statistical tests toanalysis single-subject data (Busk & Marascuilo, 1992; Center, Skiba, &Casey, 1985-1986; Crosbie, 1993; Edgington, 1992; Levin & Wampold,1999).

Four statistical considerations are required in analyzing single-subjectdata: the variability of the data, the change in level between the baselineand treatment phases, thechange in slope between baseline and treatmentphases,and the degree of autocorrelation between successive data points(Center, Skiba, & Casey, 1985-1986; Crosbie, 1987; 1993; Kazdin, 1982).Autocorrelation affects the statistical analysis of the data across the repeatedmeasures because each successive datum point is correlated withthepreceding datum point. Statistical analysis without regard to autocorrelationresults in increased error in the statistical significance (Crosbie, 1987; 1993).FBA hypotheses have been verified with subsequent treatment

manipulations in school settings using visual inspection of single-subjectdata for students with various problems (Broussard & Northup, 1995; Dunlap,dePerczel, Clarke, Wilson, Wright, White, & Gomez, 1994; Dunlap, Kern-Dunlap, Clarke, & Robbins, 1991; Lewis & Sugai, 1996; Repp & Karsh,1994; Umbreit, 1995). However, Martin, Gaffan, and Williams (1999) haveshown that different criteria set forth by researchers yield unreliable levels

75


of statistical significance. Rigorous statistical analysis of single-subject datais required to determine criterion validity.

Treatment validity andfunctional behavioral assessmentTreatment validity refers to the degree that FBA provides beneficial

treatment outcomes compared to other methods of treatment (Cone, 1997;Gresham & Noell, 1998; Hayes, Nelson, & Jarrett, 1987). This requirestransforming single-subject data into an effect size so comparisons betweenstudents in different treatment conditions can be statistically analyzed. Aftereach treatment outcome is transformed into an effect size, the statisticaldifference between the treatment effectiveness of different interventions canbe calculated using meta-analysis (Busk & Serlin, 1992).

Two meta-analytic studies comparing the treatment effects of interventionsderived from FBA and other interventions are described. In the first study,Schill, Kratochwill, and Elliott (1998) compared the treatment effectivenessof FBA to the treatment effectiveness of a self-help technical assistanceprocedure using a behavioral consultation model (BC; Bergan & Kratochwill,1990). In this study, 19 Head Start students were randomly assigned to aFBA consultation condition or a BC condition. In the FBA condition, theconsultants performed a problem identification interview (Bergan &Kratochwill, 1990) with the teacher and parent to identify the problembehavior and environmental conditions surrounding the behavior. Followingthe problem identification interview, the consultants administered theMotivation Assessment Scale (1988) to the teacher and also conducted anantecedent-behavior-consequence descriptive analysis using the FunctionalAnalysis Observation Form (1990). The consultants used the informationtaken from these data sources to develop a preliminary functional behavioralhypothesis. Then the consultant conducted a problem analysis interview(Bergan & Kratochwill, 1990) with the teacher to develop a treatment planbased on the FBA data. The teacher implemented the intervention. In theBC condition, the consultant conducted the problem identification interviewand behavioral observations in the classroom setting. Based on thisinformation, the teachers were given a self-help manual designed to providepractical classroom interventions that the teacher could use independently.

Treatment outcome was evaluated using an AB design (baseline followedby intervention). Effect size was calculated using the mean differencebetween the baseline and treatment phases divided by the baseline standarddeviation. The average effect size for the FBA condition was .84 and theaverage effect size for the BC condition was .52. However, the effect sizedifference was not statistically significant. Although the difference in the

76 S. A. Stage



magnitude of these effect sizes would suggest a significant difference, thelack of power due to the limited number of subjects and the individualvariability across the different treatments decreased the probability of findingstatistical significance. In addition, analyzing the data with the meandifference between the baseline and treatment phases does not adequatelyaddress the autocorrelation effect or change in slope.

Stage and Quiroz (1997) conducted a meta-analysis of 99 studies usingdifferent treatments to reduce disruptive classroom behavior. They used theInterrupted Time Series Autocorrelation statistical program (ITSACORR;Crosbie, 1993) with the single-subject data. The ITSACORR programprovides a statistical test for the change in level between baseline andintervention phases, a statistical test for the change in slope between thebaseline and intervention phases, as well as controlling for autocorrelation.Stage and Quiroz (1997) converted the t-score of the change in level fromeach single-subject analysis to an effect size using a formula derived byGlass, McGaw, and Smith (1981) after checking for a significant change inslope. Stage and Quiroz (1997) obtained 223 effect sizes. They tested thedifference in treatment effectiveness between 16 different interventions.When the magnitude of the effect sizes is rank-ordered across the 16

different interventions described, FBA-based interventions (ES = .51) rankedthirteenth out of sixteen in its level of effectiveness. This descriptive statisticindicates that many other interventions demonstrated a stronger influenceon the reduction of disruptive classroom behavior than the interventionsbased on FBA. The three interventions that approached one standarddeviation above the baseline condition were group contingencies (ES = 1.02),self-management (ES = .97), and differential reinforcement (ES = .95).However, the only statistically significant difference was between groupcontingencies (i.e., ES = 1.02) and cognitive-behavioral interventions (ES= .36). The meta-analysis results indicate that there is a great deal ofvariability with regard to treatment effectiveness of interventions used todecrease disruptive behavior. However, FBA has not provided greatertreatment effectiveness than other behavioral interventions usingcontingency-based reinforcement.

Taken together, the results of two studies using meta-analytic techniquessuggest that FBA-based interventions have treatment validity although theseprocedures have not evidenced greater changes in problem behavior thanother interventions. However, the number of studies using FBA with studentsin public education is quite limited at this time. Further research is requiredto determine the treatment validity with a representative group of studentswith average intellectual ability.

77


Social and Habilitative ValiditySocial validity refers to relevance of the goals of an intervention, the

acceptability of the intervention procedures, and the satisfaction of the effectsproduced by the intervention (Wolf, 1978). The most typical form ofassessment of social validity is a questionnaire or rating scale given afterthe program has been implemented. Some researchers have questioned theauthenticity of these types of assessments due to the social demands placedon the recipients of the services (Schwartz & Baer, 1991). Schwartz (1991)described the importance of studying consumer behavior as a decision-makingprocess. Authentic assessment would include assessing theacceptability of each phase of the FBA process with verbal statements madeduring the discussion of the procedures by parents, teachers, and students inaddition to paper-and-pencil assessment. During the descriptive phase, allconsumers should be involved in gathering information about the problembehavior. During the interpretative phase, consumers should providehypotheses about the predicting events and maintaining consequences ofthe problem behavior. Finally, during the verification phase consumers shouldevaluate the outcomes.

Predominantly, FBA research has evaluated teachers' acceptability of theinterventions used in their classrooms (Clarke et al., 1995; Sasso et al., 1992;Umbreit, 1995), measuring teacher acceptance at the verification phase. Onestudy investigated teachers' acceptance during the descriptive phase(Broussard & Northup, 1995). One study evaluated both teacher and students'rating of the behavior problem and intervention used, thus, assessing thesocial validity at the verification phase (Umbreit, 1995). In summary,although FBA procedures do not appear to lack social validity, the assessmenthas predominately relied on teacher rating-scales given during theverification phase of the FBA. Parents have not been involved in the decision-making process towards the development of the behavioral support plan,and therefore, the authenticity of these ratings may be questionable. It isextremely important to include parents in this process. Research isdesperately needed in this area to insure parents, teachers, and students aregiven an active role in determining the positive behavioral support plansused.

In addition to social validity, Hawkins (1991) described the concept ofhabilitative validity suggesting that goals, procedures, and outcomes shouldbe evaluated in terms of a broader context than simply a change in the targetbehavior. The habilitative validity of a treatment would suggest that theclients' success further impacted their functioning at school, at home, andin the community. In regards to FBA, reduction in problem behavior should

78 S. A. Stage



also evidence a reduction in disciplinary actions taken with the student. Byproviding adequate positive behavioral supports, the student's problematicbehavior would be reduced in all areas of the school community, therefore,reducing the number of disciplinary actions taken with him or her. Therewere no studies found that speak to the habilitative validity of positivebehavioral support plans. Research is needed to address the associationbetween FBA and discipline actions taken with students.

ConclusionThe validity of FBA with students with average intellectual ability wasreviewed. The effectiveness of FBA procedures is well established withpersons with severe and profound intellectual disabilities (Blakeslee, Sugai,& Gruba, 1994; Derby et al., 1992; Gresham, Quinn, & Restori, 1999; Iwataet al., 1994). However, the generalization of these procedures to studentswith average intellectual ability in public education settings is limited.Students of color are absent in these studies, and students in middle andhigh school are not represented. Further research with these populationswithin public education settings is needed to establish the external validityof these procedures.

The three phases of FBA were evaluated in terms of validity indicesrequired at each phase. The descriptive phase requires use ofFBA instrumentsthat cover the entire content of the behavior domains of interest, namely,setting events, antecedents, problematic behaviors, and consequences.Commonly used interviews, rating scales, and descriptive behavioralobservation techniques were reviewed. Interviews and descriptive behavioralobservation methods appear to offer the most complete coverage, andtherefore, possess content validity. However, further research should beconducted to develop other instruments. Practitioners would benefit fromhaving numerous instruments with content validity.

During the interpretive phase, functional behavioral hypotheses aredevised using the instruments used in the descriptive phase. Convergentvalidity of the hypotheses derived from the different descriptive instrumentsand different informants ensures agreement about the setting events andantecedents that predict the occurrence of the problem behavior as well asthe consequences that maintain the problem behavior. At this time, there noevidence to suggest that convergent validity has been established withdifferent informants using different methods. Research is desperately neededin this area. With little agreement between the participants using differentFBA procedures, capricious hypotheses are likely to result, rendering theassessment process useless.

79


In the verification phase, criterion validity has typically been establishedusing visual inspection of single-subject data (Kazdin, 1982). In the casewhere there is a robust difference in the level of behavior between the baselineand treatment conditions, this will suffice. However, many single-subjectdata patterns are not so obvious. Use of statistical procedures such as theInterrupted Time Series Autocorrelation statistical program (ITSACORR;Crosbie, 1993) can provide a rigorous statistical test of establishing that thecriterion has been met. In addition to criterion validity, treatment validitywas discussed in terms of the effects of FBA-based interventions versusother interventions. At this time, for students with average intellectual ability,interventions based on contingency management without prior FBA are morerobust. However, these results are based on a limited number of studies.Further research will undoubtedly determine the utility of FBA-basedinterventions with this population.

In addition to the validity indices reviewed in the three phases of FBA,social validity and habilitative validity were discussed. Given the limitedresearch to guide the practice of conducting FBA with students with averageintellectual ability, it would appear that the most ethical practice wouldincorporate social validity measures during the FBA process. As discussedby Schwartz (1991), incorporating the consumer in the decision makingprocess ensures social validity. During each phase of the FBA, the parents,student, and teacher would actively be engaged in the description of theproblem, the interpretation of the description, and the verification that thepositive behavioral support plan is providing in the desired outcome. Theultimate goal of a FBA-based positive behavioral support plan is to increasethe student's behavioral functioning at school, home, and community, andthus demonstrate habilitative validity. In regard to IDEA '97, FBA-basedpositive behavioral support plans should evidence a reduction in the numberof disciplinary actions taken with the students who receive services basedon FBA.

ReferencesAmerican Educational Research Association and National Council on Measurement in Education (1985).

Standards for educational and psychological tests. Washington DC: American PsychologicalAsociation.

American Psychological Association Ethics Committee (1992). Ethical principles of psychologists andcode of conduct. American Psychologist, 47, 1597-1611.

Anastasi, A. (1976). Psychological testing (4" ed.). NY: MacMillian.Anderson, C. M., Freeman, K. A., & Scotti, J. R. (1999). Evaluation of the generalizability (reliability

and validity) of analog functional assessment methodology. Behavior Therapy, 30, 31-50.Arndorfer, R. E., Miltenberger, R. G., Woster, S. H., Rortvedt, A. K., & Gaffaney, T. (1994). Home-

based descriptive and experimental analysis of problem behaviors in children. Topics in EarlyChildhood Special Education, 14, 64-87

80 S. A. Stage


Axelrod, S. (1987). Functional and structural analyses of behavior: Approaches leading to reduced useof punishment procedures? Research in Developmental Disabilities, 8, 165-178.

Becker, W. C., Madsen, C. H., & Arnold, C. R. (1967). The contingent use of teacher attention andpraise in reducing classroom behavior problems. Journal of Special Education, 1, 287-307.

Bergan, J. R. & Kratochwill, T. R. (1990). Behavioral consultation and therapy. New York, NY: PlenumPublishing Corporation.

Bijou, S. W., Peterson, R. F., & Ault, M. H. (1968). A method to integrate descriptive and experimentalfield studies at the level of data and empirical concepts. Journal ofApplied Behavior Analysis, 1,175-191.

Blakeslee, T., Sugai, G., & Gruba, J. (1994). A review of functional assessment use in data-basedintervention studies. Journal of Behavioral Education, 4, 397-413.

Broussard, C. D. & Northup, J. (1995). An approach to functional assessment and analysis of disruptivebehavior in regular education classrooms. School Psychology Quarterly, 10, 151-164.

Busk, P. L., & Maracuilo, L. A. (1992). Statistical analysis in single-case research: Issues, procedures,and recommendations, with applications to multiple behaviors. In T. R. Kratochwill and J. R. Levin(Eds.0), Single-case research design and analyses (pp. 159-185). Hillsdale, NJ Lawrence Erlbaum.

Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill and J.R. Levin (Eds.), Single-case research design and analyses (pp. 187-212). Hillsdale, NJ: LawrenceErlbaum.

Carr, E. G., Levin, L., McConnachie, G., Carlson, J. I., Kemp, D. C., & Smith, C. E. (1994).Communication-based intervention for problem behavior. Baltimore, MD: Brookes Publishing, Co.

Center, B. A., Skiba, R. J., & Casey, A. (1985-86). A methodology for the quantitative synthesis ofintra-subject design research. Journal of Special Education, 19, 387-400.

Clarke, S., Dunlap, G., Forster-Johnson, L., Childs, K. E., Wilson, D., White, R., & Vera, A. (1995).Improving the conduct of students with behavioral disorders by incorporating student interests intocurricular activities. Behavioral Disorders, 20, 221-237.

Cone, J .D. (1997). Issues in functional analysis in behavioral assessment. Behavioral Research &Therapy, 35, 259-275.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design an analysis issues for fieldsettings. Boston, MA: Houghton, Mifflin.

Crawford, J., Borkel, B., Schauss, S., & Miltenberger, R. G. (1992). A comparison of methods for thefunctional assessment of stereotypic behavior. Journal of the Association for Persons with SevereHandicaps, 17, 77-86.

Crosbie, J. (1987). The inability of the binomial test to control type I error with single-subject data.Behavioral Assessment, 9, 141-150.

Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal ofConsulting & Clinical Psychology, 61, 966-974.

DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal ofApplied Behavior Analysis, 12, 573-579.

Derby, K. M., Wacker, D. P., Sasso, G., Steege, M., Northup, J., Cigrand, K., & Asmus, J. (1992). Brieffunctional assessment techniques to evaluate aberrant behavior in an outpatient setting: A summaryof 79 cases. Journal ofApplied Behavior Analysis, 25, 713-721.

Dunlap, G., de-Perczel, M., Clarke, S., Wilson, D., Wright, S., White, R., & Gomez, A. (1994). Choicemaking to promote adaptive behavior for students with emotional and behavioral challenges.Journal ofApplied Behavior Analysis, 27, 505-518.

Dunlap, G., & Kern, L. (1996). Modifying instructional activities to promote desirable behavior: Aconceptual and practical framework. School Psychology Quarterly, 11, 297-312.

Dunlap, G., Kern, L., de-Perczel, M., Clarke, S., Wilson, D., Childs, K. E., White, R., & Falk, G. D.(1993). Functional analysis of classroom variables for students with emotional and behavioraldisorders. Behavioral Disorders, 18, 275-291.

Dunlap, G., Kern-Dunlap, L., Clarke, S., & Robbins, F. R. (1991). Functional assessment, curricularrevision and severe behavior problems. Journal ofApplied Behavioral Analysis, 24, 387-397.

Durand, V. M., & Crimmins, D. B. (1988a). The motivation assessment scale an administration manual.Albany, NY: State University of New York.

Durand, V. M., & Crimmins, D. B. (1988b). Identifying the variables maintaining self-injuriousbehavior. Journal ofAutism and Developmental Disorders, 18, 99-117.

Functional Behavioral Assessment 81


Edgington E. S. (1992). Nonparametric tests for single-case experiments. In T. R. Kratochwill and J. R.Levin (Eds.), Single-case research design and analyses (pp. 133-157). Hillsdale, NJ: LawrenceErlbaum.

Emerson, E., Reeves, D., Thompson, S., Henderson, D., Robertson, J., & Howard, D. (1996). Time-basedlag sequential analysis and the functional assessment of challenging behavior. Journal of IntellectualDisability Research, 40, 260-274.

Fox, J., Conroy, M., & Heckaman, K. (1998). Research issues in functional assessment of the challengingbehaviors of students with emotional and behavioral disorders. Behavioral Disorders, 24, 26-33.

Gable, R. A. (1996). A critical analysis of functional assessment: Issues for researchers and practitioners.Behavioral Disorders, 24, 36-40.

Gable, R. A. (1999). Functional assessment in school settings. Behavioral Disorders, 24, 246-248.Ghiselli, E. E., Campbell, J. P., & Zedeck,S. (1981). Measurement theory for the behavioral sciences. San

Francisco, CA: W. H. Freeman and Co.Glass, G. V., McGaw, B., & Smith, M. L.(1981). Meta-analysis in social research. Beverly Hills, CA:

Sage.Gresham, F. M., & Noell, G. H. (1998). Functional analysis assessment as a cornerstone for noncategorical

special education. In D. J. Reschly, W. D. Tilly, &, J. P. Grimes (Ed.s). Functional and noncategoricalidentification and intervention in special education (pp. 39-64). Des Moines: IA: Iowa State Dept. ofEd.

Gresham, F. M., Quinn, M. M., & Restori, A. (1999). Methodological issues in functional analysis:Generalizability to other disability groups. Behavioral Disorders, 24, 180-182.

Gunter, P. L., Jack, S. L., Shores, R. E., Carrell, E., & Flowers, J. (1993). Lag sequential analysis as atool for functional analysis of student disruptive behavior in classrooms. Journal of Emotional andBehavioral Disorders, 1, 138-148.

Hayes, S. C., Nelson, R.O., & Jarrett, R. B. (1987). The treatment utility of assessment: A functionalapproach to evaluating assessment quality. American Psychologist, 42, 963-974.

Haynes, S. N., & O'Brien, W. H. (1990). Functional analysis in behavior therapy. Clinical PsychologyReview, 10, 649-668.

Horner, R. H., & Carr, E.G. (1997). Behavioral support for students with severe disabilities: Functionalassessment and comprehensive intervention. Journal of Special Education, 31, 84-104.

Individuals with Disabilities Education Act Amendments of 1997, PL 105-17, 105th Congress,1st Session.Iwata, B. A., Pace, G. M., Dorsey, M. F., Zarcone, J. R., Vollmer, T. R., Smith, R. G., Rodgers, T. A.,

Lerman, D. C., Shore, B. A., Mazaleski, J. L., Goh, H-L., Cowdery, G. E., Kalsher, M. J., McCosh, K.C., & Willis, K. D. (1994). The functions of self-injurious behavior: An experimental-epidemiologicalanalysis. Journal of Applied Behavior Analysis, 27, 215-240.

Jones, R. R., Weinrott, M. R., & Vaught, R.S. (1978). Effects of serial dependency on the agreementbetween visual and statistical inference. Journal of Applied Behavior Analysis, 11, 277-283.

Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. NY: OxfordUniversity Press.

Kern, L. Childs, K. E., Dunlap, G., Clarke, S. & Falk, G. D. (1994). Using assessment-based curricularintervention to improve the classroom behavior of a student with emotional and behavioral challenges.Journal ofApplied Behavior Analysis, 27, 7-19.

Kern, L., Dunlap, G., Clarke, S., & Childs, K. E. (1994). Student-assisted functional assessmentinterview. Diagnostique, 19, 29-39.

Lerman, D. C., & Iwata, B.A. (1993). Descriptive and experimental analyses of variables maintainingself-injurious behavior. Journal ofApplied Behavior Analysis, 26, 293-319.

Levin, J. R., & Wampold, B. E. (1999). Generalized single-case randomization tests: Flexible analyses fora variety of situations. School Psychology Quarterly, 14, 59-93.

Lewis, T.J. & Sugai, G. (1996). Descriptive and experimental analysis of teacher and peer attention and

the use of assessment-based intervention to improve pro-social behavior. Journal of BehavioralEducation, 6, 7-24.

Lewis, T J., Scott, T. M., & Sugai, G. (1994). The problem behavior questionnaire: A teacher-basedinstrument to develop functional hypotheses of problem behavior in general education classrooms.Diagnostique, 19, 103-115.

Linehan, M. M. (1980). Content validity: Its relevance to behavioral assessment. Behavioral Assessment,2, 147-159.

82 S.A. Stage


Functional Behavioral AssessmentMace, F. C., & Lalli, J. S. (1991). Linking descriptive and experimental analyses in the treatment of

bizarre speech. Journal ofApplied Behavior Analysis, 24, 553-562.Martin, N. T., Gaffan, E. A., & Williams, T. (1999). Experimental functional analyses for challenging

behavior: A study of validity and reliability. Research in Developmental Disabilities, 20, 125-146.Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of

measurement. In H. Wainer and H. I. Braun (Eds.), Test validity (pp. 33-48). Hillsdale, NJ: Erlbaum.Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons'

responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.

National Institute of Health (1989, September). Treatment of destructive behaviors. Abstract presented atthe National Institutes of Health, Consensus Development Conference, Rockville, MD.

National Association of School Psychologists (1992). Principles for professional ethics. Silver Spring,MD: NASP Publications.

Nelson, J. R., Roberts, M. L., Mathur, S. R., & Rutherford, Jr., R. B. (1999). Has public police exceededour knowledge base? A review of the functional behavioral assessment literature. BehavioralDisorders, 24, 169-179.

O'Neill, R. E., Horner, R. H., Albin, R. W., Sprague, J. R., Storey, K. & Newton, J. S. (1997). Functionalassessment and program developmentfor problem behavior. Pacific Grove, CA: Brooks/ColePublishing Company.

O'Neill, R. E., Horner, R. H., Albin, R. W., Storey, K. & Sprague, J. R. (1990). Functional analysis ofproblem behaviors: A practical assessment guide. Sycamore, IL: Sycamore, Pub. Co.

O'Reilly, M. F. (1995). Functional analysis and treatment of escape-maintained aggression correlated withsleep deprivation. Journal ofApplied Behavior Analysis, 28, 225-226.

Reed, H., Thomas, E., Sprague, J. R., & Horner, H. (1997). The student guided functional assessmentinterview: An analysis of student and teacher agreement. Journal ofBehavioral Education, 7, 33-49.

Repp, A .C., & Karsh, K. G. (1994). Hypothesis-based interventions for tantrum behaviors of persons withdevelopmental disabilities in school settings. Journal ofApplied BehaviorAnalysis, 27, 21-31.

Sasso, G. M., Reimers, T. M., Cooper, L. J., Wacker, D., Berg, W., Steege, M., Kelly, L., & Allaire, A.(1992). Use of descriptive and experimental analyses to identify the functional properties of aberrantbehavior in school settings. Journal ofApplied Behavior Analysis, 25, 809-821.

Schill, M. T., Kratochwill, T. R., & Elliott, S. N. (1998). Functional assessment in behavioral consultation:A treatment utility study. School Psychology Quarterly, 13, 116-140.

Schwartz, I. S. (1991). The study of consumer behavior and social validity: An essential partnership forapplied behavior analysis. Journal ofApplied Behavior Analysis, 24, 241-245.

Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments: Is current practice state of art? JournalofApplied Behavior Analysis, 24, 189-204.

Shores, R. E., Jack, S. L., Gunter, P. L., Ellis, D. N., DeBriere, T. J., & Wehby, J. H. (1993). Classroominteractions of children with behavior disorders. Journal of Emotional and Behavioral Disorders, 1,27-39.

Stage, S. A., & Quiroz, D. R. (1997). A meta-analysis of interventions to decrease disruptive classroombehavior in public education settings. School Psychology Review, 26, 333-368.

Sturmey, P. (1994). Assessing the functions of aberrant behaviors: A review of psychometric instruments.Journal ofAutism and Developmental Disorders, 24, 293-304.

Tobin, T. (1994). Recent developments in functional assessment: Implications for school counselors andpsychologists. Diagnostique, 19, 5-28.

Tobin, T. J. & Sugai, G. M. (1999). Discipline problems, placements, and outcomes for students withserious emotional disturbance. Behavioral Disorders, 24, 109-121.

Touchette, P. E., MacDonald, R. F., & Langer, S. N. (1985). A scatterplot for identifying stimulus controlof problem behavior. Journal ofApplied Behavior Analysis, 18, 343-35 1.

Umbreit, J. (1995). Functional assessment and intervention in a regular classroom setting for thedisruptive behavior of a student with attention deficit hyperactivity disorder. Behavioral Disorders,20, 267-278.

U.S. Department of Education (1994). Implementation of the individuals with disabilities education act:Sixteenth annual report to congress.

83


84 S. A. Stage

Vollmer, T. R., Marcus, B. A., Ringdahl, J. E., & Roane, H. S. (1995). Progressing from brief assessmentsto extended experimental analyses in the evaluation of aberrant behavior. Journal ofApplied BehaviorAnalysis, 28, 561-576.

Wahler, R. G., & Fox, J. J. (1981). Settings events in applied behavior analysis: Toward a conceptual andmethodological expansion. Journal ofApplied Behavior Analysis, 14, 327-338.

Wolf, M. M. (1978). Social validity: The case for subjective measurement on how applied behavioranalysis is finding its heart. Journal ofApplied Behavior Analysis, 11, 203-214.

Yell, M. L. (1997). The IDEA Amendments of 1997: Implications for special and general educationteachers, administrators, and teacher trainers. Focus on Exceptional Children, 30, 1-19.

Yell, M. L., & Shriner, J. G. (1998). The discipline of students with disabilities: Requirements of theIDEA amendments of 1997. Education and Treatment of Children, 21, 246-257.

Zarcone, J. R., Rodgers, T. A., Iwata, B. A., Rourke, D. A., & Dorsey, M. F. (1991) Reliability analysis ofthe motivation assessment scale: A failure to replicate. Research in Developmental Disabilities, 12,349-360.


Canadian Journal of School Psychology 2000 Stage 67 84

Documents

validity of fba

external validity

content validity

criterion validity

convergent validity

social validity

treatment validity

habilitative validity