FOUR TYPES OF INSTITUTIONAL RESEARCHINSTITUTIONAL RESEARCH Gregg Thomson, Executive Director Office of Student Research and Campus Surveys University of California, Berkeley CAIR 35th

FOUR TYPES OF INSTITUTIONAL RESEARCH

Gregg Thomson, Executive Director Office of Student Research and Campus Surveys

University of California, Berkeley

CAIR 35th Annual Meeting November 17-19, 2010

San Diego

ALookBack

•  Beginnings….•  SidSuslow:ADeclara8ononIns8tu8onalResearch,AIR,1972“Thequalityoftheins8tu8onalresearcherliesinhisabilitytodiscernwhichmethodologiesareappropriatetotheproblemwithwhichheisconcerned:adroitapplica8onofadvancedtoolsofmanagementsciencemaybegrosslyinappropriatewhenappliedtocertainproblemsofstudentbehavior,andsubtlepsychologicalanalysismaylackmeritwhenappliedtocertainproblemsofins8tu8onalgovernance.”

DefiningCharacteris8csoftheIRRole

•  TheDuali8esofIR(Volkwein,1999)–  TheInternalVersustheExternal–  AcademicVersusAdministra8veCultures–  Ins8tu8onalVersusProfessionalRoles

•  TheIrresolvableTensionofAuthen8cIR

The Four Faces of IR and Level of Education Required (Volkwein, 1999, 2008)

Forma8veandInternal–

forImprovement

Summa8veandExternal—

forAccountability

Administra8veandIns8tu8onal

INFORMATIONAUTHORITY

[low]

SPINDOCTOR

[moderate]

AcademicandProfessional

POLICYANALYST[high]

SCHOLARANDRESEARCHER[veryhigh]

AssessingVolkwein’sTypology

•  Forma8ve/Internal/ImprovementVersusSumma8ve/External/Accountabilitytoorestric8ve

•  Rethinkingthe2X2model,especiallyinlightofrecentdevelopmentsintheprac8ceofIR

TheTransforma8onsofIR

•  Technologicaltransforma8onandresultantprolifera8onofins8tu8onalandsurveydata– KnowledgeManagerasFi\hFaceofIR

•  Accountability,repor8ngrequirements,andthewidespreaddisplayofins8tu8onaldata

•  EmergenceofStudentAffairsandstudent‐focusedIRac8vity

•  Growthofmethodologicallysophis8catedacademicresearch,especiallyonstudents

TheOppositePolesofIR

ACADEMICRESEARCHER

ACCOUNTABILITYTECHNICIAN

ADMINISTRATIVEANDINSTITUTIONALROLE

ACADEMICANDPROFESSIONALROLE

FourTypesofIns8tu8onalResearcher

ACADEMICRESEARCHER

PROGRAMEVALUATOR

ACCOUNTABILITYTECHNICIAN

CAMPUSAUTHORITY

AUTHORITYAuthorita8veresource,interpreta8onofdataanditsinterconnec8ons,discovery,debunking,reac8vespinindefenseofins8tu8on

TECHNICIANVisualdisplayofdata,precision,“error‐free”,consistency,nosta8s8calcontrols;proac8vespininpublicrepresenta8on

RESEARCHERTes8ngofhypothesis,academicresearchdesign,rigoroussta8s8calcontrols,nospin,i.e.,researchethics

EVALUATORUnbiasedevalua8onofoutcomes,effects,rigoroussta8s8calcontrols,nospininpresenta8onofresults

ApproachesoftheFourTypes

MARKET‐CENTEREDConcernedwithbroad,conceptualgoalsandpurposes(markets,posi8on,brand)oftheins8tu8on

ADMINISTRATIVEConcernedwithprac8calefficienciesandeffec8venessofprocesses

ACADEMICConcernedwithenhancementoflearningandadvancementofknowledge

STUDENT‐FOCUSEDConcernedfortheexperiencesofindividualstudentsorgroupsofstudents

Orienta8onstoEnrollmentManagement(Kalsbeek,2006,2007)

CONCEPTUALTHEORIST[INTUITIVE/THINKING]Mul8plepossibili8esandparadigms;rela8onshipsofmanyvariables;challengesassump8ons;uncertainty;valuesinteresJngness

ANALYTICSCIENTIST[SENSING/THINKING]Driventowardcertainty;equatesknowledgewithprecision,accuracy,andreliability;single,self‐consistentparadigms

CONCEPTUALHUMANIST[INTUITIVE/FEELING]Inquirytobenefithumanityaswhole;problemsdefinedbyresearcher’sownvalues;aestheJcsoftheory

PARTICULARHUMANIST[SENSING/FEELING]Personalizesknowledge;inquirytobenefitindividuals;notinterestedintheories;casestudies;acJon‐oriented

Alterna8veModesofInquiry(Mitroff&Kilmann,1978)

Educa8onandTraining

ACADEMIC

RESEARCHERCAMPUS

AUTHORITYACCOUNTABILITY

TECHNICIANPROGRAM

EVALUATOR

EDUCATIONALREQUIREMENT HIGH HIGH MODERATE HIGH

ISSUESINTELLIGENCE LOW HIGH HIGH LOW

CONTEXTUALINTELLIGENCE LOW HIGH MODERATE LOW

USEOFFORMALSTATISTICS

YES NOTUSUALLY NO YES

RoleandApproachtoData

ACADEMIC

RESEARCHERCAMPUS

AUTHORITYACCOUNTABILITY

TECHNICIANPROGRAM

EVALUATOR

WHOYOUARE AUTHOR AUTHORITY ANONYMOUS EXPERT

REFERENCEGROUP

ACADEMICDISCIPLINE

INSTITUTIONADMIN‐

ISTRATIONAPPLIED

ACADEMIC

APROACHTODATA

MULTIVARIATECONTROLS

NEWCONNECTIONS

DISPLAYS,METRICS

MEASURINGEFFECTS

PURPOSENEW

KNOWLEDGEINSTITUTIONALAWARENESS

ACCOUNT‐ABILITY

EVIDENCE

USESOFCAMPUSAUTHORITY

INTERNAL EXTERNAL

SUMMATIVE‐Genera8nginnova8vereportsandanalysis,o\eninsupportofaccountability

FORMATIVE–Exploringnewrela8onships;newunderstandinginsupportofintersubjec8vity

SUMMATIVE‐Fiercedefenseofins8tu8oninfaceofmisrepresenta8onorapack;“goto”resourceforsta8s8calandotherinforma8on

USEOFPROGRAMEVALUATION

INTERNAL EXTERNAL

SUMMATIVE‐Evalua8onofexis8ngcampusprogramsandprac8ces

FORMATIVE–Founda8onforcampusassessmentac8vi8es(Bers,2008)

SUMMATIVE‐Evalua8onforexternalagencies

FORMATIVE–Consulta8onforgrantsandproposals

Rela8onshipsAmongtheFourTypes

•  Pluralis8c,mul8plesroles,ac8vi8es•  Nega8veinfluence•  Posi8veinfluence,strengthens

GoodAcademicResearch:WhatisitGoodFor?

•  Thebepertheresearch,thelessuseful?•  Thetyrannyofmul8variateanalysis

•  Bringingtheins8tu8onbackin

AccountabilityandItsDiscontents

•  Accountabilitycenterstage:GoodandbadnewsforIR

•  Dilemmaofnocontrolsversusover‐control

•  Highlyvisibleaccountabilitydatadoesnotnecessarilymeangreaterins8tu8onalself‐understanding(orevenaccountability)

•  Makingaccountabilitymore(ac)countable

CampusAuthorityasAuthen8cIR

•  Nowthat’sinteres8ng!•  Ins8tu8onalintelligenceandmemory

•  Autonomyandtrust

•  Fromauthoritytoauthor:Crea8ngnewknowledge

•  Theenormouspoten8alofqualita8veins8tu8onalresearch

TheNeglectofProgramEvalua8on

•  Thechallengingoflookinginward•  Beingtough‐mindedinanSFenvironment

•  Getngseriousaboutprogramevalua8on

SUPPORTFORPROGRAMEVALUATION

ACADEMICRESEARCH

PROGRAMEVALUATION

ACCOUNTABILITY

CAMPUSAUTHORITY

UNBALANCEDINSTITUTIONALRESEARCH

ACADEMICRESEARCH

PROGRAMEVALUATION

ACCOUNTABILITY

CAMPUSAUTHORITY

HEALTHYINSTITUTIONALRESEARCH

ACADEMICRESEARCH

PROGRAMEVALUATION

ACCOUNTABILITY

CAMPUSAUTHORITY

SidSuslowoncemore:

•  “Theremustbewithintheins8tu8onalresearchroleasufficientdegreeofindependencewhichpermutesittofullyexamineandcri8cizetheins8tu8on,includingitspurposesandvalues.…Itsuniquenessrestsinthecombina8onofitsroleasaposiJvecriJcconcernedwiththeenhancementandadvancementofhighereduca8onanditsroleasaresearcher,quan8fier,andassessorofquan8ta8veandqualita8veinforma8onabouttheins8tu8on….Anac8veins8tu8onalresearchrolealsorequiresthesustenanceofanefficacioussymmetrybetweenac8vi8esprescribedbyitsservicetotheins8tu8onandopportuni8esforexplora8oninareasresponsivetoitsowninsightsandinterests.Crea8vityisanac8veprocesswhichrequiresadegreeofautonomythatmustberealandsufficient….Withoutthisfundamentalingredient,thecomposi8onoftheins8tu8onalresearchrolelackspotency.”

CONCLUDINGTHOUGHTS

•  Bestof8mes,worstof8mes?•  Authen8cins8tu8onalresearchandthelearningcommunity

•  Comments,Ques8ons,Sugges8ons(please!)

[email protected]