30 Years of Research: What I Have Learned Jon Kangas RP Conference, April 2006.

30 Years of Research:30 Years of Research:What I Have LearnedWhat I Have Learned

Jon KangasJon KangasRP Conference, April 2006RP Conference, April 2006

Research JourneyResearch Journey

Intelligence at Intelligence at Middle AgeMiddle Age

Vitamin E and Vitamin E and Ulceration In RatsUlceration In Rats

Self Disclosure in Self Disclosure in Small GroupsSmall Groups

Alcohol as a Cure Alcohol as a Cure for Ulcers in Ratsfor Ulcers in Rats

3500 research 3500 research studies in the studies in the SJECCDSJECCD

Referenced inReferenced inPsychology Today

Time

New Woman Magazine

Readers Digest

National Enquirer

Robin Comic Book

Start with ValuesStart with Values What is a healthful What is a healthful

individual?individual?

I outlined my values inI outlined my values inThe Psychology of The Psychology of

StrengthStrength, Prentice Hall, Prentice Hall

What Attributes Make up a What Attributes Make up a Good Citizen?Good Citizen?

Check out the accreditation standardsCheck out the accreditation standards– Personal integrationPersonal integration– EthicsEthics– AestheticsAesthetics– CommunicationCommunication

Can take care of self and othersCan take care of self and others Can participate wisely in a democracyCan participate wisely in a democracy

Personal ValuesPersonal Values

What is ultimately What is ultimately productive while productive while you are doing the you are doing the work others are work others are paying you to do?paying you to do?

Personal Logo Representing Central Value

Personal ValuesPersonal Values One thing I have One thing I have

learned:learned:– People only move People only move

forward if they start forward if they start where they are atwhere they are at

People move People move forward when they forward when they say where they are say where they are at is OK, at is OK, interesting, and interesting, and fascinating to share fascinating to share with otherswith othersPersonal Logo Representing

Central Value

The workplace is a social The workplace is a social laboratory for personal and laboratory for personal and

professional growth.professional growth.

If You Meet the Buddha on the Road Kill Him

Book by Sheldon Kopp

Start with the Big PictureStart with the Big Picture Before getting caught Before getting caught

up in details, check out up in details, check out the big picture firstthe big picture first

Your time is limited Your time is limited and deserves to be and deserves to be focused in the most focused in the most valuable wayvaluable way

Values or Bag of Tricks?Values or Bag of Tricks?

To consciously choose what you do in To consciously choose what you do in light of your values and vision of light of your values and vision of health is a value oriented enterprisehealth is a value oriented enterprise

Not to consciously know your Not to consciously know your values and make conscious values and make conscious

choices makes for achoices makes for a

“ “bag of tricks” approach to bag of tricks” approach to

researchresearch

Single Most Useful ReportSingle Most Useful Report

Success rate by gender and ethnicity Success rate by gender and ethnicity by section, course, subject, by section, course, subject, department, and college over timedepartment, and college over time

Special emphasis on Special emphasis on

English and Math English and Math

sequences leading sequences leading

to transfer Englishto transfer English

and Mathand Math

GroupGroup NN #A %A#A %A #B #B %B%B

…… # Suc# Suc % Suc% Suc

AsnAsn

MM

FF

Tot Tot

5050

5050

100100

5 5 10%10%

5 5 10%10%

10 10 10%10%

25 25

2525

50 50

50%50%

50%50%

50%50%

AfAmAfAm

MM

FF

TotTot

Tot MTot M

Tot FTot F

TotTot

Sample format for Success by Gender and Ethnicity

Note: Success = A, B, C, D, CR/All grade notations including W, D, and F

Why the English and Math Why the English and Math SequencesSequences

Need these to graduate or transferNeed these to graduate or transfer

Exception = majors requiring algebra Exception = majors requiring algebra

If you don’t achieve graduation or If you don’t achieve graduation or transfer, your chances of bridging the transfer, your chances of bridging the economic gap are low (relates to a economic gap are low (relates to a chosen value) chosen value)

Comparisons Groups Over Comparisons Groups Over Time are EssentialTime are Essential

Favorite Comparison StoryFavorite Comparison Story

Algebra Success at EVCAlgebra Success at EVC

0%

5%

10%

15%

20%

25%

30%

35%

F02

EVC



Algebra Success at EVC and StatewideAlgebra Success at EVC and Statewide

0%

10%

20%

30%

40%

50%

60%

F02

EVCState



EVC, State, and SJCCEVC, State, and SJCC

0%

10%

20%

30%

40%

50%

60%

70%

F02

EVCStateSJCC


EVC, State, and SJCC EVC, State, and SJCC

After Bob transferred from SJCC to EVCAfter Bob transferred from SJCC to EVC

Note: Bob gives 95% A GradesNote: Bob gives 95% A Grades

0%

10%

20%

30%

40%

50%

60%

70%

F02 F03

EVCStateSJCC

Comparisons: Add Ethnicity to 1A Comparisons: Add Ethnicity to 1A Persistence from 1 Level BelowPersistence from 1 Level Below

0%

5%

10%

15%

20%

25%

30%

35%

40%

Af Am Asian Hsp White

SJCCEVC

Is the converse more Is the converse more telling?telling?

How many don’t persist to How many don’t persist to

English 1A from 1LBEnglish 1A from 1LB

56%58%60%62%64%66%68%70%72%74%76%

Af Am Asian Hsp White

SJCCEVC

EnlaceEnlaceConsistently Higher than Consistently Higher than

Comparison Groups OverComparison Groups Over Time Time

EnlaceEnlaceConsistently Higher than Consistently Higher than

Comparison Groups OverComparison Groups Over Time Time

0%

10%

20%

30%

40%

50%

60%

70%

80%

F92 F93 F94

Enlace

Comparison

How to Create Comparison How to Create Comparison GroupsGroups

Construct your programs/grants around the Construct your programs/grants around the writing sequence leading to transfer English writing sequence leading to transfer English writingwriting

Place your program’s students in special Place your program’s students in special sections of writing starting 2 levels below sections of writing starting 2 levels below 1A1A

Use your success reports by gender and Use your success reports by gender and ethnicity to get your comparison dataethnicity to get your comparison data

How to Create Comparison How to Create Comparison GroupsGroups

Start with a group of 100 not equal to any other Start with a group of 100 not equal to any other groups (typical of grants)groups (typical of grants)

GPA, # of units, success rates are interesting but GPA, # of units, success rates are interesting but not very useful by themselvesnot very useful by themselves

Sort the students by writing courses 1, 2, and 3 Sort the students by writing courses 1, 2, and 3 levels below English 1Alevels below English 1A

Then compare the students 2 LB in that section with Then compare the students 2 LB in that section with all other students 2LB in that course (minus the all other students 2LB in that course (minus the exp. grp), etc.exp. grp), etc.

AdviceAdvice

Try to avoid one time studies (not over time) with no Try to avoid one time studies (not over time) with no comparison groupcomparison group

Choose studies that have the same measures over time with Choose studies that have the same measures over time with comparison groups using the same measurescomparison groups using the same measures

There are no “Remedial There are no “Remedial Students”Students”

Transfer English CompositionTransfer English Composition

1 Level Below1 Level Below Educational Futures Educational Futures LevelLevel

2 Levels Below 2 Levels Below Orientation to College Orientation to College LevelLevel

3 Levels Below 3 Levels Below Assessment LevelAssessment Level

Early Alert is too LateEarly Alert is too Late

The The

!!first two weeks!!!!first two weeks!!

are criticalare critical

When in Doubt Ask the When in Doubt Ask the StudentStudent

Every Friday for the first 4 weeks use CAT & ask:Every Friday for the first 4 weeks use CAT & ask:

Are you thinking of dropping this class?Are you thinking of dropping this class?

Do you understand the material to date?Do you understand the material to date?

Do you need a tutor?Do you need a tutor?

Would you like a study group?Would you like a study group?

Would you like to talk to the instructor?Would you like to talk to the instructor?

Are you worried about succeeding in this class?Are you worried about succeeding in this class?

Have the student sign their name.Have the student sign their name.

Or How Many Steps Can We Take to Move Or How Many Steps Can We Take to Move Away from the Student and real/relevant Away from the Student and real/relevant

intervention?intervention?Don’t ask the student anythingDon’t ask the student anythingWait a long time to infer nonsuccessWait a long time to infer nonsuccess

First ExamFirst Exam2 Absences2 Absences

Ask the instructor rather than the studentAsk the instructor rather than the studentPut the instructor response on a formPut the instructor response on a formSend form to CounselingSend form to CounselingHave counselor try to contact the studentHave counselor try to contact the studentArrange to talk to the studentArrange to talk to the studentAsk the student if they are having troubleAsk the student if they are having troubleEarly Alert is too lateEarly Alert is too lateDo a study to correlate type of problem reported with SuccessDo a study to correlate type of problem reported with SuccessIf you meet the Buddha…If you meet the Buddha…

Simple PrinciplesSimple Principles

If you want to train future behavior, practice If you want to train future behavior, practice the behaviorthe behavior

If you want to know if a student is having If you want to know if a student is having trouble, ask the studenttrouble, ask the student

If you want someone to move forward in life, If you want someone to move forward in life, accept, acknowledge, and find fascinating accept, acknowledge, and find fascinating

the place where they are at. Do not require the place where they are at. Do not require changechange

Gateway U**Gateway U**

0%

10%

20%

30%

40%

50%

60%

70%

Asian Af Am Hsp White Other

GatewayCompare

Note: N = Gateway 105; Comparison Groups 335

5 different English courses represented

Data Over Time EssentialData Over Time Essential

Affirm Success Affirm Success F01F01

0%

10%

20%

30%

40%

50%

60%

% Success

AffirmCompareBaseline

N = 19, 52, 382 in 5 courses in math, English and Chem


Affirm Success Affirm Success F04F04

0%10%20%

30%40%

50%60%

70%80%

90%

% Success


N = 45, 83, 373 in 5 courses in Math, English and Chemistry, Political Science, Guidance


Affirm Success Affirm Success F04**F04**

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

F01 F02 F03 F04


New Intervention Strategy

Note utility of success by ethnicity reports by section and course

Data Over TimeData Over Time

Enlace Success Rates over Time**Enlace Success Rates over Time**

0%

10%

20%

30%

40%

50%

60%

70%

80%

F95 F97 F99 F01

EnlaceCompare

Ns from 128 to 227, courses include English, Math, and Biology

One of top 2 studies**One of top 2 studies**Course placement success ratesCourse placement success rates

Counselor judgment vs. the computerCounselor judgment vs. the computer

50%

52%

54%

56%

58%

60%

62%

64%

66%

% Succ

CounselorsComputer

Placement Success for 1 Counselor Placement Success for 1 Counselor F89**F89**

0%

10%

20%

30%

40%

50%

60%

70%

Bob All Cs Computer

% Suc

Note: Counselor rallied to 60% in F90 and then backslide to 57% in S93. Shows the value or data over time.

Comparisons of Instructional Comparisons of Instructional Methodologies**Methodologies**

Comparison of On-Line, Hybrid, and Web Comparison of On-Line, Hybrid, and Web Enhanced Methodologies with Similar Enhanced Methodologies with Similar

CoursesCourses

0%

10%

20%

30%

40%

50%

60%

70%

80%

On Line Hybrid Enhanced

MethodCompare

Ns = 305, 71, 1117

We Don’t Really Need More We Don’t Really Need More ResearchResearch

Physiological

Safety

Love Belonging

Esteem

Actualization

We know what is neededWe know what is neededLife is not abstractLife is not abstract

FoodFood SleepSleep ShelterShelter SafetySafety Money to pay for collegeMoney to pay for college Transportation to collegeTransportation to college Sense of belongingSense of belonging

– GroupGroup– PlacePlace– Program that carries a sense of pride and Program that carries a sense of pride and

identityidentity

We know what is neededWe know what is needed SupportSupport

– Blocked classesBlocked classes– Support personnelSupport personnel– Caring follow-up when problems occur including CATs Caring follow-up when problems occur including CATs

(early alert is too late)(early alert is too late)– Assessment for realistic life/educational strategiesAssessment for realistic life/educational strategies– Sense of a home on campusSense of a home on campus– Study SkillsStudy Skills– Time managementTime management– Career planning and a clear vision of an educational and Career planning and a clear vision of an educational and

career futurecareer future– Mentors, Role ModelsMentors, Role Models– Learning CommunitiesLearning Communities

We know what is neededWe know what is needed“Love is not Enough”“Love is not Enough”

!!!Time on task (Rouche)!!!Time on task (Rouche)

with correct practice with correct practice (Jon)!!!(Jon)!!!

Add Other Instructional Add Other Instructional StrategiesStrategies

Courses with clearly articulated learning Courses with clearly articulated learning outcomes that put the grade within control outcomes that put the grade within control of the studentof the student

Interactive learningInteractive learning

High structure for low skill level studentsHigh structure for low skill level students

Peer Led Team LearningPeer Led Team Learning

Formula for Creating a New Formula for Creating a New Support Program/EffortSupport Program/Effort

Absolute Essentials: Insure thatAbsolute Essentials: Insure that– FoodFood– ShelterShelter– TransportationTransportation– Money to pay for collegeMoney to pay for college– BooksBooks– Time on TaskTime on Task

Then AddThen Add– Any of the other support/instructional variables Any of the other support/instructional variables

but include a way to create a strong sense of but include a way to create a strong sense of belongingnessbelongingness

My Sense with a “Longitudinal” My Sense with a “Longitudinal” PerspectivePerspective

New religions are needed with each new New religions are needed with each new generation: If you meet the Buddha on generation: If you meet the Buddha on the road…the road…

Funding has not changed over time so we Funding has not changed over time so we keep “shuffling the chairs on the Titanic”keep “shuffling the chairs on the Titanic”

Grants and fresh enthusiasm create new Grants and fresh enthusiasm create new requests for research for what works, new requests for research for what works, new programs and initiativesprograms and initiatives

Examples of Temporary Examples of Temporary ChangeChange

Success in Writing 2 Levels Below 1ASuccess in Writing 2 Levels Below 1A

0%

10%

20%

30%

40%

50%

60%

70%

80%

Base F01 F02 F03 F04 F05

% Suc


Success in Reading 3 Levels Below 1ASuccess in Reading 3 Levels Below 1A

60%

62%

64%

66%

68%

70%

72%

74%

Base F01 F02 F03 F04 F05

% Suc


Success in Writing 1 Level Below 1ASuccess in Writing 1 Level Below 1A

0%

10%

20%

30%

40%

50%

60%

70%

S77 BL F01 F02 F03 F04 F05

% Suc

Short Story: Gateway UShort Story: Gateway U

Chancellor’s Office GrantChancellor’s Office Grant Blocked ClassesBlocked Classes High Pride in ProgramHigh Pride in Program Emphasis on Time on TaskEmphasis on Time on Task Enthused Instructors Enthused Instructors Dubbed a Model ProgramDubbed a Model Program Success Above BaselinesSuccess Above Baselines Someone forgot to reapplySomeone forgot to reapply Here today, gone tomorrowHere today, gone tomorrow

Moral of StoryMoral of Story We know what worksWe know what works

But have to rediscover it on our ownBut have to rediscover it on our own

We don’t build on accumulated knowledgeWe don’t build on accumulated knowledge

We can’t afford to increase our efforts so we We can’t afford to increase our efforts so we make progress in the moment with our grants make progress in the moment with our grants and enthusiasm and enthusiasm

Enjoy your successful initiatives in the momentEnjoy your successful initiatives in the moment

Building on Previous KnowledgeBuilding on Previous KnowledgeReminder: Life is PracticalReminder: Life is Practical

Simple RulesSimple Rules

If you want a student to know the names If you want a student to know the names of other students, have them meet of other students, have them meet and say the names of other studentsand say the names of other students

If you want students to contact others, If you want students to contact others, construct ways for them to contact construct ways for them to contact each othereach other

Life is realLife is real


Success in Reading 1 Level Below 1ASuccess in Reading 1 Level Below 1A

52%

54%

56%

58%

60%

62%

64%

66%

68%

70%

Base F01 F02 F03 F04 F05

% Suc

Persistence to Transfer Math Persistence to Transfer Math from Intermediate Algebrafrom Intermediate Algebra

0%2%4%6%8%

10%12%14%16%18%20%

Base F01 F02 F03

% Succ

Persistence to Transfer Math Persistence to Transfer Math from Intermediate Algebra from Intermediate Algebra

SJCCSJCC

0%

2%

4%

6%

8%

10%

12%

14%

16%

Base F01 F02 F03

Int Alg

Persistence from Writing Classes below Persistence from Writing Classes below Transfer English 1A to Transfer English Transfer English 1A to Transfer English

1A1A

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

Base F01 F02

1 LB2 LB

Persistence from Writing Classes below Persistence from Writing Classes below Transfer English 1A to Transfer English Transfer English 1A to Transfer English

1A1AComparisons still helpfulComparisons still helpful

0%

5%

10%

15%

20%

25%

30%

Base F01 F02 F03

EVC 1 LBSJCC 1 LB

Why would you guess EVC higher?

College Success Rates Over College Success Rates Over TimeTime

66%

67%

67%

68%

68%

69%

69%

F02 F03 F04 F05

EVC SuccSJCC Succ

Make an Impact with DataMake an Impact with Data

People make 100s of decisions with no dataPeople make 100s of decisions with no data People don’t wait for data to make People don’t wait for data to make

decisionsdecisions People make decisions based on what they People make decisions based on what they

feel about data, not data itselffeel about data, not data itself People want to feel confident and on solid People want to feel confident and on solid

ground using the data we provideground using the data we provide So construct a strategy to make people So construct a strategy to make people

confident/comfortableconfident/comfortable

Make an Impact with DataMake an Impact with Data

Test of effective dataTest of effective data

Your primary finding will be repeated Your primary finding will be repeated at a cocktail partyat a cocktail party

The One Page ReportThe One Page Report

People will read one pagePeople will read one page

People will remember information from People will remember information from one pageone page

Our Trends Newsletters are one page Our Trends Newsletters are one page reportsreports

The One Page FormulaThe One Page Formula

Words:Words: State the Question in 1-2 sentencesState the Question in 1-2 sentences

Data:Data: Present the key data often in 8 pts. Present the key data often in 8 pts. Use notes in 6-8 pts for, caveats,Use notes in 6-8 pts for, caveats,definitions definitions Always find a way to include NsAlways find a way to include Ns

Picture:Picture: Chart Chart Words:Words: Comments, conclusions using shortComments, conclusions using short

sentences, bullets. Try to have one majorsentences, bullets. Try to have one majorfinding that can be presented like a finding that can be presented like a

soundsoundbite. bite. Make RecommendationsMake Recommendations

Make RecommendationsMake Recommendations

Outline OptionsOutline Options

Suggest Possible ProgramsSuggest Possible Programs

Suggest Possible DecisionsSuggest Possible Decisions

Differentiate Your DataDifferentiate Your Data

Green when a number/% goes upGreen when a number/% goes upRed when a number/% goes downRed when a number/% goes downUse an arrow to point to the most imp. #Use an arrow to point to the most imp. #Make important stuff big, reference info Make important stuff big, reference info

smallsmallUse fall colors for fall dataUse fall colors for fall dataBold the cell with totalsBold the cell with totalsColorize success columns but not # columnsColorize success columns but not # columnsUse music on slide showsUse music on slide showsCall Julie SlarkCall Julie Slark

Red

Impact with Data in a Impact with Data in a DocumentDocument

Put the story on the coverPut the story on the cover

Put the credits on the inside of the Put the credits on the inside of the covercover

Put the larger story as a one page Put the larger story as a one page

document on page 1document on page 1

Example of first page:

Tells the story

Tells how the program works so it can be replicated

Data, pictures, words

Notes and credits

For maximum impact with your For maximum impact with your datadata

Turn your data into a social problem, not a set of Turn your data into a social problem, not a set of numbers: Persistence from 1LB 1A to 1Anumbers: Persistence from 1LB 1A to 1A

18%

19%

19%

19%

19%

19%

20%

20%

20%

F03-4 F04-5

Persisance to1A

80% will not graduate or transfer and are not likely to bridge the economic gap

30 Years of Research: 30 Years of Research: What IWhat I

Have Learned: Have a Life Have Learned: Have a Life

EnjoyEnjoy

GrowGrow

Chart Your Own CourseChart Your Own Course

Have an ImpactHave an Impact

Value Your Impact in the MomentValue Your Impact in the Moment

JonJon

30 Years of Research: What I Have Learned Jon Kangas RP Conference, April 2006.

Documents

time slide

evc slide

f slide

sjcc slide

math slide

sjeccd slide

democracy slide

statewide slide