How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.

How we let them drive:Developing dashboards to deliver multiple

years of NSSE/FSSE dataOur Process

Southern Association of Institutional Research2010 Conference - New Orleans

Monday, September 27th4:15 p.m. - 5:00 p.m.

Robin Logan, Director of Institutional ResearchEarl Harmsen, Faculty H-E-B School of Business

Michael Schulte, Graduate Research Assistant & Doctoral Candidate

4 year, Private, Catholic

Hispanic Serving Institution

Established 1881 by the Sisters of Charity of the Incarnate Word

Carnegie: Master’s L

San Antonio: 7th largest city in the US

UGUG GRGR PRPR INTLINTL TOTTOT

5,4645,464 1,2411,241 509509 494494 7,7087,708

MainMain ADCaPADCaP VUVU INTLINTL TOTTOT

69%69% 15%15% 10%10% 6%6% 100%100%

HISPHISP MINMIN WHIWHI INTLINTL UNKUNK TOTTOT

49%49% 11%11% 21%21% 11%11% 8%8% 100%100%

UIW Demographics

Geri Hockfield Malandra. (2008) Accountability and learning assessment in the future of higher education. On the Horizon, 16(2), 57-71. Retrieved September 10, 2010, from proQuest Education Journals. (Document ID: 1484475591).

Accountability and assessment have become prominent topics of institutional and public discussion

Gonyea, R. M., & Kuh, G. D. (Eds.). (2009). Editor’s note: Using NSSE in institutional research. New Directions for Institutional Research, 141, 1-4. DOI: 10.1002/ir.282

...“increased visibility and importance of institutional research offices staffed by highly skilled and competent professionals who can provide campus leaders with objective, trustworthy data about student and institutional performance”

multiple years of data

NSSE 3 years + 2010

FSSE 2 years + 2010

"still in the binders"

various committees and assessment teams needed “meaningful” information

too much to comprehend all at one time

the challenge

• multi-campus institution

• 20 questions referenced in the QEP

• QEP population for NSSE required filters to present the correct views with FSSE data

• colleges requesting data specific to them

evaluate what had already been done

narratives...

lots of words!

lots of actions!

unclearmethodologies

data quality

NSSE Multi-Year Data Analysis Guide. (2010). Indiana University Center for Postsecondary Research. Retrieved September 23, 2010 from http://nsse.iub.edu/2010_Institutional_Report/pdf/NSSE%20MYDAG.pdf

1. Identify and focus on specific questions

2. Select and employ appropriate methods of analysis

3. Attend to data quality for each year in the analysis

4. Take into account changes in NSSE items and reports across years

5. Merge multiple years of data

how does IR...?

• deliver what multiplestakeholders need

• move towards “plug-and-play”

• keep it simple

• keep our sanity

Hughes, C. (2009). Framing the activities of institutions and academic development units in support of assessment. Innovations in Education and Teaching International, 46(2), 123-133. Retrieved September 10, 2010, from Research Library. (Document ID: 1796476001).

• Universities demonstrate their commitment to enhancing and assuring the quality of assessment through the employment of a diverse array of initiatives.

NSSE ALL

NSSE UNIT

Tableau - known!

Interactive - for UNIT - known!

Data source created in MSExcel

One file or multiple files?

FSSE ALL

QEP Combined File

NSSE ALL

stacked data for multi-year comparison

check alignment of questions

add identifier columns (YR, FY/SY, GROUPS)

decision about experimental items from prior years

• Means and Frequencies

•Means ... well-intentioned, but not enough to present to the community

•Frequency distributions... referenced for all but one benchmark in the QEP

XLSX: NSSE ALL DATA

But what about question response rate?

But what about knowing which was used in the QEP?

But what about ...

XLSX: NSSE ALL DATA

NSSE UNIT

• much easier

• same basic steps

• data for all responses - requires CALCS!

• match to campus data for campus and college **not necessary with 2010 because it was institutional column

• What do we include?

filters on the dashboard

QEP notations

benchmark notations

XLSX: NSSE UNIT LEVEL DATA

• What do we include?

means and frequencies

response rate specifically for the question

XLSX: NSSE UNIT LEVEL DATA

• What do we include?

• static tables from NSSE ALL

•comparison to NSSE, Master’s, and Peer Institutions

XLSX: NSSE BENCHMARK DATA

FSSE ALL

• Three concepts to consider:

those questions chosen for QEP

benchmark comparisons to NSSE

with

without

• Going beyond the frequencies

• *for those with NSSE crosswalks

XLSX: FSSE DATA

Incorporating the QEP goals

Crosswalk comparison

• Methodology

• FSSE: Main Campus faculty only

• NSSE: Data filtered to Main Campus respondents

XLSX: FSSE CROSSWALK

XLSX: NSSE AGG QEP CALCS

XLSX: FSSE DATA

QEP - Should it be its own workbook?

YES!

Respondent CharacteristicsXLSX: NSSE RESPONDENTS XLSX: PEER INSTITUTIONS

End Result

• NSSE ALL

• almost too big

• NSSE UNIT

• split into three interactive workbooks because of size

FSSE

split into four workbooks because of size

QEP

limited to 20 questions

End Result

Data source: MSExcel

started with multiple files

transferred to a single multi-tabbed file

Worksheets:

NSSE Benchmarks

NSSE ALL Data

NSSE UNIT Data

Peer Institutions

NSSE Respondents

FSSE Crosswalk

FSSE Data

Moore, J. (2005). Seven recommendations for creating sustainability education at the university level: A guide for change agents. International Journal of Sustainability in Higher Education 6(4), 326-339. Retrieved September 10, 2010, from Research Library (Document ID: 929185721).

Recommendation: integration of university plans, decision-making structures and evaluative measures and the integration of the research, service and teaching components of the university

Gonyea, R. M., & Kuh, G. D. (Eds.). (2009). Editor’s note: Using NSSE in institutional research. New Directions for Institutional Research, 141, 1-4. DOI: 10.1002/ir.282

...“ascendance of student engagement and other process indicators that serve as both proxy measures for institutional quality and actionable information to inform improvement efforts.”

Robin Logan, Director of Institutional ResearchEarl Harmsen, Faculty H-E-B School of Business

Michael Schulte, Graduate Research Assistant & Doctoral Candidate