1 SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html Utilizing Institutional Research Learning to Love Data Dr Barry W. Simmons Sr. Director USFA, Virginia Tech *With thanks to Bonnie Joerschke, UGA
48
Embed
1 SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. Utilizing.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Utilizing Institutional Research
Learning to Love Data
Dr Barry W. Simmons Sr.Director USFA, Virginia Tech
*With thanks to Bonnie Joerschke, UGA
2SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Presentation Outline
• Need for Research• Basic Principles and Statistics• Data Sources• Tools• Foibles• Examples• Sharing• Questions• The End
3SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Is There Time for Research?
• Reports are research– FISAP– State Reports– Institutional Reports– Common Data Set (CDS)– U S News and World Reports– National Postsecondary Student Aid Study
(NPSAS)– Others?
4SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Make Time for Research
• Strategic Planning• Fund Utilization• Grant Applications• Knowing Your Student Body• Evaluation• Compliance• Institutional Quality Assurance Program• Others?
5SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Basic Principles of Research
• Quantitative research is the systematic scientific investigation of quantitative properties and phenomena and their relationships. The objective of quantitative research is to develop and employ mathematical models, theories and/or hypotheses pertaining to natural phenomena. The process of measurement is central to quantitative research because it provides the fundamental connection between empirical observation and mathematical expression of quantitative relationships.
Source: Wikipedia
6SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Basic Principles of Research• Qualitative research attemptss to gather an in-
depth understanding of human behavior and the reasons that govern human behavior. Qualitative research relies on reasons behind various aspects of behavior. Simply put, it investigates the why and how of decision making, not just what, where, and when. Hence, the need is for smaller but focused samples rather than large random samples, Qualitative researchers, typically rely on four methods for gathering information: (1) participation in the setting, (2) direct observation, (3) in depth interviews, and (4) analysis of documents and materials [2].
Source: Wikipedia
7SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
BASIC PRINCIPLES OF RESEARCH*
• Descriptive Statistics– Describes the basic features of the data– Simple summaries– Present quantitative descriptions in a
manageable form– Descriptive vs. inferential statistics– Limitations: risk of distorting original data or
losing important detailSource: The Web Center for Social Research Methods,
www.socialresearchmethods.net/kb/statdesc/htm
8SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
DESCRIPTIVE STATISTICS*
• Most often used descriptive statistic is “central tendency or range” of a distribution of values:– Mean– Median– Mode– Range
9SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
DEFINITIONS*
• Mean: the arithmetic average
30 + 80 + 30 + 90 + 30 + 100 = 360/6 = 60
• Median: The number in the middle of an ordered list
30 30 30 80 90 100 ^
• Mode: The number that occurs most often30
• Range: The highest value minus lowest (100 – 30 = 70)
10SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
DEFINITIONS*• Standard deviation
– Measures the dispersion of your data– Outlying values can distort the average or
range of your data– Steps to calculate a standard deviation:
1. Compute the mean for the data set
2. Compute the deviation by subtracting the mean from each value
3. Square each individual deviation
4. Add up the squared deviations
5. Divide by one less than the sample size
6. Take the square root
11SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
DEFINITIONS*• Standard deviation (example)
30 – 60 = -30.0002 = 900.000000
30 – 60 = -30.0002 = 900.000000
30 – 60 = -30.0002 = 900.000000
80 – 60 = 20.0002 = 400.000000
90 – 60 = 30.0002 = 300.000000
100 – 60 = 40.0002 = 1600.000000
5000.000000/5 = 1000.0001000= 31.623
12SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
TRENDS*
• Cross-sectional Analysis– Snapshot of data at a single point in time
• Longitudinal Analysis– Study of change in data over time– Trend analysis is the simplest type of longitudinal
analysis—examines overall change over time; using the past to predict the future
• Limitation: Does not tell “why or how” the changes occur
13SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
CONSTANT DOLLARS*
• Dollars reported in terms of the value they had on a previous date. Measures data over time while controlling for changes owing to inflation.
• Usually gives a clearer view of how something is performing over time.
• Use Consumer Price Index (CPI) to adjust for inflation.
14SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
CONSTANT DOLLARS*
• Consumer Price Index– Measure used to track change in prices for
common household goods over time– Can be used to compare dollar amounts
between years using “constant dollars”– Produced by the Bureau of Labor Statistics
• www.bls.gov/cpi/
Good reference when updating COA!!!
15SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Types of Data
• Nominal: Nominal data have no order and thus only gives names or labels to various categories.
• Ordinal: Ordinal data have order, but the interval between measurements is not meaningful.
• Interval: Interval data have meaningful intervals between measurements, but there is no true starting point (zero).
• Ratio: Ratio data have the highest level of measurement. Ratios between measurements as well as intervals are meaningful because there is a starting point (zero).
16SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Data Sources
• Institution’s Administrative System• EDE Express Suite• External Databases
– Regional– State– National
• IPEDS• CPS Quarterly Sample
• Surveys• Others?
17SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research and Data Resources
18SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research Tools
• Excel
• SAS
• SPSS
• Mini-Tab
• Crystal Reports
• Access
• Others?
19SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research Foibles
• Quality of data– Reliability– Consistency– Stability
• Understanding of data– Data is data, not information, needs context– Compare and contrast appropriately– If items not well defined, develop your own definition and state it
• Application of Analysis– Use the correct measure/statistic– Don’t confuse association with correlation with causation– Basis of data…define variables
20SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Affordability Nationwide From:Higher Education
Access and Affordability in Virginia Amid the 2008 Credit Crisis: A Cursory Look, Barry W. Simmons Sr. in testimony 12/4/2008 before the Virginia Senate Finance Committee Education Subcommittee
Improvements in State Investment---state investment in need-based financial aid as a percentage of the federal investment
– Nevada 2% to 48%– North Carolina 3% to 70%– Virginia 6% to 50%– Montana 1% to 9%– Utah 1% to 8%– Washington 24% to 108%– Texas 7% to 32%– Delaware 13% to 49%– Missouri 8% to 29%– West Virginia 12% to 43%
Source: Measuring Up 2008
Needs further protocol explanation
21SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Affordability From:Higher Education Access and Affordability in Virginia Amid
the 2008 Credit Crisis: A Cursory Look, Barry W. Simmons Sr. in testimony 12/4/2008 before the Virginia Senate Finance Committee Education Subcommittee
State Tax Funds Appropriated for Higher EdOperating Expenses Per $1,000 Personal Income FY07
Virginia $6.53National $7.08North Carolina $12.52
Virginia Change from FY1980 to FY2007
($3.94) National Change from FY1980 to FY2007
($3.39)
Source: Postsecondary Education Opportunity, January 2007
Example of Good Use!!!
22SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Affordability in Virginia From:Higher Education Access and
Affordability in Virginia Amid the 2008 Credit Crisis: A Cursory Look, Barry W. Simmons Sr. in testimony 12/4/2008 before the Virginia Senate Finance Committee Education Subcommittee
Source: SCHEV
Too Low
Too High
23SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Affordability in Virginia From:Higher Education Access and
Affordability in Virginia Amid the 2008 Credit Crisis: A Cursory Look, Barry W. Simmons Sr. in testimony 12/4/2008 before the Virginia Senate Finance Committee Education Subcommittee
Source: SCHEV
24SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Foibles in Research
Others?
25SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVirginia Financial Aid Related
Metrics for Restructuring5.1 Average annual need-based student borrowing.
All Students Regardless of Enrollment or Registration Status 43482 54025
As of 9/11/2008
MSU Student Data for 2007-2008
Total Aid Disbursed by Category
Total Aid Offered
$914,843,348
31SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT Financial Statement
UNIVERSITY FINANCIAL HIGHLIGHTS For the years ended June 30, 2004—2008 (all dollars are in millions; square feet in thousands)
2003-04 2004-05 2005-06 2006-07 2007-08 REVENUES, EXPENSES, AND CHANGES IN NET ASSETS (1) Operating revenues $ 459.0 $ 500.9 $ 543.8 $ 595.9 $ 595.9 Operating expenses 97.5 741.9 815.3 883.1 883.1 Operating loss (2) $(238.5) $(241.0) $(271.4) $(287.2) $(287.2)
Non-operating revenues and expenses (2) 239.4 264.4 284.7 313.8 313.8 Other revenues, expenses, gains or losses (3) 60.3 35.5 26.1 122.3 122.3 Net increase (decrease) in net assets $ 61.2 $ 58.8 $ 39.4 $ 148.9 $ 148.9
UNIVERSITY NET ASSETS (1) Invested in capital assets, net of related debt $ 418.7 $ 465.1 $ 496.8 $ 569.7 $ 569.7 Restricted (3) $ 100.6 $ 106.4 $ 100.0 $ 159.4 $ 159.4 Unrestricted $ 49.4 $ 58.2 $ 72.3 $ 88.8 $ 88.8
ASSETS AND FACILITIES Total university assets (1, 3) $ 982.6 $1,046.9 $1,078.1 $1,296.7 $1,296.7 Capital assets, net of accumulated depreciation (1) $ 624.6 $ 698.9 $ 733.2 $ 813.5 $ 813.5 Facilities-owned gross square feet 8,001 8,147 8,454 8,498 8,498 Facilities-leased square feet 609 599 604 682 682 SPONSORED PROGRAMS Number of awards received 2,148 2,086 2,122 2,131 2,131 Value of awards received $ 147.8 $ 189.5 $ 195.9 $ 203.1 $ 203.1 Research expenditures reported to NSF (4) $ 268.8 $ 290.0 $ 321.7 $ 367.0 N/A
VIRGINIA TECH FOUNDATION Gifts and bequests received $ 53.9 $ 71.6 $ 81.8 $ 78.5 $ 78.5 Expended in support of the university $ 86.6 $ 97.8 $ 102.4 $ 107.3 $ 107.3 Total assets and managed funds $ 613.5 $ 670.4 $ 728.0 $ 808.9 $ 940.9
ENDOWMENTS (AT MARKET VALUE) Owned by Virginia Tech Foundation (VTF) $ 325.5 $ 361.7 $ 398.3 $ 469.4 $ 469.4 Owned by Virginia Tech 39.5 40.8 42.6 47.8 47.8 Managed by VTF under agency agreements 6.2 6.4 6.9 7.9 7.9 Total endowments supporting the university $ 371.2 $ 408.9 $ 447.8 $ 525.1 $ 525.1
STUDENT FINANCIAL AID Number of students receiving selected types of financial aid Loans 11,055 10,962 11,140 11,067 11,376 Grants, scholarships and waivers 14,140 14,088 14,481 15,600 16,221 Employment opportunities 7,792 7,923 8,067 8,101 8,699
Total amounts by major category Loans $ 90.7 $ 93.6 $ 101.0 $ 102.2 $ 105.9 Grants, scholarships and waivers (5) 85.4 94.8 104.1 119.8 139.2 Employment opportunities 45.7 48.7 50.5 53.4 58.4 Total financial aid $ 221.8 $ 237.1 $ 255.6 $ 275.4 $ 303.5
(1) The university adopted the new Governmental Accounting Standard Board (GASB) reporting model in fiscal year 2002 as required by GASB Statement Number 35, Basic Financial Statement—and Management's Discussion and Analysis—for Public Colleges and Universities.
(2) The university will always be expected to show an operating loss since significant recurring revenues are shown as non-operating. Major revenue sources reported as non-operating include state appropriations, gifts, and investment income. These revenue sources are used for general operations in support of the learning, discovery, and engagement missions of the university.
(3) Totals for 2006 have been restated to reflect the change in reporting guidelines for appropriations received from the Treasury of Virginia.
(4) Total research expenditures for NSF report were not available at publication date.
32SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research Example: VT Funds for the Future
33SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT Funds for the Future
In-StateFamily Income Student Count Raw Need Unmet Need % of Unmet Need Average EFC Min EFC Max EFC
Dollar Increase of Tuition & Mandatory E&G Fees $1,133 $649 $1,242 $1,325 $1,414 $1,509Dollar Increase of Mandatory non E&G Fees $79 $77 $62 $65 $68 $72
Total Dollar Increase of Tuition & Mandatory Fees $1,212 $726 $1,304 $1,390 $1,482 $1,580Increase in Tuition and Mandatory Fees 6.4% 3.7% 6.2% 6.2% 6.2% 6.2%
VA Residents Non-VA Residents VA Residents Non-VA ResidentsTuition & Mandatory E&G Fees 5,813 17,889 6,160 18,538 Mandatory Non E&G Fees 1,160 1,160 1,237 1,237 Total Tuition & Fees Annual 6,973 19,049 7,397 19,775
Per semester 3,487 9,525 3,699 9,888
Virginia Tech Tuition & Fee Rates 2007-2008 - Undergraduates
2006-2007
2006-2007
2007-2008
2007-2008
39SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT Short Narrative on
Presidential Scholarship Initiative(PSI) New Access Program
Funding Model for Virginia Tech Presidential Scholarship Initiative (PSI)
The funding model for the PSI utilizes existing aid programs with the addition of Virginia Tech funds to equal the direct cost of in-state tuition, required fees and on-campus room and board for each recipient. For 2008-09 that direct cost would be $13,674. Since the focus of the PSI is to assist low income, Pell Grant eligible students, those eligible will qualify for other in-place programs of need-based financial aid at the federal, state and institutional levels. This approach enables Virginia Tech to help the neediest students understand how tuition, fees, room and board can be addressed loan-free. The funding model estimates, on the average, that PSI participants’ direct costs will be covered by 28% federal grants, 41% state grants and 16% institutional grants and scholarships from a variety of sources including general and departmental endowed monies, unfunded scholarships and other miscellaneous sources. The remaining 15% of monies needed to meet the direct cost of each student will be covered with additional expenditures from unfunded scholarships. The bundling of multiple financial aid programs under one umbrella is designed to simplify the complexity of higher education financial aid programs thereby helping individuals more easily understand and access higher education at Virginia Tech.
40SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT PSI Access Program
41SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT PSI Access Program
42SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT PSI Access Program
43SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
44SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT College Level Summary
College Level Aid At Vt
Residency No. of Students
with Need Average
Financial Need Average
Unmet Need % of
Unmet NeedIn-State 1,342 12,228 3,372 27.58%Out-of-state 411 17,745 5,938 33.47%LAHS College Total 1,753 13,522 3,974 29.39%
Level No. of Students
with Need Average
Financial Need Average
Unmet Need % of
Unmet NeedUG 1,280 16,080 5,023 31.24%GR 473 12,576 3,586 28.52%LAHS College Total 1,753 13,522 3,974 29.39%
Gender No. of Students
with Need Average
Financial Need Average
Unmet Need % of
Unmet NeedFemale 1,063 13,515 3,993 29.54%Male 690 13,532 3,945 29.15%LAHS College Total 1,753 13,522 3,974 29.39%
45SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research ExampleVT College Level Summary
MAJOR No. of Students
with Need Admin & Superv of Special Ed 3 Apprl, Housing, & Resource Mgt 128 Career and Technical Education 38 Classical Studies 8 Communication 197 Counselor Education 45 Creative Writing 7 Curriculum and Instruction 127 Educ Ldrshp & Policy Studies 41 Educational Research & Evaluat 7 English 166 Foreign Langs, Cultures, & Lit 1 French 3 German 6 Health & Physical Education 47 History 155 Human Development 113 Humanities, Science, & Environ 11 Interdisciplinary Studies 138 International Studies 117 Liberal Arts & Human Sciences 1 Music 37 Philosophy 18 Political Science 174 Rhetoric and Writing 2 Science & Technology Studies 5 Soc, Pol, Ethical, & Cultural 4 Sociology 112 Spanish 20 Theatre Arts 22 LAHS College Total 1,753
46SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Research Example
Open Floor
Who’s doing what?
47SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html
Recap
• Need for Research• Basic Principles and Statistics• Data Sources• Tools• Foibles• Examples• Sharing• Questions• The End
48SASFAA MI 12/2008--Utilizing Institutional Research---Dr. Barry W. Simmons Sr. www.finaid.vt.edu/Presentations/SASFAA_MANAGEMENT/index.html