Page 1
1© 2000 Technology Commercialization Group LLC. All rights reserved
TECHNOLOGYCOMMERCIALIZATION
GROUP, LLCPAUL G. WAUGAMAN
LOUIS G. TORNATZKY
Higher Education PracticeWILLLIAM S. KIRBY
APPLYING BENCHMARKING TO SPONSORED PROGRAMS ADMINISTRATION
SRA - NCURASouthern Section/Southeastern Region Meeting
April 16-17, 2000
TCG
Page 2
2© 2000 Technology Commercialization Group LLC. All rights reserved
Points to Cover:
Benchmarking in research administration» Overview
The Higher Education Benchmarking Consortium» Overview
The KPMG - NACUBO - SRA Sponsored Programs Benchmarking Effort» Organization» Initial results» Next steps
Page 3
3© 2000 Technology Commercialization Group LLC. All rights reserved
Benchmarking
Systematic comparison of elements of the performance of an organization against that of other organizations, with the aim of mutual improvement.
McNair and Leibfried: “Benchmarking.” 1992
Page 4
4© 2000 Technology Commercialization Group LLC. All rights reserved
The Utility of Benchmarking
Provides a basis for comparison and self-analysis.
Accelerates the sharing of useful and novel approaches.
Energizes everybody to do better.
Page 5
5© 2000 Technology Commercialization Group LLC. All rights reserved
Purposes and Uses of Measurement in Research Administration
Measure against “industry standards”
Measure against goals» Performance drives improvement interventions.
Identify problems» Diagnose and forecast existing problems.
Build the “business case” for change or new resources.
Page 6
6© 2000 Technology Commercialization Group LLC. All rights reserved
The Problem:In Research administration .... There are no generally accepted criteria
for success. We rely on anecdotal data and informal
case study rather than systematic data collection and analysis.
Benchmarking and measurement are not part of the improvement culture
Competitive comparisons may be viewed as threats rather than improvement drivers.
Page 7
7© 2000 Technology Commercialization Group LLC. All rights reserved
The Problem (cont):
Breadth and diversity of the domain make data collection and measurement difficult.
Successful outcomes are difficult to demonstrate objectively.
There is little data available.
Who is going to do it?
Page 8
8© 2000 Technology Commercialization Group LLC. All rights reserved
The SRA Benchmarking Initiative
Benchmarking and Best Practices Task Force established in 1998.
Task Force recommends working with NACUBO and KPMG to facilitate performance data collection.
Collaboration with NACUBO and KPMG initiated in September 1998.
Data collection instrument refined with SRA Task Force input in Spring, 1999.
Page 9
9© 2000 Technology Commercialization Group LLC. All rights reserved
The KPMG Higher Education Benchmarking Consortium
The purposes of the consortium are:» To serve as a comprehensive source of
institutional comparative performance, benchmarking, and “best practice” information for the higher education community; and
» To facilitate the use of comparative performance data to meet strategic, operational improvement, and change management needs.
Page 10
10© 2000 Technology Commercialization Group LLC. All rights reserved
The KPMG Higher Education Benchmarking Consortium
Operating Principles
» Participant-driven» “E-nabled”» Focused» Leading to "best practices” » Leveraged, where possible
Page 11
11© 2000 Technology Commercialization Group LLC. All rights reserved
The KPMG Higher Education Benchmarking Consortium
Major Segments» Financial Health and Operations» Student Affairs» Physical Infrastructure» Information Technology» Human Resources» Research:
Sponsored Programs Technology Transfer
Page 12
12© 2000 Technology Commercialization Group LLC. All rights reserved
KPMG - NACUBO - SRA Sponsored Programs Survey
Purposes To provide a common set of sponsored
programs performance indicators that will allow meaningful comparisons with other organizations and with peer groups of institutions.
To facilitate the development of a benchmarking data base.
Page 13
13© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
SRA-NACUBO-KPMG Roles
NACUBO and KPMG are financing and organizing data collection and analysis, and are promoting the program with their constituencies and clients.
SRA is participating in survey design, is promoting the program with their members, and is participating in dissemination of findings.
Page 14
14© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Features of the Program
Low cost to participating institutions. Individualized reports for
participants. Expert analysis. All institution-specific data will be
“blinded.” Only general reports will be available
to non-participants.
Page 15
15© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Critical Success Factors for Research Administration
Sustaining or Enhancing Sponsored Research Activity and Funding (competitiveness)
Containing the Costs and Increasing the Efficiency of Sponsored Research Administration
Improving Service to Faculty (in order to achieve Objective 1)
Maintaining and Improving Institutional and Sponsor accountability
Page 16
16© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey Indicators:
Sustaining or Enhancing Sponsored Research Activity and Funding
Number of proposals submitted per faculty FTE. Percentage of faculty working as principal
investigators. Sponsored project dollars received per
faculty FTE. Sponsored projects funding growth rate. Number of new awards as a percent of new
proposals submitted.
Page 17
17© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Indicators:Cost and Efficiency
Number of proposals per sponsored projects admin. FTE.
Sponsored projects admin. cost per proposal. Number of awards per Sponsored projects admin
FTE. Sponsored projects admin costs as percent of
dollars received. Number of awards per research accounting FTE. Research accounting costs per award Dollars received per research accounting FTE. Research accounting cost as percent of dollars
received.
Page 18
18© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey Indicators:
Service to Faculty
Number of funded PI’s per sponsored projects FTE.
Number of active PI’s per sponsored projects FTE.
Number of funded PI’s per research accounting FTE.
Page 19
19© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey Indicators:
Institutional and Sponsor Accountability
Percent of Sponsored Projects Accounts Receivable over 120 days.
Page 20
20© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Data Needed to Develop Indicators
Fy 1997-98 Data: Number of faculty FTE. Number of central administrative FTE (sponsored
programs, research accounting). Proposal data (number, dollars, sponsor: federal,
industry, other). Award data (number, dollars, sponsor: federal,
industry, other). Sponsored program expenditure data Administrative costs allocated to research (central and
departmental costs - from indirect cost proposal).
Page 21
21© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Benefits for Participants
Access to useful comparable data for program evaluation and benchmarking.
Low cost. Ease of preparation. Uses readily available data
assembled for internal reporting or other purposes.
Page 22
22© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One Questionnaire Design (Dec 98-Jun 99)
» Survey design was peer-based» Intentions were to use available data» Initial survey requested FY 1998 data
Data Collection (Jul-Oct 99)» A high response rate was a major objective» Timing did not accommodate academic calendar» Correct POC identification was a challenge» Follow up efforts: phone calls, letters, e-mails
Page 23
23© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One Data Collection Outcomes:
» 62 institutions participated
(40% of all US college/university research expenditures)
» Definition issues arose Examples: Who are “research faculty;” new vs. renewal
awards Caused confounding metrics
Data Processing» 34 institutions chose peers» 3 sub-sets of institutions
Page 24
24© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One Reporting:
» Institution-specific reports: Customized, Institution-specific. Provided ranking
information on 16 metrics, and 4 sub-groups for each metric.
» Data feedback workshops Gave participants opportunities to discuss data, and
next steps
» Revised Reports will be prepared after participants have an opportunity to revise submissions
» Public Report in preparation
Page 25
25© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year OnePercentage of faculty working as principal investigators. Number Median
Value
Entire Sample 62 39.1%
Public 51 39.0%
Independent 11 40.0%
Research I 33 40.9%
Research II 16 40.4%
Other CarnegieClasses
13 20.8%
Land-Grant 18 44.1%
Medical Schools 27 40.4%
Page 26
26© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year OneSponsored Project dollars per Faculty FTE
Number Median Value
Entire Sample 62 63,453
Public 51 61,289
Independent 11 86,118
Research I 33 80,923
Research II 16 53,944
Other CarnegieClasses
13 23,320
Land-Grant 18 68,915
Medical Schools 27 80,923
Page 27
27© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Sponsored Projects Funding Growth - Fy 93 - 98Number Median
Value
Entire Sample 62 6.3%
Public 51 6.3%
Independent 11 6.6%
Research I 33 6.3%
Research II 16 3.7%
Other CarnegieClasses
13 8.3%
Land-Grant 18 5.5%
Medical Schools 27 6.8%
Page 28
28© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Number of proposals submitted per central sponsored projects administrative FTE
Number MedianValue
Entire Sample 61 70.7
Public 50 73.7
Independent 11 54.9
Research I 32 83.9
Research II 16 58.8
Other CarnegieClasses
13 45.1
Land-Grant 17 90.8
Medical Schools 27 86.3
Page 29
29© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Central sponsored projects administrative cost as a percent of sponsored project dollars
Number MedianValue
Entire Sample 62 1.02%
Public 51 0.97%
Independent 11 1.24%
Research I 33 0.80%
Research II 16 1.27%
Other CarnegieClasses
13 2.11%
Land-Grant 18 0.81%
Medical Schools 27 0.82%
Page 30
30© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Central post-award financial administrative cost per active project
Number MedianValue
Entire Sample 60 $389
Public 49 $369
Independent 11 $477
Research I 32 $310
Research II 15 $470
Other CarnegieClasses
13 $628
Land-Grant 18 $289
Medical Schools 25 $375
Page 31
31© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Central post-award financial administrative cost as a percent of sponsored project dollars
Number MedianValue
Entire Sample 61 0.70%
Public 50 0.65%
Independent 11 0.71%
Research I 32 0.49%
Research II 16 0.88%
Other CarnegieClasses
13 0.95%
Land-Grant 18 0.56%
Medical Schools 27 0.57%
Page 32
32© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Year One
Number of active Principal investigators per central sponsored projects admin. FTE
Number MedianValue
Entire Sample 60 33.3
Public 49 34.2
Independent 11 23.6
Research I 31 37.3
Research II 16 27.7
Other CarnegieClasses
13 23.5
Land-Grant 18 44.5
MedicalSchools
26 38.1
Page 33
33© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Next Steps
Complete Fy 1998 round» Reopen data base for revisions and
additional enrollments (April-May, 2000)
» Complete a brief public report (May, 2000)
» Complete revised individual reports (June, 2000)
» Brief Best Practices study (June - August, 2000)
Page 34
34© 2000 Technology Commercialization Group LLC. All rights reserved
Sponsored Programs Survey
Next Steps
Initiate Fy 2000 round» Target date to begin data collection: September-
November, 2000» Activate Web Site for data collection on-line
(September, 2000) and report preparation by participants (January, 2000)
» Reports: January, 2001
Web site (for informational purposes): www.us.kpmg.com/highered/benchmark/
Page 35
35© 2000 Technology Commercialization Group LLC. All rights reserved
Contact us ...
TECHNOLOGY COMMERCIALIZATION GROUP, LLC2237 OXFORD HILLS DRIVE
RALEIGH, NORTH CAROLINA 27608-1672 USA
TEL: 919-833-2569
FAX: 919-833-3277
E-MAIL: [email protected]
PAUL G. WAUGAMAN, PRINCIPAL
WWW.T-C-GROUP.COM
TCG