Top Banner
ASG Cost Model ASG Cost Model Presentation Presentation CCSSO National Conference CCSSO National Conference on Student Assessment on Student Assessment
22
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

ASG Cost Model ASG Cost Model PresentationPresentation

CCSSO National Conference on CCSSO National Conference on Student AssessmentStudent Assessment

Page 2: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 2

OverviewOverview Model description/StructureModel description/Structure Key inputs and outputsKey inputs and outputs DatabasesDatabases ReportsReports UsesUses QuestionsQuestions

Page 3: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 3

Description/StructureDescription/Structure

The ASG Assessment Cost Model is a variable The ASG Assessment Cost Model is a variable input, metric based calculation output modelinput, metric based calculation output model Specific assessment program variables are input to the Specific assessment program variables are input to the

model and applied against cost factors to derive model and applied against cost factors to derive assessment costassessment cost

Several hundred variables are contained in the Several hundred variables are contained in the modelmodel Numerous detailed input variables are assembled for each Numerous detailed input variables are assembled for each

of the functions and elements involved in constructing an of the functions and elements involved in constructing an assessment (test development, production and assessment (test development, production and manufacturing, logistics, editing and scoring, reporting, manufacturing, logistics, editing and scoring, reporting, psychometrics, program management, quality assurance, psychometrics, program management, quality assurance, information technology, etc.) information technology, etc.)

These variables are input into the software and matched These variables are input into the software and matched to metric based calculations/formulas and/or databases to to metric based calculations/formulas and/or databases to convert the assessment program variables into dollar convert the assessment program variables into dollar costs costs

Page 4: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 4

Description/Description/Structure Structure (cont.)(cont.)

The cost for the entire assessment is built The cost for the entire assessment is built from the from the ground upground up, element by element, , element by element, function by functionfunction by function

The model also contains “efficient cost The model also contains “efficient cost metrics” for each key cost area and metrics” for each key cost area and function so that a state’s assessment function so that a state’s assessment program cost data can be compared to program cost data can be compared to those of an “efficient” assessment those of an “efficient” assessment This allows states to see where their This allows states to see where their

assessment program is cost effective and assessment program is cost effective and where there are opportunities to improve where there are opportunities to improve costs costs

Page 5: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 5

Description/Structure Description/Structure (cont.)(cont.)

Proprietary functions and calculations have Proprietary functions and calculations have been developed based on our years of been developed based on our years of experience in the industryexperience in the industry

Model outputs have been Model outputs have been tested against real tested against real datadata

The model is currently built for ASG use only. The model is currently built for ASG use only. We are in the process of developing a web We are in the process of developing a web based application that could be shared with based application that could be shared with states.states.

ASG has partnered with CCSSO and ten states ASG has partnered with CCSSO and ten states on an on an Enhanced Assessment GrantEnhanced Assessment Grant to study to study assessment efficiency in the participating assessment efficiency in the participating states. The ASG cost model will be the key states. The ASG cost model will be the key tool used in the analysis.tool used in the analysis.

Page 6: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Cost Cost Model-MainModel-Main Menu Menu

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 6

Page 7: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

AdministrationAdministration

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 7

Administrative Center

Key Functions:

- Update Rate Tables- Define Security Settings- Create Custom Reports

Page 8: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Assessment MapAssessment Map

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 8

Page 9: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 9

Key Inputs and OutputsKey Inputs and Outputs

Numerous assessment component inputs Numerous assessment component inputs are available for each key area, function are available for each key area, function etc.etc. Assessment cost can be developed using Assessment cost can be developed using

detailed data or higher level, summary datadetailed data or higher level, summary data Cost outputs by function, area, cost type, Cost outputs by function, area, cost type,

etc. are generatedetc. are generated Key metrics are presented to allow Key metrics are presented to allow

comparison to model efficient cost data comparison to model efficient cost data for that function/area/componentfor that function/area/component

Canned and custom reports are available Canned and custom reports are available at the summary and detailed levelat the summary and detailed level

Assessment Cost ModelAssessment Cost Model

Page 10: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Key InputsKey Inputs

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 10

This is a sample of the key inputs.

There are over 200 different inputs that are captured in the assessment model.

Page 11: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Data Input ScreensData Input Screens

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 11

Page 12: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Data Input Screens - Data Input Screens - cont’dcont’d

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 12

Page 13: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 13

Databases/TablesDatabases/Tables

The Model Contains the FollowingThe Model Contains the Following 70+ Tables to Manage Rates & Data Capture70+ Tables to Manage Rates & Data Capture Rate Tables for Each Cost Component (examples):Rate Tables for Each Cost Component (examples):

Design/Composition RatesDesign/Composition Rates Printing RatesPrinting Rates Freight/Shipping RatesFreight/Shipping Rates Labor Rates – by State, by OccupationLabor Rates – by State, by Occupation Rates for Delivery of On-Line AssessmentsRates for Delivery of On-Line Assessments Open Ended Scoring Reader RatesOpen Ended Scoring Reader Rates Item Acquisition/Permission RatesItem Acquisition/Permission Rates

Page 14: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Key Key DatabasesDatabases - Labor - Labor

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 14

Page 15: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Key Key DatabasesDatabases - Travel - Travel

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 15

Travel Table - Includes Airfare, Lodging and Meal per diem rates for all 50 States and the District of Columbia

Page 16: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 16

Reports Reports

Data presented in report and graphical formatsData presented in report and graphical formats Canned reports include:Canned reports include:

Assessment Cost SummaryAssessment Cost Summary Assessment Cost by FunctionAssessment Cost by Function Detail Assessment Functional CostDetail Assessment Functional Cost Fixed and Variable Cost DataFixed and Variable Cost Data Assessment Price Composition (direct, indirect and Assessment Price Composition (direct, indirect and

margin)margin) Key Metrics Summary & Functional Area Metrics Key Metrics Summary & Functional Area Metrics

SummarySummary Assessment Costs vs. “efficient costs”Assessment Costs vs. “efficient costs” Assessment Metrics vs. “efficient metrics”Assessment Metrics vs. “efficient metrics” Other reports are currently in developmentOther reports are currently in development

Reports can also be easily customized per customer Reports can also be easily customized per customer requestrequest

Assessment Cost ModelAssessment Cost Model

Page 17: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

ReportingReporting

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 17

Page 18: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Reporting – cont’dReporting – cont’d

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 18

Page 19: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Reporting – cont’dReporting – cont’d

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 19

Page 20: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

Reporting – cont’dReporting – cont’d

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 20

Page 21: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 21

UsesUses The model can be used in several ways to The model can be used in several ways to

improve assessment cost and efficiency:improve assessment cost and efficiency: Developing a new RFPDeveloping a new RFP

ASG cost sheets key in discerning if vendors understand ASG cost sheets key in discerning if vendors understand your program and are proposing apples to apples solutionsyour program and are proposing apples to apples solutions

Making tradeoffs between various assessment componentsMaking tradeoffs between various assessment components Develop the best assessment for the available dollarsDevelop the best assessment for the available dollars

Review of an existing program to determine areas of Review of an existing program to determine areas of inefficiency and opportunityinefficiency and opportunity

Assessment costs tend to be too high because of Assessment costs tend to be too high because of inefficiencies in the program designinefficiencies in the program design

Working with the assessment vendor to reduce/eliminate Working with the assessment vendor to reduce/eliminate efficiencies and lower costs is a win/win for both the efficiencies and lower costs is a win/win for both the vendor and customervendor and customer

Devising scope changesDevising scope changes

Page 22: ASG Cost Model Presentation CCSSO National Conference on Student Assessment.

June 22/23, 2009Assessment Solutions Group www.assessmentgroup.org 22

QuestionsQuestions

ASG would be happy to answer any ASG would be happy to answer any questions or provide a state specific questions or provide a state specific demo of the model on request.demo of the model on request.

Contact information:Contact information: Barry Topol - 210-859-9920 Barry Topol - 210-859-9920

[email protected]@assessmentgroup.org Pat Hennon - 210-376-7654 Pat Hennon - 210-376-7654

[email protected]@assessmentgroup.org John Olson - 210-859-5593 John Olson - 210-859-5593

[email protected]@assessmentgroup.org Ed Roeber - 517-614-4877 Ed Roeber - 517-614-4877

[email protected]@assessmentgroup.org