Back to Basics, The Application of Common Measures Lane Kelly Performance Specialist USDOL – ETA, Region 3 - Atlanta May 22, 2008 Ritz-Carlton Hotel Atlanta,

Back to Basics, Back to Basics,

The Application of Common The Application of Common MeasuresMeasures

Lane KellyLane KellyPerformance SpecialistPerformance Specialist

USDOL – ETA, Region 3 - AtlantaUSDOL – ETA, Region 3 - Atlanta

May 22, 2008May 22, 2008Ritz-Carlton Hotel Ritz-Carlton Hotel Atlanta, GeorgiaAtlanta, Georgia

Focus of Today’s SessionFocus of Today’s Session

Clarification of Terms and Concepts Related to these Topics:

Accountability for Grantees Required Definitions Exiting Participants -- Completion vs. Exit Gap in Service Basic Rules and Concepts Regarding

Common Measures Incumbent Worker vs. Employed Worker Calculating the Common Measures

Accountability for GranteesAccountability for Grantees

General performance accountability for grantees consists of: Grant-specific training outcomes Capacity building impacts Common measures Other information to tell the story of your

grant’s accomplishments

Translation: Common Measures are a part of grantee accountability


Common measures are required for ETA formula-funded grants and most discretionary grants. They improve the ability to:

describe in a similar manner the core purposes of the workforce system

collaborate between programs and reduce duplicative data entry

analyze program outcomes and develop effective program strategies

compare results of similar government programs

Training and Employment Guidance Letter No. 17-05 details DOL policy for common measures

See TEGL 17-05

5


Which common measures apply to discretionary grantees?

Adult Common Measures Youth Common Measures

Entered Employment Rate Placement in Employment or Education

Employment Retention Rate Attainment of Degree/Certificate

Average Earnings Literacy/Numeracy Gains

Required Definitions—ParticipantRequired Definitions—Participant

Participant: An individual determined eligible to participate in the program who receives a service funded by the program in either a physical location (e.g., One-Stop Center) or remotely through electronic technologies

1. Determined eligible to participate

2. Receives a funded service

3. In physical location or remotely You need to determine what services trigger participation in

your grant You may need to capture information about services provided

remotely

Required Definitions—Required Definitions—Other Terms Related to ParticipationOther Terms Related to ParticipationParticipation Date:

Date of first program-funded service

Participation Quarter: Calendar quarter containing the participation date

Participant Cohort: Group of individuals who share the same

participation quarter

Required Definitions—ExiterRequired Definitions—Exiter

Exiter: A participant who has not received a program or partner-funded service for 90 consecutive days and no future services are scheduled has exited the program

1. Has not received a service

2. No service received for 90 consecutive days

3. No future services scheduled

Example: The last day of class was 2/19/08, and it’s now 5/19/08 – if no services were provided between 2/19 and 5/19, the exit date is 2/19/08

Required Definitions—Required Definitions—Other Terms Related to ExitOther Terms Related to Exit

Exit Date: Date of last program-funded service

Exit Quarter: Calendar quarter containing the exit date

Exit Cohort: Group of individuals who exit during the same

calendar quarter

Additional Reasons for Exiting a Additional Reasons for Exiting a ParticipantParticipant

Six reasons for excluding an individual from performance calculations:

Institutionalized;

Missing or invalid SSN;

Death;

Reservist called to active duty;

Relocation to a mandated program (for youth only); and

Health/medical or family care

Exiting ParticipantsExiting Participants

Completion vs. ExitCompletion vs. Exit -- Not the same!

Completion – a finishing or concluding action.

(Complete services from the program; still in the program)

Exit – the last date of service.

(Exit the program; no longer in the program)

It is the reference point for all adult common measures.

Completion vs. ExitCompletion vs. Exit

Question: What are the consequences of equating the two concepts?

Classic Example – A grantee is delivering 2 “levels” of training; Level 1 must be completed before Level 2. After completion of both levels, the expectation is employment


Scenario 1: – Participant is exited after completion of their service plan; in other words, the participant is exited after completion of Level 2

When are you accountable for reporting on the common measures for this participant?

Scenario #1Scenario #1Participant

completes Level 1, Level 2 and is

exited; this is their

EXIT QTR

Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit

Accountable for Entered

Employment

Accountable for Employment

Retention and Average Earnings


Scenario 2: – Participant is exited after completing Level 1; they are then re-enrolled and exited after completing Level 2

When are you accountable for reporting on the common measures for this participant?

Scenario #2Scenario #2Participant

completes Level 1 and is exited; this is

their EXIT QTR

Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit

Participant completes Level 2

and is exited; this is their EXIT QTR

Q1 Post-Exit Q2 Post-Exit

The grantee is accountable for Employment (based on

the first exit) when the individual is still in training, decreasing the likelihood of

a positive outcome

““Gap in Service”Gap in Service”

A gap in service occurs if one of three allowable circumstances takes place that is expected to last more than 90 days:

Training is delayed, A health/medical reason related to the participant

or a family member, or A temporary move from the area

NOTE: A gap in service prevents an unintentional exit from taking place.

““Gap in Service” -- Things to KnowGap in Service” -- Things to Know

A gap in service … is dependent on the participant’s intent to continue

services

Should not exceed 180 calendar days. However, a subsequent gap of an additional 180 days is possible to resolve issues

Must be documented in the participant case file, including the beginning and expected end date

Is optional; it’s a judgment call.

Utilizing “Gap in Service”Utilizing “Gap in Service”

Example: Part I of a training program ended in 12/07, and Part II is not scheduled until 4/08; no services are provided between Part I and Part II

After 90 days of inactivity (e.g., end of 3/08), participants would have to be exited unless there is a gap in service

The gap in service stops the 90-day clock

A Gap in Service is A Gap in Service is OptionalOptional

Scenario: Project participant has a health issue that will preclude further participation at this time for at least 90 days.

What do you do?

Does s/he intend to continue services?

Based on the answer, you have an option – to either document a gap in service or exit the participant.

Exit or Continue Gap in ServiceExit or Continue Gap in Service

Scenario: A participant has been placed in a gap in service of 120 days; at the 100th day, the individual decides they no longer wish to continue services

What do you do?

In this case, you have an exit, with the exit date being the last date of service.

Exit or Gap in ServiceExit or Gap in Service

Scenario: A participant finishes one class, disappears and then resurfaces later to attend another class.

Can a “gap in service” be used for the time in between the participant’s disappearance and resurfacing?

Generally, an exit would be more appropriate because the “delay before training” was due to the participant and not the training schedule.

Common Measures — Common Measures — Basic Rules and ConceptsBasic Rules and Concepts Only participants are included in performance

calculations Exception: If one of six conditions exist, individuals are

excluded from common measure calculations Exception: Statutory exclusion applying to WIA adults

receiving self-service and informational activities only

Calculations are based on exiters Only exception is Youth Literacy/Numeracy, which is

based on one year from participation date

Specific data sources must demonstrate the outcomes

24

Incumbent Worker vs. Employed WorkerIncumbent Worker vs. Employed Worker

Incumbent Worker–

Incumbent Worker Training Programs --Typically… targets specific employers that are either experiencing a decline and have the potential to undergo workforce reductions, or are experiencing a skills gap that impacts their ability to compete, retain workers, and expand operations

Employed Worker --

Talent development training activities – Target unemployed, underemployed, and employed persons seeking career advancement

25

Incumbent Worker TrainingIncumbent Worker Training

What is Incumbent Worker Training (IWT)?

Customized training to meet the needs of the employer and delivered to workers employed by the targeted business

Employer is typically responsible for the assessment of workers and the design and implementation of the training

Supports career advancement; promotes high skill, high wage positions; and promotes economic development within a region

Delivered to targeted employers via contracts or written agreements

IWT contracts between employers and the grantee should require employers to provide participation and outcome data

Entered Employment RateEntered Employment Rate

Of those who are not employed at the date of participation:

Number of adult participants who are employed in the 1st quarter after the exit quarter

Number of adult participants who exit during the quarter

Entered Employment RateEntered Employment Rate

Employed atParticipationDate?

Employed in1st quarterafter exit?

Excluded from Measure

Numerator

Denominator

YES

NO

NO

YES

Employment Retention RateEmployment Retention Rate

Of those who are employed in the first quarter after the exit quarter:

Number of adult participants who are employed in both the second and third quarters after the exit quarter


Employment Retention RateEmployment Retention RateEmployed in 1st quarter after exitquarter?

Employed in2nd and 3rd

quarters afterexit quarter?


Numerator

Denominator

NO

YES

NO

YES

Average Six-Month EarningsAverage Six-Month Earnings

Of those adult participants who are employed in the first, second and third quarters after the exit quarter:

Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter


Average Six-Month EarningsAverage Six-Month Earnings

Employed in 1st quarter afterexit quarter?

Employed in2nd and 3rd

quarters afterexit quarter?


Numerator = Wages from quarters 2 & 3, post-exitincluded

Denominator = Number ofadult participants who exitduring the quarter

No

Yes No

Yes

The Bigger Picture– The Bigger Picture– Adult Common MeasuresAdult Common Measures

Participation Quarter(s)

Exit Quarter Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit

Because the adult common measures are

exit-based, participation

dates, participation quarters, and participation

cohorts are not needed for calculations

The 3 adult common

measures are exit-based.

Therefore, the exit quarter is

critical

Entered Employment Rate is based

on employment in Q1 post-exit; this is where we

expect the positive outcome

Employment Retention Rate is based on employment in Q1 and Q2 post-exit for those employed

Q1 post-exit; we expect employment in both post-exit

quarters

Average Earnings is based on gross earnings in Q2 and Q3 post-

exit for those employed Q1, Q2 and Q3 post-exit

Example -- Adult Common MeasuresExample -- Adult Common Measures

Scenario: You have a participant who was employed at participation, employed in the first quarter after exit, and not employed in the third quarter after exit.

1. For which adult common measure (s) would you be held accountable?

2. For which adult common measure (s) would there be a positive outcome?

More on the Common MeasuresMore on the Common Measures Training and Employment Guidance Letter (TEGL) 17-05: http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2195

Back to Basics Webinars: Part I – Back to Basics, Part I: Data Collection and Adult

Common Measures http://www.workforce3one.org/view.cfm?id=4809&info=1

Part II – Back to Basics, Part II: Data Processing and Reporting http://www.workforce3one.org/view.cfm?id=4824&info=1

Part III – Back to Basics, The Application of Common Measures to Community-Based and High-Growth Grants

http://www.workforce3one.org/public/_shared/detail.cfm?id=4857&info=1&simple=false

Questions?Questions?

THANK YOU

Back to Basics, The Application of Common Measures Lane Kelly Performance Specialist USDOL – ETA, Region 3 - Atlanta May 22, 2008 Ritz-Carlton Hotel Atlanta,

Documents