Click here to load reader
Nov 10, 2014
MQ/PQ Best Practices:
Valid Selection at the First Hurdle
Lisa J. Lewen, Ph.D., Aon Hewitt
Master Tutorial
April 12, 2013
28th Annual Conference for the Society of Industrial and
Organizational Psychology – Houston, TX
1
MQ/PQ Best Practices: Agenda Section 1 Introduction (5 minutes)
Audience Poll and Baseline Knowledge Check
Objectives
Background
Section 2 Court Cases and Legal Guidelines (20 minutes)
Court Cases Through the Years
Employment Law and Guidelines
Summary
Section 3 Development of MQPQs: Best Practices (30 minutes)
Qualifying Questions and Rules of Thumb
Legal Issues related to Minimum Educational/Degree MQs (time allowing)
Process Checks during Development of MQPQs
Best practices
Possible Derailers
Section 4 Content Validation of MQPQs (10 minutes)
Rating Scales
Content Validation Template
Decision Rules
Section 5 MQPQ Exercise (5 minutes)
Section 6 Questions and Answers (10 minutes)
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
2
Supporting Material – Handout
Slides are available for download at My.SIOP Member
Community webpage
Additional Content available in Supporting Material
Handout (My.SIOP Member Community webpage)
(my.SIOP Community Library » SIOP 2013 Conference
Presentations » Legal Issues/Employment Law )
Expanded MQPQ Content Validation Data Collection
Template
Complete MQPQ Content Validation Rating Scales,
including anchors/definitions
MQPQ Numerical Criteria and Decision Rules
References
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
3
Base-Line Knowledge and Experience with MQPQs
Audience Poll
1. Academics?
2. Internal Consultants?
3. External Consultants?
4. How many have ever developed:
Job Descriptions
MQPQs
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
4
Base-Line Knowledge and Experience of MQPQs
What is your current MQPQ knowledge?
Job Description 1: Plant Accountant (Accounting/Finance/Insurance)
Job Requirements:
Cost accounting and manufacturing background
Bachelors degree
5+ years of business/operations experience
Cross-functional understanding of Operations, Accounting and
Finance
Solid knowledge of SOX and GAAP accounting
Project Management experience
Excellent communication and interpersonal skills
Ability to make decisions and give directions
Ability to manage multiple priorities and tasks
Must be a self-sufficient and self-motivated individual
Rate Appropriateness on a scale of 1(meaningless, subjective)
to 5 (highly useful and objective):
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
5
Base-Line Knowledge and Experience of MQPQs
What is your current MQPQ knowledge?
Job Description 2: Financial Auditor (Finance)
Minimum Qualifications:
BA/BS degree in Finance, Accounting or Business; in lieu of degree, 6 years of relevant
work experience.
Preferred Qualifications:
Experience in Internal Audit projects in the areas of supply chain, revenue and 3rd-party assessments
CPA/CA/CIA with 6 years of audit experience.
Understand how Information Technology impacts overall financial and operational risk
Demonstrated ability to partner with technical resources.
Excellent communication and interpersonal skills
Ability to work with cross-functional teams
Ability to surface and resolve conflicts diplomatically
Ability to manage and coordinate multiple project assignments in a deadline-driven environment,
accepting ownership and accountability of the process and deliver on commitments.
Leadership skills (ability to take charge, motivate others, confidence to interact with all levels, set
objectives, and drive to results)
Ability to work in a fast paced environment and navigate through ambiguity
Rate Appropriateness on a scale of 1(meaningless, subjective) to 5 (highly useful and
objective):
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
6
Base-Line Knowledge and Experience of MQPQs
What is your current MQPQ knowledge?
Job Description 3: Product Manager (GeoCommerce) Minimum Qualifications:
BA/BS in Computer Science or a related technical field; in lieu of
degree, 4 years of relevant work experience.
Preferred Qualifications:
MS, MBA or PhD
5 years relevant work experience
Product management or design experience with a focus on software
products and technologies
Excellent communication and interpersonal skills
Strong technical abilities
Rate Appropriateness on a scale of 1(meaningless, subjective)
to 5 (highly useful and objective):
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
7
MQ/PQ Best Practices: Objectives
This Master Tutorial is designed to help you:
1. Describe relevant case law and legal guidelines pertaining to the development
and validation of MQs/PQs.
2. Develop MQs/PQs according to accepted legal and professional standards.
3. Identify appropriate and inappropriate MQs/PQs from applied examples.
4. Describe methods to content validate MQs/PQs in order to meet accepted
legal and professional standards.
Grand Take-Aways
1. Learn what MQPQs are (and are NOT)
2. Understand the legal guidelines and requirements surrounding MQPQs
(enough so that you may explain this to your internal and external clients)
3. Acquire the skills to develop and validate your own legally defensible MQPQs
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
8
MQ/PQ Best Practices: Background
What are Minimum and Preferred Qualifications (MQPQs)?
Minimum Qualifications (MQs)
Impact applicant pool by initially screening out
applicants determined to be less than minimally
competent
Preferred Qualifications (PQs)
Help focus applicant pool by differentiating among
applicants considered minimally competent
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
9
MQ/PQ Best Practices: Background
What are MQPQs?
Tools used to evaluate applicant experience, education,
training, and certifications to make employment decisions
Level /Amount of education, experience, or combination
Criteria intended to screen out applicants who are unable to
perform the job at a minimally acceptable level
Those who meet or exceed proceed through to the next hurdle
MQs are among the most commonly used selection procedure
in employment settings
Subject to the same set of procedures as are other selection
tests under the Uniform Guidelines
Reason why ensuring legal defensibility is so important
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
10
MQ/PQ Best Practices: Background
Other Clear Benefits:
Minimizes Time
Applicant - especially when not qualified
Employer + Increased ROI
To fill jobs
Saves Money
Reduces the applicant pool
Especially important in difficult economic times
Directed focus towards most competitive applicants
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
11
Court Cases and Legal Guidelines
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
12
MQ/PQ Best Practices: Court Cases through the Years
Griggs v. Duke Power, 1971
MQ = High School Diploma; IQ test; Disparate Impact based on
Race
Lessons Learned: Job-Relatedness and Specificity
Albemarle Paper Co. v. Moody, 1975
MQ = High School Diploma; Two IQ tests; Disparate Impact
based on Race
Lessons Learned: Job-Relatedness and Specificity
Reynolds v. ALDOT, 1994 (additional dates)
MQs = Misc; Disparate Impact based on Race
Lessons Learned : Documentation, Job-Relatedness
Hudson v. Department of Veteran Affairs, 2010
MQ = Years of Experience; Claim of Gender Discrimination
Lessons Learned: Job-Relatedness and Specificity
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
13
MQ/PQ Best Practices: Court Cases through the Years
Griggs v. Duke Power, 1971
Background: African Americans originally limited to Duke‟s
lowest paying jobs; revised after enactment of the Civil Rights
Act
MQs = High School Diploma and IQ test for higher paying
jobs
Result: Adverse Impact for African Americans
Evidence: Neither the high school diploma requirement nor
the IQ test showed a demonstrable relationship to successful
job performance.
Employees who had not completed high school or taken
the IQ tests continued to perform satisfactorily and make
progress
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
14
MQ/PQ Best Practices: Court Cases through the Years
Griggs v. Duke Power, 1971
Defense: Company claimed no intention to discriminate
Offered to help undereducated employees through
Company financing of two-thirds the cost of tuition for high
school training.
Legal Guidelines: Businesses must demonstrate that tests
are job-related.
Use of employment tests are prohibited for selection
purposes if they are not a "reasonable measure of job
performance,
Regardless of the absence of actual intent to
discriminate.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
15
MQ/PQ Best Practices: Court Cases through the Years
Griggs v. Duke Power,1971, cont‟d.
Court Findings: High school diploma and IQ test minimum
requirements (MQs) were:
Broad-based
Not directly related to the jobs performed
Not shown to be related to job performance
Duke Power's selection procedure found to be in violation of
the Act.
Implications: Employer has the burden of producing and
proving the business necessity of a test.
Employer has the burden of showing that any given
requirement (e.g., MQs) has a manifest relationship to the
job in question.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
16
MQ/PQ Best Practices: Court Cases through the Years
Albermarle Paper Co v. Moody, 1975
Background: Class action suit by African Americans
MQs = High School Diploma and two IQ tests
Result: Adverse Impact for African Americans
Court Findings: Employer could not show job relatedness of
MQs
Court Rule: “Even assuming that non-high school graduates
do not perform as well as high school graduates, the question
should be whether non-high school graduates perform
adequately”
For only if the diploma-less are not adequate for a job, may
their exclusion from that job be deemed a business
necessity (530 F.2d at 1180)
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
17
MQ/PQ Best Practices: Court Cases through the Years
Hudson v. Department of Veterans Affairs , 2010
Background: Plaintiff worked as a Health Technician
Applied for “Readjustment Counseling Therapist”
position
Was informed that he did not meet the MQ requirements
MQs: Prior Experience:
Four years of experience as a counselor, readjustment
counselor, therapist, or social worker
Result: Plaintiff claimed he was denied the position due to
Gender Discrimination
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
18
MQ/PQ Best Practices: Court Cases through the Years
Hudson v. Department of Veterans Affairs , 2010, cont’d.
Defense: Selected applicant had extensive experience as a
social worker
Court Findings: Plaintiff failed to prove that the Agency
subjected him to discrimination
Plaintiff was not qualified because he lacked the
requisite counseling experience necessary for the
Counseling Readjustment Therapist position
Implications: Minimum Qualifications of specific prior
experience were:
Job-related
Necessary for performance at a minimally acceptable level
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
MQ/PQ Best Practices: Employment Law and Guidelines
Lack of literature on how to develop and content validate MQs
The EEOC publishes the Uniform Guidelines on Employee
Selection Procedures (1978)
Addresses use of prior training and experience in selection
procedures
Courts have tended to give these procedures “great deference”[1]
“A requirement for or evaluation of specific prior training or
experience based on content validity, including a specification of
level or amount of training or experience, should be justified on
the basis of the relationship between the content of the training
or experience and the content of the job for which the training or
experience is to be required or evaluated.
[1] 95 S. Ct. at 2378 (quoting Griggs, 91 S. Ct. at 854).
19
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
20
MQ/PQ Best Practices: Employment Law and Guidelines
According to the EEOC – “Uniform Guidelines” (UG):
Content Validating Job Qualifications Requires:
Specifying the level of the qualification
Specifying the amount of the qualification
Specifying the linkage between the qualifications and important
work activities and behaviors and characteristics required for
successful performance on the job:
“The critical consideration is the resemblance between the specific
behaviors, products, knowledges, skills, or abilities in the
experience or training and the specific behaviors, products,
knowledges, skills, or abilities required on the job, whether or not
there is close resemblance between the experience or training as a
whole and the job as a whole.”
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
21
MQ/PQ Best Practices: Employment Law and Guidelines
EEOC indicated that MQs requiring a high school diploma
may violate the Americans with Disabilities Act (2011).
Informal letter and response from the agency: (http://www.eeoc.gov/eeoc/foia/letters/2011/ada_qualification_standards.html )
“Thus, if an employer adopts a high school diploma requirement
for a job, and that requirement “screens out” an individual who is
unable to graduate because of a learning disability that meets
the ADA‟s definition of “disability,” the employer may not apply
the standard unless it can demonstrate that the diploma
requirement is job related and consistent with business
necessity. The employer will not be able to make this showing,
for example, if the functions in question can easily be performed
by someone who does not have a diploma.”
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
22
MQ/PQ Best Practices: Employment Law and Guidelines
EEOC‟s standing on Criminal Background Checks (Arrest-
Conviction) as MQs
Pepsi Settlement - $3.13 million and offer jobs and training to black
applicants impacted by its criminal background check policy
Pepsi refused to hire anyone for a permanent position who had been
arrested pending prosecution, whether convicted or not
Arrests v. Convictions
Time since crime occurred
Correlation with SES
As a best practice, and consistent with applicable laws, EEOC
recommends that employers not ask convictions on job applications
And, if and when they make such inquiries, inquiries be limited to
convictions for which exclusion would be job related for the and consistent
with business necessity.
Key: Determining whether applicant credit history is job related
and consistent with business necessity
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
23
MQ/PQ Best Practices: Employment Law and Guidelines
EEOC‟s standing on Credit Check issues as MQs
Inquiry into an applicant's current or past assets, liabilities, or
credit rating, including bankruptcy or garnishment, refusal or
cancellation of bonding, car ownership, rental or ownership of a
house, length of residence at an address, charge accounts,
furniture ownership, or bank accounts generally should be
avoided because they tend to impact more adversely on
minorities and females. Exceptions exist if the employer can
show that such information is essential to the particular job in
question.
Key: Determining whether applicant credit history is
job related and consistent with business necessity
http://www.eeoc.gov/laws/practices/inquiries_credit.cfm
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
24
MQ/PQ Best Practices: Employment Law and Guidelines
Office of Federal Contract Compliance Programs (OFCCP,
2005)
Recommendations for developing and validating
MQs/PQs:
“The proposed definition [of MQPQs] further provided
that „„advertised, basic qualifications‟‟ must be
noncomparative, objective, and job-related (69 FR
16449–450).”
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
25
MQ/PQ Best Practices: Employment Law and Guidelines
Why are professional guidelines relevant to minimum
qualifications?
Guidelines specify professionally accepted best practices for
validating qualifications
Principles for the Use and Validation of Personnel Selection
Procedures (2003)
Content validating job qualifications requires:
Establishing a relationship between content of
experience and content of work requiring that
experience
Should be more than a superficial resemblance
(e.g., job titles, course titles)
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
26
MQ/PQ Best Practices: Case Law and Court Cases
Legal Guidelines Summary:
MQPQs must be based on legally defensible processes
and procedures
Inappropriate MQs may be challenged, leaving an
organization vulnerable
While requirements and expectations for MQPQs do
exist in courts and enforcement agencies:
May be difficult to locate and follow
However, doing so is critical:
Will result in a higher legal defensibility and support
from the courts
Ensure all MQPQs are based in solid Job Analysis
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
27
MQ/PQ Best Practices: Court Cases and Employment Law
Legal Guidelines Summary, cont’d.:
Document, Document, Document:
Develop standardized procedures
Document standardized procedures in a manual
Ensure accessibility to current and future individual
involved in the process
Support Your Decisions
Retain documentation of support for your procedures (e.g.,
relevant case law, professional literature, etc.)
Communicate the basis for MQPQs
Consider legal precedent
Become familiar with case law
Consult with counsel
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
28
MQ/PQ Best Practices: Court Cases and Employment Law
Legal Guidelines Summary, cont’d.: Be “Risk Averse”:
Use most well supported route
Don‟t use untested or experimental procedures
Involve outside experts early
Experts should review their own prior writings and testimony to ensure
there is no significant conflict
Know Your Limits (and Guidelines):
Adhere to the Uniform Guidelines
The Uniform Guidelines are federal regulations, and remain an important
legal standard that courts may apply to determine compliance with the law
Choose Your Support team Carefully:
Limit the number of staff members
Only highly trained and well qualified people should perform most of the
development work
Less experienced assistants - closely supervised and refrain from
makingjudgment calls
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
MQPQ Best Practices:
The Development Process
29
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
30
MQ/PQ Best Development Practices: A Closer Look
MQPQ – Content
Different types (e.g., training/education & experience,
competency-based, task-based)
Focus today is on MQPQs containing:
Education/Training
Previous work experience
Certifications
Most important qualifying question:
What is the least amount or type required for an
incumbent to perform the job at a minimally
acceptable level?
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
31
MQ/PQ Best Development Practices: A Closer Look
Rules-of-Thumb:
1. Must not involve comparing qualifications of one person to
another
e.g., Cannot be: Passed certification in top 10% of scores (vs. 3
years of experience in finance and accounting)
2. Must be job-related
e.g., Cannot be: Leadership experience for an entry level job
3. Must be objective (measurable)
e.g., Cannot be: “Effective written and verbal communication skills”
“a third party, unfamiliar with the employer's decision
process, should be able to evaluate whether the job
seeker possesses the qualification without more
information about the employer's judgment”
- Governmental inter-agency task force, including among others the Office of Federal Contract Compliance Programs and the Equal
Employment Opportunity Commission supplemental OFCCP proposed “common definition”.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
32
MQ/PQ Best Development Practices: Rules of Thumb
Avoid General Requirements:
“High School Education”
“2 years of experience”
Develop Specific Requirements:
“Bachelor's Degree in Statistics, Math, Computer
Science, Engineering or other quantitative field”
“2 years work experience in accounting, finance, or
related field”
“2 years work experience in accounting, finance, or
related field, performing duties such as: accounts
receivable, payroll, and bank reconciliation”
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
33
MQ/PQ Best Development Practices: Rules of Thumb
Process Checks:
Could a new hire perform the essential functions of the job at a
minimally acceptable level without this qualification? If yes, qualification should not be considered for an MQ
Would training or on-the-job experience enable the new hire to
acquire this qualification within the first 6 months? If yes, qualification should not be considered for an MQ
Does the qualification relate to some legal, professional, or other
externally required pre-requisite for this position (e.g., license,
certification, degree, etc.)?
If yes, qualification would be considered for an MQ
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
34
Legal Issues Related to Education/Degree MQs
Some jobs do require a minimum education/degree to function at
a minimally acceptable level
Whenever possible, create alternate MQs with experience as a
substitute
e.g. MQ: 3 years work experience in accounting, finance, or related field,
performing duties such as: accounts receivable, payroll, and bank
reconciliation
e.g. PQ: Bachelor‟s degree in Finance, Accounting, or related field. Alternate:
Legal vulnerability exists when the only MQ is a minimum degree
If required, the degree MQ should be as specific (job related) as
possible
Possible issues/consequences: Adverse impact in the applicant pool
Impediments to career progression
Immigration via sponsorship
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
35
Legal Issues Related to Education/Degree MQs
To proactively address issues when utilizing a sole degree MQ: 1. Justification: Document and Approve
What about this job has changed in order to now require a
degree?
What is it about the job that requires a degree (technical, etc.)?
2. 360O Review: Peers, and those incumbents in positions one level
above and below
Should a degree be required at the immediate peer level to this
position?
Should a degree also be required above or below this position?
Do the associates above or below this position have degrees?
Would requiring a degree at this position prevent current
associates from promoting/demoting into this position?
Would requiring a degree prevent overall career progression
and/or succession planning for multiple levels of associates?
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
36
Legal Issues Related to Education/Degree MQs
To proactively address issues when utilizing a sole degree MQ, cont’d.:
3. Exit Strategy: Review all current associates to identify those not meeting
new degree requirements
If any of the current associates do not meet the degree requirement,
an exit strategy must be developed in order to move those
associates out of that position within the next 18-24 months. If this
is not done, then a degree requirement is not required of the
position.
4. Liaison for Sponsorship: Identify an HR representative to act as liaison
with Legal and Selection & Assessment Teams to ensure compliance
with Immigration Law should sponsorship be necessary for any
candidates
Provide additional training for both the Liaison and Hiring Managers
to ensure understanding of sponsorship limitations and
requirements.
HR representative must work with the Legal Department prior to
moving a candidate to another position to ensure compliance with
Immigration Law.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
37
Legal Issues Related to Education/Degree MQs
Possible Consequences of Using a Degree as a Stand-Alone MQ:
1. Limitation of the Applicant Pool
Unintentionally excluding applicants otherwise qualified
for the job
Creating such small pools that you cannot fill position
Impacting the number of diverse candidates
2. Limiting associate career progression
Associates cannot progress into the position until they
have received a degree.
Those in the position today who are successful, but
have no degree, must exit the position.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
38
Legal Issues Related to Education/Degree MQs
Possible Consequences of Using a Degree as Stand-Alone MQ, cont’d.: 3. Implications for Immigration visa sponsorship (Specialty Occupation Workers: H-
1B Visa) Reserved for certain professionals (who come to the U.S. for a limited amount of
time
Invites the government to audit the organization‟s hiring practices, including MQs
for the position occupied by the foreign national.
Subject to yearly numerical limitations imposed by US Customs and Immigration
Services (USCIS)
Degree must be a usual requirement for the position
When it is time to apply for a green card for the foreign applicant, the position must
be reposted internally and externally. This includes advertising in newspapers, on job boards, on the internal & external job
portals, etc. inviting qualified U.S. applicants to compete for the position.
Open positions must be given to U.S. applicants over foreign incumbent even if
U.S. applicant is not as qualified as the foreign national who has been performing
in the position.
Once a company has brought an employee to the U.S. on an H-1B visa, should
the company dismiss that employee before the expiration of the visa, the company
is liable for any reasonable costs that the employee incurs in moving
himself/herself and his/her effects, back to his/her last foreign residence.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
39
MQ/PQ Best Development Practices: The Process
Job Analysis:
Identify the important work activities and behaviors
(WABs) performed on the job
Identify the important knowledge, skills, and abilities
(KSAs), and other characteristics involved in the job
Review and summarize the information
Enlist the help of SMEs to draft MQ and PQ statements
Goal is to standardize MQPQs by job level across the
organization
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
40
MQ/PQ Best Development Practices: The Process
Talking Points to begin process: 1. How many current incumbents would meet the proposed
qualification(s)
Or would be excluded from consideration?
2. Are the qualifications the same or comparable to those of
jobs at the same level? If not, is there a good rationale for the differences?
3. Are there differences in degree or experience
requirements for jobs above and below target job? If so, what is the rationale?
4. Will the MQs restrict movement of associates from level
to level?
5. How long would a person need to stay in a given position
before moving to the next level?
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
41
MQ/PQ Best Development Practices: The Process
Best Practice Approach
Step 1: Select SMEs: representative, qualified,
experienced
Step 2: SMEs review important WABs and KSAs (imp
and RUE) for the job in question
Step 3: SMEs determine MQs for each job
Step 4: Job analysts compile MQs, revise as appropriate
and ensure consistency with legal guidelines
Step 5: SMEs rate MQ appropriateness and linkage to
important WABs and KSAs (validation)
Step 6: Retain MQs that meet criteria for inclusion
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
42
MQ/PQ Best Validation Practices: The Process
Potential Derailers During the MQPQ Development Process:
1. Identifying important WABs and KSAs
2. Linking WABs to KSAs
3. Grouping jobs into job families:
Career progression - MQPQs
4. Number of facilitators
Too many cooks
non-SMEs (e.g., HR, Legal)
5. SME-related issues:
Facilitating SME progress and content
“Raising the Bar”
Desire to vent rather than produce
6. Scheduling Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
43
MQ/PQ Best Development Practices: The Process
SME-related issues: Facilitating MQPQ development progress
Helpful Talking Points for SME Discussion:
1. If an individual had no prior experience, is there a level of education
that would allow him/her to perform minimally? If so, which education
level?
What courses/area would that individual have to have experience
in to ensure that he/she would be successful at the job at a
minimally acceptable level? (specificity)
2. If an individual had no education at all, how much experience would
he/she need to perform minimally?
What type of experience would that individual need to have that
was related to the job?
3. How much experience would the individual need with each additional
level or type of education?
4. Is there a licensure or certification required to perform at this job
level?
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
44
MQ/PQ Best Development Practices: The Process
MQPQ Examples Good:
Bachelors Degree in Accounting, Finance, or related field OR four years work experience in Accounting, Finance, or related field.
Bad:
3 years work experience and 2 years project management experience OR 5 years work experience; OR equivalent education.
Ugly:
Internal Applicants: Finance and Sr. Project Management Experience Required.
External Applicants: Must have led Finance and Strategy or Business teams. Seeking individual who can think analytically, build/lead a team to accomplish a strategic vision, and communicate exceptionally well.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
MQPQ Best Practices:
The Key to Legal Defensibility: The Validation Process
45
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
46
MQ/PQ Best Validation Practices: The Process
Ratings are CRITICAL to the validation process
“Importance” ratings
How important is the qualification to performing the job
effectively?
3=Critical; 2=Important; 1=Minor Impact; 0=Not Required
“Level” ratings
Is this qualification a reasonable expectation of a minimally
acceptable candidate? (Minimum Qualifications)
Is this qualification a reasonable expectation of an applicant
beyond the minimum requirements? (Preferred Qualifications)
3=Too Much; 2=Reasonable Expectation; 1=Not Enough
“Linkage” ratings (Is MQ/PQ required to perform imp. WABs?)
(X=Yes)
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
47
MQ/PQ Best Validation Practices: The Process
Data Collection Template: Content Validation Ratings
Job
Importance
Rating
(0-3)
Level Rating
(1-3)
Is MQ Required
to Perform Imp.
WABs? (X=Yes)
Organization ID IEC - International eCommerce
Job ID ABC1234
Job Title Senior Director, Insights &
Analytics
Position Level Senior Director
Minimum Qualification 1
Minimum Qualification 2
Minimum Qualification 3
Minimum Qualification 4
Minimum Qualification 5
Job
Importance
Rating
(0-3)
Level Rating
(1-3)
Is PQ Required to
Perform Imp.
WABs? (X=Yes)
Preferred Qualification 1
Preferred Qualification 2
Preferred Qualification 3
Preferred Qualification 4
Preferred Qualification 5
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
48
MQ/PQ Best Validation Practices: The Process
Decision Rules During the MQPQ Validation Process:
1. Retain Important MQPQs (ratings ≥ 2)
Discard those MQPQs that are not important or
of minor importance
2. Retain MQPQs at levels of Reasonable
Expectation (ratings must = 1)
Discard those MQPQs too easy or too difficult
3. Retail only those MQPQs that link to important
WABs
If an MQPQ is not related to important work
activities and behaviors, it does not qualify
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
49
MQPQ Exercise
Apply Your MQPQ Knowledge :
Exercise: Production Engineering Manager – Job Description Qualifications:
Bachelor‟s Degree in Industrial Engineering or a related field.
Previous Industrial Engineering experience in the manufacturing
industry.
Excellent written and verbal communication skills
Must be able to demonstrate effective people skills
Advanced computer skills and proficiency with office software
(Microsoft Word, Excel, Outlook)
Must be a self-starter and team player
How would you change the above qualifications to be consistent with
the MQPQ development and validation best practices you have
learned?
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
50
MQPQ Exercise
Apply Your MQPQ Knowledge : Objective and Valid version!
Exercise: Production Engineering Manager – Job Description Minimum Qualifications: Bachelor‟s Degree in Industrial Engineering or a related field
X years experience in Industrial Engineering
X years of supervisory experience
Excellent written and verbal communication skills
Must be able to demonstrate effective people skills
Advanced computer skills and proficiency with office software
Must be a self-starter and team player
Preferred Qualifications: X years experience in Industrial Engineering in the manufacturing industry
X years experience in a Production role to include preparing layouts of
equipment, and/or coordinating and conducting production line trials of new
or modified equipment
X years experience in developing and executing equipment training
programs
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
51
MQ/PQ Best Practices: Questions and Answers
In Closing:
Thoughts?
Questions?
Experiences You Would Like to Share?
Feel free to contact me with any questions or
requests for guidance!
THANK YOU!!!!
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
52
Base-Line Knowledge and Experience of MQPQs
What is your current MQPQ knowledge?
Job Description 1: Plant Accountant (Accounting/Finance/Insurance)
Job Requirements:
Cost accounting and manufacturing background →½ point
Bachelors degree → worst offender
5+ years of business/operations experience →½ point
Cross-functional understanding of Operations, Accounting and
Finance
Solid knowledge of SOX and GAAP accounting →½ point
Project Management experience →½ point
Excellent communication and interpersonal skills
Ability to make decisions and give directions
Ability to manage multiple priorities and tasks
Must be a self-sufficient and self-motivated individual
Rate Appropriateness on a scale of 1(meaningless, subjective)
to 5 (highly useful and objective): 1
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
53
Base-Line Knowledge and Experience of MQPQs
What is your current MQPQ knowledge?
Job Description 2: Financial Auditor (Finance)
Minimum Qualifications:
BA/BS degree in Finance, Accounting or Business; in lieu of degree, 6 years of work
experience. →½ point
Preferred Qualifications:
Experience in Internal Audit projects in the areas of supply chain, revenue and 3rd-party assessments
→ 1 point
CPA/CA/CIA with 6 years of audit experience. → 1 point
Understand how Information Technology impacts overall financial and operational risk
Demonstrated ability to partner with technical resources.
Excellent communication and interpersonal skills
Ability to work with cross-functional teams
Ability to surface and resolve conflicts diplomatically
Ability to manage and coordinate multiple project assignments in a deadline-driven environment, accepting
ownership and accountability of the process and deliver on commitments.
Leadership skills (ability to take charge, motivate others, confidence to interact with all levels, set objectives,
and drive to results)
Ability to work in a fast paced environment and navigate through ambiguity
Rate Appropriateness on a scale of 1(meaningless, subjective) to 5 (highly useful and
objective):
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
54
Base-Line Knowledge and Experience of MQPQs
What is your current MQPQ knowledge?
Job Description 3: Product Manager (GeoCommerce) Minimum Qualifications:
BA/BS in Computer Science or a related technical field; in lieu of
degree, 4 years of work experience. →½ point
Preferred Qualifications:
MS, MBA or PhD
5 years work experience
Product management or design experience with a focus on software
products and technologies
Excellent communication and interpersonal skills
Strong technical abilities
Rate Appropriateness on a scale of 1(meaningless, subjective)
to 5 (highly useful and objective):
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
55
Biographies
Lisa J. Lewen, Ph.D. is a Consultant in Aon Hewitt‟s Selection & Assessment Service
Line located in Atlanta, GA, providing services in the design, validation, and
implementation of employee selection procedures. Her focus is on delivering
technically sound and impactful business solutions that enable organizations to hire,
retain, and promote top performers while decreasing employee turnover and
maximizing legal defensibility to include products such as: work simulations, job
knowledge tests, structured interviews, and computer adaptive tests.
Dr. Lewen is known for her expertise in the area of litigation support, including providing
internal and external consulting on maximizing the legal defensibility of selection
practices; conducting adverse impact analyses and writing reports targeted for
attorneys, EEO representatives, and the OFCCP; and supporting clients during OFCCP
audits and investigations. Dr. Lewen has worked with public sector and Fortune® 100
clients across an extensive range of industries, ranging from law enforcement and fire
and rescue to retail, pharmaceutical, telecommunications, and power generation.
Dr. Lewen received her M.S. and Ph.D. in Industrial/Organizational Psychology from
the Georgia Institute of Technology, with a minor in Quantitative Psychology. She also
received her B.S. degree from McGill University in Psychology. Lisa has presented her
research in the areas of methods of unproctored internet testing (UIT), employee
selection procedures, cognitive aging, as well as discrimination and adverse impact at
national conferences for the Society for Industrial and Organizational Psychology
(SIOP), the American Psychological Association (APA), and the American Telemedicine
Association, as well as at the Cognitive Aging Conference (Atlanta, GA).
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
56
Contact List
Lisa J. Lewen, Ph.D.
Consultant, Performance, Reward & Talent
Aon Hewitt
3350 Riverwood Pkwy, Ste 80, Atlanta, GA 30339
Office: 1.770.690.7767; Mobile: 1.404.859.6998
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013
57
Legal Disclaimer
This Master Tutorial, including any handouts, is intended
only for SIOP 2013 conference participants. Unauthorized
individuals or entities are not permitted access to this
information. Any dissemination, distribution, disclosure, or
copying of this information is unauthorized and strictly
prohibited.
Selection & Assessment | Performance, Reward and Talent
Proprietary & Confidential | April 12, 2013