This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
• Demonstrated by empirical data showing that the selection procedure is predictive of, or significantly correlated with, important elements of work behavior
• Relies on “correlations” between tests and job criteria
The closer to +1.00 or -1.00 the The closer to +1.00 or -1.00 the stronger the relationship between the stronger the relationship between the variablesvariables
The stronger the relationship between The stronger the relationship between two variables, the better the ability to two variables, the better the ability to predict one if given the otherpredict one if given the other
VG = Meta Analysis Applied to Test Validation Research
• VG applies meta-analysis techniques to combine the results of several validation studies to form general theories about relationships between variables across different situations
• Schmidt & Hunter (1977) opened the gate to VG techniques in the personnel testing field
Factors That Can Influence Validity From “moving” Between SituationsFactors Before/At Testing Situation Factors Occurring After Testing
• Sample Size• Base Rate (% of applicants who “show
up qualified”)• Competitive Environment• Other Selection Procedures Used
Before/After the Test• Test Content• Test Administration Conditions
(proctoring, time limits, etc.)• Test Administration Modality (e.g.,
written vs. online)• Test Use (ranked, banded, cutoffs
used)• Test Reliability (e.g., internal
consistency)• Test Bias (e.g., culturally-loaded
content)
• Job Content Comparability • Job Performance Criteria• Reliability of Job Performance Criteria • Level of Supervision/Autonomy• Level/Quality of Training Provided• Org./Unit Demands & Constraints• Job Satisfaction• Management Styles and Role Clarity• Reward Structures and Processes• Organizational Citizenship, Morale, and
Commitment of the Workforce• Organizational Culture, Norms, Beliefs, Values,
Expectations Surrounding Loyalty and Conformity
• Organizational Socialization Strategies for New Employees
• Formal and Information Communication (Style, Levels, and Networks)
• Centralization and Formalization of Decision-Making
Test Validation & Adverse Impact Civil Rights Act of 1991
Amends Section 703 of the 1964 Civil Rights Act (Title VII)(k)(1)(A). An unlawful employment practice based ondisparate impact is established under this title only if:
• A(i) a complaining party demonstrates that a respondent uses a
particular employment practice that causes a disparate impact on the basis of race, color, religion, sex, or national origin, and the respondent fails to demonstrate that the challenged practice is job-related for the position in question and consistent with business necessity; OR,
• A(ii) the complaining party makes the demonstration described in subparagraph (C) with respect to an alternate employment practice, and the respondent refuses to adopt such alternative employment practice.
• “. . . the expert failed to visit and inspect the Atlas office and never studied the nature and content of the Atlas clerical and office jobs involved. The VG theory utilized by Atlas with respect to this expert testimony under these circumstances is not appropriate. Linkage or similarity of jobs in dispute in this case must be shown by such on site investigation to justify application of such a theory.”
• The premise of the VG theory . . . is that intelligence tests are always valid. The first major problem with a VG approach is that it is radically at odds with Albemarle Paper v. Moody, Griggs v. Duke Power, relevant case law within this circuit, and the EEOC Guidelines, all of which require a showing that a test is actually predictive of performance at a specific job. The VG approach simply dispenses with that similarity or manifest relationship requirement . . . (emphasis added) (EEOC v. Atlas Paper, 868 F.2d. at 1499).
• When the courts evaluate criterion-related validity evidence, four basic elements are typically inspected: – Statistical significance– Practical significance– Type and relevance of the job criteria– Evidence to support the specific use of the test
• VG has a difficult time answering these questions…
Recommendations for Applying VG in Personnel Testing Research
• Recommendation #1: Address the evaluation criteria provided by the Uniform Guidelines, Joint Standards, and SIOP Principles regarding the evaluation of the internal quality of the VG study. This will help insure that the VG study itself can be relied upon for drawing inferences.
• Key Factors:– Publication Bias– Corrections Made and Underlying
Assumptions/Justifications– Similarities of Tests and Criteria
Recommendations for Applying VG in Personnel Testing Research
• Recommendation #2. Address the criteria provided by the Uniform Guidelines, Joint Standards, and SIOP Principles regarding the similarity between the VG study and the local situation.– Helps to insure that the VG study can be relied upon and the
research is relevant to the local situation (similarities between tests, jobs, job criteria, etc.).
– The most critical factor evaluated by courts when considering VG evidence is the similarity between jobs (see also 7B of the Uniform Guidelines).
– VG evidence is the strongest where there is clear evidence that the job duties between the target position and those in the positions in the VG study are highly similar as shown by a job analysis in both situations.
Recommendations for Applying VG in Personnel Testing Research
• Recommendation #3: Only use VG evidence to supplement other sources of validity evidence (e.g., content validity or local criterion-related validation studies) rather than being the sole source. – Supplementing a local criterion-related validity study
with evidence from a VG study may be useful if an employer has evidence that statistical artifacts (not situational moderators) suppressed the actual validity of the test in the local situation (provided that the job comparability criteria of 7B UGESP has been met).
Recommendations for Applying VG in Personnel Testing Research
• Recommendation #4: Evaluate the test fairness evidence from the VG study using the methods outlined by the Uniform Guidelines, Joint Standards, and SIOP Principles.
• Recommendation #5: Evaluate and consider using “alternate employment practices” that are “substantially equally valid” (as required by the 1991 Civil Rights Act Section 2000e-2[k][1][A][ii] and Section 3B of the Uniform Guidelines).