1 Non-Response Bias Analyses of the Survey of Workplace Violence Prevention Andrew Kato, Kathy Downey, William McCarthy, and Samantha Cruz U.S. Bureau of Labor Statistics The opinions expressed here are those of the authors and do not represent official policy of the Bureau of Labor Statistics.
23
Embed
1 Non-Response Bias Analyses of the Survey of Workplace Violence Prevention Andrew Kato, Kathy Downey, William McCarthy, and Samantha Cruz U.S. Bureau.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Non-Response Bias Analyses of the Survey of Workplace Violence Prevention
Andrew Kato, Kathy Downey, William McCarthy, and Samantha Cruz
U.S. Bureau of Labor Statistics
The opinions expressed here are those of the authors and do not represent official policy of the Bureau of Labor Statistics.
2
Survey of Workplace Violence Prevention
Special 2005 study conducted for NIOSH (National Institute for Occupational Safety and Health)
Workplace Violence Prevention (WVP) Prevalence of security features, The risks facing employees, Employer policies and training, and Related topics associated with maintaining a
safe work environment
3
WVP: Sampling Sample taken from respondents to 2003
SOII (Survey of Occupational Injuries and Illnesses) SOII - Private industry, State and local
government public sector units from the fourth Quarter 2003
Longitudinal Database (LDB) and mining and railroad establishments
Total of 39,998 units Randomly selected units proportional to size and
oversampled within specific industries Used respondents so have prior relationship
4
WVP: Methodology Hardcopy was 12 pages, envelope and
insert; also available to non-respondents in Word via e-mail
Voluntary Protocol
Initial mailing to SOII respondent (Sept 05), Follow-up mailing to non-respondents, Address corrections for post office returns, and Telephone follow-ups to non-respondents Close-out June 06
Final response rate was 61%
5
Purpose of Non-Response Analyses
OMB requirement Examine potential bias due to non-
response since predicted response rate might be low
6
Predominant Approaches to Conducting Non-Response (Olson, 2006)
Comparing characteristics with a benchmark survey
Comparing frame information between respondents and non-respondents*
Simulating statistics based on restricted protocol (“level of effort” analyses)*
Mounting experiments to produce varying response rates across groups
7
Data Available for Non-Response Analyses
2003 SOII frame data Size class
1: 1-10 2: 11-49 3: 50-249 4: 250-999 5: 1000+
Sector (industry) – combined private and public
2003 SOII Rate (per 10,000 FTE hours) of job transfer or
restriction Rate (per 10,000 FTE hours) of days-away-from-
work cases
8
Data Analyses
Compare non-respondents and respondents on 2003 frame data and survey responses
Level of effort analyses: response propensity models
9
Comparing WPV Respondents and Non-Respondents: Size Class, by % NR
*p<.0001 for overall
5149
56.1
43.9
60.2
39.9
58.3
41.8
55.3
44.7
0
10
20
30
40
50
60
70
1 2 3 4 5
Respondents Non-Respondents
10
Comparing WPV Respondents and Non-Respondents: Size Class, by Average Days Away from Work Rate
*p<.003 for class 1, p<.0001 for class 3
0.4
1
1.2
1.4
1.9
2.1
1.71.8
1.5 1.5
0
0.5
1
1.5
2
2.5
1 2 3 4 5
Respondents Non-Respondents
11
Comparing WPV Respondents and Non-Respondents: Size Class, by Average Job Transfer or Restriction Rate
*p<.0001 for classes 3, 4, and 5
0.2
0.3
0.50.5
1.4
1.2
1.8
1.3
1.5
1.1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
1 2 3 4 5
Respondents Non-Respondents
12
Comparing WPV Respondents and Non-Respondents: Selected Sectors, by % NR
*p<.0001 for selected sectors above
36
64
38.6
61.4
34.9
65.1
30.9
69.1
37.9
62.1
39.5
60.5
0
10
20
30
40
50
60
70
Agric Constr Healthcare
Manuf PubAdmin
Educ
Respondents Non-Respondents
13
Comparing WPV Respondents and Non-Respondents: Selected Sectors, by Average Days Away from Work Rate
*p<.02 transport, p<.004 rest
2.2
2 2
1.3
1.8
1.5
3.6
3.8
2.3
1.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Health RealE Retail Trans Whole
Respondents Non-Respondents
14
Comparing WPV Respondents and Non-Respondents: Selected Sectors, by Average Job Transfer or Restriction Rate
*varying significant p values
2
1.3 1.3
0.9
1.61.5
1.8
2.1
0.6
0.4
2.4
2.1
0.8
0.4
0.8
0.5
0
0.5
1
1.5
2
2.5
Whole Retail Health Trans Info Manuf PubAd AdminS
Respondents Non-Respondents
15
Comparing WPV Respondents and Non-Respondents: Conclusions
Dealing with four projects when discussing: 2003 SOII sample and analyzed 2003 SOII
estimates of DAW and JTR (injuries/illness) WPV survey – existence of violence
prevention programs This NR project 2001 one-time Respirator Survey – used
1999 SOII sample
16
Comparing WPV Respondents and Non-Respondents: Conclusions
Non-respondents versus respondents Higher size class showed curvilinear
effect with size 3 highest Industry had some variation
possibly those with more public units
17
Comparing WPV Respondents and Non-Respondents: Conclusions
60.8
39.2
57.2
42.8
0
10
20
30
40
50
60
70
Public Private
Respondents Non-Respondents
*p<.0001
18
Comparing WPV Respondents and Non-Respondents: Conclusions – Days Away from Work, Job Transfer
Higher rates are responding more Size
Days away from work rate: highest class 3, lowest 1
Job transfer or restriction rate: highest 3, 4, 5 Industry
Days away from work rate: wholesale, retail, real estate, health care, transportation
Job transfer or restriction rate: same plus public admin, admin support, manufacturing
19
Level of Effort: Response Propensity Models for Contact and Cooperation
Predicting Contact
Predicting Cooperation
Coeff SE Coeff SE
Intercept 5.09** 0.16 5.59** 0.21
Size 1 0.07 0.17 -0.07 0.20
Size 2 -0.04 0.08 -0.04 0.09
Size 3 0.09 0.05 0.14* 0.06
Size 4 -0.01 0.05 0.01 0.06
Hours (FTE worked) 0.00 0.00 0.00 0.00
DART case rate* 0.01 0.00 0.01* 0.00
1st NR mail – Nov 05 -0.19 0.21 -0.82* 0.26
1st round calling 0.29 0.15 -0.83** 0.18
2nd NR mail – Apr 06 -5.10** 0.17 -3.88** 0.13
2nd round calling 12.46 163.1 -5.53** 0.15
*p<.05
**p<.0001
20
Level of Effort: Response Propensity Strata for Contact and Cooperation
Response Propensity Strata
Low Grp 2 Grp 3 Grp 4 High
Contact
Actual rate (n)
51% (5021)
55% (5021)
88% (5021)
99% (5021)
97% (5021)
Est. Non-contacts
51% 54% 58% 99% 99%
Est. Contacts
51% 54% 92% 99% 100%
Cooperation
Actual rate
36% (1410)
55% (2179)
98% (3883)
99% (3921)
100% (3935)
Est. Ref. 32% 55% 98% 100% 100%
Est. Coop 43% 56% 99% 99% 100%
21
Conclusions
Some differences between WPV respondents and non-respondents Size Industry SOII estimates
Respondents to WPV have more programs and more SOII incidents
Not sure what impact to WPV, possibly respondents have more programs and more risks (over-reporting?)
22
Limitations
Limitation: only as good as phone logs from vendor (contact/non-contact) Learned over surveys to not have
subcontractors so can have more detail in phone logs, manage mailings/contacts
23
Future Research
Level of effort analyses tied into data estimates How data might change for key WPV
estimates at different levels of effort (truncation)
Huge work to re-weight, though More WPV analyses – 3-digit NAICS,