Gamification of On-line Surveys for Public Health Data Collection by Corey P. J. Alexander A Thesis Presented to The University of Guelph In partial fulfilment of requirements for the degree of Masters of Science in Computer Science Guelph, Ontario, Canada c Corey P. J. Alexander, January, 2017
96
Embed
Gami cation of On-line Surveys for ... - University of Guelph
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Gamification of On-line Surveys for Public Health Data Collection
Gamification of Online Surveys for Public Health Data
Collection
Corey P. J. Alexander Advisor:
University of Guelph, 2016 Dr. Daniel Gillis
Standard survey methods have many issues associated with them, especially when
they contain a large number of questions, or take a long time to complete. These
problems come in the form of user fatigue, a lack of interest in the survey, or inaccurate
and incomplete answers. We hypothesized that the use of badging - a self element
of gamification - would increase completion rates, increase time spent on the survey,
and have no effect on straight-lining behaviour. This research describes the specific
tools that have been developed to investigate the use of Gamification (specifically
badging) and answer these hypotheses. Results indicated that badging had no effect
on completion rates, although this could be an artifact of participant selection bias.
However, the study does suggest that gamification, specifically badging, increases the
time a user spends on the survey.
iii
DEDICATION
For CPJA, because you know.
iv
ACKNOWLEDGEMENTS
I would like to thank my advisor Dr. Daniel Gillis of the School of ComputerScience at the University of Guelph. I view Dr. Gillis as a mentor and a friend.I would like to thank him for the wisdom and guidance that he has provided mewhenever I needed it, not only in my research and academics but also life in general.I am grateful to have had the pleasure to work with and learn from Dr. Gillis. Thankyou.
I would also like to acknowledge Dr. Judi McCuaig of the School of ComputerScience at the University of Guelph as the second reader of this thesis. Dr. McCuaig,much like Dr. Gillis, was always there to support me and guide me in my researchand academics but also in my life. Thank you.
Without the support and guidance of both Dr. Daniel Gillis and Dr. Judi McCuaig,I would not have made it to where I am today. I would like to sincerely thank bothof them for all the support they have given me during my time with them.
I would like to thank Dr. Stacey Scott for being my external examiner and for thevaluable comments she has provided me.
I would like to thank Colin Howes and Dominic Gagne for their help with buildingparts of the experiment. It was a learning experience for all of us and I wouldn’t havemade it through as easily as I did without their help and dedication.
Finally, I would like to thank all of my friends and family who have supported meduring my degree.
B.1 Lookup for the Category ID and the Name of the Category. . . . . . 82B.2 Wilcoxon Rank Sum Results for Question Delta Timers . . . . . . . . 85B.3 Wilcoxon Rank Sum Results for Question Relative Delta Timers . . . 88
Table 4.1: Completion data collected for each type of survey given.
people who started a survey, 91 (82%) completed the survey, and 20 left the website
before finishing their survey. Table 4.1 breaks down further this information based on
the type of survey a user was randomly assigned; badged or non-badged. Specifically,
54 of the 111 people who started a survey were given a badge survey to complete,
while 57 were given a non-badged survey. Of those who started a badge survey, 45 of
them were able to finish the given survey leaving only 9 unfinished. Similarly, 46 of
57 people were able to complete a non-badged survey. That is, approximately 83%
and 81% of respondents completed the respective surveys.
Respondents who answered the non-badged survey spent an average of 553.53 sec-
onds on the survey, while badged respondents spent an average of 635.34 seconds.
That is, badged survey respondents spent on average 81.81 seconds longer answering
the survey. Median time spent on the surveys was 519.81 seconds and 553.53 seconds
for non-badged and badged surveys, respectively. The difference in medians is 33.72
seconds.
43
By design, respondents who were randomly assigned a badged survey received a
random number of badges. Table 4.2 outlines some basic statistics describing the dis-
tribution of badges received. The maximum number of badges given to a participant
during the experiment was 13, the minimum number of badges given was 0, and the
median number of badges that participants earned was 5. Table 4.2 also shows the
maximum estimated time that earning a badge could add to a participant’s survey
completion time if they were to allow the badge pop up to stay on screen for the full 3
seconds. Recall that the badge pop ups were designed to appear for at most 3 seconds.
Based on this, the longest time added to the total survey completion time (assuming
the extra time is only related to the badge pop ups) was an extra 39 seconds. That is,
we’d expect that the badged survey would be on average 14.7 seconds longer, and no
more than 39 seconds longer than the non-badged survey, if the badges had no effect
on the time a user spends on the survey. Further, we’d expect the median difference
between survey completion times to be approximately 15 seconds.
A Wilcoxon Rank Sum test was preformed on each of the survey types to determine
if there was a difference between the time to complete each of the surveys. The results
of the test showed a p-value of 0.057. Although this p-value was not statistically
significant, further investigation was explored to determine if there were differences
based on different factors. Specifically, we analyzed the time to complete the survey
based on the survey categories, the different answer types, the length of text in each
44
Statistic Number of badges collected Estimated extra time (s)
Mean 4.9 14.7
Median 5.0 15.0
Maximum 13.0 39.0
Minimum 0.0 0.0
Table 4.2: Summary of badges collected by participants in the badged survey, andthe expected extra time these would add to the total survey if the user allowed thebadge pop up to remain on screen for 3 seconds per badge as designed.
question, and by question, to see if there were any differences between the time to
complete the two surveys for any of these factors. A Wilcoxon Rank Sum test was
performed for each of the factors. Wilcoxon Rank Sum tests were applied to both the
delta times, and the relative delta times for each of the factors. Results are provided
in Sections 4.1.1-4.1.4.
4.1.1 Results by Question Category
The first factor examined was the question categories. Wilcoxon Rank Sum tests
were performed to see if there was a significant difference between the time required
to complete each question category. The null hypothesis of each test was that there
was no significant difference between the non-badged and badged category times. A
full list of categories (including the number of questions per category) can be found
in Appendix B.1.
45
Tables 4.3 and 4.4 both show the results from the Wilcoxon Rank Sum tests that
were performed on the aggregated delta times and the relative delta times per cate-
gory, as well as the medians of both sets of times for both surveys. The tables also
include the p-values associated with each of the test statistics, where p-values that
are less than 0.05 represent statistically significant results and are indicated with bold
face font. It should be noted that in the delta times result set, the demographics and
food habits categories have p-values less than 0.05, where in the relative delta timers,
the demographics and food insecurity categories are both close to 0.05, with their
values being 0.065 and 0.078 respectively. Further, it should be noted that the me-
dian times for the badged survey are greater for every category than the non badged
survey, except for food insecurity category.
In the case of the significant results for the tests performed on the delta timers, the
badged survey took longer than the non-badged survey. In particular, badged survey
respondents spent approximately 20 seconds longer on the demographics category,
and 7 seconds longer on the food habits category questions than non-badged survey
respondents. That is, the first two categories presented to the respondents had median
timers that were significantly different between badged and non-badged surveys. In
both cases, the badged survey had a higher median timer.
Table 4.5 provides another view of the data. Here we divide the median time per
category for both badged and non-badged surveys by the number of questions in each
46
category. For both badged and non-badged surveys, the proportion of median time
spent per question seems to increase towards the end of the survey.
Category p-value Median B Median NB
Demographics 0.007 104.165 84.708
Food Habits 0.020 33.784 26.344
Food Insecurity 0.633 45.328 45.442
Fruits 0.145 58.378 56.043
Vegetables 0.086 62.625 59.543
Meat and Alternatives 0.336 78.868 77.833
Dairy and Alternatives 0.377 48.050 45.221
Grains and Alternatives 0.586 36.057 33.530
Other Foods 0.158 48.106 41.795
Table 4.3: Wilcoxon Rank Sum results for both surveys, of the delta times percategory. Where Median Badged and Median Non-Badged are the medians of eachcategory for the badge survey and non-badge survey respectively.
4.1.2 Results by Survey Question Type
The questions were then categorized by answer type: radio button, checkbox, or
textfield. In both surveys, the only textfield was for the postal code. All other
questions were either a radio button, or a checkbox, with 26 answers being radio
47
Category p-value Median B Median NB
Demographics 0.065 2.684 2.558
Food Habits 0.921 0.256 0.256
Food Insecurity 0.078 0.263 0.289
Fruits 0.853 0.265 0.268
Vegetables 0.805 0.214 0.221
Meat and Alternatives 0.412 0.218 0.230
Dairy and Alternatives 0.312 0.106 0.112
Grains and Alternatives 0.249 0.072 0.081
Other Foods 0.780 0.087 0.091
Table 4.4: Wilcoxon Rank Sum results for both surveys, of the relative delta timesper category. Median Badged and Median Non-Badged are the medians of eachcategory for the badge survey and non-badge survey respectively.
48
Category A = Number Questions MedianBA
MedianNBA
Demographics 15 6.94 5.65
Food Habits 6 5.63 4.39
Food Insecurity 8 5.67 5.68
Fruits 5 11.68 11.21
Vegetables 7 8.95 8.51
Meat and Alternatives 9 8.76 8.65
Dairy and Alternatives 6 8.01 7.54
Grains and Alternatives 4 9.01 8.38
Other Foods 5 9.62 8.36
Table 4.5: Total number of questions per category (A), with median badge time (Me-dian B) and median non-badged time (Median NB) divided by number of questionsper category.
49
buttons and 38 answers being checkboxes. Again, we explored whether or not there
was a difference in the timers for each question type between the different surveys
using the Wilcoxon Rank Sum test.
Tables 4.6 and 4.7 both summarize the results from the Wilcoxon Rank Sum tests
that were performed on the various answer types that are used in the two surveys.
In the delta times, radio buttons (answer type 0) have a p-value less than 0.05, and
checkboxes (answer type 1) have a p-value that close the 0.05, at 0.075, while in the
relative deltas set, the radio buttons p-value is close to 0.05 (at 0.062).
AnswerType p-value Median B Median NB
0 0.034 142.722 132.446
1 0.075 395.225 347.945
2 0.335 8.627 8.488
Table 4.6: Wilcoxon Rank Sum results for both surveys, of the delta times per answertype (0 = Radio Button, 1 = Checkbox, 2 = Textfield). Where Median Badged andMedian Non-Badged are the medians of each answer type for the badge survey andnon-badge survey respectively.
4.1.3 Results by Survey Question Length
It should be noted that each of the questions were broken into 3 categories for word
length. The categories were short, [0 - 25), medium, [25 - 35), and long, [35, 50). The
question with the shortest text contained 11 words in both the question text and the
50
AnswerType p-value Median B Median NB
0 0.062 2.746 2.543
1 0.696 1.220 1.282
2 0.789 0.264 0.265
Table 4.7: Wilcoxon Rank Sum results for both surveys, of the relative delta timesper answer type (0 = Radio Button, 1 = Checkbox, 2 = Textfield). Where MedianBadged and Median Non-Badged are the medians of each answer type for the badgesurvey and non-badge survey respectively.
answer text, while the question with the longest text contained 49 words in both the
question text and the answer text. The median word length for both question and
answer of the both surveys was 33 words.
Tables 4.8 and 4.9 sum up the results of the Wilcoxon Rank Sum tests of each
survey for the word length category of each question. For the delta times of the word
length categories, there is one result that is of interest, the questions that fell into the
short category ([0 - 25) total word length) have a p-value of 0.013, while the p-values
for the other categories are greater than 0.05. The relative deltas do not have any
p-values that are less than 0.05.
4.1.4 Results by Survey Question
We further investigated to determine if there was a significant difference in times
spent per question between the two surveys. Wilcoxon Rank Sum tests were per-
51
Word Category p-value Median B Median NB
1 0.013 75.146 59.389
2 0.121 247.014 235.212
3 0.103 225.088 212.318
Table 4.8: Wilcoxon Rank Sum results for both surveys, of the delta times per wordlength category (1 = [0 - 25) total words, 2 = [25 - 35) total words, 23= [35 - 50)total words). Where Median Badged and Median Non-Badged are the medians ofeach word length category for the badge survey and non-badge survey respectively.
Word Category p-value Median B Median NB
1 0.100 1.660 1.514
2 0.216 1.486 1.367
3 0.532 1.205 1.175
Table 4.9: Wilcoxon Rank Sum results for both surveys, of the relative delta timesper word length category (1 = [0 - 25) total words, 2 = [25 - 35) total words, 23= [35- 50) total words). Where Median Badged and Median Non-Badged are the mediansof each word length category for the badge survey and non-badge survey respectively.
52
formed for each of the questions. A full summary of results can be found in Appendix
B.3.
Tables B.2 and B.3 show the results for both surveys, for the delta times and the
relative deltas per question. These results show that there are questions with p-values
less than 0.05 for the delta times per question, specifically questions 2, 11, 22, 24,
29, 54, 57, 60. And that the relative delta times also have a couple of questions with
p-values less than 0.05, specifically questions 9, 16, 22, 54, 63. It should be noted
that questions 22 and 54 show up in with p-values less than 0.05 for both the delta
times and relative deltas.
53
4.2 Discussion
4.1 outlines the various results that have been gathered throughout the experiment.
This section will discuss what the results mean with respect to the experiment [and
hypotheses] outlined in Section 2.4. As Section 4.1 points out, the overall results
from the experiment show that of the 457 people who came to the website, only 111
actually took part in the experiment, which is approximately 24% of the people who
came to the site actually participating in the experiment, with approximately 48.6%
taking part in a badged survey and 51.3% taking part in a non-badged survey. From
here, it can also be seen that 91 people who took part in the experiment were able
to complete it, leaving only 20 unfinished surveys between the two types of surveys.
That is, approximately 81% of the people who took part in the experiment were able
to finish.
Breaking this down by survey type, approximately 83.3% of people who started a
badged survey completed it, and 80.7% of people who started a non-badged survey
completed it. A simple test of proportions indicates that there is no significant dif-
ference (p-value=0.91) in the completion rates for those who took the badged versus
the non-badged survey. Interestingly, this contradicts Hypothesis 1 in Section 2.4.
That is, badging does not appear to have influenced the completion rate.
While the Wilcoxon Rank Sum tests indicate that there was no statistically signif-
icant difference between the time it took to complete either types of surveys, some
54
important observations need to be noted. Upon taking a deeper look at the survey
results, we can see that for the badge survey, the median number of badges that were
given during the experiment was 5, the average number 4.7, and the maximum given
out being 13. Since each badge would produce a pop up window that would appear
for a maximum of 3 seconds (unless the participant closed it), this suggests that the
badged survey should take around 15 seconds longer (using either the median or av-
erage number of badges received) for a user to complete, while at most it should take
a participant around 39 seconds longer. In reality, we observed the following:
• The average time for a non-badged survey was 553.53 seconds.
• The average time for a badged survey was 635.34 seconds.
• The difference in the averages is 76.39 seconds.
• The median time for a non-badged survey was 519.81 seconds.
• The median time for a badged survey was 553.53 seconds.
• The difference in medians was 33.72 seconds.
Referring to Figure 4.1, the difference in the time it takes to finish a badged survey,
on average, is much higher than the time taken to finish a non-badged survey. That
is, the average difference of 76.39 seconds is 5.19 times as much as what would be
expected if we only considered the additional time required to open and close the
badge pop up windows. This hints that participants may be more engaged during a
55
badged survey than with their non-badged counterparts. Further, the median times
between the two survey types also shows that participants appear to be taking long to
complete the badged survey than what would be expected. Specifically, the difference
in median times to complete the survey is 2.25 times longer than what would be
expected if the difference were only related to the badge pop ups. While referring to
Figure 4.1, the orange bar represents the non-badged survey results with the addition
of the time required to receive the average number of badges that were assigned. This
shows that even by adding this extra time to the non-badged survey, there is still a
difference between the two surveys that is larger than might be expected. Again,
this trend suggests that the badged survey respondents are spending more time on
the surveys, and are possibly more engaged. This provides support for Hypothesis 2
outlined in Section 2.4, although it is not statistically significant.
As described in Section 4.1.1, the demographics and food habits categories showed
a significant difference in the delta time taken to complete them, while only the
demographics category showed a significant difference in the relative time taken to
complete it. This suggests that the distribution of times for the demographics cat-
egory is shifted more to the right than the distribution of times for the non-badged
survey. When the categories are ordered, these two categories become the first that
are presented to the participant. The differences observed might be associated with
the participant becoming familiarized with the survey itself.
56
Figure 4.1: A plot of the average and median times to complete the survey for thebadged (in light blue) and non-badged (in dark blue) surveys and the non-badgedsurvey with the addition of the time to earn the average number of badges given (inorange).
57
Other significant results between the surveys were limited to delta timers for radio
button type questions, and questions with fewer than 25 words. No other significant
differences were identified. That is, if straight-lining were occurring, it’s occurring in
the same way for both surveys. However, it should be noted that the median results
for delta timers were always larger for the badged survey than the non badged survey.
That is, despite a lack of statistical significance, it appears that the respondents were
taking longer to complete badged survey questions.
However, when considering the proportion of median time spent per question for
each category, it appears as if the respondents for both badged and non-badged
surveys spent more time answering questions closer to the end of the survey. That is,
it does not appear as if straight-lining occurred, or if it did, it occurred in the same
way. Since none of the categories near the end of the survey were found to have times
that were significantly different between the survey types, this neither supports nor
contradicts Hypothesis 3 in Section 2.4.
58
Chapter 5
Conclusion
As a consequence of issues with pen and paper (or phone) surveys, such as a lack of
user engagement, premature survey termination, and transitions from from traditional
landline phones to computers and hand held devices such as tablets and phones,
investigation into new methods of data collection becomes necessary. Alternative data
collection methods, e.g. on-line surveys, provide a method to collect data without the
use of standard pen and paper, or telephone surveys. Self elements of Gamification
were used to increase user engagement and completion rates during the online survey,
by adding badges that participants can earn during the survey.
5.1 Findings
We now review the hypotheses that this thesis was attempting to answer.
59
1. Gamification can be used to increase participant completion rates.
More specifically, the use of self elements (such as badges) will in-
crease participants completing the survey.
In regards to hypothesis 1, it was found that both types of surveys had very
similar completion rates, with the badge survey having 83.3% of participants
finish it, while 80.7% of non-badge survey participants were able to finish. As
Table 4.1 shows, the number of user’s that started for each survey is close,
with the badged survey having 54 participants start and the non-badged survey
having 57. Table 4.1 also shows that the number of participants that com-
pleted were close as well, with the badge survey having 45 participants finishing
and the non-badged survey having 46, with 9 and 11 participants not finishing
the badged or non-badged survey respectively. That is, we can not conclude
that Gamification was able to increase completion rates, as both the badged
and non-badged surveys have similar completion rates. Evidence suggests that
hypothesis 1 is FALSE.
2. The participant will spend more time on the gamified survey as op-
posed to the non-gamified survey. More specifically, the use of self
elements will engage the participant and they will take more time to
answer the survey.
The average and median times to complete a survey was greater for the badged
60
survey than the non badged survey. The difference, although not statistically
significant, was even larger than what would have been expected when taking
into account the time a badge stays on screen when it is earned. This trend
seems to suggest that hypothesis 2 is TRUE, and perhaps with further investi-
gation and a larger sample size, the true results will be known. Despite the fact
that statistical significance was not observed, the trend suggests that Gamifi-
cation may increase the amount of time that participants were spending on the
questions.
3. There will be no difference between a gamified survey and a non-
gamified survey in terms of participants straight-lining the survey.
More specifically, the participant is just as likely to straight-line a
gamified survey as they are a non-gamified survey.
The results indicate that straight-lining likely did not occur due to the amount
of time that participants of the badged survey spent on each question. However,
since there were mostly non significant results between the two surveys when
compared by delta timers and relative delta timers for category, question type,
question length, question, and overall survey, there is insufficient evidence to
suggest the time spent on each survey differed, or that participants were behav-
ing differently for each survey. That is, the evidence suggests that hypothesis 3
is INCONCLUSIVE.
61
To summarize, hypothesis 1 was found to be true, hypothesis 2 was found to be
false, and hypothesis 3 was found to be inconclusive. Regardless, it appears as if
badging has at least in some way contributed to improving participant engagement.
As such, it is recommended that badging be considered for future online public health
surveys, and that further study be done.
5.2 Future Work
In the future, further investigation of other elements of Gamification, and other
types of Self Elements should be looked at to see if they have an effect on user
engagement. More specifically, Social Elements of Gamification should be investigated
such as leaderboards or being able to share results from the survey if the participant
chooses. While other types of Self Elements that should be further looked at are
Pointsification, the use of numeric scores or points, or immediate feedback as a user
finishes a question, section/category, or survey, to see if these have an impact on the
engagement of users.
Another element of Gamification to look into would be Branching Elements, by
sending the user to different questions or sections of the survey depending on the
answer that was given for a specific question. This would help avoid premature
termination or fatigue for the user by having users skip specific questions that may
be unnecessary for the user to answer based on a previous answer. An example of
62
this would be asking the user if they have eaten a category of food such as fruits; if
the user answers ‘no’ then there is no reason to ask more questions about the type
of fruits a user has eaten just to force them to select an answer such as “I have not
eaten this.”
In regards to the data that was collected, further future work would include taking
a look at the subgroups, such as gender, age, location, within the demographics of
the data and seeing if the same conclusions can be drawn. It should be noted that
there will be biases present in this data due to the biases outlined in section 3.1
As well, since the contents of the survey were modelled to be similar to that of the
survey that the PHAC was running, other future work would include comparing the
data they collected to the data that was collected by this experiment. This would
help to further determine the impact of Gamification by comparing the number of
user’s that completed the survey, timing data if available, or if data collected are of
the same quality.
63
References
C. Alfi. The world in 2011: Ict facts and figures, 2011. URL http://www.slideshare.net/
cokyfauzialfi/ict-facts-figures2011.
Jerald G. Bachman, Lloyd D. Johnston, Patrick M. O’Malley, and John E. Schulenberg. Monitoring
the future. Technical report, Ann Arbor, MI: Institute for Social Research, 2006.
Philip Batterham. Recruitment of mental health survey participants using internet advertising:
content, characteristics and cost effectiveness. International Journal of Methods in Psychiatric
Research, doi: 10.1002/mpr.1421, 2014.
V. Bhaskaran. Increasing confidence in responses to electronic surveys, January 3 2013. URL
https://www.google.com/patents/US20130004933. US Patent App. 13/539,180.
Dan Birman. Ux flows: How to drive deep user engagement, 2014.
Marloes Bults, Desiree JMA Beaujean, Onno Zwart, Gerjo Kok, Pepijn Empelen, Jim E. Steenber-
gen, Jan Hendrik Richardus, and Helene ACM Voeten. Perceived risk, anxiety, and behavioural
responses of the general public during the early phase of the influenza a (h1n1) pandemic in
the netherlands: results of three consecutive online surveys. BMC Public Health, 11(1):1–13,
Since this thesis required experiments involving human participants, it was necessary to review
the experimental design to ensure that risks and harm were to any participant was minimized.
As indicated in Figure A.1, the Research Ethics Boards of the University of Guelph reviewed the
proposed study, and provided the Certification of Ethical Acceptability of Research Involving Human
Participants, and REB Number 15AU004.
69
Page 1 of 1
APPROVAL PERIOD: February 10, 2016 EXPIRY DATE: February 10, 2017 REB: NPES REB NUMBER: 15AU004 TYPE OF REVIEW: Delegated Type 1 PRINCIPAL INVESTIGATOR: Gillis, Daniel ([email protected]) DEPARTMENT: School of Computer Science SPONSOR(S): None TITLE OF PROJECT: Alternative Data Collection to Understand Food
Behaviours
The members of the University of Guelph Research Ethics Board have examined the protocol which describes the participation of the human participants in the above-named research project and considers the procedures, as described by the applicant, to conform to the University's ethical standards and the Tri-Council Policy Statement, 2nd Edition. The REB requires that researchers:
Adhere to the protocol as last reviewed and approved by the REB.
Receive approval from the REB for any modifications before they can be implemented.
Report any change in the source of funding.
Report unexpected events or incidental findings to the REB as soon as possible with an indication of how these events affect, in the view of the Principal Investigator, the safety of the participants, and the continuation of the protocol.
Are responsible for ascertaining and complying with all applicable legal and regulatory requirements with respect to consent and the protection of privacy of participants in the jurisdiction of the research project.
The Principal Investigator must:
Ensure that the ethical guidelines and approvals of facilities or institutions involved in the research are obtained and filed with the REB prior to the initiation of any research protocols.
Submit a Status Report to the REB upon completion of the project. If the research is a multi-year project, a status report must be submitted annually prior to the expiry date. Failure to submit an annual status report will lead to your study being suspended and potentially terminated.
The approval for this protocol terminates on the EXPIRY DATE, or the term of your appointment or employment at the University of Guelph whichever comes first. Signature: Date: February 10, 2016
RESEARCH ETHICS BOARDS Certification of Ethical Acceptability of Research Involving Human Participants
A. Papdopoulos Chair, Research Ethics Board-NPES
Figure A.1: Ethics approval from the University of Guelph Research Ethics Board,February 10, 2016
70
A.1.2 Ethics Letter Of Information & Consent Form
The following Letter of Information and Consent Form (Figures A.2-A.4) was presented to the
participants of the survey prior to beginning the survey.
71
Alternative Data Collection to Understand Food Behaviours
Letter of Information If you have access to the internet and are 18 years of age or older, we’re inviting you to take part in a study being conducted by Dr. Daniel Gillis (an Assistant Professor and Statistician in the School of Computer Science at the University of Guelph). The project has two major goals:
1. We want to know how people across the country consume food (including how often they typically eat, and the kinds of food they eat), and what challenges they might face that could get in the way of them eating the types of foods that they want to eat.
2. We want to figure out how to make surveys more enjoyable for the person answering questions, and in such a way as to provide researchers with better, more complete information. To do this we’re asking some people to answer regular surveys, and others to answer surveys that give them feedback on the information they’ve provided in the form of badges that they can share on Facebook or Twitter (if they choose to).
As a participant, you’re invited to answer a series of questions about yourself, about your food habits, your understanding and level of food insecurity (that is, not being able to eat the way you’d like to eat because of a lack of money), and food consumption. The questionnaire consists of 9 sections: demographics, food habits, food insecurity, fruits, vegetables, meats and alternatives, dairy and alternatives, grains and alternatives, and other foods. To proceed from one section to the next, you’ll need to provide an answer to every question within a section. However, you are not obliged to answer every question. Instead, you have the option of answering “I don’t know”, or “I’d prefer not to answer” on any question. Every question is multiple choice: some requiring a single answer, others that allow for multiple answers. The research program also investigates the use of new technologies (mobile phones, tablets, gesture control devices, etc.) for collecting data for research. While the results are not directly commercializable, they could be used to in the development of tools for marketing or business data collection, and thus might be commercializable in the future. The first question of the survey will ask for your consent for our team to analyze the data provided, as well as share and publish results based on the aggregate answers collected. All of your data are completely anonymous. We won’t (and don’t) ask for your name, address (other than the first three digits of your postal code), or email. To ensure anonymity, please do not enter your name anywhere in the survey.
Figure A.2: Page 1 of the Letter of Information and Consent Form provided to theparticipants prior to participating in the experiment.
72
Alternative Data Collection to Understand Food Behaviours
Letter of Information If you have access to the internet and are 18 years of age or older, we’re inviting you to take part in a study being conducted by Dr. Daniel Gillis (an Assistant Professor and Statistician in the School of Computer Science at the University of Guelph). The project has two major goals:
1. We want to know how people across the country consume food (including how often they typically eat, and the kinds of food they eat), and what challenges they might face that could get in the way of them eating the types of foods that they want to eat.
2. We want to figure out how to make surveys more enjoyable for the person answering questions, and in such a way as to provide researchers with better, more complete information. To do this we’re asking some people to answer regular surveys, and others to answer surveys that give them feedback on the information they’ve provided in the form of badges that they can share on Facebook or Twitter (if they choose to).
As a participant, you’re invited to answer a series of questions about yourself, about your food habits, your understanding and level of food insecurity (that is, not being able to eat the way you’d like to eat because of a lack of money), and food consumption. The questionnaire consists of 9 sections: demographics, food habits, food insecurity, fruits, vegetables, meats and alternatives, dairy and alternatives, grains and alternatives, and other foods. To proceed from one section to the next, you’ll need to provide an answer to every question within a section. However, you are not obliged to answer every question. Instead, you have the option of answering “I don’t know”, or “I’d prefer not to answer” on any question. Every question is multiple choice: some requiring a single answer, others that allow for multiple answers. The research program also investigates the use of new technologies (mobile phones, tablets, gesture control devices, etc.) for collecting data for research. While the results are not directly commercializable, they could be used to in the development of tools for marketing or business data collection, and thus might be commercializable in the future. The first question of the survey will ask for your consent for our team to analyze the data provided, as well as share and publish results based on the aggregate answers collected. All of your data are completely anonymous. We won’t (and don’t) ask for your name, address (other than the first three digits of your postal code), or email. To ensure anonymity, please do not enter your name anywhere in the survey.
Figure A.3: Page 2 of the Letter of Information and Consent Form provided to theparticipants prior to participating in the experiment.
73
2. Understand that your participation is voluntary and you are free to withdraw at any time 3. Understand the provisions for confidentiality
§ I agree to participate.
§ I do not agree to participate.
Figure A.4: Page 3 of the Letter of Information and Consent Form provided to theparticipants prior to participating in the experiment.
74
A.2 Survey Questions
Survey questions and answers presented to the participants of the survey are outlined in Figures
A.5-A.10.
75
Questionnaire Category Category
Description/Instructions Question Answer Set
Demographics The following questions allow us to get to know you a little better. The information you provide here will be used to identify patterns in food consumption, food habits, and food insecurity. If you are unsure of your answer, please select "don't know" or follow the specific instructions of the question. If you prefer not to answer, please select that option, or follow the specific instructions of the question. You can leave the survey at any time by simply closing your browser.
Please enter your gender:
{"0":"Male", "1":"Female", "2":"Other", "3":"prefer not to answer"}
Please enter your age:
{"0":"<18", "1":"18 to 29","2":"30 to 39", "3":"40 to 59", "4":"60+", "5":"prefer not to answer"}
In the past 12 months, were you a part time or full time university or college student?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Which of the following best describes your current
{"0":"Single", "1":"Married/Partnered", "2":"Separated/Divorced/Widowed", "3":"Other", "4":"prefer not to answer"}
relationship status?
Do you have any dependents that live with you?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Please enter the first 3 digits of your postal/zip code. Enter XXX if you prefer not to answer. Enter YYY if you do not have a postal code. Enter ZZZ if you do not know your postal code.
{"0":"Postal Code"}
What is your annual income?
{"0":"<$30000","1":"$30000-$59999","2":"$60000-$79999","3":"$80000-$99999","4":"$100000+","5":"don't know","6":"prefer not to answer"}
What is your highest level of education?
{"0":"Primary","1":"Secondary","2":"University/College Degree/Diploma","3":"Graduate Degree/Diploma","4":"prefer not to answer"}
Which of the following best describes your current university student status?
{"0":"I am not currently a university student","1":"Undergraduate","2":"Master's","3":"PhD","4":"Post Doc","5":"don't know","6":"prefer not to answer"}
If you are a student in university or college, are you an international student?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Do you identify as any of the following?
{"0":"First Nations Status","1":"First Nations Non-Status","2":"Metis","3":"Inuit","4":"Senior Citizen (65+)","5":"Newcomer to Canada (<5yrs in Canada)","6":"I do not identify as any of the above","7":"don't know","8":"prefer not to answer"}
Figure A.5: Page 1 and 2 of the survey questions and answers provided to the par-ticipants
76
In general, how would you describe your physical health?
{"0":"Excellent","1":"Very good","2":"Good","3":"Fair","4":"Poor","5":"Very poor","6":"don't know","7":"prefer not to answer"}
In general, how would you describe your mental health?
{"0":"Excellent","1":"Very good","2":"Good","3":"Fair","4":"Poor","5":"Very poor","6":"don't know","7":"prefer not to answer"}
Thinking about the amount of stress in your life, would you say that most days are:
{"0":"Not at all stressful","1":"Not very stressful","2":"A bit stressful","3":"Quite a bit stressful","4":"Extremely stressful","5":"don't know","6":"prefer not to answer"}
Identify all sources of income for the previous year.
{"0":"Government student loan","1":"Scholarship/Bursary","2":"Bank loan","3":"Research/Teaching Assistantship","4":"Personal savings","5":"Family","6":"Part time employment","7":"Full time employment","8":"Child tax benefits","9":"Social assistance","10":"don't know","11":"prefer not to answer"}
Fruits The following questions are related to different types of fruit you've eaten in the last 7 days. Do not include fruit that was consumed as part of a prepared dessert (such as a pie or a tart), unless it was, for example, a bowl of cherries with whipped cream. Note that each answer includes an option for "I didn't eat any of these foods". If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can
Did you eat any of the following fresh or frozen tropical fruits?
{"0":"Bananas","1":"Mangos","2":"Papayas","3":"Kiwis","4":"Pomegranates","5":"Avocado","6":"Other tropical fruits","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
leave the survey at any time by simply closing your browser.
Did you eat any of the following fresh or frozen tree fruits?
{"0":"Apples","1":"Pears","2":"Plums","3":"Peaches","4":"Nectarines","5":"Apricots","6":"Other tree fruits","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
Did you eat any of the following citrus fruits?
{"0":"Lemons","1":"Limes","2":"Grapefruits","3":"Oranges","4":"Tangerines","5":"Clementines","6":"Other citrus fruits","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
Did you eat any of the following fresh or frozen melons?
{"0":"Cantaloupe","1":"Watermelon","2":"Honey Dew","3":"Other melons","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Did you eat any of the following fresh or frozen berries?
{"0":"Strawberries","1":"Blueberries","2":"Raspberries","3":"Blackberries","4":"Cherries","5":"Other berries","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Vegetables The following questions are related to different types of vegetables you've eaten in the last 7 days. Note that each answer includes an option for "I didn't eat any of these foods". If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can leave the survey at any time by simply closing your browser.
Did you eat any of the following fresh or frozen leafy vegetables?
{"0":"Lettuce","1":"Cabbage","2":"Spinach","3":"Swiss Chard","4":"Kale","5":"Other leafy vegetables","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Did you eat any of the following fresh or frozen
{"0":"Carrots","1":"Beets","2":"Potatoes","3":"Turnips","4":"Parsnips","5":"Radish", "6":"Other root vegetables/tubers","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
Figure A.6: Page 3 and 4 of the survey questions and answers provided to the par-ticipants
77
root vegetables/tubers?
Did you eat any of the following fresh or frozen strong vegetables?
{"0":"Onions","1":"Leeks","2":"Garlic","3":"Shallots","4":"Scallions","5":"Other strong vegetables","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Did you eat any of the following fresh or frozen stalk vegetables?
{"0":"Celery","1":"Broccoli","2":"Cauliflower","3":"Bok choy","4":"Other stalk vegetables","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you eat any of the following fresh or frozen pod vegetables?
{"0":"Beans","1":"Peas","3":"Other pod vegetables","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Did you eat any of the following fresh or frozen squash vegetables?
{"0":"Acorn","1":"Butternut","2":"Spaghetti","3":"Pumpkin","4":"Other squash","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you eat any of the following fresh or frozen vegetables?
{"0":"Cucumbers","1":"Tomatoes","2":"Mushrooms","3":"Zucchini","4":"Other vegetables","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Meat and Alternatives
The following questions are related to different types of meat (or meat alternatives) that you've eaten in the last 7 days. Note that each answer includes an option for "I didn't eat any of these foods". If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can
Did you eat any of the following fresh or frozen unprocessed red meats (e.g. steaks, roasts, chops, etc.)? Do not include ground meats or deli meats here.
{"0":"Beef","1":"Deer","2":"Elk","3":"Bison","4":"Lamb","5":"Goat","6":"Other red meat","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
leave the survey at any time by simply closing your browser.
Did you eat any of the following fresh or frozen ground meats?
{"0":"Ground beef","1":"Ground pork","2":"Ground chicken","3":"Ground lamb","4":"Ground turkey","5":"Other ground meat","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Did you eat any of the following fresh or frozen white meats (e.g. breasts, legs, chops, etc.)?
{"0":"Chicken","1":"Pork","2":"Turkey","3":"Other white meat","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Did you eat any of the following fresh or frozen unprocessed fish or seafood?
{"0":"Fish","1":"Mussels","2":"Shrimp","3":"Scallops","4":"Clams","5":"Lobster","6":"Crab","7":"Other seafood or fish","8":"I didn't eat any of these foods","9":"don't know","10":"prefer not to answer"}
Did you eat any of the following nuts or seeds?
{"0":"Pecans","1":"Walnuts","2":"Almonds","3":"Cashews","4":"Hazelnuts","5":"Sunflower seeds","6":"Other nuts or seeds","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
Did you eat any of the following raw or cooked bean/protein replacements?
{"0":"Lentils","1":"Chickpeas","2":"Kidney beans","3":"Black eyed peas","4":"Soy or soy products (e.g. tofu)","5":"Other meat replacement proteins","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Did you eat any of the following processed/deli meats?
{"0":"Sliced ham","1":"Sliced beef","2":"Sliced turkey","3":"Sliced chicken","4":"Salami","5":"Bacon","6":"Other processed meats","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
Did you eat any of the following veggie/vegan protein replacements?
{"0":"Veggie/vegan burger","1":"Veggie/vegan hot dog","2":"Ground veggie meat substitute","3":"Other veggie/vegan meat replacements","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Figure A.7: Page 5 and 6 of the survey questions and answers provided to the par-ticipants
78
Did you eat any of the following nut butters?
{"0":"Peanut butter","1":"Cashew butter","2":"Almond butter","3":"Hazelnut butter","4":"Other nut butters","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Dairy and Alternatives
The following questions are related to different types of dairy (or dairy alternatives) that you've consumed in the last 7 days. Note that each answer includes an option for "I didn't eat any of these foods". If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can leave the survey at any time by simply closing your browser.
Did you drink any of the following milks?
{"0":"Milk (cow, goat, etc.)","1":"Almond milk","2":"Soy milk","3":"Cashew milk","4":"Rice milk","5":"Coconut milk","6":"Other milks","7":"I didn't eat any of these foods","8":"don't know","9":"prefer not to answer"}
Did you eat any of the following medium to hard cheeses?
{"0":"Cheddar","1":"Parmesan","2":"Romano","3":"Colby","4":"Monterey Jack","5":"Emmental","6":"Gouda","7":"Edam","8":"Other medium to hard cheeses","9":"I didn't eat any of these foods","10":"don't know","11":"prefer not to answer"}
Did you eat any of the following soft or semi-soft cheeses?
{"0":"Cream cheese","1":"Brie","2":"Havarti","3":"Munster","4":"Other soft or semi-soft cheeses","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you eat any of the following frozen dairy products? Include frozen ice cream bars here.
{"0":"Ice cream (dairy based)","1":"Rice milk ice cream","2":"Soy milk ice cream","3":"Coconut milk ice cream","4":"Other milk ice cream","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you eat any of the following yogurts?
{"0":"Yogurt (dairy based)","1":"Rice milk yogurt","2":"Soy yogurt","3":"Cashew yogurt","4":"Coconut yogurt","5":"Other milk yogurts","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Did you eat any of the following processed dairy based products?
{"0":"Cheese slices","1":"Cheese strings","2":"Pre-made milk based beverage","3":"Other processed dairy","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Grains and Alternatives
The following questions are related to different types of grains (or grain alternatives) that you've eaten in the last 7 days. Note that each answer includes an option for "I didn't eat any of these foods". If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can leave the survey at any time by simply closing your browser.
Did you consume any of the following processed grain items?
{"0":"White bread","1":"Hot dog/hamburger buns","2":"White pasta","3":"White rice","4":"Other similar processed carbohydrates","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you consume any of the following whole grain products?
{"0":"Rye bread","1":"Whole wheat bread","2":"Whole wheat pasta","3":"Brown or wild rice","4":"Other complex carbohydrates","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you eat any of the following cold cereals?
{"0":"Bran flakes or other high fibre cereal","1":"Cheerios or other grain based cereals","2":"Granola based cereals","3":"Rice puff cereal","4":"Sugary cereal","5":"Other processed cereals","6":"I didn't eat any of these foods","7":"don't know","8":"prefer not to answer"}
Did you eat any of the following hot cereals?
{"0":"Oatmeal", "1":"Cream of wheat","2":"Multigrain cereal","3":"Other hot high fibre low sugar unprocessed cereals","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Food Insecurity
The following questions allow us to understand your current food security situation. Your answers will help us understand any issues you might have faced
In the past 12 months, have you NOT HAD enough food to eat because of a lack of money?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Figure A.8: Page 7 and 8 of the survey questions and answers provided to the par-ticipants
79
trying to obtain and eat the types of food you want to eat. If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can leave the survey at any time by simply closing your browser.
In the past 12 months, have you NOT eaten the quality or variety of foods that you wanted to eat because of a lack of money?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 12 months, have you worried that there might not be enough to eat because of a lack of money?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 12 months, have you sought assistance from a food bank or other food provider?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
If you needed to, do you know where you could go to find food in an emergency?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 12 months, have you ever not eaten balanced meals because you couldn't afford to?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 12 months, have you ever reduced your food consumption because there wasn't enough to eat because you couldn't afford it?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 12 months, did you experience weight loss because you couldn't afford to buy food?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Other Foods The following questions are related to other types of food or drinks that you've consumed in the last 7 days. Note that each answer includes an option for "I didn't eat any of these foods". If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can leave the survey at any time by simply closing your browser.
Did you eat any of the following items?
{"0":"Cake/Cupcakes/Muffins/Dessert Bread","1":"Pie/Tarts","2":"Cookies","3":"Pastries/Croissants","4":"Other desserts or pastries","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Figure A.9: Page 9 and 10 of the survey questions and answers provided to theparticipants
80
Did you consume any of the following beverages?
{"0":"Latte","1":"Capuccino","2":"Milkshake","3":"Other tea/coffee milk based beverage","4":"I didn't eat any of these foods","5":"don't know","6":"prefer not to answer"}
Did you consume any of the following beverages?
{"0":"Soda/Pop","1":"Energy drink","2":"Vitamin water","3":"Sports drink","4":"Other energy drinks","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you consume any of the following alcoholic beverages?
{"0":"Beer/cider","1":"Wine","2":"Clear liquor (e.g. vodka, gin, etc.)","3":"Other liquor (e.g. rye, scotch, etc.)","4":"Other alcoholic beverages","5":"I didn't eat any of these foods","6":"don't know","7":"prefer not to answer"}
Did you consume any of the following items?
{"0":"Chocolate bar","1":"Popcorn","2":"Granola bars","3":"Energy bars","4":"Potato chips","5":"Corn chips","6":"Crackers","7":"Other snack foods","8":"I didn't eat any of these foods","9":"don't know","10":"prefer not to answer"}
Food Habits The following questions give us a better idea of your current eating habits. These will be used to better understand your food consumption habits, which you'll be asked about in following sections. If you are unsure of your answer, please select "don't know". If you prefer not to answer, please select "prefer not to answer". You can leave the survey at any time by simply closing your browser.
In the past week, have you eaten breakfast every day?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past week, have you eaten lunch every day?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past week, have you eaten dinner every day?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Are you a vegetarian or vegan?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 7 days, were you fasting for religious, medical, or other purposes?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
In the past 7 days, have your eating habits changed for any reason?
{"0":"Yes","1":"No","2":"don't know","3":"prefer not to answer"}
Figure A.10: Page 11 and 12 of the survey questions and answers provided to theparticipants
81
Appendix B
Code & Further Tables
B.1 Code
The code to create and implement the surveys used in this thesis can be found in the following