Developing an African Youth Psychosocial Assessment: An Application of Item Response Theory Theresa S. Betancourt, Sc.D., M.A. Associate Professor, Department of Global Health and Population Director, Research Program on Children and Global Adversity Department of Global Health and Population Harvard T.H. Chan School of Public Health Co-Authors: Frances Yang, Paul Bolton, & Sharon-Lise Normand
67
Embed
Developing an African Youth Psychosocial Assessment: An ...pubdocs.worldbank.org/en/628571433957235883/APYA... · Developing an African Youth Psychosocial Assessment: An Application
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Developing an African Youth Psychosocial
Assessment: An Application of Item Response
Theory
Theresa S. Betancourt, Sc.D., M.A.Associate Professor, Department of Global Health and Population
Director, Research Program on Children and Global Adversity
Department of Global Health and Population
Harvard T.H. Chan School of Public Health
Co-Authors: Frances Yang, Paul Bolton, & Sharon-Lise Normand
Overview
□ Measuring Mental Health
Constructs Cross-Culturally
□ Challenges
□ Methods
□ Northern Uganda: Context
□ Qualitative and other studies
□ Refining the AYPA using IRT
□ Implications for Future
Research
Psychosocial Impact of Armed Conflict
□ More than 1billion children worldwide live in areas affected by armed conflict
□ War-affected children experience direct and indirect exposure to violence, disrupted family functioning, damaged social structures, etc. all of which increase risks for mental health problems
□ LMICS lack of monetary and human resources devoted to accurate measurement of mental health problems in children and adolescents
□ Result: Limited data to support claims about the burden of mental health disorders in young people, or the outcomes of intervention research
Culture in Assessment/Measurement and
Intervention Development
“Ethnographic studies
demonstrate convincingly
that concepts of emotions,
self, and body, and general
illness categories differ so
significantly in different
cultures that it can be said
that each culture’s beliefs
about normal and
abnormal behavior are
distinctive”
(Kleinman 1988, p.49)
Typical Use of Questionnaires in Assessment
and Evaluation
1. Select or create questionnaire/select standard measure to adapt
• Usually developed outside the local culture/situation
2. Translate into local language (no validity tests)
3. Individual interviews with survey
4. Determine need based on frequency of responses
5. Choice of problem and therefore intervention is based on quantitative results
6. Repeat individual surveys before and after intervention to assess program impact
Problems with Relying on Western Measures in Cross-
Cultural Research
□ Cultural validity: How closely concepts in a questionnaire match local concepts; Western/outside concepts may not apply locally
□ Unknown local concepts: Are there important local issues/concepts unknown to us? How to include questions we don’t know we should be asking?
□ Translation problems: Who translates? Translation- back translation methods inadequate, can result in semantic equivalence but real-world insignificance (i.e. lighting fires)
□ Function/tasks of young people in family and community
Main Problem Themes Emerging from Free
Listing Exercises (N=45)
Theme Number reporting Percentage
Lack food 34 74%
Lack clothing 31 67%
Lack school fees, uniforms, books, etc. 30 65%
Insecurity/fear of abduction 18 39%
Diseases (sexually transmitted, due to poor
hygiene, malaria) 14 30%
Poor hygiene (latrines, bathing, soap etc) 13 28%
Lack parents 9 20%
Lack of safe housing/shelter 9 20%
Males disrupting females/girls staying with
soldiers/rape 9 20%
Lack money (general) 8 17%
Dropping out of school 7 15%
Stubborn, don't listen to parents 7 15%
Fighting 6 13%
Rude or spoilt (children) 6 13%
□ Prolonged interviews with local experts
□ N=32 adults, N=25 10-17 year olds
□ Investigate selected free list issues in detail (nature, causes, treatments)
□ Look for other syndromes missed on free lists
□ Ability to conduct repeat interviews
Key Informant Interviews
Results – Northern UgandaMood problems (Two Tam, Par and Kumu)
□ sad, cries continuously, sits with cheek in palm,
□ constant worries, forget what they are thinking, loses interest in school
□ think they are of no use, thinks about suicide, don’t care whether they live or die, talks about problems constantly, sits alone, don’t feel like talking to others
□ pain all over body, headache, loses appetite, weak
□ doesn’t sleep
Conduct problems (Gin Lugero/Kwo Maraco)
□ loses interest in school, sexual misbehavior, fights, use bad language, drinks alcohol, disrespectful, misbehaves, disobedient
Anxiety-like problems (Ma Lwor)
□ Clings to elders, wants to be alone, doesn’t greet people,
□ constantly running around, doesn’t sleep, thinks people are chasing them,
□ fast heart rate, loss of appetite, think they have no future,
□ doesn’t like loud noise
□ Developed from the qualitative study
□ Categorized them into 5 local syndromes similar to 3 DSM-IV/ICD-10 domains:1. Depression : Par, Kumu, Two Tam2. Anxiety : Ma Lwor3. Conduct Problems: Kwo maraco/Gin Lugero
□ Additional Qualitative work added prosocial subscale
Items Comprising the Assessment of Functioning Scales
Phase II: Validity Study
□ Validity: whether instrument really
measures the construct of interest
□ Major problem with much cross-cultural
research is criterion validity
□ Comparison of results against a gold
standard
□ Accepted gold standards usually not
available for these populations
Testing Validity
Diagnosis
□ Local appraisal of illness present or absent
as judged by local people can be used as
an alternative ‘gold standard.’
□ Requires an understanding of local
perceptions of mental disorders or illness
□ Triangulation of informant necessary when
gold standard is lacking
□ Multiple-informants used to identify ‘cases’
and ‘non-cases’ and look for agreement
To Compare The Locally Derived Measure
(The APAI) To Several Validity Criteria:a) Self Report
b) Caregiver Report
c) Standardized Western Measure of Emotional and Behavioral Problems in Children and Adolescents: The Strengths and Difficulties Questionnaire (Goodman, 1997)
Phase II: Validity Study
14-17 year-olds
1. Generate cut-off score for study eligibility (N=178 in validity study)
2. Evaluate the psychometric properties of the instrument using full sample screened (N=667)
Purpose of Instrument Validation Study
□ Three levels of stringency in defining a local case of two tam, par, kumu, gin lugero/kwo maracoand ma lwor:
1. Caregiver reported their child having the syndrome for at least one month (least stringent)
2. Child reported having the syndrome for at least one month
3. Both child and caregiver agreed on presence of syndrome for at least one month (most stringent).
□ All cases as defined above also demonstrated some degree of functional impairment.
Determination of Local “Caseness”
*1 out of 3 depression syndromes**1 out of 5 APAI syndromes
Syndrome Specific Analyses
Concordance Discordance
Norm Presence of
Syndrome
Absence of
Syndrome
Possible
Syndrome
Apai Subscale Range Mode M (SD) M (SD) M (SD) p M (SD)
Two Tam 1-43 14 16.38 (7.69) 21.36 (7.87) 13.25 (6.42) *** 17. 02 (7.25)
by higher or lower levels of latent psychological distress will endorse each possible response
option
□ Additionally, IRT models can estimate item bias or differential item functioning across different subgroups (such as age, race, gender, differing syndrome types or severity).
friends” (a=0.90), whereas items such as “I deceive”
were less discriminating (a=0.60).
□ Prosocial attitudes and behavior, the most
discriminating item was “I cooperate with others”
(a=0.73), while items like “I play together with others”
showed poorer discrimination (a=0.40).
□ Somatic complaints: “I have pain all over my body”
showed the best discrimination (a=0.76), while the
item “I get headaches” was less discriminating
(a=0.60).
Item response theory parameters (a = discrimination)
(b = thresholds) for the 41 AYPA items
Chronbach’s Alpha of subscales
in this sample
□ Internalizing problems (α=0.88)
□ Externalizing problems (α=0.83)
□ Prosocial attitudes/behaviors (α=0.72)
□ Somatic complaints without medical cause
(α=0.74)
Limitations
□ Reliance on youth self-reports remains an important
limitation.
□ Finding that the subscale for somatic complaints
without medical cause was weakest must be
considered in the context of a setting with high disease
burden and limited services --even with adequate
instructions, participants may find it challenging to
distinguish somatic complaints not due to a medical
cause from those due to illnesses such as malaria. No
independent medical evaluation was able to be
conducted to make this determination
□ In future research on the AYPA in SSA, it would be
valuable to assess validity using comparison to the
ratings of a child and adolescent psychiatrist familiar
with the culture and setting.
Broader Applicability of APYA?
□ Final refined and validated AYPA measure demonstrates potential for broader applicability to other African settings
□ Qualitative data on expressions of child mental health problems and prosocial behaviors in Rwanda and Sierra Leone share many similarities with items in the AYPA’s internalizing and externalizing problem scales.
□ Phrasing of items in the AYPA is more comparable to how mental health problems of children and adolescents may be expressed in SSA□ Expressions of sadness (i.e. having “pain in the heart” and “sitting
with cheek in palm”)
□ Social withdrawal (“staying away from others”) and anxiety or rumination (“thinking too much” )
□ AYPA was used successfully in other RCTs among war-affected youth in the Democratic Republic of Congo (O’Callaghan & McMullen, 2013)
Conclusions
□ Refined and shortened AYPA measure is a promising tool for assessing emotional and behavioral and prosocial attitudes and behaviors among youth in SSA
□ IRT-driven analyses can be applied to the refinement of instruments on emotional and behavioral problems in children derived from qualitative data in a war-affected and LMIC setting
□ Approach taken to develop APYA and examine its psychometric properties can be replicated in future research to expand measures available for use in LMICs
Thank you
Acknowledgements
□ Analysis was funded by the National Institute of Mental
Health, the National Center for Minority Health and Disparities, and by the Francois-Xavier Bagnoud Center
for Health and Human Rights
□ Primary data collection was supported by World Vision
Uganda and War Child Holland
□ The authors are grateful to the children, youth and
families of the Awer and Unyama IDP camps who
participated in this research
References
□ Betancourt T.S. (2010) A longitudinal study of psychosocial adjustment and community reintegration among former child soldiers in Sierra Leone. International Psychiatry, 7(3), 60–62.
□ Betancourt T.S., Bass J., Borisova I., Neugebauer R., Speelman L., Onyango G., Bolton P. (2009a) Assessing local instrument reliability and validity: a field-based example from northern Uganda. Social Psychiatry and Psychiatric Epidemiology, 44(8), 685–692.
□ Betancourt T.S., Speelman L., Onyango G., Bolton P. (2009b) A qualitative study of mental health problems among children displaced by war in northern Uganda. Journal of Transcultural Psychiatry, 46(2), 238–256. DOI: 10.1177/ 1363461509105815
□ Betancourt T.S., Meyers-Ohki S., Stevenson A., Ingabire C., Kanyanganzi F., Munyana M.C., Mushashi C., Teta S., Fayida I., CyamatareF.R., Stulac S.N., Beardslee W. (2011a) Using mixed-methods research to adapt and evaluate a family strengthening intervention in Rwanda. African Journal of Traumatic Stress, 2(1), 32–45.
□ Betancourt T.S., Rubin-Smith J., Beardslee W.R., Stulac S.N., Fayida I., Safren S.A. (2011b) Understanding locally, culturally, and contextually relevant mental health problems among Rwandan children and adolescents affected by HIV/AIDS. AIDS Care, iFirst, 1–12. DOI: 10.1080/09540121.2010.516333
□ Bolton P. (2001) Cross-cultural validity and reliability testing of a standard psychiatric assessment instrument without a gold standard. Journal of Nervous and Mental Disease, 189(4), 238–242. DOI: 10.1097/00005053-200104000-00005
□ Bolton P., Tang A.M. (2002) An alternative approach to cross-cultural function assessment. Social Psychiatry and Psychiatric Epidemiology, 37(11), 537–543. DOI: 10.1007/s00127-002-0580-5
□ Bolton P., Wilk C.M., Ndogoni L. (2004) Assessment of depression prevalence in rural Uganda using symptom and function criteria. Social Psychiatry and Psychiatric Epidemiology, 39(6), 442–447. DOI: 10.1007/s00127-004-0763-3
□ Bolton P., Bass J., Betancourt T.S., Speelman L., Onyango G., Clougherty K.F., Neugebauer R., Murray L., Verdeli H. (2007) Interventions for depression symptoms among adolescent survivors of war and displacement in northern Uganda: a randomized controlled trial.Journal of the American Medical Association, 298(5), 519–527. DOI: 10.1001/jama.298.5.519
□ Chan E.Y.Y., Griffiths S., Gao Y., Chan C.W., Fok T. F. (2008) Addressing disparities in children’s health in China. Archives of Disease in Childhood, 93, 346–352. DOI: 10.1136/adc.2007.130146
□ Cheng T.A., Williams P. (1986) The design and development of a screening questionnaire (CHQ) for use in community studies of mental disorders in Taiwan. Psychological Medicine, 16(02), 415–422. DOI: 10.1017/S0033291700009247
□ Child Rights International Network. (2001) Rwanda Country Report, New York, Better Care Network.
□ Chisholm D., Flisher A.J., Lund C. Patel V., Saxena S., Thornicroft G., Tomlinson M. (2007) Scale up services for mental disorders: a call for action. Lancet, 370(9594), 1241–1252.
□ Deng Q. (2010) Natural disasters, migration and urban insecurity in China. In Interconnection Among Global Problems in NortheastAsia, San Francisco, CA, Nautilus Institute for Security and Sustainability.
□ Dorit R.-S. (2001) Reducing risk for children in changing cultural contexts: recommendations for intervention and training. Child Abuse & Neglect, 25(2), 231–248. DOI: 10.1016/s0145- 2134(00)00242-8
References
□ Guha-Sapir D., van Panhuis W.G., Degomme O., Teran V. (2005) Civil conflicts in four African countries: a five-year review of trends in nutrition and mortality. Epidemiologic Reviews, 27(1), 67–77. DOI:
□ 10.1093/epirev/mxi010
□ Jacob K.S., Sharan P., Mirza I., Garrido-Cumbrera
□ I.M., Seedat S., Mari J.J., Saxena S. (2007) Mental health systems in countries: where are we now? Lancet, 370, 1061–1077. DOI: 10.1016/S0140-6736(07)61241-0
□ Newman D.A. (2003) Longitudinal modeling with randomly and systematically missing data: a simulation of ad hoc, maximum likelihood, and multiple imputation techniques. Organiza- tional Research Methods, 6(3), 328–362. DOI: 10.1177/1094428103254673
□ Newnham E.A, Doyle E.L., Sng A.A.H., Hooke G.R., Page A.C. (2012) Improving clinical outcomes in psychiatric care with touch-screen technology. Psychological Services, 9(2), 221–223.
□ Patel V. (2003) Cultural issues in measurement and research. In Prince M., et al. (eds) Practical Psychiatric Epidemiology, pp. 43–64, Oxford, Oxford University Press.
□ Patel V. (2007) Mental health in low- and middle- income countries. British Medical Bulletin, 81, 81–96. DOI: 10.1093/bmb/ldm010
□ Patel V., Simunyu E., Gwanzura F., Lewis G., Mann A. (1997) The Shona Symptom Questionnaire: the development of an indigenous measure of common mental disorders in Harare. Acta Psychiatrica Scandinavica, 469–475. DOI: 10.1111/j.1600-0447.1997.tb10134.x
□ Patel V., Araya R., Chatterjee S., Chisholm D., Cohen A., De Silva M., Hosman C., McGuire H., Rojas G., van Ommeren M. (2007a) Treatment and prevention of mental disorders in low-income and middle-income countries. The Lancet, 370(9591), 991–1005. DOI: 10.1016/S0140-6736(07)61240-9
□ Patel V., Flisher A.J., Hetrick S., McGorry P. (2007b) Mental health of young people: a global public health challenge. Lancet, 369, 1302–1313. DOI: 10.1016/S0140-6736(07)60368-7
□ Prince M., Patel V., Saxena S., Maj M., Maselko J., Phillips M.R., Rahman A. (2007) No health without mental health. The Lancet, 370(9590), 859–877. DOI: 10.1016/S0140-6736(07) 61238-0
□ Psychosocial Working Group. (2003) Psychosocial Intervention in Complex Emergencies: A Conceptual Framework, Edinburgh, Centre for Inter- national Health Studies, Queen Margaret University College.
□ Pynoos R.S., Nader K. (1988) Psychological first aid and treatment approach to children ex- posed to community violence: research implications. Journal of Traumatic Stress, 1(4), 445–473. DOI: 10.1002/jts.2490010406
□ Saraceno B., van Ommeren M., Batniji R., Cohen A., Gureje O., Mahoney J., Sridhar D., Underhill C. (2007) Barriers to improvement of mental health services in low-income and middle-income coun- tries. The Lancet, 370(9593), 1164–1174. DOI: 10.1016/S0140-6736(07)61263-X
□ Saxena S., Thornicroft G., Knapp M., Whiteford H. (2007) Resources for mental health: scar- city, inequity, and inefficiency. The Lancet, 370(9590), 878–889. DOI: 10.1016/S0140- 6736(07)61239-2
References
□ Srinivasan T.N., Suresh T.R. (1990) Non-specific symptoms and screening of non-psychotic morbidity in primary care. Indian Journal of
Psychiatry, 32(1), 77–82.
□ UNICEF. (2009) Machel Study 10-Year Strategic Re- view: Children and Conflict in a Changing World, New York, UNICEF.
□ Van Ommeren M. (2003) Validity issues in transcultural epidemiology. The British Journal of Psychiatry, 182(5), 376–378. DOI:
10.1192/bjp.182.5.376
□ Wang X., Gao L., Shinfuku N., Zhang H., Zhao C., Shen Y. (2000) Longitudinal study of eathquake-related PTSD in a randomly selected
community sample in North China. American Journal of Psychiatry, 157, 1260–1266. DOI: 10.1176/appi.ajp.157.8.1260
□ Yang H., Stanton B., Cottrel L., Kaljee L., Galbraith J., Li X., Cole M., Harris C., Wu Y. (2006) Pa- rental awareness of adolescent risk
involvement: implications of overestimates and underestimates. Journal of Adolescent Health, 39(3), 353–361. DOI: 10.1016/j.
jadohealth.2005.12.008
□ Yu X., Lau J.T.F., Zhang J., Mak W.W.S., Choi K. C., Lui W.W.S., Zhang J., Chan E.Y.Y. (2010) Posttraumatic growth and reduced suicidal
ideation among adolescents at month 1 after the Sichuan earthquake. Journal of Affective Disorders, 123, 327–331. DOI: 10.1016/j.