Selecting and Prioritizing Intervening Variables for Your SAPC Grant December 14, 2015 – 10:30am to 12:00pm Webinar Presentation Scott Formica Social Science.

Slide 1

Selecting and Prioritizing Intervening Variables for Your SAPC Grant

December 14, 2015 10:30am to 12:00pmWebinar PresentationScott FormicaSocial Science Research & Evaluation, Inc. 1Moderator and PresenterModerator: Ben Spooner, Technical Assistance Provider, MassTAPP

Presenter: Scott FormicaSocial Science Research & Evaluation, Inc.2

2Pause to celebrate the first 6 months of the grant. Okay. Moving on.There are several important dates and deadlines approaching:Dec. 18, 2015: Touch base with MassTAPP on progress on Step 1 and Step 2. Jan. 11, 2016: MIS and Narrative Report Webinar Part II. Jan. 29, 2016: Draft of Section 1 and 2 of Strategic Plan due to BSAS & MassTAPP. Jan. 31, 2016: Narrative Report and MIS for Quarter 2 due.Apr. 8, 2016: Full Strategic Plan and logic model due to MassTAPP. Apr. 29, 2016: Full Strategic Plan and logic model due to BSAS. 3Welcome Important DatesExamine requirements and expectations for the 1/29/16 deliverable.

Assessment of UAD and Other Drug Use and Intervening Variables.

Capacity Building.

Explore the concept of intervening variables (IV) including the prioritization process that will occur in Step 3 following the 1/29/16 deliverable.

Touch upon linkages between problem statements, IVs, and strategies.

Provide an opportunity to ask questions directed to MassTAPP TA providers, the state evaluators, and/or BSAS.4Webinar ObjectivesOverview/Abstract - NO

Step 1: Assessment (sections 1.1 1.3) YES Focus of today

Step 2: Capacity Building (sections 2.1 2.5) YES

Step 3: Strategic Planning (sections 3.1 3.5) NOSection 3.4 (Logic Model) - NO

Step 4: Implementation (sections 4.1 4.2) NO

Step 5: Evaluation (sections 5.1 5.2) - NO5What Is Due on 1/29/16?6ASSESSMENT: Sections 1.1-1.3

66The SAPC strategic plan template asks you to write about your process and findings related to underage drinking and other drug use across the cluster.

How did you conduct the assessment? (write about all the steps you took)What data sources and techniques for data collection did you use? Include numbers/rates/percentages based on your best data sources. Identify where any data you use came from. Are any subpopulations disproportionately affected? Evidence? Data gaps and how you plan to address these. Additional information relevant to the issue or the assessment. Sustainability and cultural competence of the process. 7SAPC Strategic Plan Requirements (1.1)Was a comprehensive process used to systematically review data on underage drinking and, as needed, other drug use?

Does the plan clearly describe the data sources that were used?

Does the plan include data on underage drinking and, as needed, other drugs?

Were different subgroups considered?

Were data gaps identified along with plans to address them?

Were sustainability and cultural competence discussed?

8SAPC Strategic Plan (1.1) ExpectationsConvince the readers that a comprehensive assessment was conducted.

Honesty is the best policy we KNOW that there are going to be data gaps across the cluster and that not every cluster is going to have the same data.

Section 1.1 should be limited to describing consumption data. Do not get into the suspected reasons for use or the potential ways to address use.

This is a long-term grant. Be prepared to discuss how you will revisit assessment over the life of the grant and enhance the assessment over time.

Whatever appears here as a plan to address data gaps should most likely make its way into your capacity building plans in Section 2.

9SAPC Strategic Plan (1.1) TipsStep 1.1 (Assessment Data on Underage Drinking and Other Drug Use) needs to occur before you move on to Step 1.2 (Assessing Intervening Variables).

You dont want to look for reasons why until you have a thorough understanding of what the issue is and which groups, if any, are disproportionately affected.

Per Page 3 of the Guidance Document, decisions to focus on issues in addition to underage drinking require substantial data to support this decision.

It is important to consider the use pattern of interest (e.g., initiation, experimentation, current use, moderate use, high-risk use). This will influence both your IVs and your strategy selection.

10SAPC Strategic Plan (1.1) TipsAny questions specific to Section 1.1 Assessment Data on Underage Drinking and Other Drug Use?

Does anyone want to discuss any successes they have had during the assessment that could benefit others?

Any interesting or novel data sources or findings that others may want to explore?

Any challenges that others may encounter?

11Questions About Section 1.1Intervening Variables: [what is contributing to the consumption pattern?]Factors that have been identified through research as being strongly related to and influencing the occurrence and magnitude of substance use and related risk behaviors and their subsequent consequences. These variables are the focus of prevention interventions, changes in which are then expected to affect consumption and consequences. (SAMHSA, 2009)Includes risk and protective factors, but not limited to individual-level factors. Emphasis on population-levelchange across multiple domains and systems. 12Intervening Variables

The SAPC strategic plan template asks you to write about your process and findings related to intervening variables (IVs) for underage drinking and other drugs across the cluster.

How did you conduct the IV assessment? (write about all the steps you took)What data sources and techniques for data collection did you use? List all IV investigated, including data on each variable and the source(s).Data gaps and how you plan to address these. Additional information relevant to the issue or the assessment. 13SAPC Strategic Plan Requirements (1.2)Was a comprehensive process used to systematically review data on IVs related to underage drinking and, as needed, other drugs?

Does the plan clearly describe the data sources/sources of evidence used?

Does the plan list ALL of the intervening variables that were considered?

Were data gaps identified along with plans to address them?

14SAPC Strategic Plan (1.2) ExpectationsConvince the readers that a comprehensive assessment was conducted.This should not be based on convenience/availability alone (i.e., our student survey asks about perception of risk).

Again. Honesty is the best policy.

Consider that IVs may vary across cluster communities more than consumption patterns it is okay to say that.

Again. This is a long-term grant. Be prepared to discuss how you will revisit IV assessment over the life of the grant and enhance the assessment over time.

IVs are really where cultural competence plays a key role.

15SAPC Strategic Plan (1.2) TipsAgain, whatever appears here as a plan to address data gaps should most likely make its way into your capacity building plans in Section 2.

You probably want to consider around 6 IVs for UAD consumption (and an equivalent number for any other drugs) and plan to narrow this down to 3 for each consumption pattern later in Section 3 of the plan.

Do not prioritize or choose IVs in section 1.2. This is where you list everything even those IVs that arent ultimately retained.

If the evidence for the IV is based on key stakeholder interviews or focus groups, we need to know how, who, when, how many, etc. Are these based on the responses of 1 or 2 individuals? Were there corroborating data?

16SAPC Strategic Plan (1.2) TipsIntervening Variables: [what is contributing to the consumption pattern?]17Intervening Variables: In-Depth

What can we learn from looking back at MassCALL2, PFS-II, and MOAPC sites about identifying and collecting IVs?

I came up with three big questions in this area

How did sites approach IV data collection? What questions guided these efforts?

What consumption IVs did sites identify and pursue? 18IVs Identification and CollectionWhere did former grant sites turn to identify/generate IV data?Key stakeholder interviewsFocus groupsSurveysYouthParentsCommunityKey stakeholderGeographical mapping (e.g., outlet density)School administrative dataHospital dataPolice dataTx provider data19IVs Identification/CollectionTreatment ProvidersSocial service providers/outreach workersEducation and High Education administrators, guidance, wellness staffHospital/ED personnel including SBIRTLaw EnforcementMental Health Service ProvidersLocal GovernmentParents and GuardiansRecovery CommunityCultural SpokespersonsFaith CommunityJustice/Corrections

20IVs Sample Key StakeholdersCommunity A: Describe the situation in terms of how severe the problem is; how the problem has changed over time; the priority groups including age, gender, race/ethnicity who are engaging in the behavior; when and where the behavior occurs; and risk factors that drive the behavior.

Community B: Is there something about this area that makes people more likely to engage in the behavior? Is there a community attitude or characteristic that you think contributes and, (b) Have you noticed patterns in availability of the substance?

Community C: Assess the problem in community; what is currently occurring to prevent the behavior?

Community D: What is the nature and extent of problem; who is the priority population(s); what factors are driving the problem?21IVs Sample Key Stakeholder QuestionsWhat are some consumption IVs?Low levels of law enforcementCommunity norms favorable to usePerception of parental disapproval or attitudeFamily communicationParental monitoring or family management practicesAvailability/Ease of Access economic, retail, socialPerception of harm/riskPerception of peer disapproval or attitudeClose friends who use/abuseAttachment to school and family

Representation across individual, peer, family, and community domains.

22IVs Sample Consumption IVsAny questions specific to Section 1.2 Assessment Data on Underage Drinking and Other Drug IVs?

Does anyone want to discuss any successes they have had during the IV assessment that could benefit others?

Any interesting or novel data sources or findings that others may want to explore?

Any challenges that others may encounter?

23Questions About Section 1.2 or IVs24CAPACITY BUILDING: Sections 2.1-2.5

24242.1: Community and Key Stakeholder InvolvementKey sectors currently collaborating; colleges and universities; general community; sectors not yet represented.2.2: Structure and FunctioningOrg chart; team functioning; decision-making; challenges and solutions.2.3: Core Planning CommitteeMembership; challenges and solutions2.4. Capacity BuildingStrengths; areas of growth; capacity-building action plan2.5. TA Needs25SAPC Strategic Plan Requirements (2.1-2.5)2.1: Are there actual concrete plans to engage the community and sectors not yet involved? Is there good representation? Which sectors are missing?

2.2: What structures have been put into place and how are they functioning?

2.3: Who is on the core planning committee? How are they supporting the development of the strategic plan?

2.4: Have strengths and areas of growth been identified? Is there a concrete capacity building action plan?

Were gaps/capacity needs identified in Section 1 carried forward here as needed areas of growth?

26SAPC Strategic Plan (2.1-2.5) Expectations and Tips27Prioritizing Intervening Variables

2727This step should not occur until after the 1/29/16 deliverable when you write section 3.2 of the strategic plan.

The first bullet in Section 3.2 asks for the final set of intervening variables from section 1.2 that you selected, including how this list was selected (prioritized) from among the larger list of variables considered.

This assumes that you conducted a comprehensive assessment of IVs and that you need to pare down this list into a more manageable/practical subset.

Critical step that ultimately helps point to the strategies you will put in place.

28IVs PrioritizationImportanceChangeabilityFeasibilityCapacityReadinessResource availabilitySupported by data qualitative and quantitativeData quality29Corroborated by different sourcesCultural CompetenceEvaluability data availableFit relevant to issue/communityPotential for unintended consequencesSustainabilityWisdom of PracticePresent in multiple communities

It is helpful to generate a list of criteria on which each IV can be examined and compared. Potentially relevant criteria include:

IVs Sample Prioritization CriteriaThis is an example from a former grantee on how you might frame a set of questions on multiple criteria:Is the IV identified independently by multiple sources?How reliable and valid are the data supporting it?Have local changes in the variable produced changes in use?How actionable is the variable?Is it feasible to address the variable within the time frame of this grant?Are other efforts in place to change this variable? Are there capacity and resources available or that could be developed to address the intervening variable?

30IVs Sample Prioritization Criteria QuestionsWhich groups/bodies should make the final decision?

Small group of cluster/steering committee representatives attempt to reach consensus based on available evidence?

Develop an IV Workgroup that reports to the steering committee?

Each cluster solicits input from key stakeholders and brings this to the SC?

How, if at all, are community members included?

How, if at all, are principles of cultural competence woven in?31IVs Sample Decision-Making ProcessesWork from the assessment to the literature.

There is a danger that starting from a list will be self-fulfilling you will find what you are looking for rather than finding the major factors that operate in your community.

Once you have identified the major IVs in your site then turn to reviews of the literature to determine whether or not it has been linked to consumption.

32IVs Ideas, Tips, and ResourcesYou probably want to consider around 6 IVs for each consumption pattern and narrow this down to 3.

IVs may differ from community to community and gaps are OK.

If an IV does not appear in a list or a review, it is worth going into the literature yourself or asking for help from BSAS/MassTAPP.

Terminology will sometimes differ across articles and reviews. 33IVs Ideas, Tips, and ResourcesSAMHSAs CAPT websiteRisk and Protective FactorsSample Risk Factors for Underage DrinkingR&P Factors for Drinking Among Adolescents and Young AdultsRisk and Protective Factors for Marijuana Use

NIDA websitePreventing Drug Abuse Among Children and Adolescents

BSAS websiteRisk and Protective Factors

Literature General Causal Model for Alcohol, Tobacco, and Illicit Drug Prevention

34IVs What Lists or Resources Are Out There?Step 3 in the SAPC Guidance Document (pg 31-32) discusses prioritizing IVs.

Emphasis is on Importance and Changeability the latter includes many of the sub-categories we have discussed (e.g., readiness, resources, suitable interventions, time required for change to occur)

Other potential criteria include: Directionality can you interpret the change?Resource/Need gap are resources already going to this? Political will is there political/public will to address this factor?Distance is the IV proximal enough to affect change/measure?35IVs Revisiting Prioritization36ImportanceCHANGEABILITYHighLowHighLowIntervening VariablesCriteriaFeasibilityData AvailabilityPreventabilityLow peer disapprovalLack of parental knowledgeHigh Social AvailabilityIVs Revisiting Prioritization37IVs Revisiting Prioritization

Efficiency and SynergyLinkages between intervening variablesMultiple IVs that can be impacted by a single interventionIs the IV related to other health or behavioral health issues?

AppropriatenessHas the IV been shown to be linked to consumption with the group that you are targeting? Age? Gender? Race/Ethnicity? SES? Urban, Suburban, Rural? Etc.

AcceptabilityDid the process and final decision-making process provide adequate opportunity for community input? Are there missing segments? Was the assessment comprehensive and inclusive? 38IVs Additional Things to ConsiderAny questions specific to the IV prioritization process?

39Questions About IV Prioritization

40When in Doubt Call Them Not Me

Melissa WardPicture to Come!