Evidence based social work

Stephen A. Webb

Glasgow Caledonian University, Scotland

Interest in the issue of ‘research quality’ is at an all time high. Undoubtedly, one of the key spurs to the quest for higher standards in social research is the evidence-based policy movement. The chosen instrument for figuring out best-possible, future interventions in policy domains is the systematic review of all first-rate, bygone evidence from previous studies in that realm. A key step in the logic is to provide an ‘inclusion criterion’ as a means identifying those existing studies upon which most reliance should be placed.

Pawson notes “The very idea of evidence-based policy rests on a pair of remarkably brave claims. The first, is the proposition that evidence can be heard amidst the political clamour of modern policy-making. The second, is that evidence should have a privileged voice in policy formation because there are objective methods available to judge and justify the quality of the advice provided by social research”. The starting point for the quality assessment in evidence-based policy, lies with this assertion that there are objective means of detecting factual ‘truths’. Truth is “out there” waiting to be discovered.

It isn’t good data as such that makes science honest. It is the ceaseless scientific interrogation of its significance that makes data honest. It is now commonly accepted that we have no direct observational access to reality and that all observation is ‘theory-laden’. (Pawson)

When your doctor suspects you have the ‘flu and pops a thermometer in your mouth, she is not observing temperature directly. She is utilizing a theory confirmed a century previously about the linear expansion of mercury. A theory about your illness is tested against a different, well-established theory.

The modernisation of social care places a high premium on evidence. At the level of central government, commitment to service reform is increasingly based on evidence about effectiveness. At the level of citizens, acceptance of professional expertise is increasingly tempered by a well informed critique, supported by improved access to high-quality information. At the level of service providers, accountable, regulated services means ensuring that practice is based on evidence rather than on past practice or current patterns of service.

See https://www.sheffield.gov.uk/.../corporate.../professionals/.../Supporting-...

Within the various kinds of evidence required to inform social care (including citizens’ views, practitioners’ experience, and organizational audit and inspection), that provided by research plays a special role. The best research is specifically designed to be as free as possible from bias in favour of any interest group or policy position, and potentially provides the most secure basis to inform national policy.

Evidence-based health care originated because of gaps among evidentiary, ethical, and application concerns. Sackett et al. (1997) estimated that about two questions arose for every three patients and that 30 percent of all questions remained unanswered. (We do not know how many questions arise in the course of social work remain unanswered.) Evidence-based practice (EBP) suggests and explores ways to decrease gaps both at the level of clinical practice and decision making about groups or populations, for example purchasing services (Gray, 2001). It is as much about the ethics of educators and researchers as it is about the ethics of practitioners and agency administrators.

Ethics and accreditation standards, ethical and evidentiary issues are often far apart in practice. Consider gaps between obligations described in the Code of Ethics of the National Association of Social Workers (1996) and everyday practice regarding informed consent and drawing on practice and policy-related research. For example, research findings suggest that social workers do not draw on practice-related research findings (e.g. Rosen, 1994; Rosen, Proctor, Morrow-Howell, & Staudt, 1995). The survey conducted by Sheldon and Chilvers (2000) found that 18% of the total number of social workers surveyed (n=2,285) had read nothing related to practice within the last 6 months.

If professionals are not familiar with the evidentiary status of alternative practices and policies, they cannot pass this information on to their clients; they cannot honor informed consent obligations. If some alternatives are effective in attaining outcomes clients and significant others value and practice proceeds based on ignorance of this information, clients are deprived of maximizing opportunities to achieve hoped for changes in their lives.

EBP involves a shift in paradigm. Intuition and unsystematic clinical expertise are considered insufficient grounds on which to make decisions. On the other hand, the "value laden nature of clinical decisions" (Guyatt & Rennie, 2002, p. 4) implies that we cannot rely on evidence alone: Thus, knowing the tools of evidence-based practice is necessary but not sufficient for delivering the highest quality of [client] care. The philosophy of evidence-based practice encourages practitioners to be effective advocates for their clients: "physicians concerned about the health of their patients as a group, or about the health of the community, should consider how they might contribute to reducing poverty" (Guyatt & Rennie, 2002, p. 9).

EBP originated within the medical school of McMaster University, Toronto, in the early 1990s (Evidence-Based Medicine Working Group [EBMWG], 1992). By definition, EBM involves the conscientious, explicit, and judicious application of best research evidence to a range of domains: clinical examinations, diagnostic tests, prognostic markers, and the safety and efficacy of interventions whose purposes may be therapeutic, rehabilitative, or preventative, with therapeutic interventions understandably getting most of the attention.

“EBM is laid out in a neat and orderly way, with a painstakingly described set of five steps that compose its practice, a list of questions to answer when following each of these steps, flow charts, a classification of evidence in terms of its relevance and value, and careful descriptions of blind, randomized clinical trials (RCTs) as the gold standard for deciding the efficacy of interventions” (Jane Gilgun, 2005) RCTs are called experimental designs in the social sciences.

EBM relies heavily on “measurement”. That is the quantification of indicators, such as confidence intervals, effect size, experimental event rate, control over event rate, and number needed to be treated to prevent one event. Guyatt et al. (2000) recommended the quantification of both evidence and values, stating this is “the most rigorous approach to making recommendations” (p. 1839). Evidence about diagnosis, prognosis, or harm can arise from other forms of research besides RCTs, including case studies and qualitative research. Evidence about the efficacy of interventions whose face validity is self-evident and whose withholding poses ethical issues do not require RCTs (Ellis, Mulligan, Rowe, & Sackett, 1995).

1. Converting information needs related to practice decisions into answerable questions.

2. Tracking down, with maximum efficiency, the best evidence with which to answer them.

3. Critically appraising that evidence for its validity, impact (size of effect), and applicability (usefulness in practice)

4. Applying the results of this appraisal to practice and policy decisions. This involves deciding whether evidence found (if any) applies to the decision at hand (e.g., Is a client similar to those studied?) and considering client values and preferences in making decisions as well as other applicability concerns

5. Evaluating our effectiveness and efficiency in carrying out steps and seeking ways to improve them in the future (Sackett et al., 2000, pp. 3-4).

Clarifying the question for review about the effectiveness of a particular treatment

Searching for primary studies that address this question

Appraising the quality of these studies in terms of their ability to answer the efficacy question

Extracting the data from each proficient study on the outcomes of the treatment in that particular trial

Synthesizing the data by aggregating the results of all competent trials

Disseminating the findings about the overall efficacy of the treatment.

If it is assumed that there is only one research design that permits authoritative statements to be made on treatment efficacy, then it follows that appraisal of research quality can be made directly and early in the review process. This is exactly the function of quality standards articulated by Oxman and now well established in evidence-based medicine. Meta-analysis seeks to establish a causal linkage between a particular treatment and a specific outcome and it is deemed that RCTs are the only permissible design for making such inferences.

“The awesome logic of experimental control is brought to bear here. If subjects are randomly allocated into experimental and control conditions, and the treatment is applied to the former but not to the latter, then any subsequent differences between the two groups must be the result of the only matter on which they differ – namely, the application of the treatment” (Pawson, ESRC Working Paper 1).

“Do not mistake science for procedural uniformity. Science is not all control and calculation, checking and double-checking. It also proceeds though insight and imagination, speculative hunches and bold conjectures” (Pawson). Polanyi’s (1966) felicitous phrase, ‘we can know more than we can tell’ means that any particular scientific inquiry has thousands of decision points and that their resolution often relies on the experience, judgement and tacit wisdom of the researcher. This is especially so when it comes to the matter of hypothesis generation and inference making.

So science is truly value-laden !

Post empiricist philosophy no longer dwells on technical definitions of science but seeks its identification in such matters as its propositional, critical and normative structure. What matters is not research practice but research programmes (Lakotos, 1970). What counts is the logic of scientific discovery (Popper, 1959). It seems that science “is science” because it balances theory and method, concepts and evidence, guesswork and surveillance. With Haack (1993), we may say that findings are justified when they accord with the evidence and are consonant with other theories.