Top Banner
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=ccpq20 Download by: [108.45.47.222] Date: 20 March 2016, At: 06:06 Counselling Psychology Quarterly ISSN: 0951-5070 (Print) 1469-3674 (Online) Journal homepage: http://www.tandfonline.com/loi/ccpq20 Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation Kristen G. Pinto-Coelho, Clara E. Hill & Dennis M. Kivlighan Jr. To cite this article: Kristen G. Pinto-Coelho, Clara E. Hill & Dennis M. Kivlighan Jr. (2016) Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation, Counselling Psychology Quarterly, 29:1, 29-52, DOI: 10.1080/09515070.2015.1072496 To link to this article: http://dx.doi.org/10.1080/09515070.2015.1072496 Published online: 01 Sep 2015. Submit your article to this journal Article views: 133 View related articles View Crossmark data Citing articles: 1 View citing articles
25

Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

May 14, 2023

Download

Documents

Nil Satana
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=ccpq20

Download by: [108.45.47.222] Date: 20 March 2016, At: 06:06

Counselling Psychology Quarterly

ISSN: 0951-5070 (Print) 1469-3674 (Online) Journal homepage: http://www.tandfonline.com/loi/ccpq20

Therapist self-disclosure in psychodynamicpsychotherapy: A mixed methods investigation

Kristen G. Pinto-Coelho, Clara E. Hill & Dennis M. Kivlighan Jr.

To cite this article: Kristen G. Pinto-Coelho, Clara E. Hill & Dennis M. Kivlighan Jr. (2016)Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation,Counselling Psychology Quarterly, 29:1, 29-52, DOI: 10.1080/09515070.2015.1072496

To link to this article: http://dx.doi.org/10.1080/09515070.2015.1072496

Published online: 01 Sep 2015.

Submit your article to this journal

Article views: 133

View related articles

View Crossmark data

Citing articles: 1 View citing articles

Page 2: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

Therapist self-disclosure in psychodynamic psychotherapy: A mixedmethods investigation

Kristen G. Pinto-Coelhoa*, Clara E. Hilla and Dennis M. Kivlighan Jr.b

aDepartment of Psychology, College of Behavioral and Social Sciences, University of Maryland,College Park, MD, USA; bDepartment of Counseling, Higher Education, and Special Education,

College of Education, University of Maryland, College Park, MD, USA

(Received 12 April 2015; accepted 10 July 2015)

We investigated 185 therapist self-disclosure events in 16 cases of open-endedpsychodynamic/interpersonal psychotherapy. One disclosure occurred approxi-mately every other session, with the most frequent type being disclosure offacts. Therapists initiated 73%, and focus returned to the client following93%, of disclosure events. Disclosures of feelings and insight were judged tobe more intimate and higher in quality than were disclosures of facts. Overalldisclosure occurrence was positively related to client-rated working allianceand feelings disclosure occurrence was positively related to client-rated realrelationship. Factual disclosure occurrence was negatively related to workingalliance and real relationship. Results suggest that disclosure is multifaceted,nuanced, and complex. Implications for practice, training, and research arediscussed.

Keywords: therapist self-disclosure; psychodynamic psychotherapy; immediacy;real relationship; working alliance

The question of whether therapists should share personal information with clients dur-ing psychotherapy has long been debated by theorists and practitioners. Historically,psychoanalytic theorists argued that therapist self-disclosure (TSD) can impede treat-ment (e.g. Curtis, 1982; Freud, 1958/1912; Greenson, 1967), whereas humanistic theo-rists argued that it can enhance therapy’s effectiveness (e.g. Bugental, 1965; Derlega,Hendrick, Winstead, & Berg, 1991; Jourard, 1971). Current thinking across theoreticalorientations is converging on the belief that TSD can have a variety of beneficial effectsif used intentionally and judiciously, and that avoiding disclosure in all circumstancesmay have detrimental effects on both the client and the therapy (Eagle, 2011; Farber,2006; Henretty & Levitt, 2010; Hill & Knox, 2002).

In their review, Hill and Knox (2002) defined TSD as “therapist statements thatreveal something personal about the therapist” (p. 256). For this paper, we use this def-inition, explicitly excluding nonverbal self-disclosures (such as wearing a wedding ring)and immediacy (also known as self-involving disclosures in which the therapist shareswith the client “here and now” feelings about the client or the therapeutic relationship).We make these distinctions because past research suggests that nonverbal disclosures

*Corresponding author. Email: [email protected]

© 2015 Taylor & Francis

Counselling Psychology Quarterly, 2016Vol. 29, No. 1, 29–52, http://dx.doi.org/10.1080/09515070.2015.1072496

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 3: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

and immediacy seem to have quite different characteristics, functions, and consequencesthan TSD (Knox & Hill, 2003).

Although TSD is used infrequently compared to other therapist responses, account-ing for about 3.5% of interventions (Hill & Knox, 2002), more than 90% of therapistsreported having disclosed to clients at some time (Henretty & Levitt, 2010). In addition,in one study (Hill et al., 1988), clients rated TSD as the most helpful of all therapistresponse modes.

Most researchers have used analog methods (see reviews in Henretty & Levitt,2010; Hill & Knox, 2002) to study TSD. In these studies, researchers asked non-client(student) participants to respond to brief (e.g. 6-, 10- or 12-min) recorded or written cli-ent and/or therapist stimuli, neither of which is a close approximation of actual therapy.Students would not necessarily respond the same way clients would, and whenKushner, Bordin, and Ryan (1979) compared therapists’ response to filmed and real cli-ents, they found that therapists’ responses to analogs differed significantly from andwere minimally predictive of their behavior with real clients. These authors concludedthat results from analog studies should not be extrapolated to real therapy settings, thatnaturalistic settings are preferable to analogs for studying therapy, and that therapistsare very responsive to situational factors. It seems it is difficult, if not impossible, tosimulate contextual variables such as client and therapist background, much less theinterplay between the two unique individuals (i.e. the moment-by-moment interactionalsequence of therapist intentions and interventions with client reactions, perceptions, andchanging needs and goals), which is crucial to investigating TSD occurrence, given thatTSD is an intervention that relies heavily on the therapeutic relationship (see Hill &Knox, 2002). Nonetheless, analog studies offer the benefit of experimentally controllingvariables of interest (e.g. client presenting issues, nature of the working alliance, andcontent and frequency of TSD). Results from such studies have provided mixed results:(a) student participants’ previous experience in therapy may or may not affect their rat-ings of TSD types, “therapist,” and “therapy” session (i.e. results were mixed); (b)beliefs about the strength of the working alliance prior to TSD may be an importantcontextual factor that affects students’ ratings of TSD events, therapists, and therapysessions; and (c) TSD (in its varying types) may or may not be related to (i.e. resultswere mixed) students’ perceptions of “therapist” trustworthiness, expertise, empathy,warmth, credibility, attractiveness, professionalism, and ability to inspire hope; students’levels of self-disclosure; students’ ratings of session smoothness, depth, and positivity;and students’ helpfulness ratings for TSD (Dowd & Boroto, 1982; Hoffman-Graff,1977; McCarthy & Betz, 1978; Myers & Hayes, 2006; Reynolds & Fischer, 1983; Yeh& Hayes, 2011). The mixed results from these studies suggest that our understanding ofhow TSD operates is incomplete and that new methods are needed to better examinethis phenomenon.

Research from naturally occurring psychotherapy (Ain & Gelso, 2008; Audet, 2011;Audet & Everall, 2010; Barrett & Berman, 2001; Hanson, 2005; Hill, Mahalik, &Thompson, 1989; Hill et al., 1988; Knox, Hess, Petersen, & Hill, 1997) has providedrich accounts of clients’ views on the positive and negative effects of TSD. Someresearchers have compared disclosure in a dichotomous way (e.g. self-involving vs.self-disclosing and helpful vs. non-helpful), whereas others have examined the helpful-ness, amount, and relevance of TSD, and the relationships of these variables to treat-ment process and outcome variables. These findings have led to several conclusions: (a)

30 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 4: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

despite it occurring infrequently, client ratings of helpfulness were higher for TSD thanfor all other therapist response modes; (b) clients indicated that therapist disclosureswere likely to be helpful, whereas non-disclosures on the part of the therapist werelikely to be unhelpful; (c) TSD can have positive or negative consequences, thoughpositive effects seem more prevalent; (d) TSD is related to the quality of the therapeuticrelationship (e.g. increased but moderate levels of TSD were found to improve the qual-ity of the relationship in one study, whereas too much TSD negatively affected clients’perceptions of the relationship in another study); (e) TSD is related to client involve-ment in therapy and to treatment progress and outcome; (f) failure to disclose may bedetrimental to the therapeutic alliance; (g) consequences of TSD may be affected bycontextual factors such as client expectations and preferences about TSD, the strengthof the working alliance before the TSD, and the skill level with which TSD is deliv-ered; and (h) clients assess the therapist’s intentions for disclosing and evaluate TSDsfor relevance to clients’ issues and therapeutic needs.

Although this research in naturally occurring psychotherapy is promising, a majorproblem is that researchers have defined TSD in many different ways and have includedmany diverse behaviors under the umbrella of TSD. Given that TSD can range frompurely factual data to deeply intimate revelations, it seems important to distinguishamong the variety of interventions that comprise TSD. Knox and Hill (2003), in theirreview of the literature, suggested that TSD could be divided into several types: (a)facts, (b) feeling, (c) insight, (d) strategy, (e) reassurance/support, (f ) challenge, and (g)immediacy. Although an interesting speculation, this proposal about types needs to beempirically investigated.

Our overall purpose was to investigate and describe the occurrence of TSD in open-ended psychodynamic psychotherapy conducted by advanced doctoral students. Asdescribed above, examining the use of TSD in a naturalistic setting is critical so thatwe can represent how TSD actually occurs in psychotherapy. Specifically, we examinedthe frequency, duration, initiator, intimacy, and quality of TSD events, regardless oftype. Our second purpose involved investigating the different types of TSD (excludingnon-verbal disclosures and immediacy) as suggested by Knox and Hill (2003). Third,we wanted to examine whether certain types of TSD were more likely to be judged ashigher in intimacy and quality than other types. Finally, we wanted to predictclient-rated real relationship and working alliance scores based on occurrence versusnon-occurrence, TSD type, and intimacy level.

In an effort to achieve in-depth examination of and a rich perspective on TSDoccurrence, we adopted techniques from consensual qualitative research (CQR; Hill,2012) to code each TSD (i.e. to consensually determine type, intimacy level, and qual-ity level). CQR was designed to examine complex, nuanced phenomena, and it allowsfor detailed discussion among judges to reach conclusions. We then used quantitativeanalyses so that we could link TSD with measures of session outcome.

Method

Data-set

In this study, we analyzed 16 cases conducted within a psychology department clinicthat provided open-ended psychodynamic/interpersonal psychotherapy to community

Counselling Psychology Quarterly 31

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 5: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

clients for a low fee (the same 16 cases were analyzed for immediacy by Hill et al.,2014). Cases were selected in which therapists and clients met for at least eight sessionspast intake (given that we wanted to examine open-ended therapy), and that had termi-nated. The data-set consisted of video recordings from the 16 therapy dyads (with 9therapists, each seeing 1–3 clients), comprising a total of 360 sessions (ranging from 11to 60 sessions per case [M = 25.87, SD = 12.89], including intake sessions). For thecase with 60 sessions, sessions 21–25, 31–35, 41–45, and 51–55 were eliminated tomake the number of sessions across cases more equivalent.

Participants

Therapists

Nine (5 female, 4 male; 5 European Americans, 1 Asian American, 1 Asian interna-tional, 1 Chilean international, and 1 African-American) doctoral students ranging inage from 26 to 50 years (M = 31.78, SD = 7.46) and in their 2nd–5th year of a counsel-ing psychology doctoral program served as therapists in this study. Therapists workedin the clinic for 1–3 years. All had completed at least two psychotherapy practicasbefore working in the clinic. All indicated at least a moderate affiliation with a psy-chodynamic/interpersonal orientation. All participated in weekly individual supervisionand bi-weekly group supervision.

Clients

Sixteen (8 female, 8 male; 11 European American, 3 Hispanic American, 1 African-American, and 1 Asian American) clients, ranging in age from 21 to 60 years(M = 34.56, SD = 12.77) at the time of intake, participated in the current study. Formaldiagnoses were not determined, but presenting problems described during screening(some described more than one) included relationship concerns (11), anxiety or depres-sion (5), career concerns (1), coming out (1), and immigration issues (1).

Judges

Twenty-one (15 females and 6 males; 18 psychology majors, 3 unknown majors;19–25 years old; 12 European Americans, 1 African-American, 1 Hispanic/WhiteAmerican, 2 Middle Eastern, 2 Asian Americans/Pacific Islanders, 2 unknown; 3sophomores, 7 juniors, and 11 seniors) undergraduate psychology majors and one doctoralstudent (the first author, a 41-year-old female master’s student in counseling psychology)served as judges. Undergraduate judges were selected on the basis of grade point average(minimum: 3.25), interest in psychotherapy, professionalism, and motivation. Judges hadcompleted at least three psychology courses and an online basic ethics course for conductinghuman subject research. All 21 undergraduate judges served in Phase 1 of the codingprocess, and a subset of 6 female judges and the doctoral student served in Phase 2 of thecoding process (phases to be described in detail in the Procedures section).

In CQR (Hill, 2012), researchers discuss their opinions and personal biases relatedto the research topic as a way of attempting to “bracket” them, or set them aside (Hill,Knox, Thompson, Williams, & Hess, 2005). Phase 2 judges’ expectations and biases

32 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 6: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

(collected before coding began and discussed throughout the coding process) arereported here briefly to allow readers to evaluate the context of these findings. Of thesix undergraduate Phase 2 coders, all had heard about TSD in psychology classes andwhile serving as Phase 1 judges. Of these, four reported having learned it was notalways a good thing for therapists to do, but that it could be beneficial on occasion.The remaining two judges viewed TSD more favorably. All judges believed that exces-sive disclosure on the part of the therapist could be harmful.

Measures

Real Relationship Inventory–Client

The Real Relationship Inventory–Client (RRI-C) is a 12-item version of the original 24-item measure (RRI-C; Kelley, Gelso, Fuertes, Marmarosh, & Lanier, 2010), which wasdesigned to assess client perceptions of the real relationship. Clients use a five-pointLikert-type scale from strongly disagree (1) to strongly agree (5) to describe their viewsabout the strength of the real relationship, its realism, and its genuineness. The originalRRI-C has been found to relate to other variables and to treatment progress and out-come in theoretically predicted ways (Lo Coco, Gullo, Prestano, & Gelso, 2011), andhigh internal consistency has been found in previous studies (e.g. Fuertes et al., 2007;Marmarosh, Gelso, Markin, & Majors, 2009). Hill et al. (2014) reported that the shortform was correlated .91 with the longer form. Internal consistency (alpha) for the pre-sent sample was .91.

Working Alliance Inventory–Short Revised

The Working Alliance Inventory–Short Revised (WAI-SR; Hatcher & Gillaspy, 2006), arevision developed using extensive factor analyses of the 36-item Working AllianceInventory (Horvath & Greenberg, 1989), is a 12-item self-report measure designed toassess client perceptions of the working alliance. Clients use a five-point Likert-typescale from seldom (1) to always (5) to describe their views about the alliance. Hatcherand Gillaspy (2006) reported that the total scale was related to other alliance measuresand had adequate internal consistency (α > .85). The internal consistency (alpha) for thepresent study was .91.

Self-disclosure definition

As per Knox and Hill (2003), TSD was defined as “therapist statements that revealsomething personal about the therapist.” In keeping with our research questions, bothnonverbal self-disclosures (such as wearing a wedding ring) and immediacy (also knownas self-involving disclosures, in which the therapist shares with the client “here andnow” feelings about the client or the therapeutic relationship) were explicitly excluded.

Types

Knox and Hill (2003) suggested that TSD could be divided into several types: (a) facts,(b) feeling, (c) insight, (d) strategy, (e) reassurance/support, (f ) challenge, and (g)

Counselling Psychology Quarterly 33

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 7: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

immediacy. As noted earlier, we did not include immediacy in the current study. Fur-thermore, early in coding, we discovered that two of the proposed types (reassur-ance/support and challenge) were not distinct from the other types. Any disclosure(whether of facts, feelings, insight, or strategy) could be reassuring, challenging, neitherreassuring nor challenging, or both reassuring and challenging. Accordingly, we catego-rized TSDs using four types: (a) facts (factual information such as the institution fromwhich the therapist earned his or her degree, e.g. “I have master’s from Penn State.”),(b) feelings (sharing actual or hypothetical emotions experienced when in a similar sit-uation to what the client is experiencing, e.g. “I can imagine feeling sad in thoseyounger days and wanting the relationship to be different.”), (c) insight (realizationstherapists had about themselves when facing a similar situation to what the client isexperiencing, e.g. “The expectation serves a protective function. I know that myexpectations for people affect how I am in relationships.”), and (d) strategy (methodsused to handle similar difficulties or solve problems, e.g. “Sometimes I think it helpsme to think ‘What’s good enough?’ as opposed to ‘What’s perfect?’”). A coding manualwas developed to provide clear definitions of each of the types, examples of the types,and hints for distinguishing among types.

Duration

The therapist’s actual speaking time was recorded for each TSD. If a TSD continued onthe same topic across speaking turns, all therapist speaking time devoted to the topicwas included.

Initiation of TSD and focus return following TSD

We coded whether the client initiated the disclosure (i.e. asked the therapist to disclose,e.g. “Have you seen the movie about Van Gogh and his brother?”) or the therapist initi-ated the disclosure without being prompted (e.g. “I am an advanced doctoral student incounseling psychology supervised by an experienced, licensed psychologist.”) Whetherthe focus of the session turned back to the client following the disclosure or not wasalso coded.

Intimacy

An overall judgment of intimacy was based on the amount of information shared, theemotional context of the TSD, the therapist’s vulnerability (i.e. whether the therapistseemed comfortable sharing the information and whether the information might put thetherapist at risk by possibly altering the client’s view of the therapist), and the level ofintimacy of the disclosure content (e.g. low intimacy: everyday mundane such as latebecause of traffic or parking, or professional or educational history; medium intimacy:insecurity or needing help, fear of public speaking, personal hardship, or emotionalstruggle; and high intimacy: suicide, drug addiction, sexuality, illegal activity, or any-thing related to social stigma). A nine-point Likert-type scale was used (1 = low or notat all intimate, e.g. “I know a little about the process of applying to law school. I knowa lot of people who have applied to law school and am familiar with the process, but Idon’t have personal experience with it.”; 3 = medium or moderately intimate, e.g. “I am

34 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 8: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

surprised you have a canvassing job. ... I see people doing that all the time and I think,‘Wow, I don’t think I could do that.’ You have to really put yourself out there. ... Idon’t know how I would be at that job.”; 7 = on the higher end of the scale “I am in asimilar process of trying to figure out the broader picture of who I am, what the mean-ing of life is, what kind of person I want to be, what kind of person I am striving tobecome. I can understand and relate to your urgency, because I have a sense of reallywanting to find my answer.”); and 9 = high or very intimate; no highly intimate TSDswere judged as having occurred in this sample, so an example of the highest level ofintimacy is not provided here. This measure, modeled after the Helpfulness Scale(Elliott, 1985), an instrument widely used in counseling research, was previously usedby Kim et al. (2003) in a study of TSD and counseling process with East Asian clients.

Quality

The quality (i.e. the judged helpfulness) of each disclosure event was rated on a three-point scale (0 = low quality, e.g. “I go there too. I know Allrecipes.com. It’s a greatsite;” 1 = medium quality, e.g. “Coming from an Asian culture, the adult is supposed tobe respected and the child is not supposed to speak back or argue back;” and 2 = highquality, e.g. “I am single and do not have experience with extramarital affairs but mem-bers of my family have, and they have affairs for a reason.”). In order to be rated ashigh quality by judges, a disclosure needed to meet the three criteria in our definitionof quality: (a) it was reciprocal (i.e. was in response to a similar client disclosure, asdefined by Barrett & Berman, 2001), (b) it seemed to benefit the relationship betweenthe client and therapist or contribute to the therapeutic bond, and (c) it was relevant tothe therapeutic work (i.e. the client’s issues).

Procedures

Recruiting

At the clinic where the study took place, clients were recruited through announcementsin local newspapers, flyers sent to local therapists and local agencies, an internet web-site, and word of mouth; therapists were recruited via announcements in the doctoralprogram housing the clinic. All clients and therapists were aware that the clinic con-ducted psychotherapy process and outcome research and signed informed consent formsacknowledging that session recordings would be used for research purposes. Under-graduate research assistants (RAs) were recruited via announcements from the psychol-ogy department. Clients, therapists, and judges were not aware of the purposes of thecurrent study.

Screening and intake

When potential clients contacted the clinic, they were briefly screened by phone todetermine preliminary eligibility. Eligible persons were scheduled for an intake, duringwhich time they signed a consent form, talked about their presenting problems, andwere screened more thoroughly for eligibility for services (willing to be videotaped,

Counselling Psychology Quarterly 35

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 9: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

willing to work on relational aspects of her/his problems, willing and able to pay thefee, and willing to work with the therapist who completed the intake). Clients andtherapists were assigned code numbers for all data to protect confidentiality. Thosepeople who were not eligible at any step of the process were offered referrals to othermental health providers.

Treatment

Therapists conducted 45–60-min videotaped sessions using a psychodynamic/interper-sonal orientation. No limit was placed on the number of sessions, other than therapistavailability (7 of the 16 cases were terminated when therapists’ externships in the clinicended). Following all sessions (including intake), clients completed the RRI-C andWAI-SR.

Phase 1: pre-study identification of events

Undergraduate RAs viewed recorded therapy sessions and identified the occurrence ofTSD and other events not relevant to the present study (e.g. silence, crying, laughter,anger, therapist apology, and immediacy). To prepare them to identify these events,Phase 1 judges read definitions of the different types of events. They then participatedin a 3-h workshop where they viewed DVDs of therapy sessions not included in thisstudy and practiced identifying events. They asked clarifying questions to ensure theyunderstood what constituted each event and discussed how events should be recorded.Additional individual meetings were held as needed to review questions.

Following training, each judge was then assigned to individually view DVDs oftherapy sessions included in the study to identify and record when the various eventsoccurred. The judges recorded start times, stop times, and duration for each event (TSDand others). Finally, for each TSD event, defined as a therapist verbal statement thatrevealed something personal about the therapist, they wrote a brief narrative descriptionsummarizing the event.

Phase 2: coding TSD events

Because we wanted Phase 2 judges to be bright, reliable, and generally informed aboutpsychotherapy but able to evaluate TSD with as few preconceptions as possible (i.e.without a major bias for or against TSD or a detailed understanding or opinion regard-ing the role it might play in therapy), as is important in qualitative research (Hallberg,2006), we decided to use upper level undergraduate psychology majors to serve asjudges. Accordingly, Phase 2 judges were selected from the group of undergraduatePhase 1 coders based on diligence, attention to detail, ability to understand abstract con-cepts, and background in psychotherapy-related coursework.

Phase 2 judges were informed that the study was examining the occurrence andquality of TSD in psychotherapy, but they were not informed about the research ques-tions in the study. To prepare for coding, they read summary articles describing TSD(Hill & Knox, 2002; Knox & Hill, 2003). Two separate teams, each with three judgesand the first author, then met for six hours to discuss how to identify TSD and how it

36 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 10: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

might be used in psychotherapy, and to practice consensually coding TSD events notincluded in the present study.

In an effort to achieve in-depth examination of TSD occurrence and a rich perspec-tive on the data, we adopted some guidelines used in consensual qualitative research forcase studies (CQR–C; Jackson, Chui, & Hill, 2012). Each team began by watching theintake session or reading the therapist’s session notes for the case to gain an overallunderstanding of the case. Then, the team viewed the TSD events identified in Phase 1sequentially in the order in which they occurred in the case. After watching an event,judges first confirmed it was indeed a TSD event, then watched the 10 min before theevent occurred, and then re-watched the event itself.

Once a TSD was identified, the first author confirmed from the Phase 1 coding thestart and end time, as well as speaking time for both client and therapist duringthe event. Next, the judges consensually wrote a description of their perceptions of theevent (i.e. what was happening just before the event, the content of the TSD itself, andhow the judges observed that the client responded to the TSD) to ensure that judgeshad a clear understanding of the event before coding.

Each judge then independently coded events for the following variables in the fol-lowing order: type, intimacy, quality, who initiated, and whether the focus returned tothe client. Independent judgments were then discussed, with team members rotatingwho spoke first to avoid having the same team member always go first or last and tosafeguard against domination of the conversation by more persuasive or opinionatedmembers. Each team member explained her rationale for her judgment, without anyreaction from other team members. Once all team members had spoken, they discussedtheir perspectives until they reached consensus. The first author, who participated onboth teams, made a special effort to ensure that each team member (herself included)expressed her perspective fully and listened to others respectfully.

Adapting a constant comparative process from grounded theory research (Hallberg,2006), the event being coded was compared to previously coded events to ensure con-sistency of ratings for each variable. This step was especially useful for the intimacyand quality scales (e.g. “Do we all agree that this event was higher in intimacy than Xevent but lower than Y event?”).

As with many interventions in therapy, we found that each TSD had to be consid-ered in the context of the therapy relationship, the client’s issues, and what was happen-ing in the moment. For example, without context, “Two years ago, when I first startedpracticing, I would have been uncomfortable [discussing sex and infidelity], but now Iam more comfortable.” may seem inappropriate for a therapist trainee to share with aclient. However, this client was hesitant to talk about an extramarital affair, and thetherapist was trying to encourage and reassure him. The client had hinted that he wor-ried that the subject might make the therapist uncomfortable, and had refrained fromdiscussing it for that reason. Thus, judges used the context as well as the content ofTSDs when making the ratings.

Relationships among TSD variables and client-rated post-session measures

Because clients were nested within therapists, we ran a series of “empty” models foreach of the variables examined in this study. The variance components from the emptymodels were used to calculate therapist-level intraclass correlation coefficients (ICCs).

Counselling Psychology Quarterly 37

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 11: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

The therapist-level ICC indexes the amount of variance in a variable (e.g. RRI) that isbetween therapists. The therapist ICCs for all of the variables were small (< .01) and notsignificant (all ps > .39). Therefore, therapist variance was not included in our mainanalyses. We analyzed relationships among TSD variables (type, quality, and intimacy)and client post-session measures using hierarchical linear modeling (HLM; Raudenbush,Bryk, & Congdon, 2011), a statistical software package designed to analyze nested(multilevel) models. HLM accounts for multilevel data (i.e. TSD events within therapysessions within clients) and for the nonindependence of the data (e.g. probable depen-dence of a client’s RRI-C ratings across sessions) (Bryk & Raudenbush, 2002). HLMallowed us to simultaneously investigate TSD-level variables at the lowest level of themodel (level 1: type, intimacy, and quality), session-level variables (level 2: RRI-C,WAI-SR, and whether or not TSD occurred – yes/no, for a specific session), and client-level variables at the highest level of the model (average RRI-C and WAI-SR for eachclient [i.e. slope]). Level 1 (TSD event level) variables are impacted by level 2 (session-level) variables because they are nested within them and therefore share common vari-ance (i.e. event-level variables are impacted by session-level variables). Likewise, level2 (session-level) variables are nested within and impacted by level 3 (client-level) vari-ables. Following the procedures described by Curran and Bauer (2011), the variance inthe predictor variables was partitioned into between events (TSDs), between sessions,and between clients. We found the greatest variance in disclosures for all variables at theevent level. Less variance was accounted for by sessions and clients than events,although in some cases, session-level and client-level variance was small but significant,validating the use of the hierarchical model for analyzing the data.

Because HLM cannot handle missing data, three sessions that lacked data fromclient RRI-C and/or WAI-SR ratings were not included in the HLM analyses. Inaddition, disclosures of strategy were not included because there were only six suchevents. Significant results for each independent variable reported below include theHLM regression coefficient (μ100 – the coefficient of the first group compared to thereference group, μ200 – the coefficient of the second group compared to the referentgroup, and so on), the regression’s standard error (SE), the regression’s degrees of free-dom and T-statistic, the p-value at which the regression results were statistically signifi-cant, and predicted (for continuous variables) or estimated (for categorical variables)ratings for the relevant variable for each type.

Results

Description of TSD events

There were 185 TSD events, ranging from 1 to 7 events per session and 1 to 29 eventsper case. Of the 360 sessions in the sample, 115 sessions included at least one TSDevent. For the descriptive data (see Table 1), we first computed means for each caseand then computed averages across cases (thereby controlling for the number ofsessions per case).

Events averaged 11.03 s (SD = 12.33, ranging from 0 to 75 s) in duration. Thera-pists initiated 73% (SD = 7%) of the TSDs (ranging from 82 to 100% per case), andthe focus turned back to the client following 93% of the disclosures. On a nine-pointLikert-type scale ranging from 1 (least intimate) to 9 (most intimate), the average

38 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 12: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

intimacy rating across cases was 2.46 (SD = .65; range = 1.00–3.67). The majority(82%) of events were rated as low intimacy (levels 1–3 on a nine-point scale). On athree-point Likert-type scale ranging from 0 (low quality) to 2 (high quality), the aver-age quality rating across cases was .70 (SD = .43; range = .00–1.56). The majority(57%) of events were rated as low in quality. The average numbers of TSD events persession per case across the 16 cases, by type, were facts = .27 (SD = .70); feelings = .11(SD = .36); insight = .07 (SD = .30); and strategy = .01 (SD = .12). Thus, the typicalTSD event was brief, infrequent (approximately one every other session), initiated bytherapists, low in intimacy, and low in quality. The typical event was a disclosure offacts.

However, describing the “typical” TSD only gets at a small part of the picture. Themajority of sessions did not include any TSD, but two sessions included seven each.TSDs varied widely in terms of content, quality, intimacy, and clients’ responses, bothacross and within types. An example of each type follows, with quality and intimacyratings to illustrate differences: “I don’t watch Mad Men.” (type = facts, intimacy = 1,and quality = low); “If I were you and she threatened divorce … I would feel kind ofparanoid.” (type = feelings, intimacy = 4, and quality = high); “I guess, for me, the wayI understand myself is based on what’s inside me, and beliefs stay outside.” (type =insight, intimacy = 4, and quality = high); and “What I did when I was studying for thecollege entrance exam was I got two different books and practiced A LOT.”(type = strategy, intimacy = 4, and quality = low).

Some TSDs were rated by judges as likely to have therapeutic impact (i.e. highquality), whereas others seemed unrelated to clients’ content and core issues (i.e. lowquality). Intimacy ranged from 1 (low) to 7 (high) for TSD events; the average TSDintimacy by case ranged from 1 to 3.67. There was also variation across and withintherapists. For example, one therapist had low average intimacy levels and averagequality levels across three cases, with the therapist initiating 0, 28, and 100% of TSD,respectively, whereas a different therapist had an average intimacy in the low range forthree clients, but varied in quality by case such that one case was low, one medium,

Table 1. Description of TSD events across 16 cases.

Type Mean SD Min. Max. Range of scale

Facts .27 .70 .00 .88 0 = did not occur, 1 = did occurFeelings .11 .36 .00 .46 0 = did not occur, 1 = did occurInsight .07 .30 .00 .23 0 = did not occur, 1 = did occurStrategy .01 .12 .00 .13 0 = did not occur, 1 = did occurDuration 11.03 12.33 1.00 75.00 0–75 sIntimacy 2.46 .65 1.00 3.67 1 = low or not at all intimate,

5 = medium or moderatelyintimate,9 = high or very intimate

Quality .70 .43 .00 1.56 0 = low quality,1 = medium quality,2 = high quality

RRI 4.05 .29 3.53 4.53 1 = weakest, 5 = strongestWAI 3.79 .37 2.28 4.22 1 = weakest, 5 = strongest

Notes: RRI = Real Relationship Inventory. WAI = Working Alliance Inventory.

Counselling Psychology Quarterly 39

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 13: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

and one high in quality, and the therapist initiated almost all the disclosures across allthree cases (88, 92, and 81%).

Predicting client post-session measures

In this first set of analyses we examined, all sessions and TSDs were treated as dichoto-mous predictor variables to determine whether sessions with TSD differed from sessionswithout TSD. This section addresses results from analyses conducted using all therapysessions in the sample (whereas the next section addresses results of analyses excludingsessions without TSD). HLM analyses comparing sessions with TSD events to thosewithout TSD events were conducted using the 357 sessions for which complete datawere available for the 16 cases.

The stronger the WAI-SR averaged across sessions for a client was, the more likelythat a TSD of any type (facts, feelings, or insight) occurred (μ01 = 1.05, SE = .42,t(df = 14) = 2.49, p = .026). For every one-point increase in the WAI-SR score averagedacross sessions for a client, the odds of a therapist making a disclosure increased 286%.Thus, a client’s overall or average working alliance was positively related to the numberof TSDs.

The stronger the RRI-C was for a session, the less likely that a disclosure of factshad occurred in that session (μ10 = −1.21, SE = .50, t(df = 15) = −2.43, p = .028). Forevery one-point increase in the RRI-C score, the odds of a therapist making a disclosureof facts decreased by 70%. Therefore, a client’s real relationship in a session was nega-tively related to the number of factual TSDs in that session.

In the next set of analyses, we only examined those sessions in which one or moreTSD occurred. In other words, sessions without TSD were excluded. Data consisted ofa total of 183 disclosures nested within 113 sessions within 16 clients.

When a client’s scores were high on RRI-C for a session, the therapist was lesslikely to have made disclosures of facts in that session than when RRI-C scores werelow (μ010 = −1.70, SE = .56, t(df = 15) = −3.02, p = .009), so RRI-C and disclosures offacts were negatively related. A one-point increase in the RRI-C score decreased theodds of a disclosure of facts by 82%. Thus, therapists disclosed facts more in sessionswhen there was a weak real relationship, or there was a weak real relationship in ses-sions in which therapists disclosed facts more.

Similarly, when a client’s scores were high on the WAI-SR, the therapist was lesslikely to have made disclosures of facts than when the client WAI-SR was low(μ010 = −1.78, SE = .57, t(df = 15) = −3.10, p = .007), so WAI-SR and disclosures offacts were negatively related. A one-point increase in the WAI-SR score decreased theodds of a therapist making disclosure of facts by 83%. Thus, therapists disclosed morefacts when there was a poor working alliance (or vice versa).

Frequency of disclosures of feelings was positively related to clients’ RRI-C ratings(μ010 = 1.81, SE = .64, t(df = 15) = 2.83, p = .013). With a one-point increase in theRRI-C score, the likelihood of a therapist disclosing feelings increased 611%, thus,therapists disclosed more feelings when clients reported a strong real relationship (orvice versa).

TSD event intimacy and client RRI-C ratings were positively correlated (μ010 = .98,SE = .28, t(df = 15) = 3.47, p = .003). For every one-point of increase in the RRI-C, theintimacy ratings of TSDs increased by .98 points (on the nine-point Likert-type

40 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 14: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

intimacy scale ranging from 1 to 9). Hence, therapists disclosed with more intimacy (allin the moderate range) in sessions when clients reported a strong real relationship, orclients reported a strong real relationship when therapists disclosed with more intimacy.

Similarly, TSD event intimacy and client WAI-SR ratings were positively correlated(μ010 = .89, SE = .24, t(df = 15) = 3.79, p = .002). Specifically, for every one-pointincrease in WAI-SR, the intimacy ratings of TSDs increased by .89 points (on the nine-point Likert-type intimacy scale ranging from 1 to 9). Hence, therapists disclosed withmore intimacy (all in the moderate range) in sessions when clients reported a strongworking alliance, or clients reported a strong working alliance when therapists disclosedwith more intimacy.

Discussion

Our primary aim in this study was to enhance our understanding of verbal TSD (notincluding immediacy) in naturally occurring psychotherapy with doctoral student psy-chodynamic/interpersonal therapists and community clients. As previous research hasexamined TSD globally, we wanted to examine its variations, subtypes, and characteris-tics in hopes of learning more about TSD occurrence. We found that TSDs were infre-quent and brief; most often were therapist initiated; most often returned the focus to theclient; on average were at a low level of intimacy; on average were at a low level ofquality; and were most often disclosures of facts rather than of feelings, insight, orstrategies. Disclosures of fact were rated as lower in quality and were associated withlower client-rated real relationship and working alliance than were other types of TSDs.In contrast, disclosures of feeling and disclosures of insight were rated as high in qual-ity, and disclosures of feelings were positively associated with client-rated real relation-ship. The stronger the client-rated working alliance, the more TSDs there were; thestronger the client-rated real relationship, the fewer factual TSDs there were. Finally,results varied widely across sessions, suggesting that therapists adjusted their use ofTSDs to be responsive to the situation.

The results confirm that TSD is indeed a complicated, multifaceted intervention thatdiffers widely from one instance to the next. TSDs ranged from a straightforward state-ment of fact that was not rated as being high in quality (e.g. “I am not a doctor. I don’thave that label yet because I’m still a graduate student.”) to a high-quality nuancedchallenge (e.g. “If you said that to me, I would think you were irritated and upset withme.”) to a disclosure of strategy (e.g. “When I was in your situation, I joined a bereave-ment support group and it was very helpful.”). Hence, TSDs often mimic other inter-ventions (e.g. facts, reflections of feelings, and strategies in the above examples), butare offered with a personal rather than an impersonal manner, which may make themeasier for clients to hear.

It is important to keep in mind that all of our findings were correlational rather thancausal, and thus directionality is unclear. Changes in clients’ ratings of the therapeuticrelationship could have led to changes in TSD, changes in TSD could have led tochanges in how clients rated the therapeutic relationship, or an unidentified third vari-able could have caused changes in both clients’ ratings of the therapeutic relationshipand in TSD. Findings are discussed in more detail below.

Counselling Psychology Quarterly 41

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 15: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

Frequency and duration of TSD

In terms of overall frequency, approximately one TSD occurred every other session.The average therapist speaking time (duration) for a TSD event across cases was rela-tively brief (11.03 s). This rarity and brevity of occurrence was expected, given recom-mendations in the literature for infrequent use of TSD (Knox & Hill, 2003) andprevious findings that TSD is used rarely (Hill & Knox, 2002).

Initiation

Therapists initiated the majority (135) of the 185 disclosures in this sample. Thus,therapists were usually the ones who decided when and how to use TSDs rather thanclients asking for disclosures. None of the client-initiated TSDs were rated as being ofhigh quality, suggesting that therapists were not able to disclose therapeutically whenclients requested information. Interestingly, 2 of the 16 clients accounted for more thanhalf of the client-initiated disclosures (28 of 50) in the sample, whereas 5 clients didnot request any TSDs, suggesting that there may be a client factor involved in whysome clients ask for TSDs. For example, in one of the cases where the client frequentlyrequested TSDs, the client seemed to want more disclosure than the therapist was will-ing to give. Throughout the work, there was tension when the client would ask personalquestions and then make follow-up queries, even though it was evident that the thera-pist preferred not to answer these questions. During termination, when the therapist saidshe was going on internship, the client asked a series of rapid-fire questions to learnmore about the therapist’s plans. The therapist felt extremely uncomfortable with thesequestions. This example underscores calls in the literature for therapists to think deeplyin advance about what, how, and when they might disclose (Henretty & Levitt, 2010),as well as how to engage the client in a dialog surrounding requests for TSDs.

Return of focus

The focus of the session turned back to the client following the majority of disclosures(93%), either explicitly by the therapist (e.g. “How does that fit for you?”) or by the cli-ent responding with more exploration. This finding is consistent with suggestions in theliterature (Henretty & Levitt, 2010; Knox & Hill, 2003) that therapists should avoidshifting the focus away from the client. Because these therapists were trained usingHill’s (2009) helping skills book, they were all aware of the importance of returning thefocus of a session to the client following a TSD. In addition, because these therapistswere in supervision, it is possible that they were influenced by the fact that supervisorsmight view session recordings.

Intimacy

The average intimacy rating for a TSD event across cases was relatively low (2.46 on anine-point Likert-type scale where 1 = least intimate and 9 = most intimate). There wereno TSDs rated 8 or 9 on the scale, and there were no disclosures that the judges consid-ered to be extremely or inappropriately intimate. These findings are consistent withrecommendations in the literature that TSDs should not be too intimate, but should be

42 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 16: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

moderately intimate to achieve the beneficial effects of making therapists more real orhuman or establishing trust between therapist and client (Knox & Hill, 2003). It islikely that the low intimacy levels in our sample were due to the majority of disclosuresbeing of facts (e.g. “Yes, I read Miss Manners.”).

The intimacy of TSDs (which ranged from low to moderately intimate) was posi-tively related to client ratings of both the real relationship and the working alliance. Itmakes intuitive sense that therapists would not disclose even moderately intimate mate-rial in the absence of a strong therapeutic relationship. Similarly, as the relationshipstrengthens, we would expect an increase in the intimacy of TSD content as wasreflected in the data (recalling that there were no highly intimate or inappropriatedisclosures).

Quality

The average level of TSD quality was low (.70; 0 = low and 2 = high quality), whichmakes sense, given that the majority of disclosures were factual. The preponderance offactual disclosures (M = .27) was somewhat surprising, given the emphasis in the litera-ture (Henretty & Levitt, 2010; Knox & Hill, 2003) on the importance of using moder-ately intimate disclosures with therapeutic intention. It is possible that disclosures offacts were less risky than other types, particularly for student therapists because (for themost part) they tended to be innocuous. Therapists in our sample may have felt morecomfortable with them, and as a result, used them relatively frequently.

Given that disclosures of facts typically (though not always) conveyed basicinformation that was not necessarily related to client issues (e.g. “Yes, I have heard theexpression ‘dog and pony show’”), it is not surprising that they were judged as lowestin quality among the types. Judges did not perceive that they contributed to the bond orenhanced the relationship between the client and the therapist, which was part of thedefinition of quality for the present study. In most cases, disclosures of fact lacked boththe emotional arousal “hook” necessary for change to occur (Frank & Frank, 1991) andthe possibility of increasing the client’s understanding, and therefore were less likely tobe rated as therapeutic.

In contrast, disclosures of feelings and disclosures of insight were more likely to berated as higher in quality because they were more directly relevant to client material.All 48 high-quality TSDs were initiated by the therapist and most of these high-qualityTSDs were coded as disclosures of feelings (27) or insight (16). The intimacy level forthe majority (44) of these high-quality disclosures was moderate, in the 3–5 range ofthe scale. Thus, these TSDs rated high in quality aligned with guidance in the literature(Henretty & Levitt, 2010; Knox & Hill, 2003) and findings from qualitative client inter-views. Knox and Hill (2003) advised therapists to only disclose material that is relevantto the client. Similarly, Audet and Everall (2010) found that clients assessed TSD (forsuitability, relevance, and context appropriateness) and used it to determine both howattuned therapists were to their issues and needs and how responsive they felt therapistswere.

An example of a high-quality disclosure of insight (intimacy = 6) is, “I’m beingjudgmental, but from my perspective that’s extremely harsh for a family member to dothat to you.” This client had experienced significant trauma, was socially isolated, haddifficulty connecting with others, and had a hard time getting in touch with his

Counselling Psychology Quarterly 43

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 17: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

emotions. He expected the worst from others and seemed to take bad treatment byothers as a matter of course. The therapist prefaced this disclosure by admitting that hewas “being judgmental,” implying, perhaps, that he had an emotional reaction tothe incident the client had described. Though the words and the tone were tempered,the therapist seemed to be expressing outrage at the treatment his client had received.He went on to provide normalization for the client in terms of the kind of response hebelieved would be warranted in the situation. Assuming the client was ready to acceptthe therapist’s assessment of the situation (and he seemed to be), the therapist’s risk insharing his emotional reaction and his judgment of the situation was directly related tothe client’s core issue of feeling disconnected from others and also modeled an appro-priate emotional response to what was happening in the client’s life at the moment. Hill(2009) suggested that disclosures of feelings encourage increased client experiencingand emotional arousal, and disclosures of insight encourage client exploration andunderstanding (Hill, 2009). Furthermore, if used effectively, both types (feelings andinsight) may serve to convey to the client that the therapist understands his or her issuesdeeply (Audet & Everall, 2010), thereby strengthening the therapeutic relationship andfacilitating client change.

TSD in relation to client-rated post-session measures

Overall TSD occurrence

There was a positive association between client-reported working alliance and TSDoccurrence. Thus, a strong therapist–client working bond and agreement on treatmentgoals and tasks (Bordin, 1979; Gelso, 2011) was related to TSD use. Perhaps, the moretherapists were attuned with clients’ issues and therapeutic goals, the more comfortablethey were using TSD. Clarity about treatment goals and effective collaboration towardachievement of those goals may have provided therapists with a level of comfort aboutthe direction of therapy, which may have mitigated the potential risk of shifting thefocus away from the client and allowing the therapist more leeway to engage in TSDs.

These findings coincide with conclusions drawn by Audet and Everall (2010) for alink between TSD and the working alliance. Specifically, these researchers found that“[therapist] disclosure can influence the extent to which clients are willing to share andprocess information that is therapeutically relevant to them” and also that therapists’attunement with clients’ issues and needs, as reflected in TSD, had “a bearing on clientconfidence in the therapist’s abilities and the working relationship, creating conditionsthat either supported or hindered engagement” (Audet & Everall, 2010, p. 339). It isalso possible that the directionality is reversed: more frequent TSD may build a strongerworking alliance between client and therapist. Perhaps a cyclical pattern exists, in whicheffective TSD engenders client confidence in the therapist and in treatment, which leadsto increased engagement on the client’s part, and a subsequent strengthening of theworking alliance. As Audet and Everall (2010) noted, the cycle could also move in theopposite direction, harming the working alliance, should the therapist disclose inappro-priate or irrelevant material.

Our finding that the working alliance (averaged across sessions), but not the realrelationship, was positively associated with occurrence of TSD seems to contradictAin’s (Ain, 2011; Ain & Gelso, 2008) findings that the real relationship correlated with

44 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 18: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

the amount of TSD. However, in these studies, Ain compared clients’ perceptions ofthe real relationship with clients’ ratings of the amount of TSD, whereas in the currentstudy, we examined actual occurrence of observer-coded TSD. It may be that when cli-ents rate the overall amount of TSD, they are conflating it with their general feelingsabout the therapy and treatment as a whole, and providing an overall satisfaction rating.

Facts

When a client reported a stronger real relationship and working alliance, the therapistwas less likely to make disclosures of facts than when a client reported a weaker realrelationship and working alliance. If we consider the common factors that contribute tothe effectiveness of mainstream approaches to therapy (Frank & Frank, 1991; Hill,2009), it stands to reason that disclosures of facts (which tend to convey basic informa-tion not necessarily related to client issues) were less likely than other types to con-tribute to the therapeutic relationship, and therefore were not likely to be associatedwith stronger RRI-C or WAI-SR scores. Alternatively, it is possible that when thetherapist made disclosures of facts, the client was less likely to report a stronger realrelationship and working alliance.

Because the vast majority of the variance in factual TSD was between sessions(with little between-client variance in factual TSD), sessions, rather than clients, wererelated to whether or not factual TSD occurred. In other words, whether or not a thera-pist made a factual TSD appears to be related to what was happening in a particularsession (as opposed to being related to a particular client). These findings suggest thattherapists were attuned to clients in the moment and deciding whether or not to dis-close. In their book on countertransference, Gelso and Hayes (2007) posited that “ …the therapist takes into account factors such as timing, content, tone, affect, and thepatient’s capacity to receive what the therapist wishes to convey, and then engages inempathic expression” (p. 110).

Feelings

Disclosures of feelings were positively related to clients’ ratings on the RRI-C, suggest-ing that therapists were more likely to make disclosures of feelings when clientsreported a strong rather than a weak real relationship. On the other hand, it is possiblethat when therapists made disclosures of feelings, clients were more likely to report astrong relationship. A disclosure of feeling can be a risky intervention because if thetherapist chooses the wrong emotion (i.e. an emotion far from the client’s experience),s/he risks alienating the client. We posit that a therapist would be unlikely to disclosefeelings about something important unless the therapist (a) feels confident that s/heknows the client and his/her emotional state well enough to disclose a feeling that willresonate strongly with the client’s experience in the moment, (b) has considered thecontext of the therapeutic relationship and believes it is strong enough to support apotential gaffe, and (c) feels close enough to the client that s/he is willing to be vulnera-ble by sharing his/her feelings. When feelings (whether the client’s or the therapist’s)are being shared accurately and received empathically, the two people become closer,with fewer boundaries. It follows that when the therapist shares his/her feelings, theclient would be more likely to see the therapist as a real person and to rate the real

Counselling Psychology Quarterly 45

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 19: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

relationship as stronger. Alternatively, the therapist may be more likely to disclosefeelings in the context of a strong real relationship in which the client sees the therapistas a real person.

Comparison of TSD and immediacy

Hill et al. (2014) investigated the uses and consequences of immediacy, defined as dis-closure about the therapeutic relationship (similar to self-involving disclosures). Wehighlight this study because we used the same sample of 16 cases for the present studyand because we want to compare the two interventions. These authors found thatimmediacy events were relatively brief (M = 285 s), infrequent (5–14% of total time intherapy), and generally initiated by the therapist (85%). Immediacy events focused onhelping clients explore unexpressed feelings about the therapeutic relationship (59%),negotiating tasks and goals (26%), discussing parallels to other relationships (26%), anddiscussing ruptures (8%). The two most common consequences of immediacy, as deter-mined by trained judges, were clients expressing feelings about the therapist or therapyand gaining insight.

Immediacy occurred more often per session than did TSD (M = .61, SD = .31 vs.M = .45, SD = .31, d = .52), and also lasted longer than did TSD events (M = 285,SD = 114 s vs. M = 11, SD = 13 s, d = 3.38). Therapists initiated 85% of immediacyevents in contrast with 73% of TSD events, which is slightly more, although we cannotdetermine if the percentage is meaningfully different.

Because the clinic was a psychodynamic/interpersonal psychotherapy clinic thatemphasized interpersonal techniques, we were not surprised that there were moreimmediacy events than TSD events. Given that immediacy tends to focus on the thera-peutic relationship, it also makes sense that the immediacy events lasted longer thanTSD events. With immediacy, the emphasis is on what is going on between the thera-pist and the client, so the focus stays there long enough for the dyad to process what’shappening in the moment and how that may be relevant to the client’s behavior bothwithin and outside the therapy room. For example, the question “Are you building wallsin our relationship to prevent any conflict or confrontation from arising?” prompted adiscussion about the therapeutic relationship and the client’s tendency to push othersaway. However, high-quality TSD is used more in service of focus on the client to helphim/her deepen emotions or insight, so brevity and an immediate return of focus to cli-ent material is warranted. For example, the therapist’s statement, “I don’t think extra-marital affairs are good or bad.” was used to encourage the client to feel morecomfortable talking in detail about his infidelity to his partner.

Finally, the finding that clients initiated more TSD than immediacy events may sug-gest that clients are more interested in learning something about the therapist than indiscussing the therapeutic relationship. It may be that clients don’t fully understandinterpersonal therapy and do not see the therapeutic relationship as something that war-rants examination and is relevant to their issues. Thus, clients may be less likely toinitiate exploration of the relationship. On the other hand, (at least some) clients arelikely curious about their therapists and may feel uncomfortable with the one-sided nat-ure of disclosing in therapy. This may cause clients to be more inclined to solicitinformation. Indeed, more than one client in our sample specifically stated they needed

46 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 20: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

“a little give and take” (i.e. chit-chat) at the beginning of sessions to ease into theprocess.

Although TSD and immediacy occurred in every case, they did not occur at thesame time and did not seem to be connected with one another in any way in any of theinstances. Thus, the two interventions were distinct. This finding aligned with our suspi-cion that TSD and immediacy are very different interventions with different intentions,different qualities, and different consequences, though these questions remain open toempirical investigation.

Strengths and limitations

The most significant strength of this study was that it was conducted in a naturalisticclinical setting with actual clients. An additional strength was the rigorous process ofanalysis undertaken by the judges, which involved at least four judges coding eachTSD and a constant comparative process for placing each TSD event in the context ofthe others. In addition, TSD was clearly defined and broken down into various compo-nents, rather than treated as a one-dimensional intervention. Two final strengths werethe mixed method nature of the study (i.e. analyzing the data both quantitatively andqualitatively) and the use of HLM to analyze the multilevel data.

The first set of limitations relate to the sample. With only 16 therapeutic dyads, thesample was relatively small. The sample of both therapists and clients included goodrepresentation from both genders. The majority of clients (11) and therapists (5) wereEuropean American, but other ethnicities/races were also represented. In addition, thecases, ranging from 11 to 60 sessions, were relatively brief when compared with thetypical length of psychodynamic psychotherapy in private practice (recall that onlycases with at least 8 sessions that had already terminated at the time of the study wereevaluated). Also, narrowing the representativeness of the results, the therapists weredoctoral trainees and were all trained in the same psychodynamically oriented counsel-ing psychology doctoral program. In addition, these therapy sessions were videotapedand viewed by supervisors, so it is possible that the TSDs in this sample might havebeen different from those found in a sample of practicing clinicians. Finally, the ter-mination of these cases may not have been representative of successfully completedpsychodynamic psychotherapy cases, given that of the 16 cases, 7 terminated when thetherapist left the clinic, 2 when the client moved out of the region, 3 clients terminatedagainst their therapists’ advice, 3 clients dropped out without explaining why, and 1 cli-ent dropped out to begin couples therapy. Accordingly, results may not generalize toother populations and settings.

Another limitation is that the only source of data regarding the TSD events wasjudges’ observations. Although the judges worked to identify and bracket their biasesand discussed each TSD to consider as many perspectives as possible, the judges them-selves may have played a significant role in the construction of the findings. Becauseconsensual qualitative research is primarily a constructivist process, ratings are alwayssubjective, and results obtained from one group’s consensual coding probably do notexactly match those that would be obtained from another group. In addition, the judgeswere undergraduate psychology majors who were motivated and invested in the pro-cess. They were generally informed about psychotherapy, but also able to evaluate TSDwith as few preconceptions as possible (i.e. without a major bias for or against TSD or

Counselling Psychology Quarterly 47

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 21: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

a detailed understanding or opinion regarding the role it might play in therapy).Doctoral students or postdoctoral psychologists may have had different perspectives,given their therapeutic training.

A third limitation is that the scales used to assess intimacy and quality were devel-oped for this study. Further research is needed to establish their reliability and validity.

In addition, there were a few TSDs that did not fit well into the typology (facts,feelings, insight, and strategy). Disclosures about beliefs, opinions, and value judgments(all of which occurred infrequently) were “forced” into the existing categories.

Finally, although our analytic approach allowed for high-powered statistical analy-ses, it must be noted that these analyses provided a very broad-brush stroke picture ofthe occurrence and consequences of TSDs (i.e. for therapists who generally do or donot disclose what the effects are). We did not examine in detail the immediate conse-quences of TSDs, and so cannot make conclusions about how clients responded in themoment to therapists’ disclosures.

Implications

Given that clients have reported TSDs to be helpful (Hanson, 2005; Hill et al., 1988)and that non-disclosure on the part of the therapist has been found to be unhelpful(Hanson, 2005), it seems important that therapists include TSD as part of their reper-toire of skills to be used when appropriate. We emphasize that we are not talking aboutdisclosures of facts (or other types) that do not have therapeutic benefit, but rather thoseof high quality and moderate intimacy that are focused on feelings and insight. How-ever, as articulated by Gelso and Hayes (2007) in their book about countertransferenceand the therapist’s inner world, questions of “ … just how and how much of [the thera-pist’s] world should be expressed during the [therapy] hour, and in which ways thisinner world should be expressed” (p. 91) remain unanswered with regard to TSD. Weconcur with earlier theorizing (Hill & Knox, 2002) that quantity does not equal quality;TSD can be a particularly powerful intervention when used sparingly with forethoughtand intention. To maximize the benefits of TSD, we join others in encouraging thera-pists to disclose in ways that are “therapeutically meaningful to the client” (Audet &Everall, 2010, p. 339). This caution is particularly compelling, given findings that cli-ents assess TSDs for relevance and therapeutic intent (Audet & Everall, 2010). Whenconsidering using TSD, therapists should thus contemplate the context of the session,the strength of the therapeutic relationship, timing, and how the client will perceivewhat the therapist shares.

What kind of disclosures should therapists make? The highest quality disclosures inthis sample were disclosures of insight and disclosures of feelings that were directlyrelevant to client concerns and client material. Therapists should keep in mind that it islikely that TSD intimacy, like frequency, is likely “good up to a certain point” (Gelso& Palma, 2011, p. 347), but beyond that point, it may cease to be useful or therapeutic.Consistent with Knox and Hill’s (2003) recommendations, we suggest that therapistsdisclose with moderate levels of intimacy to maximize beneficial effects. We alsorecommend that therapists return the focus of the session to the client following self-disclosure and assess the effect of each disclosure.

Therapists can also use their urges to self-disclose as a gauge for what is happeningin the relationship. For example, if a therapist feels pulled to disclose facts, this may be

48 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 22: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

a clue that the relationship needs strengthening. The association of factual disclosureswith weaker client ratings of the real relationship and working alliance suggests thattherapists should think carefully before using disclosures of facts.

For the purposes of both training and research, it is important to conceptualize TSDas complex and multifaceted, rather than as a monolithic intervention. Put concisely,TSD is not one therapeutic intervention, but many (often mimicking other interventionssuch as reflections of feelings, challenges, and direct guidance), and should be treatedas such. We recommend that each type of TSD be taught and researched separately,with attention paid to differing intentions, impacts, and relationships with other vari-ables (e.g. session outcome measures) based on disclosure type.

Future researchers might consider including data from multiple perspectives (i.e. cli-ent, therapist, and observers) to provide a more comprehensive view of TSD and allowfor triangulation of findings. Obtaining post-session feedback from therapists and cli-ents, respectively, about intentions for and effects of TSD would provide particularlyuseful information. In addition, an examination of therapists’ intentions for disclosingand a qualitative investigation of therapists’ attitudes toward and use of TSD may leadto insights into how high-quality TSDs are formulated. Finally, a temporal analysis ofdisclosure occurrence and client ratings of the relationship would move us one step clo-ser toward understanding the outcomes of TSDs.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributorsKristen G. Pinto-Coelho is a doctoral student in counseling psychology at the University ofMaryland. Kristen G. Pinto-Coelho’s research interests include psychotherapy process and out-come, therapist factors, and dream work in psychotherapy.

Clara E. Hill is a professor in the Department of Psychology at the University of Maryland.Clara E Hill’s current areas of interest are the identification and training of counseling skills, pro-cess and outcome studies of psychotherapy, working with dreams, and qualitative research.

Dennis M. Kivlighan, Jr., is a professor in the Department of Counseling, Higher Education andSpecial Education at the University of Maryland. He is interested in applying new statisticalmethods to better understand the therapeutic processes in individual and group treatments.Currently, he is an associate editor for the Journal of Counseling Psychology and was the formereditor of Group Dynamics: Theory, Research and Practice.

References

Ain, S. (2011). The real relationship, therapist self-disclosure, and treatment progress: A study ofpsychotherapy dyads. College Park, MD: University of Maryland. Retrieved from http://hdl.handle.net/1903/11995

Ain, S., & Gelso, C. J. (2008). Chipping away at the blank screen: Self-disclosure, the real rela-tionship, and therapy outcome (Master’s thesis). University of Maryland, College Park, MD.Retrieved from http://drum.lib.umd.edu/bitstream/1903/8722/1/umi-umd-5451

Counselling Psychology Quarterly 49

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 23: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

Audet, C. T. (2011). Client perspectives of therapist self-disclosure: Violating boundaries orremoving barriers? Counselling Psychology Quarterly, 24, 85–100.

Audet, C. T., & Everall, R. D. (2010). Therapist self-disclosure and the therapeutic relationship:A phenomenological study from the client perspective. British Journal of Guidance & Coun-selling, 38, 327–342.

Barrett, M. S., & Berman, J. S. (2001). Is psychotherapy more effective when therapists discloseinformation about themselves? Journal of Consulting and Clinical Psychology, 69, 597–603.

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance.Psychotherapy: Theory, Research & Practice, 16, 252–260.

Bryk, A. S., & Raudenbush, S. W. (2002). Hierarchical linear models: Applications and dataanalysis methods (2nd ed.). Thousand Oaks, CA: Sage.

Bugental, J. F. T. (1965). The search for authenticity. New York, NY: Holt, Rinehart, & Winston.Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person

effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619.doi:10.1146/annurev.psych.093008.100356

Curtis, J. M. (1982). Principles and techniques of non-disclosure by the therapist during psy-chotherapy. Psychological Reports, 51, 907–914.

Derlega, V. J., Hendrick, S. S., Winstead, B. A., & Berg, J. H. (1991). Psychotherapy as a per-sonal relationship. New York, NY: Guilford Press.

Dowd, E. T., & Boroto, D. R. (1982). Differential effects of counselor self-disclosure, self-involving statements, and interpretation. Journal of Counseling Psychology, 29, 8–13.

Eagle, M. N. (2011). From classical to contemporary psychoanalysis: A critique and integration.New York, NY: Routledge.

Elliott, R. (1985). Helpful and nonhelpful events in brief counseling interviews: An empiricaltaxonomy. Journal of Counseling Psychology, 32, 307–322.

Farber, B. A. (2006). Self-disclosure in psychotherapy. New York, NY: Guilford Press.Frank, J. D., & Frank, J. B. (1991). Persuasion and healing: A comparative study of psychother-

apy (3rd ed.). Baltimore, MD: Johns Hopkins University Press.Freud, S. (1958). The dynamics of transference. In J. Strachey (Ed.), The standard edition of the

complete psychological works of Sigmund Freud, Vol. 12. (pp. 97–108). London: HogarthPress (Original work published 1912).

Fuertes, J., Mislowack, A., Brown, S., Gur-Arie, S., Wilkinson, S., & Gelso, C. (2007). Correlatesof the real relationship in psychotherapy: A study of dyads. Psychotherapy Research, 17,423–430.

Gelso, C. J. (2011). The real relationship in psychotherapy: The hidden foundation of change.Washington, DC: American Psychological Association.

Gelso, C. J., & Hayes, J. A. (2007). Countertransference and the therapist’s inner experience:Perils and possibilities. Mahwah, NJ: Lawrence Erlbaum Associates.

Gelso, C. J., & Palma, B. (2011). Directions for research on self-disclosure and immediacy:Moderation, mediation, and the inverted U. Psychotherapy, 48, 342–348.

Greenson, R. R. (1967). The technique and practice of psychoanalysis, Vol. 1. New York, NY:International Universities Press.

Hallberg, L. R.-M. (2006). The “core category” of grounded theory: Making constant compar-isons. International Journal of Qualitative Studies on Health and Well-being, 1, 141–148.doi:10.1080/17482620600858399

Hanson, J. (2005). Should your lips be zipped? How therapist self-disclosure and non-disclosureaffects clients. Counselling and Psychotherapy Research, 5, 96–104.

Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short versionof the working alliance inventory. Psychotherapy Research, 16, 12–25.

Henretty, J. R., & Levitt, H. M. (2010). The role of therapist self-disclosure in psychotherapy: Aqualitative review. Clinical Psychology Review, 30, 63–77. doi:10.1016/j.cpr.2009.09.004

50 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 24: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

Hill, C. E. (2009). Helping skills: Facilitating exploration, insight, and action (3rd ed.).Washington, DC: American Psychological Association.

Hill, C. E. (Ed.). (2012). Consensual qualitative research: A practical resource for investigatingsocial science phenomena. Washington, DC: American Psychological Association.

Hill, C. E., Gelso, C. J., Chui, H., Spangler, P., Hummel, A., Huang, T., … Miles, J. R. (2014).To be or not to be immediate with clients: The use and perceived effects of immediacy inpsychodynamic/interpersonal psychotherapy. Psychotherapy Research, 299–315.

Hill, C. E., Helms, J. E., Tichenor, V., Spiegel, S. B., O’Grady, K. E., & Perry, E. (1988). Effectsof therapist response modes in brief psychotherapy. Journal of Counseling Psychology, 35,222–233.

Hill, C. E., & Knox, S. (2002). Self-disclosure. In J. C. Norcross (Ed.), Psychotherapy relation-ships that work: Therapist contributions and responsiveness to patients (pp. 255–265).New York, NY: Oxford University Press.

Hill, C. E., Knox, S., Thompson, B. J., Williams, E. N., & Hess, S. A. (2005, April). Consensualqualitative research: An update. Journal of Counseling Psychology, 52, 196–205.

Hill, C. E., Mahalik, J. R., & Thompson, B. J. (1989). Therapist self-disclosure. Psychotherapy,26, 290–295.

Hoffman-Graff, M. A. (1977). Interviewer use of positive and negative self-disclosure and inter-viewer-subject sex pairing. Journal of Counseling Psychology, 24, 184–190.

Horvath, A. O., & Greenberg, L. S. (1989). The development of the working alliance inventory.In L. S. Greenberg & W. M. Pinsof (Eds.), The psychotherapeutic process: A research hand-book (pp. 529–556). New York, NY: Guilford Press.

Jackson, J. L., Chui, H. T., & Hill, C. E. (2012). The modification of consensual qualitativeresearch for case study research: An introduction to CQR–C. In C. Hill (Ed.), Consensualqualitative research: A practical resource for investigating social science phenomena(pp. 285–303). Washington, DC: American Psychological Association.

Jourard, S. M. (1971). The transparent self. New York, NY: Van Nostrand.Kelley, F. A., Gelso, C. J., Fuertes, J. N., Marmarosh, C. L., & Lanier, S. H. (2010). The real

relationship inventory: Development and psychometric investigation of the client form.Psychotherapy: Theory, Research, Practice, Training, 47, 540–553.

Kim, B. S. K., Hill, C. E., Gelso, C. J., Goates, M. K., Asay, P. A., & Harbin, J. M. (2003).Counselor self-disclosure, East Asian American client adherence to Asian cultural values, andcounseling process. Journal of Counseling Psychology, 50, 324–332.

Knox, S., Hess, S., Petersen, D., & Hill, C. E. (1997). A qualitative analysis of client perceptionsof the effects of helpful therapist self-disclosure in long-term therapy. Journal of CounselingPsychology, 44, 274–283.

Knox, S., & Hill, C. E. (2003). Therapist self-disclosure: Research-based suggestions for practi-tioners. Journal of Clinical Psychology, 59, 529–539.

Kushner, K., Bordin, E. S., & Ryan, E. (1979). Comparison of Strupp’s and Jenkins’s audiovisualpsychotherapy analogues and real psychotherapy interviews. Journal of Consulting and Clini-cal Psychology, 47, 765–767.

Lo Coco, G. G., Gullo, S., Prestano, C., & Gelso, C. J. (2011). Relation of the real relationshipand the working alliance to the outcome of brief psychotherapy. Psychotherapy, 48, 359–367.

Marmarosh, C., Gelso, C., Markin, R., & Majors, R. (2009). The real relationship in psychother-apy: Relationships to adult attachments, working alliance, transference, and therapy outcome.Journal of Counseling Psychology, 56, 337–350.

McCarthy, P. R., & Betz, N. E. (1978). Differential effects of self-disclosing versus self-involvingcounselor statements. Journal of Counseling Psychology, 25, 251–256.

Myers, D., & Hayes, J. A. (2006). Effects of therapist general self-disclosure and countertransfer-ence disclosure on ratings of the therapist and session. Psychotherapy: Theory, Research,Practice, Training, 43, 173–185.

Counselling Psychology Quarterly 51

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6

Page 25: Therapist self-disclosure in psychodynamic psychotherapy: A mixed methods investigation

Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2011). HLM 7.0: Hierarchical linear modeling(student). Lincolnwood, IL: Scientific Software International.

Reynolds, C. L., & Fischer, C. H. (1983). Personal versus professional evaluations of self-disclosing and self-involving counselors. Journal of Counseling Psychology, 30, 451–454.

Yeh, Y., & Hayes, J. A. (2011). How does disclosing countertransference affect perceptions of thetherapist and the session? Psychotherapy, 48, 322–329.

52 K.G. Pinto-Coelho et al.

Dow

nloa

ded

by [

108.

45.4

7.22

2] a

t 06:

06 2

0 M

arch

201

6