This training provides employees with information on basic … · 2009. 3. 4. · This training provides employees with information on basic interviewing skills. The content for this

This training provides employees with information on basic interviewing skills.

The content for this training draws from the work of:Tara Lennon, Division of School Audits, Office of the Auditor General, ArizonaJenny Frank and Scott Sager, Wisconsin Legislative Audit Bureau

1

2

Background/Overview—What are the major policies and procedures of the program? Who are the major actors? What is the scope of the program?

Technical—How does the program or procedure operate? Why is the situation occurring?

Issue Identification Is there something worth evaluating or auditing?Issue Identification—Is there something worth evaluating or auditing?

Problem Identification—What are the specific problems with the program? What are the potential solutions? Who is harmed by the problem?

Follow-Up—Is my understanding of program policies and procedures accurate?Follow Up Is my understanding of program policies and procedures accurate? What would be the costs and benefits of making certain changes based on our recommendations?

3

Testimonial evidence gathered through interviews is generally not, in and of itself, reliable, conclusive, or sufficient.

All categories of evidence contain bias. But interviews can have significant bias -your own, the interviewees, the effects of interpersonal dynamics, and interpretation issues.

Bias makes your evidence less valid and erodes your ability to compare it against other forms of information or examples.

However, we can conduct interviews in a way that controls for some of the inherent bias in the method.

4

The type of interview you conduct exposes you to more or less bias – both your own and the other

persons.

Four types of interviews---on a continuum of potential bias and flexibility

Background and Issue ScopingBackground and Issue Scoping

Informal conversational interview—hallway conversation, very flexible & very biased, not comparable to other interviews or

other sources of information

Interview guide approach—preliminary survey (background)---still biased

Criteria and Condition

Standardized open ended interview Can be confidently compared to other interviews particularly if questions are asked inStandardized open-ended interview—Can be confidently compared to other interviews, particularly if questions are asked in

the same way and in the same order.

Closed, fixed response interview—Survey - less biased when questions are written correctly, very comparable to other

interviews or other sources of information

Each method may be useful during an audit, but not every method is the most appropriate for certain audit questions. While

informal and guided interviews can often be useful for gathering background information for project planning and scoping,

5

they are not appropriate when you seek to obtain direct information that will be used to establish condition and create

findings.

Background questions allow you to determine whether the person is new to the job or a veteran of the program. This can help you determine whether the person is a reliable source of information. For example, people with more experience can often be invaluable in understanding the historical development of an agency or program.

BACKGROUND – what is the background of the person you are interviewing or the history of the program that your are evaluatingKNOWLEDGE – what does the person knowTIMEFRAME – when did something occur, when did the person do somethingAGENCY – what they do and how they do itSENSORY – what do they see or hear, is it consistent with policy or other audit criteriaTHINKING – what they think FEELING – what they feel

6

Create a list of questions that reflect your office’s standard format. Using questions from prior evaluations can be helpful.

7

Question order can has significant effects on the progress and outcome of your i t i ti l l if i ti i I f t finterviews, particularly if you are using surveys or questionnaires. In a face-to-face interview, question order can help create a flow of information that allows the person being interviewed to get comfortable with the topic and situation.

Background questions are generally good non-controversial items. It gets the person talking about themselves and what they do. You can also take this time to establish criteria.estab s c te a

Current condition: frame questions around knowledge, timeframe, and agency. This establishes the control.

Opinion about condition: bring in thinking, feeling, agency, sensory questions. This allows the subject to voice their thoughts and opinions. These will be biased, butallows the subject to voice their thoughts and opinions. These will be biased, but can be compared against their assessment of the current condition.

Past condition: both knowledge, timeframe, agency, and opinion questions

Future condition: opinion questions

8

Ask them if there is anything else they would like to discuss.

Follow up questions should be asked throughout the interview if you do not understand the information that has been provided or when you need additional detail. For example, if someone tells you that a process changed in the past, but does not provide the date ask for the date Good follow up questions asked during

Unintentional primes include “think”, “feel”, or words or statements that increase the likelihood that a subjects answer will contain bias a result of the prime, rather than their own inherent bias. All questions have primes, but you want to control where and when those primes appear.

Dichotomies. Unless the point of the question is to pin down a subject on a certain answer, these questions generally do not provide sufficient information.

Why questions. Why questions can raise defenses? “Why do you do it that way?” Consider “What factors contributed you to make that decisions?”

Implied critiques or judgments: Questions should be designed to elicit information, and not indicate hat e or others think of a program or policand not indicate what we or others think of a program or policy.

9

Writing good questions is not always easy. But they should always be clear and concise, contain single ideas, and have correct grammar.

PresuppositionsIncorrect: What is the most time-consuming aspect of the licensing process?Correct: As we go through the licensing process, could you tell me how long each step takes such as days or hours?step takes, such as days or hours?

Implied DimensionIncorrect: How is it that so many of your employees deny clients for income maintenance?Correct: What are your policies for approving or denying requests for incomeCorrect: What are your policies for approving or denying requests for income maintenance?

SingularityIncorrect: Do you have controls in place to guarantee timely and accurate billing to licenses?

Correct: I want to ask you about your controls but I need to go one piece at a time so----what controls are in place to guarantee timely billing to licensees?…accurate billing?

10

Similar to Class Exercise #1, create a list of questions and examples that best reflect the style of your organization, or use questions from prior evaluations.

11

Priming can be used to help guide an interviewee to think about or respond to questions in certain ways. Some great research has been done on “priming effects,” or the effect that certain words or phrases can have on people’s responses to questions.

For example, questions that ask how one “feels” versus how one “thinks” about an issue elicit different responses. Without getting into the neuroscience and psychology, paying attention to the words you use can help you direct the interviewee in providing the information you want. If you don’t care how they “felt”, don’t use the term.

Priming can also have negative or unintended consequences. Be sure that you are aware of the primes contained in your questions and understand the effect they may y y yhave on responses.

12

Probes help auditors control the completeness and quality of the information they collect. The art of asking good probing questions takes time and mentoring to develop, so ask you colleagues to point out areas where you could have asked probing questions. And, practice makes perfect. Keep doing more interviews.

There are at least three types of probes.

Elaboration on the topic being discussed.

Could you elaborate on the “manager review” of case files?

Clarification of a statement.

When you said that mangers review files, could you clarify which files?

Detail on a particular topic or statement.When did your agency implement that change? How often do perform audits of a sample of payments made to child care providers?

13

If applicable, ask “anything else that I should know but haven’t asked?” This can be a very effect, albeit general, probe.

This reinforces your control over the interview through the use of prompts, such as:-I see-Uh huh-Thank you-That’s interesting. Could you tell me more about…

Prompts can also be transmitted non-verbally.-Stop writing-Stop making eye contact

14

Unless we really concentrate on doing so, most of us do not naturally “listen”.

Listening is not about just being aware that there is someone other than you making noise. Good listening techniques allow you to understand what the speaker is saying and analyze

it within the context of the discussion.

Be engaged-make eye contact as you ask questionsBe engaged make eye contact as you ask questionsShow you are listening – nod, verbal cuesAsk for clarification of unclear statements – restate what you heardPay attention to nonverbal cues – posture, smiling, hands, fidgety, breathingTolerate silence – gather your thoughts, provide a break for you and the

person being interviewed

If you have some expertise in a policy area or have been auditing an agency for several years, do not assume that you know more than the person you are interviewing or know enough to start sharing “niche” knowledge. Just listen.

Suspend judgment of the speaker – listen to the messageDon’t rehearse the next question as you listen

15

Don t rehearse the next question as you listenDon’t assume you know what they are talking about. Ask questions when they use

an acronym you do not know or discuss something you do not understand.

Typically, we do not use tape recorders or bring laptops to record what is said during an interview. Rather, we use old fashioned pen, pencil, and paper. There are many methods of taking notes, and some are more effective than others. Whether you record information in an outline, or create free form diagrams to link thoughts, concepts, and ideas, here are some tips on what should be included in your notes.

Make attributions correctly – while this is not always a critical matter, knowing who said what is important information particularly if the information will be used to develop auditwhat is important information, particularly if the information will be used to develop audit findings and conclusions.

Note follow-up questions and information requests – find a spot on your paper, such as the margins, to jot down follow-up questions and information requests. It ensures that you remember these items, and makes these items easy to identify later.

Shorthand – a short hand style can save you time in writing notes, such as using “w/” or “bc”, for “with” and “because”. This saves time and allows you to focus on recording the main points and prevents words from cluttering the page.

It goes without saying that your notes should be legible. If you cannot read them, then you will lose valuable information and time.

16

Stay within the established time limitsIf you are going longer than you thought, and either ask if they can continue

to schedule a second interview.Look for non-verbal cues that interviewee is tiring or distracted.

Be prepared to offer a wrap-up of the audit process at the end of the interview and answer any additional questions, especially if they are unfamiliar with your organization (non-state government staff new state staff line staff)organization (non state government staff, new state staff, line staff)

Ask if there are follow-up questions“Did we leave out anything important”Ask for additional names to talk withThank the interviewee.

17

Document your interviews as quickly as possible so you remember the discussion. If you cannot do it quickly, try to review notes and add comments and decipher handwriting to aid in write-up sometime later.

When writing the workpaper, make sure to distinguish between facts learned in the interview and the interviewee’s stated personal views.

Contact interviewee by e-mail or phone (as discussed during the interview) for any clarifications or further questions.

If more than one auditor attended the interview, you may want to let the other auditor review the workpaper.

Remember, workpapers will be public documents in most instances, therefore they should be objective and accurate with no extraneous comments about the interviewee or the program. However, don’t “water them down”—you need to present all information that will aid in making conclusions and supporting findings

18

Depending on your audit’s needs, different methods of documenting interviews can highlight some information or subordinate other types of information. For example, if you wish to analyze responses to particular questions, writing workpaper in the Q&A sequence method may be the most beneficial. If you want to get an overall view of a program or policy, the topical sequence may be the best method to identify key theme and salient issues.

19

Group exercise where team members take turns interviewing other team members who take on different personalities.

The Proud – “You Don’t Have the Expertise to Evaluate Us”“You don’t have the expertise to evaluate us” Explain areas in which we do have experienceExplain you are part of a larger teamRemind that you are the liaison between program and LegislatureUse complimentary language to remind interviewee of reason you are there to interview—his/her many

years of experience

The Disengaged – Would Rather be Talking to Anyone but Youg g g yIf you’ve done a good job of preparing, you should be able to be polite, but persistentYou may need to ask for or identify another person to interview

The Quiet One – Responds Non-Verbally or with a “Yes” or “No” to Open-Ended QuestionsWork to put the interviewee at easeUse persistence and your background knowledge to “prime the pump”May need to schedule additional meetings to build rapportExplain why you need interviewee to explain things

The Talker – Talks About Everything. Everything!Try to gently keep interviewee on target

Restate points “Let’s go back to something you mentioned…”“I’m interested in something you said about x…”Switch to closed ended questions

The Whistleblower – “I Have Secret Information”Explain that we do not quote individuals in the report (but be careful to be clear on process—

20

p a t at e do ot quote d dua s t e epo t (but be ca e u to be c ea o p ocessworkpapers will be public record)

If they insist on confidentiality, reschedule and consult with supervisor and director (this strategy may vary depending on the nature of your audit or the particular functions of your office).

Don’t make promises and get over-enthusiastic