Mixing Methodologies: A Sliding Continuum or an … Methodologies: A Sliding Continuum or ... Creswell describes the ... Deduction and induction have been considered a key difference

Mixing Methodologies: A Sliding Continuum or an Iterative Cycle?

Introduction

Mixed methods’ time came a decade ago according to Johnson and Onwuegbuzieare (2004);

however eleven years on, and although mixed methods is still popular in educational research, mixed

methodologies seems to be a far less discussed approach. Should research in a particular field

follow the traditional or favoured methodologies associated with that field, or, if it is desirable for

the empirical methods of research to be mixed, can the same not be said for the theoretical

standpoint of the research design? Does mixing methodologies imply that methodologies can be

placed on a sliding scale to create a new methodology from combining elements of the old; or does

it imply an iterative, cyclical process using a suitable methodology for the stage in the research? This

essay will explore what combining qualitative and quantitative methods actually means in terms of

social and educational research and how this can assist in developing a mixed methodological

approach.

The essay will first present a brief history of qualitative and quantitative approaches in educational

research and review the literature pertaining to why a combination of qualitative and quantitative

methods are believed to enhance research, particularly with regards to triangulation of findings and

providing a fuller picture of the situation. It will draw on relevant literature to distinguish between

methods and methodology and outline the complexities in defining methodology in educational

research, including the misuse of the term by researchers and authors. It will consider the role of

paradigms on methodological choices and their link to the qualitative and quantitative nature of

inductive and deductive reasoning to generating and verifying theory. It will argue that, for the very

reason that combining methods gives a more holistic view of the situation being researched, so does

mixing methodological approaches in terms of inductive and deductive reasoning. It considers a

methodological continuum between the extremes of induction and deduction. Finally it will

propose a new term for combining methodologies: ‘omniduction’, which encompasses both

induction and deduction in an iterative, cyclical methodological approach, which it concludes is a

favourable approach in educational research.

History of Qualitative and Quantitative Approaches in Educational Research

Educational research began in the late nineteenth century (Jarvis, 2005) and focused predominantly

on quantitative methods and experimental research (ibid). Educational research was very much

psychological in the early days which continued into the early twentieth century (ibid). One

researcher who used these quantitative approaches, Thorndike, was thought to be very influential on

educational research, however much of his research was not conducted anywhere near a classroom

(Wellington, 2015). This has naturally been a criticism of early educational research (ibid). How can

a researcher really understand the field if they are not in direct contact with issues they are

researching?

By the middle of the twentieth century, sociology became the main influence on research in

education. Since then, ideas about the nature of educational research and how it should be

conducted has evolved. From the 1970s onwards there has been a rise in research which favours

qualitative approaches (Jarvis, 2005). According to Denzin and Lincoln (2005), qualitative research

is “a situated activity that locates the observer in the world” (p. 3?) which tackles the criticism of the

more distant early educational research. Some more modern critics of educational research however

believe that solely using qualitative methods without triangulation lacks rigour (Oancea, 2005; Jarvis,

2005).

Experimental research is still prevalent in the USA in social and educational research, which is

perhaps the influence of policy makers who look for answers which are generalisable as opposed to

researchers themselves who seek to investigate wider issues in education. For results to be

generalisable, policy makers have come to expect evidence based “causal analysis by means of

experiment as the only way to improve educational research” (Erickson and Gutierrez, 2002, p.22).

Indeed at the start of the 21st century, emphasis from the National Research Council in the USA on

producing scientifically based research caused issues for qualitative researchers (Denzin and Lincoln,

2005).

There has been much debate over the merits of a qualitative approach over the more quantitative,

scientific approach and whether educational research should “mirror or mimic so-called scientific

methods” (Wellington, 2015, p.11). Wellington gives three reasons why it is in fact a nonsensical

debate whether educational research should use the scientific approach: firstly he states that there is

in fact not just one ‘scientific method’ used in the sciences; secondly that not all research in science

is driven by hypothesis, nor is it all experimental research employing control variables; and thirdly

that quantitative and scientific do not have exactly the same meaning, that is rejecting a scientific

method in research does not mean that quantitative approaches are not able to be used (ibid). Since

the last quarter of the twentieth century the notion of mixing qualitative and quantitative methods

has been widely debated , the supporters of which believe that combining methods takes advantage

of “social science’s full methodological repertoire”(Greene, 2005, p.274).

Distinguishing between Methods and Methodology

Defining the term methodology is a necessity if we want to define what it means to mix

methodologies. The Collins Dictionary (n.d.) gives the following definition of methodology:

1. the system of methods and principles used in a particular discipline

2. the branch of philosophy concerned with the science of method and procedure

The definition of methodology above seems straightforward enough, however in educational

research and indeed in academic publications about research methods, the term is often used

interchangeably with ‘methods’ (Opie, 2004). In many research papers and essays, methodology is

simply the title of a chapter or section of the write up where the methods are described. According

to Cohen et al. (2007): “[t]oo often methods are confused with methodology and methodology is

confused with design” (p. 165). Clough and Nutbrown (2007) argue that a “critical design attitude”

should be “at work throughout a study, rather than confined within a brief chapter called

‘Methodology’” (p. 35).

Hartas defines methods as “a strategy for data collection and analysis” and methodology as “the

study of research methods” (ibid, p. 445); however ‘the study of research methods’, if not

ambiguous, is certainly a broad term encompassing a range interpretations. Figure 1 shows a

selection of the range of definitions and distinctions between methods and methodology. Although

the exact definitions of methodology between researchers differ, they all share the common theme

of justification of the empirical (Clough and Nutbrown, 2007).

Researcher/Author Methods Methodology

Hartas (2010) A strategy for data collection and analysis

The study of research methods

Cohen et al. (2007) Range of approaches used to gather data; the techniques and procedures used in the process of data-gathering

Describes approaches to, kinds and paradigms of research

Clough & Nutbrown (2007)

The ingredients of research Reasons for using a particular research recipe.

Opie (2004) Procedures Theory of getting knowledge

Greene (2005) Either integrated or component designs

Theoretical stance and research design

Denzin & Lincoln (2005)

The best means for acquiring knowledge

Figure 1 – Researcher definitions of methods and methodology

The term methods is more agreed upon as the tools used for data collection, for example

questionnaires, interviews or classroom observation. However methodology is more than just

describing the methods used to conduct research, as it incorporates theory which “focuses on the

specific ways, the strategies or methods we use to understand social reality” (Hartas, 2010, p.17).

However just as the term methodology has different interpretations, the term theory means different

things to different people. Robson (2002) cites two conflicting positions with regards to the place

of theory in social research: those who believe that theory is stifling in planning and carrying out

research; and those who believe that without theory the outcome of any research has little value,

even if the research process is quicker. Creswell (2003) describes three levels of inquiry: firstly the

nature or theory of knowledge, known as epistemology and lastly the specific methods used to carry

out the inquiry. Nestled between these, Creswell describes the middle level being the approach to

empirical research, which could be interpreted as how methodology links epistemology and

methods. Perhaps the term ‘methodology’ needs to be broad in definition if the aim of

methodology is to help us “to understand, in the broadest possible terms, not the products of

scientific inquiry but the process itself.” (Cohen et al., 2007, p.47).

The terms qualitative and quantitative are used extensively in research to describe researchers,

paradigms, methodologies, methods and data (Holland and Campbell, 2005). Howe (1992) makes

two distinctions in the way that the terms qualitative and quantitative are used. The first is the literal

use in terms of the type of data collected; the second is the derivative use concerning research

paradigms. Deduction and induction have been considered a key difference between qualitative and

quantitative methodologies (Cohen et al., 2007). Deduction is thus called as it deduces theory from

empirical research, where quantitative approaches are often preferred. Induction, on the other hand

is more commonly employed in qualitative research, where observations come before the theory so

hypotheses may be induced or generated from this approach (Lobe et al, 2007).

Research generally starts with an observation or a theory: either a question to be answered or a

hypothesis to be tested. From here the researcher needs to make decisions about the research

methodology. If it is a hypothesis to be tested then it needs to be researched through deductive

reasoning for theory verification. If the question is more open-ended then inductive reasoning can

be used for theory generation, in order to identify themes in the data to generate new theories. Next

the researcher needs to make decisions about how to collect the information required. Will it be

theoretical or empirical research? If empirical, will the data come from primary or secondary

sources? If primary, will the methodology selected be primarily quantitative or qualitative? These

could all be considered methodological issues, however confusion occurs due to how the term

methodology “is also used to denote the overall approach” (Sikes in Opie, 2004, p.16), for example

case study, action research etc. In an attempt to avoid this confusion, Denzin and Lincoln (2005)

refer to this meaning of the term methodology as research strategies. The following is a list, which is

by no means exhaustive, of these methodologies or research strategies available to the educational

researcher:

action research

case study

ethnography

experimental research

grounded theory

narrative inquiry

secondary data analysis

survey research

To add a further level of ambiguity in the definition of the term methodology, each of these

research strategies may be longitudinal, cross-sectional, historical, correlational or ex post facto or

indeed a combination of two or more of these. Again these could all be considered methodological

issues.

This essay has described two interpretations of the term methodology:

the theory linking epistemology to methods in terms of deductive and inductive reasoning;

the overall approach to a study (e.g. action research, case study, ethnography).

These do not include the definitions which the author believes are misinterpretations of the term

methodology: for example the interchangeability with the term methods and the title of a research

design chapter or section. While the author does not dispute the bulleted definitions of

methodology above, for the sake of this essay, methodology will refer to the theory of generating and

verifying knowledge through inductive and deductive reasoning, rather than the overall approach to

a study. Methods will refer to the data collection techniques employed by the researcher, for example

interviews or questionnaires; research strategies will refer to the overall approach to the research, for

example action research or ethnography; and research design will refer to the chapter or section of

research used to describe the methods, research strategies and methodology.

What’s mixed in Mixed Methods?

Combining methods in research is designed to optimise the strengths and limit the weaknesses of

the research (Lindsey, 2013). Rossman and Wilson (1985) state three main ways this is achieved:

corroboration, elaboration and initiation. Corroboration is the triangulation of data, where different

methods agree with one another giving more confidence in the findings. Elaboration is where one

method can probe deeper into a phenomenon revealed by another method. Initiation is where

consideration of alternative methods can offer a “fresh perspective” (Hartas, 2010, p. 278) on the

research. Robson (2002) describes a benefit of mixing methods as a “reduction of inappropriate

certainty” (p.370) through triangulation, enhancing interpretability and supporting the development

of explanations.

But what does mixed methods mean? Is it one method combining both qualitative and quantitative

data collection; or within one piece of research, using one method associated traditionally as a

qualitative method for investigating emerging themes, for example interviews, and another more

quantitative data collection method for statistical analysis, for example questionnaires? It can in fact

be both of these and indeed a combination of the two. For example an interview which is

traditionally perceived as a qualitative research method can be semi-structured, which can allow

both deductive and inductive reasoning: the structured element seeks theory verification, however

the willingness to allow the interview to be shaped by the views of the interviewee allows for the

generation of new theory. Similarly a questionnaire, traditionally associated with quantitative

inquiry, can be composed of both closed and open questions, the closed questions providing theory

verification through deduction and the open questions allowing theory generation. On the other

hand, mixing methods for some researchers could simply mean using both interviews and

questionnaires in the same study.

In a mixed methods inquiry, the researcher needs to make the decision whether the chosen methods

should be integrated throughout the study, or if they should be presented separately then the

outcomes of the different methods compared, which Greene (2005) refers to as “integrated” and

“component” designs respectively (p. 276). Lindsay (2013) refers to the way methods are mixed as

full or partial mixed methods, and perhaps for this reason he favours the term combined methods.

Denzin (1989) refers to both of these mixed method designs as methodological triangulation, one of

his categorisations of triangulation in social research. The first example is triangulation within

methods, concerned with the ability to replicate the study; the second example is between methods

triangulation, seeking convergence between different methods (Cohen et al., 2007). However this

view is contested by Lincoln and Guba (see Cohen et al., 2007), who raise concern over the notion

of theoretical and methodological triangulation as they believe this notion is incoherent in terms of

epistemology. This raises the question: “when social inquirers mix methods, are they also mixing

philosophical assumptions, and should they?” (Greene, 2005, p.275).

The Role of Positivistic and Interpretive Paradigms

Denzin and Lincoln (2005) define the term paradigm as the “basic set of beliefs that guides action”

(p.19) and describe two main paradigms which have influenced educational research: positivist and

interpretive. The positivist paradigm is scientific and objective as opposed to the interpretive

paradigm which is naturalistic and subjective (ibid). Quantitative methods stem from the positivist

paradigm, the scientific and objective nature of which, imply deductive methodologies setting out to

verify a theory, in pursuit of the universal truth. Qualitative methods, on the other hand, are linked

to the interpretive paradigm, where the subjective nature lends itself to inductive methodologies

(Greene, 2005) and generating theory through research. Opie (2004) however, argues that all

research is essentially interpretive as it can never be “an actual replica of the world” (p.18).

Opie (2004) places positivist and anti-positivist (interpretive) approaches on a continuum (see

Figure 2), implying there is scope for less extreme approaches in the middle of this scale. This raises

the question: does a study need to be wholly experimental or naturalistic?

Figure 2 - Comparison of positivistic and anti-positivistic approaches to educational research (Opie, 2004, p.8)

However in the same publication, Opie (2004) also produced a flow diagram with two distinct

routes to take in educational research depending on the researcher’s view of the nature of

knowledge (see Figure 3). This seems to be in contrast to the idea of a paradigmatic continuum.

Figure 3 – Flow diagram for defining views of knowledge (Opie, 2004, p.13)

Purists believe that different paradigms encompass assumptions which are not compatible with one

another (Smith and Heshusius, 1986). Pragmatists, on the other hand, believe that paradigmatic

positions can be brought together. In the middle of this continuum are situationists who believe

that paradigmatic integrity is important, however they also appreciate that taking a variety of

perspectives on a situation has the potential to enhance the meaningfulness of the research findings

(Hartas, 2010). Hartas states that “paradigmatic purity is not a prerequisite to the completion of a

research project” (ibid, p.278). If it is not necessary to remain rigid to one paradigm, and a mixed

methods approach is deemed the optimal approach in educational research, then would combining

qualitative and quantitative methodologies be a natural consequence?

As a result of the complexities of individuals or communities, a simplistic linear view of

methodology may not deal with their diverse nature. Complexity theory, however, looks for

“multiple causality and multidirectional causes and effects” (Cohen et al., 2007, p.34). Complexity

theory aims to look at phenomena through as many eyes as possible which allows for multiple

perspectives and hence multiple causality to be investigated. According to Cohen et al. (2007)

complexity theory in educational research challenges conventional methodological approaches and

hence “provides an emerging new paradigm for research” (ibid, p.34)

Combining Qualitative and Quantitative Methodologies

Beyond method, what else is mixed in mixed method inquiry? (Greene, 2005, p.275)

As discussed earlier, a mixed methods approach has been popular since the end of the twentieth

century; that is combining both qualitative and quantitative methods to triangulate findings. In fact

this mixed method triangulation may use a combination of both positivist and interpretive

techniques (Cohen et al., 2007). However there seems to be an expectation for research students to

choose which epistemology and methodology to frame their research and stick to only that. But

could this be damaging to the research as any findings are shoe-horned into one chosen ‘ology’?

Maxwell and Loomis (2003) define mixed method inquiry as far more than utilising a combination

of data collection procedures and list five components of a mixed method inquiry: purposes of the

inquiry; the conceptual framework; the research questions; specific methods; and validity. These

five components should influence each other for a holistic mixed method inquiry. Greene (2005)

adds to this definition stating that “flexibility, creativity, resourcefulness – rather than a priori

methodological elegance – are the hallmarks of good mixed-method design” (p.277). Furthermore,

according to Hartas (2010), due to the complex nature of educational research, “multiplicity and

polyphony in the modes of inquiry are required to respond to the different needs of individuals and

groups in society” (p.27)

According to Opie (2004) the “relationship between methodology and methods and knowledge and

truth is controversial” (p.21). The interaction of conceptual and methodological issues in mixed

method research is complex and constantly evolving, but central to the concept, is that the methods

selected should be based on what is appropriate to the research questions or hypotheses posed

(Johnson and Onwuegbuzieare, 2004). This stance is shared by Greene (2005) who states that

methods should be “selected and implemented in the service of inquiry questions rather than vice

versa” (p.277).

The fundamental reason for conducting research is to acquire and communicate knowledge (Opie,

2004). However there are disagreements among researchers for how to obtain that knowledge due

to disagreements in methodology and epistemology (Griffiths, 1998). Many researchers label

themselves as a qualitative researcher or a quantitative researcher. Denzin and Lincoln (2005)

describe five ways in which qualitative and quantitative researchers differ:

1. Uses of positivism and post positivism

Positivism for quantitative researchers and post positivism or interpretive for qualitative

researchers.

2. Acceptance of postmodern sensibilities

i.e. how different methods tell different types of story.

3. Capturing the individual’s point of view

Both types of research want to achieve this, but approach it differently. Qualitative

researchers do so through a detailed, in depth study; quantitative research use statistical

analysis.

4. Examining the constraints of everyday life

Qualitative researchers confront it, while quantitative researchers are more distant due to the

large numbers involved.

5. Securing rich descriptions

Qualitative researchers believe this is valuable, while quantitative researchers want to make

generalisations which the detail can hinder (pp.11-12).

While Denzin and Lincoln (2005) seem to appreciate the value of mixed methods research, they are

primarily qualitative researchers and this is perhaps noticeable in the differences described above as

the quantitative standpoint is portrayed in a more negative light to the qualitative researcher’s view

of research. Denzin and Lincoln (2005) describe qualitative researchers as bricoleurs, from the

assumption that there is lots going on at one time in qualitative research in terms of different voices,

perspectives and points of view. Denzin and Lincoln (2005) view qualitative research as not

favouring a single methodological practice as it does not belong to one theoretical discipline.

However quantitative researchers often regard qualitative research as “informal and lacking rigour”

(Jarvis 2005, p.209). Miles and Huberman (1994) state that the quantitative-qualitative argument is

unproductive and that it is not necessary to “tie the distinction to epistemological preferences”

(p.41). Salomon (1991) argues that it is in fact not an argument of quantitative or qualitative but a

case of whether an analytic or systematic approach is required or preferred.

There are researchers at both extremes on the qualitative and quantitative continuum. On one

extreme lies the quantitative researcher Kerlinger, who believed there was no qualitative data as

everything can be interpreted as simply 1 or 0. At the other extreme is the qualitative researcher

Berg, who believed that all data is qualitative (Miles and Huberman, 1994). Miles and Huberman

state that “numbers and words are both needed if we are to understand the world (1994, p.40). As

self-confessed qualitative researchers they believe that qualitative research requires methods which

are “credible, dependable, and replicable in quantitative terms” (ibid, p.2). Howe (1988) believes

qualitative and quantitative methods are naturally intertwined and cannot be presented in isolation

and Jarvis (2005) believes there is a place for both types of method in educational research. A

quantitative approach can support a qualitative study by supplying background information and

producing a representative sample. Conversely, a qualitative approach can support a quantitative

study through conceptual development (Miles and Huberman, 1994). Nevertheless once a

researcher has chosen to combine methodologies are the qualitative and quantitative approaches

given equal status in the research or is there a main methodology employed? (Greene, 2005)

As discussed earlier, in a good mixed method inquiry the methodologies and methods will be solely

dictated by the research questions or hypothesis. However according to Opie (2004) the most

significant influence on the choice of methods and methodology employed in a study is the

researcher’s ontological and epistemological stance. This stance is often driven by past experience

and dictates the strategies and approaches used by the researcher (Robson, 2002). In reality,

however, education researchers will also be constrained by practicalities, for example access to

schools, funding etc.

Combining Inductive and Deductive Reasoning

Mixing methods lends itself to mixing more than one methodology. Punch (2009) describes two

common mixed-method designs: explanatory, which moves from quantitative to qualitative

methods; and exploratory, which starts with qualitative methods and moves onto quantitative

methods. These fit well with the deductive and inductive extremes to methodological approaches.

Robson (2002) however argues that quantitative research can generate theory and qualitative

research can verify theory. For example, as discussed above, semi structured interviews allow both

deductive and inductive reasoning; questionnaires which mix quantitative and qualitative methods

by combining closed and open questions also allow for both deductive and inductive reasoning.

There is a less discussed, third type of reasoning in research, that of abduction. Abductive reasoning

is more commonly known and used in scientific research where a best-fit method of educated

guesses seeks the most likely explanation for the tested hypothesis. Thagard (1980) represents this

neatly as:

Phenomenon P is puzzling.

Hypothesis H would explain P.

∴ H is plausible, and should be subjected to test (Thagard, 1980, p.188)

The scientific definition of abduction fits with idea of abductive reasoning in educational research,

as it is combining deductive reasoning from testing a hypothesis and inductive reasoning by seeking

the causal explanation for the phenomena investigated. Is abductive reasoning a means of mixing

methodologies? If so, then it implies perhaps that inductive and deductive reasoning are extremes

on a methodological continuum where abduction is positioned as a more liberal centre on the scale

(see Figure 4). However in practice is it helpful to combine the methodologies in this manner or is

it more favourable to simply move from one methodology to another depending what the current

stage of the research demands?

Figure 4 – A possible continuum of methodological approaches to educational research

While positivist and interpretive paradigms fit well on a continuum, perhaps due to their opinion

base, placing the methodological approaches on a continuum seems less convincing. Therefore this

essay proposes another way to describe combining inductive and deductive methodologies:

omniduction. As the prefix implies, omniduction encompasses all the methodological approaches and

employs each when appropriate to the stage of the research. In this sense it is an iterative or cyclic

methodological process (see Figure 5).

Research Paradigms Positivist Post-positivist Anti-positivist/Interpretive

Methodological Approaches

Deduction Abduction Induction

Explanatory Causal Exploratory

Quantitative → Qualitative Qualitative → Quantitative

Figure 5 – Mixing Methodologies through Omniduction

Whether you start with an observation or a hypothesis, research will involve theory which is either

generated or requires verification. Either way, implications should be drawn from the findings

which may lead to further observations required or to new hypotheses or research questions being

formed. Theory informs practice, however theory can also emerge from practice. Therefore simply

choosing a deductive or inductive approach seems inappropriate as the approach is dependent of

the stage of the research, just as the qualitative or quantitative nature of the methods should be

dependent on the research questions and what the research requires rather than a favoured

approach on behalf of the researcher.

Conclusion

A mixed method inquiry seeks to combine the best of what qualitative and quantitative approaches

have to offer, and can add rigour to research. Quantitative methods can enhance qualitative

findings, and vice versa. Combing methods can give a more holistic view of the situation and

maximise validity through the triangulation of research findings. However, it is the research

question and the stage of the research which should dictate these methods, rather than a favoured

approach by the researcher. Similarly whether a researcher considers themselves to follow a

positivist or interpretive paradigm should not be the deciding influence on whether quantitative,

deductive methodologies are employed as opposed to qualitative, inductive methodologies.

Research methodology is inherently iterative or cyclic, as whether theory is generated or verified,

implications will subsequently arise in the form of new questions or theory. Omniduction is a

mixed methodological approach which aims to capitalise on this process as it encompasses both

Observation Hypothesis

Implications

Exploratory

Explanatory

Causal

Causal

inductive and deductive methodologies. With omniduction, the methodology is dictated by the

research, not the researcher.

References

Clough, P., & Nutbrown, C. (2012). A student's guide to methodology, London, Sage. Cohen, L., Manion, L. and Morrison, K. (2007) Research Methods in Education. London, Routledge. Collins Online Dictionary (n.d.) http://www.collinsdictionary.com/dictionary/english/methodology Creswell, J. W. (2003) Research design: Qualitative, quantitative, and mixed methods approaches (second edition), Thousand Oaks, CA, Sage. Denzin, N. and Lincoln, Y. (Eds) (2005) Handbook of Qualitative Research (third edition), London, Sage. Denzin, N. K. (1989) The Research Act (third edition), Englewood Cliffs, NJ, Prentice Hall.

Erickson, F., & Gutierrez, K. (2002) Comment: Culture, rigor, and science in educational research. Educational Researcher, Vol 31, No. 8, pp. 21-24. Greene, J. (2005) ‘Combining Qualitative and Quantitative Methods in Social Inquiry, Key Concepts’ in Somekh, B. & Lewin, C. Research Methods in the Social Sciences, Sage, pp. 274-277. Griffiths, M. (1998) Educational research for social justice: Getting off the fence, McGraw-Hill International. Hartas, D. E. (2010) Educational Research and Inquiry Qualitative and Quantitative Approaches. London: Continuum. Campbell, J., & Holland, J. (2005) Methods in development research: combining qualitative and quantitative approaches, Rugby, ITDG Publishing. Howe, K. R. (1992) ‘Getting over the quantitative-qualitative debate’, American Journal of Education, pp. 236-256. Howe, K. R. (1988) ‘Against the quantitative-qualitative incompatibility thesis or dogmas die hard’, Educational researcher, Vol. 17, No. 8, pp. 10-16. Jarvis, M. (2005). The psychology of effective learning and teaching. Cheltenham, Nelson Thornes. Johnson, R. B., & Onwuegbuzie, A. J. (2004) ‘Mixed methods research: A research paradigm whose time has come’, Educational Researcher, Vol. 33, No. 7, pp.14-26.

Lindsay, G. (2013) ‘The benefits of combined (mixed) methods research’, Social Work and Social Sciences Review, Vol. 16, No. 2, pp. 76-87. Lobe, B., Livingstone, S. & Haddon, L. (2007). ‘Researching children's experiences online across countries: issues and problems in methodology’. Report from the EU Kids Online network, EU Kids Online Deliverable D4.1 Methodological Issues Review, EC Safer Internet plus Programme. The London School of Economics and Political Science. Avaliable at: https://lirias.kuleuven.be/bitstream/123456789/200390/1/ReportD4.1MethodologicalIssuesCover.pdf [Accessed 30 March 2015]

Maxwell, J. A. & Loomis, D. M. (2003) ‘Mixed methods design: An alternative approach’, Handbook of mixed methods in social and behavioral research, 1, pp. 241-272.

Miles, M. B. & Huberman, A. M. (1994) Qualitative Data Analysis: An Expanded Source Book, Second Edition, Thousand Oaks, Sage. Oancea, A. (2005) ‘Criticisms of educational research: key topics and levels of analysis’, British educational research journal, Vol. 31, No. 2, pp. 157-183. Opie, C. (2004) Doing educational research, London, Sage. Punch, K. (2009) Introduction to Research Methods in Education. London: Sage. Robson, C. (2002) Real World Research: A Resource for Social Scientists and Practitioner-Researchers, 2nd edition, London, Blackwell.

Rossman, G. B. & Wilson, B. L. (1985) ‘Numbers and words: Combining quantitative and qualitative methods in a single large-scale evaluation study’, Evaluation Review, 9, pp. 627-643.

Salomon, G. (1991) ‘Transcending the qualitative-quantitative debate: The analytic and systemic approaches to educational research’, Educational Researcher, Vol. 20, No. 6, pp. 10-18.

Silverman, D. (2004) Qualitative research: Theory, Method and Practice (second edition), London, Sage. Smith, J. K. & Heshusius, L. (1986) ‘Closing down the conversation: The end of the quantitative-qualitative debate among educational inquirers’, Educational researcher, Vol. 15, No. 1, pp. 4-12. Thagard, P. (1980) ‘Against evolutionary epistemology’, in PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association (pp. 187-196), Philosophy of Science Association. Wellington, J. (2015). Educational research: Contemporary issues and practical approaches (second edition), London, Bloomsbury Publishing.