What is MIS?. Two Specific Questions How can MIS be identified within academia? What differentiates high and low quality MIS research?

What is MIS?

Two Specific Questions

• How can MIS be identified within academia?• What differentiates high and low quality MIS

research?

Method

• Determine fields related to MIS (Katerattankul, Han, & Rea, 2006)

• Gather article attributes from the Top 6-9 journals in each of these related fields and MIS

DisciplinesMIS

EducationAccounting

Computer ScienceEconomicsSociology

PsychologyLibrary Science

HealthcareCommunication

ManagementMarketing

Electrical Engineering

Data

• Scrape ISIknowledge.com• 102,388 articles• Attributes analyzed included

– Title– Publication– Abstract– Keywords– Citations per Year– References to other articles– Many more

Coded Articles

• 50 citation classics were randomly chosen from the MIS articles

• Matched with 50 non-citation classics on journal and publication year

• Coded each of these 100 articles in groups of 3 after a training session and 2 trials

• Attributes coded– Theoretical contribution– Type of article (Empirical, Theoretical, Review,

Methodological)– Type of study

How can MIS be identified within academia?

Abstract Analysis

Jaebong and John

9

Analysis of Research Paper Abstracts

• Determine disciplines similar to MIS– Comparative definition of MIS discipline

• 13 Disciplines– MIS, Accounting, Communication, …

• Variables– 3 Numeric variables

• No. of authors• No. of pages (end page – start page = no. of pages)• No. of total citations (received to date)

– 817 Text variables - nouns and noun phrases• Extracted from abstracts

10

Descriptive Statistics

• 13 Disciplines; 38,642 Papers

DisciplineNo of

Papers1 Accounting 8152 Communication 6693 Computer Science 1,2254 Economics 5,2865 Education 2,2646 Electrical Engineering 4,2467 Healthcare 1,8768 Library Science 3,6969 Management 3,510

10 Marketing 3,79211 MIS 5,78812 Psychology 3,28213 Sociology 2,193

Total 38,642

11

Framework for Analysis

MIS Mgmt Psychology Computer Science…

Global Vocabulary (817 distinct terms)

Bag-of-Words for Each Paper

Cluster Analysis

Extract nouns and noun phrases by term frequency (TF) for each discipline

Extract most frequent 150 terms from each disciplineResult: 817 distinct terms

Build a bag-of-words model for each paper

Apply cluster analysis to bag-of-words from papers

12

5 Naturally Formed Clusters

Total # of papers: 38,642

• No. of papers / cluster

IS for Deci-sion

Support; 5524

Or-ga-nizati

onal

Be-havior, 6196Ele

c Eng & Heal-thcare,9270

Econ & Acct; 9555

What MISis NOT,8097

13

1 Info Systems for Decision Support

10 Top Keywords from Abstracts

Decision Support System (DSS)

Information

System

Software

Organization

Database

Web

Collaboration

Knowledge

Information retrieval

● Core: Library Science● Communication-based● Not psychology

14

2 Organizational Behavior

10 Top Keywords from Abstracts

Transformational leadership

Leader-member exchange (LMX)

Relational uncertainty

Organizational citizenship behavior

(OCB)

Organizational commitment

Leadership

Satisfaction

Culture

Meta-analysis

Social movement

● Human side● Sociology in business school● Collaborative

15

3 Electrical Engineering & Healthcare

10 Top Keywords from Abstracts

Inverter

Induction motor

Sensor

Topology

Mobile robot

Neural network

Architecture

System

Support vector machine (SVM)

Genetic algorithm (GA)

● Technical side● Data-driven● Not human

16

4 Economics & Accounting

10 Top Keywords from Abstracts

Earnings announcement

Financial Accounting Standard Board (FASB)

Sarbanes-Oxley Act (SOX)

Audit fee

Equilibrium

Valuation

Private information

Bidder

Earnings forecast

Incentive

● Econ & Acct very similar● No psychology● Numbers-based

17

5 What MIS is NOT

10 Top Keywords from Abstracts

Somatic symptom

Body mass index

Bipolar disorder

Anxiety disorder

(Major) depression

(Psychiatric, Mental) disorder

Physical activity

Medication

Blood pressure

Competitive intelligence (CI)

● Outside business school● Stress related● MIS does not research

Percent of Each Discipline in Clusters

MIS in Clusters

Keyword Analysis

John and Yu-Kai

Keyword Analysis in a Nutshell

• Questions to be asked and addressed:– How to represent a discipline?

• Vector Space Model

– Based on the representation, how to compare the relations/similarities among different disciplines?

• Cosine Similarity

– How’s the relations/similarities between MIS and the other disciplines evolve over time?

Vector Space Model

= <w11, w12, … , w1x>

= < w21, w22, … , w2x >

MISComputer Science

MIS

Computer Science

K1 KxK2

w11 w12 w1x

w21 w22 w2x

Dn

D1D1D2

D1

K1, K2, …, Km

Dn

D1D1D2

D1

K1, K2, …, Km

…

…

…

Cosine Similarity

∂1

∂2

v

v1

v2

Illustration of cosine similarity

Similarity of MIS with the other Areas(measurement unit: each year)

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100

0.1

0.2

0.3

0.4

0.5

accounting communication computer science economics educationelectrical engineering medical informatics management marketing psychologysociology

Sim

ilarit

y

Similarity of MIS with the other Areas(measurement unit: every two years)

1991-1992

1993-1994

1995-1996

1997-1998

1999-2000

2001-2002

2003-2004

2005-2006

2007-2008

2009-2010

0.0

0.1

0.2

0.3

0.4

0.5

accounting communication computer science economics educationelectrical engineering healthcare management marketing psychologysociology

computer science

marketing

management

healthcare

sociologyeducationpsychology

electronical engineeringaccounting

economics

Sim

ilarit

y

Reference Analysis

Justin G., Devi, Shan

Interaction of MIS vs others

• Indicators:– MIS Contribution (CMIS)

– MIS Consumption (MISC)

Contribution to MIS

Who are buying ideas?

CMIS

MIS Contribution

1971~1975 1976~1980 1981~1985 1986~1990 1991~1995 1996~2000 2001~2005 2006~20100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Accounting

Communication

Computer Science

Economics

Education

Electrical Engineering

Healthcare

Library Science

Management

Marketing

Psychology

Sociology

Management

Marketing

MIS

Con

trib

ution

MIS Consumption

1971~1975 1976~1980 1981~1985 1986~1990 1991~1995 1996~2000 2001~2005 2006~20100

0.2

0.4

0.6

0.8

1

1.2

Accounting

Communication

Computer Science

Economics

Education

Electrical Engineering

Healthcare

Library Science

Management

Marketing

Psychology

Sociology

Healthcare

Marketing

Library sci-ence

Electrical Engineering

Education

MIS

Con

sum

ption

MIS Consumption

1986~1990 1991~1995 1996~2000 2001~2005 2006~20100

0.1

0.2

0.3

0.4

0.5

0.6

Accounting

Communication

Computer Science

Economics

Education

Electrical Engineering

Healthcare

Library Science

Management

Marketing

Psychology

Sociology

Marketing

MIS

Con

sum

ption

Healthcare

Education

Library science

Citation Analysis using Google Motion Charts

1970 - 2009

Number of citations received by a discipline

1970 - 2009

Number of references given by a discipline

1970 - 2009

Number of self citations by a discipline

1970 - 2009

Number of citations receivedVs

Number of references given

1970 - 2009

Market share of total citations received by a discipline

1970 - 2009

Market share of total references given by a discipline

1970 - 2009

What differentiates high quality and low quality articles in MIS?

Dan, Julian, and Justin W.

Overview

• Identify factors that determine high quality MIS articles

• “High quality” = 100 or more citations• Logistic regression models• Dependent variable is binary variable called “quality”

– 1 = high quality– 0 = not high quality

Analysis

• Analysis used 6 models– 2 “standard” models

• 5 or 6 explicit variables from ISI data set

– 4 “conceptual phrase” models• Numerous phrases derived from article title, author

keywords and ISI keywords generated by text mining

Two “Standard” Models

“Standard” model• Years since publication• Number of references• Number of authors• Number of pages• Type of document

“Standard” + name model• Years since publication• Number of references• Number of authors• Number of pages• Type of document• Name of journal*

* Name of journal suspected of dominating “standard” model

Four “Conceptual Phrase ” Models

Steps to find new possible “conceptual phrase” variables 1. Text-mine fields for most frequently used terms in

– Article titles– Author keywords– ISI keywords

2. Group terms into conceptual phrases3. Add conceptual phrases to “standard” models

– “standard” + title– “standard” + author keywords– “standard” + ISI keywords– “standard” + title + author keywords + ISI keywords

Compare Model Performance

Type Model Performance (R2)

Standard Standard 0.135

Standard Standard + Journal Name 0.304

Conceptual Phrase Standard + Article Title 0.302

Conceptual Phrase Standard + Author Keyword 0.232

Conceptual Phrase Standard + ISI Keyword 0.247

Conceptual Phrase Standard + Title + Author + ISI 0.479

Key Factors in “High Quality”

Factors Evidence

Number of pages Coefficient = 0.053

Number of references Coefficient = 0.023

Age of paper Coefficient = 0.052

Keywords (see table)

Name of journal (see table)

Factor: KeywordsResearch Contributions• study• Investigation• research

Theoretical Background • theory • perspective • building

Research Domains• computing• commerce

Title Author Keyword ISI Keywordcomputing cost economics

building online researchinfluence computing userssuccess quality computer

commerce user theoryresearch research strategytheory technology information

technology mediainvestigation perspective

study

Title Author Keyword ISI Keywordsupport decision design

information systems impact

Positive Factors

Negative Factors

Factor: Name of JournalJournal Citations

MeanCitationsMedian

Citations Mode

MIS Quarterly 60.31 31 0

Information Systems Research 49.11 30 12

Journal of Management Information Systems 24.48 13 0, 13

Information & Management 14.31 8 0

Journal of Strategic Information Systems 12.86 7 0

European Journal of Information Systems 12.55 8 0

Decision Support Systems 12.17 7 2

Information Systems 8.68 3 0

*626 articles published in MISQ received a total of 37,754 citations. The top 10 most cited MISQ articles received more than 20% of the total citations of MISQ articles.

10 Most Cited Articles – MISQ

Article Name Citations Received

Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology 2298

User acceptance of information technology: Toward a unified view 897

Review: Knowledge management and knowledge management systems: Conceptual foundations and research 693

Computer Self-Efficacy – Development of a Measure and Initial Test 499Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology – A Replication 454

Trust and TAM in online shopping: An integrated model 448

The Measurement of End-User Computing Satisfaction 413A set of principles for conducting and evaluating interpretive field studies in information systems 404

Task-Technology Fit and Individual-Performance 398

The Case Research Strategy in Studies of Information Systems 395

Article Name Journal Citations Received

Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology MISQ 2298User acceptance of information technology: Toward a unified view MISQ 897Understanding Information Technology Usage - A Test of Competing Models ISR 760Review: Knowledge management and knowledge management systems: Conceptual foundations and research MISQ 693Computer Self-Efficacy – Development of a Measure and Initial Test MISQ 499The DeLone and McLean model of Information Systems Success: a Ten-Year Update JMIS 463Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology – A Replication MISQ 454

Trust and TAM in online shopping: An Integrated Model MISQ 448

The Measurement of End-User Computing Satisfaction MISQ 413

A set of principles for conducting and evaluating interpretive field studies in information systems MISQ 404

10 Most Cited Articles – All Journals

High-Quality Articles

Determinants of Highly-Cited MIS Papers

Jeff Proudfoot & Ryan Schuetzler

Analyses

• Logistic Regression• Matched Pair Logistic

Regression

Best Model

Theory Comparison

Matched Pair Logistic Regression

• Articles paired on year and journal• One highly cited (>=100 citations)• One non-highly cited (<100 citations)• Analyze variables to determine what is

significant in predicting the highly cited paper

Most Significant Main Effect

coef exp(coef) se(coef) z pTheoryTesting 0.512 1.67 0.339 1.51 0.13

Theory Testing > Theory Building?

• Technology evolves, so a lot of work is done in applying old theories to new technological paradigms

• Many theories used in IS are borrowed from other fields, so building is not as prevalent as testing it in the IS domain

Conditional Logistic Regression

  coef exp(coef) se(coef) z pTheory Testing -1.924 0.1460 1.26 -1.53 0.130Theory Building -2.380 0.0926 1.33 -1.79 0.074Testing*Building 0.944 2.5706 0.47 2.01 0.044

Possible Reasons

• Neither Theory Testing nor Theory Building is enough to constitute a valuable paper alone.

• A strong combination of Testing and Building produce the most valuable works

Matched Pairs Logistic Regression Management Empirical Articles

coef exp(coef) se(coef) zBuild New Theory*** 1.153236 3.168429 0.107841 10.694Test Existing Theory*** 0.660693 1.936134 0.104642 6.314

Refs*** 0.016899 1.017043 0.004754 3.555

Reading Comp* 0.197758 1.218667 0.077736 2.544

Validity*** 1.133152 3.105429 0.165915 6.83

Rsquare= 0.277 (max possible= 0.454 )Likelihood ratio test= 492.1 on 6 df, p=0Wald test = 173.1 on 6 df, p=0Score (logrank) test = 349.6 on 6 df, p=0

Non-Empirical Articlescoef exp(coef) se(coef) z Pr(>|z|)

Theoretical Contribution*** 0.289556 1.335835 0.05806 4.987 6.13E-07

Refs** 0.010592 1.010648 0.003714 2.852 0.00435

Reading Comp* 0.243718 1.275984 0.110311 2.209 0.02715

Rsquare= 0.128 (max possible= 0.39 )Likelihood ratio test= 63.33 on 4 df, p=5.783e-13Wald test = 36.87 on 4 df, p=1.920e-07Score (logrank) test = 51.62 on 4 df, p=1.656e-10

Conclusions

• MIS seems to be a multidisciplinary and maturing field

• There seem to be at least two identifiable areas within MIS – Behavioral and Technical

• Over time MIS seems to be becoming more behavioral and less technical

• Although theory testing has been more important than theory building in the past, as the discipline matures it is likely that theory building will emerge as the dominant paradigm for research in MIS