How Does Data-Driven Decision-Making Affect Firm ...misrc.umn.edu/wise/papers/1a-1.pdf · How Does Data-Driven Decision-Making Affect Firm Productivity and CEO Pay? ... We seek to

How Does Data-Driven Decision-Making

Affect Firm Productivity and CEO Pay?

Erik Brynjolfsson, MIT Sloan

Lorin Hitt, Wharton, School, University of Pennsylvania

Heekyung Kim, MIT Sloan

September 15, 2010

Preliminary and Incomplete Draft

For consideration at WISE 2010

Comments welcome. Please do not quote.

Abstract

While there is a great deal of anecdotal evidence that firms can boost their performance by

adopting a data driven decision-making (DDD) approach, there is little data or systematic

analysis of these claims themselves. We gather detailed information on the business practices

and information technology investments of 165 large publicly traded firms and find that DDD

can explain a 3-5% increase in their output and productivity, beyond what can be explained by

traditional inputs and IT usage. Furthermore, firms with more DDD tend to have a higher degree

of consistency in business practices across business units and geography and stronger linkages

between business and technology. In addition, we find that DDD is correlated with a significant

increase in CEO pay even after controlling for the average worker’s wage, suggesting that the

data-driven decision-making may further widen the gap between top managers and average

workers. Intriguingly, the quadratic effect of DDD is positive for CEO compensation, but not for

firm productivity, echoing Rosen’s (1982) model of increasing rewards to “superstars”.

1. Introduction

Each revolution in science is typically preceded by a revolution in measurement and data

gathering. The invention of microscope, for example, allowed the exploration of an entire new

world that had been invisible, transforming biology and medicine. Today’s data revolution

promises to transform management in an equally fundamental way. Businesses now can

measure consumer behavior, business operations and myriad other aspects of their environment

with unprecedented precision. For instance, clickstream data provides an unprecedented digital

trace of customer actions and ERP systems generate terabytes of data as byproducts of their

normal activities, while email, RFID, mobile phones, and many other information technologies

are all contributing to a doubling of the quantity of business data roughly every 1.2 years.

Data by itself does not change anything. It must be analyzed and used to change decisions to

have any value. The ability to collect and analyze the enormous amount of data is, therefore,

becoming of critical importance for firms to stay competitive and gain market share. Ironically,

although there is a great deal of anecdotal evidence of firms’ using data to gain a competitive

edge in the business press in popular books, there has been virtually no systematic data analysis

of the productivity effects of data-driven decision making using statistical methods. We seek to

address this gap by examining detailed business practices and information technology investment

of publicly traded large 165 firms in the US. We find that approximately 3-5% increase in the

productivity can be explained by data-driven decision-making (DDD) and that the top executives

in data driven firms, particular CEOs, have reaped disproportionately large rewards in terms of

personal compensation. Our results can help explain not only the relatively higher productivity

growth rates that IT-intensive firms have recently experienced, but also the growing gap between

CEO pay and the compensation of ordinary workers.

As information technology has penetrated every step of business practices, business decisions are

increasingly relying on data and experiments. Consumer-facing companies rich in transaction

data are routinely testing innovations and randomized testing is standard procedure in online

firms such as Amazon, eBay, and Google. For example, Amazon.com continually experiments

with their websites for the best appearance. “We have been implementing changes on <online

site> based on opinion, gut feeling or perceived belief. It was clear that this was no way to run a

successful business…Now we can release modifications to the page based on purely on

statistical data” (Kohavi, Longbotham et al., 2009). Kohavi et al. (2009) illustrates how data can

overrule the traditional approach of relying on intuition and “the highest paid person’s opinion”

(HiPPO) for corporate decision-making. “Greg Linden at Amazon created a prototype to show

personalized recommendations based on items in the shopping cart. Linden notes that while the

prototype looked promising, a marketing senior vice-president was dead set against it, claiming it

will distract people from checking out. Greg was forbidden to work on this any further.

Nonetheless, Greg ran a controlled experiment” to prove that the new feature would bring in a lot

more sales at a statistically significant level and the HiPPO was wrong.

This suggests that information technology, once thought of a major driver in widening income

gap between college and high school graduates by replacing workers of routine and simple

clerical jobs (i.e., Autor, Katz et al., 1998) is now poised to replace HiPPOs. On the contrary,

however, we have been witnessing that a further widening income gap in every sector and every

income percentile of the population in recent years (i.e., Piketty and Saez, 2003). In this study

we report that the data-driven decision-making is correlated with the increase in CEO’s pay even

after controlling for the average worker’s wage, suggesting that DDD is further widening the

income gap between the top manager and other workers. Our interpretation of this finding is that

information technology is eroding the high-level managers’ work as it has replaced the workers

of routine jobs (Autor, Katz et al., 1998), shrunken the ranks of middle managers (Pinsonneault

and Kraemer, 1997), and flattened hierarchies (Rajan and Wulf, 2006). At the same time,

however, it is complementing the CEO’s job. A CEO’s vision for the future and steering her

firm to the future cannot be experimented and replaced by data. Rather, talented CEOs are even

more in demand to make sound decisions based on the ever-increasing amount of information, to

shape her firm to be data-driven and encourage her employees to experiment and collect and

analyze data, to exploit the capability of information technology to gain a competitive advantage.

2. Theory

The starting point for economic theories of the value of information typically begins with the

seminal work of Blackwell (1953). Blackwell’s theorem (Blackwell, 1953) states that decision

makers observe signals correlated to the state of nature prior to their choice of action and hence

“update” the probability distribution before the “optimal” action is chosen. In this framework

the value of information (in terms of expected utility) is always positive. Since the advent of

computers in the 1980s, firms have deployed enormous resources to computerize work and

collect information. Many researchers, however, showed that only organizations with certain

characteristics such as high endowments of human capital and decentralized work practices tend

to reap the benefit of their information technology investment (i.e., Bresnahan, Brynjolfsson et

al., 2002). Our study is related to the stream of literature, uncovering another organizational

characteristic that may be associated with this increase in productivity. Controlling for firms’

human capital and information technology investments, we show that data-driven decision-

making practices (DDD) significantly affect firm productivity, and are also highly correlated

with the consistency of business practices across business units.

The increase in the amount of available information may not be always valuable when acquiring

and processing the information is too costly. “The real issue is not the lack of data, but our

ability to properly process and distribute the information quickly. If we cannot properly process

and distribute the information quickly, we run the risk of destroying the value of the information

we are striving so hard to attain.” (Collins, 2010). CIOs have expressed concerns that after

investing in data collection, they lack the capability and personnel to analyze these data.1 Our

study explores the question by empirically examining whether there is an optimal level of DDD.

While the answer to the question is inconclusive due to our limited sample size, a striking result

emerged – the reward to CEOs shows increasing returns to investments in DDD. For instance,

we find that firms at two standard deviations above the mean of our DDD scale have CEO

compensation four times larger than the average firm.

CEO compensation might be related to DDD because it affects the ability of organizations to

transfer and process knowledge. In particular, information technology makes some types of

information alienable from workers or mid-level managers (see e.g. Brynjolfsson, 1993) but not

necessarily the most senior managers. This increases the marginal value of decisions made by

CEOs.

1 CIO forum, MIT center for digital business, 2010.

Numerous researchers have noted that some information is not easily transferable because it

“tacit” (i.e., Polanyi, 1958; Rosenberg, 1982) or “specific” (i.e., Jensen and Meckling, 1976) or

“sticky”(i.e., Von Hippel, 1994). Advancement in information technology has, however, brought

in a wave of codification of knowledge. Balconi, 2002) presented an illustrative example of the

changes in the progressive codification of technological knowledge occurred over the last 30

years formerly embodied in skilled workers in the steel industry. Until the end of the 1960s

instruments to measure the temperature and chemical composition of liquid steel were not widely

available so measurements were carried out based on some physical characteristics observable by

sight. For example, “in order to know the temperature of liquid steel, a sample was taken out of

the furnace, poured upon an iron plate and the temperature was deduced by observing the

forming of the spot, its shape and the way it solidified and attached itself to the plate.” The

ability to recognize the temperature by sight was acquired with 5 years of experience or more.

By the 1970s the content and the temperature of liquid steel was routinely conducted by

automated measurement tools but the process was still controlled by workers and line managers.

By the end of 1980s the whole processing cycle was automated. Since the mid-90s, workers are

principally involved in monitoring automated data collection and processing equipment, and

approving process changes recommended by the processing algorithms. Over time, the

knowledge once embodied in experienced workers has been supplanted by machine monitoring

and largely automated decision making. A consequence is that the specialized knowledge only

possessed by the most skilled individuals can now be embedded in computer instruments and

available to distant operators in the operating rooms throughout the entire company. More

broadly, firms now gather and propogate knowledge not only from production workers, but from

their consumers, suppliers, alliance partners, and competitors much faster and more cheaply with

the aid of information technology, make information available for distant decision makers.

Knowledge generation does not stop at passive analysis of data. New and broadly available

software has enabled managers to conduct active experiments with their new business ideas and

base their decisions on scientifically valid data. From banks such as PNC, Toronto-Dominion,

and Wells Fargo to retailers such as CKE Restaurants, Famous Footwear, Food Lion, Sears, and

Subway to online firms such as Amazon, eBay, and Google, firms test many business ideas

through a randomized test before launch, called as “information-based strategy” (Davenport,

2009). This information-based strategy alienates the high-level manager’s tacit knowledge – the

knowledge of what was or was not likely to be a successful business innovation. Just as the

process of controlling steel processes was transformed by improved measurement of temperature,

the innovative process in an online business is now being transformed by the improved

measurement of customer preferences. However, this process automation cannot substitute for

strategic decision making, although it does allow strategic decisions to be moved from managers

at the level of a task or a business unit, to a manager with responsibility for the entire firm.

Consequently, a small difference in the talent of top manager who has an enormous amount of

information to process may make a huge difference to the fortune of the firm.

Our study is also related to the effects of IT on command, control, coordination, and organization

of firms (see Leavitt and Whisler, 1958;Rule and Attewell, 1989;Gurbaxani and Whang,

1991;Malone, Yates et al., 1987;Brynjolfsson and Mendelson, 1993;Brynjolfsson, Malone et al.,

1994;Brynjolfsson and Hitt, 2000 and the literature reviews cited therein). In theory, IT could

shift power either toward the center or away from it, leading to centralization or decentralization

of a firm. If IT can transfer knowledge to central managers, a firm can become more centralized

which would suggest higher CEO relative pay. On the other hand, if IT facilitates lateral

communication of local knowledge between employees, employees can coordinate tasks among

themselves more easily with the need for senior management involvement, suggesting lower

CEO relative pay. Researchers have found evidence of decentralization at task level, but the

increase in the scope of CEO as well (Rajan and Wulf, 2006). Our finding is consistent with the

increase in the scope of CEO, but not as a substitute for decentralized work practices. Data

driven firms generally invest in both decentralized work practices, but with greater overall

centralized coordination of these activities, making them consistent across the firm.

Our study is also related to the literature on IT and wage inequality (i.e., Autor, Katz et al., 1998).

Our view on the role of IT in increasing CEO pay is close to those of Garicano (Garicano and

Rossi-Hansberg, 2006;Garicano, 2000). Their model suggests a hierarchy of knowledge. Some

knowledge resides in lower level employees and is used to solve routine problems, while other

knowledge resides in higher-level employees that is primary directed at solving non-routine

problems. As the cost of communication among agents decreases, the complicated problems that

lower-ranked employees cannot solve can be easily passed to their superiors. Once solved, the

solution can be disseminated throughout the firm and become part of routine work practice. This

can lead to the dependency of the problem-solving on a few “superstars” (Rosen, 1981) and thus

a higher wage for the superstars. Their explanation is consistent with our view that the

availability of data allows a distant few problem solvers to provide a solution to non-trivial

problems and increase their marginal productivity disproportionately. Therefore, data driven

firms become dependent on a small number of superstars which should be associated with

greater pay. CEOs may or may not be one of these problem solvers, but they are nonetheless

responsible for coordinating, managing, and rewarding their activities, which also increases the

marginal value of their effort.

Finally, our study is related to the literature on CEO compensation, especially studies that

document or explain the observed increase in CEO pay. Following Gabaix and Landier’s (2008)

summary of the literature on the topic, we recapitulate the literature in four views. The agency

view (Jensen and Murphy, 1990;Dow and Raposo, 2005;Holmstrom and Kaplan, 2001) suggests

that increased expected compensation is necessary to provide incentives in an increasingly

volatile business environment and to reward difficult to measure activities like innovation. The

market failure view (Yermack, 1997;Bertrand and Mullainathan, 2001;Bebchuk and Fried,

2005;Bebchuk, Fried et al., 2002;Hall and Murphy, 2003) suggests that the rise in CEO pay is a

consequence of failure of corporate governance mechanisms. A third possibility is that the CEO

job has changed (Frydman, 2005) to become more general which has increased their outside

options and put upward pressure on pay. Finally, the market equilibrium view (Gabaix and

Landier, 2008;Tervio, 2008) suggests that CEO pay increases simply reflect an increase in firms

size. In equilibrium, the most talented CEO is correctly matched to the largest firm and paid the

highest because the marginal value of their decisions is greatest there.

Our paper is most closely linked to the market equilibrium view; the increased marginal

productivity of CEOs resulted from the increase of firm size is a major determinant of the recent

rise of CEO pay. We articulate that what is relevant to the marginal productivity of CEO is an

effective size, of which data-driven firm is greater than others of the same nominal size. The

more data-driven a firm is, the easier the flow and access of knowledge and thus the larger the

effective size of the firm that distant managers can control. As a result, the marginal productivity

of CEO increases and this drives an increase in CEO pay.

Our argument is related to the study by Baker and Hall (Baker and Hall, 2004), who attributed to

a specific mechanism of increased marginal productivity. In particular, they argue that when

CEOs perform tasks that affect the entire firm, optimal compensation should be connected to the

proportional change in value rather than the absolute change in value. As firms become larger,

this implies larger CEO pay. Since modern investments in DDD tend to lead to firm wide

changes or standardization of processes across the firm, the Baker and Hall argument would

suggest that DDD leads to greater CEO compensation.

One challenge to investigating these theories is that different firms, in different vary in the

marginal productivity and therefore wages of all workers (Katz and Summers, 1989;Krueger and

Summers, 1987;Krueger and Summers, 1988;Gibbons and Katz, 1992). For example, the

median workers’ wage in the industry of computer systems and related services, $65,000, was

over 3-times higher than that in the industry of food services, $18,000, in 2005. This variance is

at least partly due to the fact that DDD can raise the productivity of all workers. However, our

theory predicts not just increases in wages, but disproportionate increases in the wages of top

executives. Therefore, to isolate this effect, we will control for wages of other workers in our

analysis.

Data

Our business practice and information system measures are estimated from a survey

administered to senior human resource (HR) managers and chief information officers (CIO) from

large publicly traded firms in 2008. We received matched responses (both HR and CIO) from

127 firms, HR only responses from 122 firms, and only CIOs-only responses from 81 firms. The

survey asks over 80 questions on business practices as well as organization and usage of

information systems. The questions extend a previous wave of surveys on IT usage and

workplace organization administered in 1995-1996 and 2001 (Hitt and Brynjolfsson, 1997

Tambe, Hitt et al., ), but adds additional questions on organizational structure, the usage of

information for decision making, and the consistency of their organizational practices. To test

our hypothesis, we used the survey response to construct measures of firms’ organizational

practices. We combine these measures with publicly financial data and CEO compensation.

This yielded 164 firms with complete data for an analysis of firm productivity, and 165 firms

during the same period with the variables needed to analyze CEO compensation. Our sample

spans manufacturing, retail/wholesale trade, information, and finance/insurance industries over

the period from 2005 to 2009.

Business Practices

We constructed our key independent variable, data-driven decision-making (DDD), from three

questions of the survey: 1) the usage of data for the creation of a new product or service, 2) the

usage of data for business decision-making in the entire company, and 3) the existence of data

for decision-making in the entire company. Two other measures from the survey were also

constructed to indicate the general employee’s human capital; 1) the importance of typical

employee’s education and 2) the average of % of employees using e-mail and % of employees

using PCs, workstation, or terminal. All measures except the ones based on percentages were

capture in 5-point Likert scales.

We created DDD by first standardizing each factor with mean of zero and standard deviation of 1

and then standardizing the sum of each factor for each composite measure:

DDD = STD(STD(use of data for creation of a new product/service) + STD(use of data for

business decision in the entire company) + STD(existence of data for such a decision))

To examine the factors which may be correlated with DDD, we constructed a number of other

organizational measures listed in Table 1. “Consistency” refers to the consistency of business

practices in the entire company. The consistency measure was constructed from a composite of

responses to six survey questions on consistency of business practices across operating units,

within business units, across functions, and across geographies (4 questions); the effectiveness

of IT for supporting consistent practices; and consistency of prioritization of projects (Table 1).

“BT-link” captures the integration of business strategy with information technology and is

derived from size survey questions focusing on the interaction of employees from both groups,

understanding of each other, and the importance of technology in strategy. “Dissemination” was

based on two questions focusing on embedding processes in technology and the ability of the

firm to disseminate business practices across the firm.. “IT-governance” is a measure on how IT

project is governed in terms of prioritization, involvement with business, and managing demand

and benefit. “Find” is the effectiveness of employees’ finding knowledge or a colleague with a

particular expertise or broadcasting their expertise. “Exploration” is the capability of innovating

and creating a new business capability, product or service, a composite from 7 survey questions.

“Exploitation” is the ability of improving their existing capability, a composite from 15 survey

questions. The number of instances of IT systems on ERP/SCM, CRM, HR, and Financial

Systems and the level of centralization of IT resources on application development and

maintenance was used to construct a composite designated “Centralized IT system”. We

capture employee human capital a question about the importance of typical employee’s

education and the percentage of employees using emails or pc were included.

Financial data

Financial measures were derived from Compustat. Measures of physical assets, employees, sales

and operating income were taken directly from the Industrial annual file from 2005 to 2009.

Materials were estimated by subtracting operating income before tax and labor expense from

sales. In the case that labor expense was not available, the industry-average was used with the

industry average based on average labor expense for all firms that report labor expense and

employees in the same industry at the most detailed industry breakdown available.

CEO compensation

We used Compustat database for Executive pay for the period from 2005 to 2009 which provides

data on compensation of as many as 13 senior executives from each company. The Executives

database is compiled from proxy statements filed by the companies in compliance with

Securities and Exchange Commission (SEC).

The two measures of CEO compensation were used; one is the variable, tdc1, from Compustat

Executives data set and the other is the sum of salary and bonus. The tdc1 includes salary, bonus,

other annual, restricted stock grants, LITP payouts, all other, and value of option grants valued at

the day of grants.

For a variable for firm’s nominal size, we used three variables; total employee number, sales, or

market capitalization. Market capitalization was estimated in the same way as Gabaix and

Landier (2008) as the market value of equity at the end of the fiscal year, plus debt less deferred

taxes.

IT-employee data

The survey included the questions about IT budgets, outsourcing, change of IT budgets from

2008 to 2009, and full time IT employment. The number of full-time IT employees for the year

2008 was asked in the survey, but for the year 2009 it was estimated from the questions on IT

budget. Using the change of IT budget from 2008 to 2009, the percentage of outsourcing, and IT

FTE for 2008, we were able to estimate the IT FTE for the year 2009. The year from 2005 and

2006, we used data collected in a previous study (Tambe and Hitt, ). For the year 2007, a

moving average from 2005, 2006, 2008 and 2009 was used. The number of non-IT employees is

equal to the number of employees reported on Compustat less our computed IT employment

measure.

3. Methods

Productivity Tests

We use the Cobb-Douglas specification, the most commonly used model in information

technology and productivity literature (e.g., Brynjolfsson and Hitt, 1993, 1996; Dewan and Min,

1997). Our primary regression model can be written as the following:

ln (�����)� = � + � ln(�)�� + � ln(�� − ��������)�� + � ln(��� − �� ��������)�� + (DDD)��

+ "(DDDxDDD)��

Where k is physical capital, IT-Employee is the number of IT employees, Non-IT Employee is

the number of Non-IT employees, DDD is our data-driven decision-making variable, and

DDDxDDD is the square term of DDD. The last term, DDDxDDD, was included to examine the

potential for increasing marginal productivity effects of DDD. In some specifications, we

included variables indicating the firm’s human capital, such as importance of typical employee’s

education, to rule out some alternative explanations for our results.

Although our data on IT and other production inputs and outputs are longitudinal, our main

independent variable, DDD, is based on a single survey conducted in 2008. We constructed a 5-

year panel (2005-2009) by making the assumption that DDD was the same for the years from

2005 to 2009. Similar assumptions have been used in previous studies (Bresnahan, Brynjolfsson

et al., 2002), and many researchers have reported that organizational practices change very

slowly, especially for large firms (i.e., Milgrom and Roberts, 1990 Hannan and Freeman, 1984).

For a robustness check, we categorized the firms into 5 groups based on their response to one of

the survey questions about the change in the consistency of their business practices. The

productivity tests were conducted controlling for the group, examining the effect of DDD within

the same group which is likely to have a similar degree of change in the consistency of business

practices. The rationale for this test was based on the observation that the consistency of

business practices was highly correlated with DDD in our sample firms. Therefore, the change

in the consistency of business practices is likely to be correlated with the change in DDD.

Estimating the effect of DDD within the same group mitigates the bias that our assumption may

cause.

CEO Compensation Tests

A robust and high correlation between the nominal size of a firm and its CEO compensation has

been supported by theoretical and empirical models in numerous studies (i.e., Baker and Hall,

2004; Gabaix and Landier, 2008) . We extend the model in prior literature by including our

organizational variable, DDD and the square term of DDD to capture nonlinear effects. Our

empirical model specification is therefore:

ln($���) = � + � ln(%&'� �&(�)�� + �(DDD) + (DDDxDDD)

where ceoit is CEO pay of firm i and year t. We used output as the firm size in most analysis but

results are similar when other measures of size are used (employees, assets and market

capitalization). In some specifications, we include other variables to control for firm’s human

capital to address other potential omitted variables explaining the result.

4. Results and Discussion

Productivity Tests

The key result for the productivity was shown in Table 2; the first column is the productivity test

only with capital, employee, and materials. The coefficients associated with IT employee and

non-IT employee are consistent with previous studies (i.e., Tambe, Hitt et al., ). The second

column shows the model including our key independent variable, DDD. These coefficients

suggest that firms with a one standard deviation higher in on our measure of data-driven decision

making (DDD) have a 5% greater productivity than the mean firm. In the third column we add

the square of DDD. We find it to be negative but not significant, suggesting the possibility of

declining marginal benefits. Our results are largely unchanged when we include a variety of

human capital controls.

In table 3, we examine the relationship between our DDD measure and other organizational

practices. One surprising result was that individual IT use (employees using PCs or e-mail) was

not correlated with DDD at all. It seems that the widespread usage of computers and e-mail do

not separate high IT-usage firms from low-IT usage firms by this measure. The consistency of

business practices across business units was positively correlated with the output in some

industries at a statistically significant level (results not shown). The complementarity between

DDD and the consistency measure was not, however, statistically significant (results not shown).

We assumed that DDD was the same for the tested period (2005-2009). To check the robustness

of this assumption, we categorized our sample firms in 5 groups based on their answers on the

change of the consistency of business practices over the last 3 years because the change of the

consistency of business practices is likely to be correlated with the change of DDD. The DDD

coefficient was qualitatively the same for both models; one controlling for the group, the other

not controlling for the group (Table 4).

Overall, our results are consistent with a positive relationship between DDD and productivity

that is robust to a variety of controls and checks.

CEO Compensation and DDD

Table 5 reports our analysis of the relationship between CEO pay and firm size. The first 3

columns use the total compensation as the dependent variable including options and the next 3

models only the sum of salary and bonus. This result is robust to using physical capital as a size

control, which leads to small change reduction in the DDD coefficient. When market

capitalization was used as firm size variable, the coefficient of DDD is reduced by half. By

construction, the market capitalization value includes the stock price and reflects the expected

value of future sales. Firms with high DDD are likely to have a potential to grow faster and their

stock price may reflect the growth potential . Indeed, DDD is correlated with the market

capitalization at a statistically significant level. When the growth potential of a firm is captured

through its market capitalization, DDD coefficient may become smaller.

Following the norm in the literature of CEO compensation where sales are most commonly used

as size variable, we selected sales as size variable in the next model specifications (Table 6). In

the models in Table 6 the nonlinearity of CEO pay on DDD was first examined. Although the

coefficient of the square term of DDD in the first model in Table 4 is not statistically significant

at 90% confidence level, it becomes statistically significant when more controls are included. It

is a striking result that CEO pay increases not only with DDD but the square term of DDD. This

result echoes Rosen’s result (1981) that top talent’s earing increases with the quadratic term of

her talent as both price and the size of market increases with her talent. Further study will

formulate a model to link our result to Rosen’s economics of superstars (1981). DDD coefficient

was still statistically significant after controllling for firms’ human capital measured in the

importance of education and percentage of employees.

Conclusion

Overall, a variety of literature suggests a potential connection between data driven decision

making, productivity and executive compensation. Using survey data, our initial analysis

suggests that DDD is associated with productivity. Moreover, we find a positive and increasing

effect of DDD on executive compensation, consistent with the idea that DDD increases the value

of central decision makers and “superstar” individual decision makers. More analysis will be

conducted to strengthen this result by considering more complex models that control for firm

heterogeneity and potential reverse causality.

Table 1. Construction of Measure of Organizational Practices

Range N Mean Std. Dev.

Chronbach’ Alpha

Measure 1: Data-Driven Decision-making (DDD) 0.59

Typical basis for the decision on the creation of a new product or service (HR survey q13a)

1-5 (Experience/expertise =1 Data = 5)

160 3.01 1.10

We depend on data to support our decision making (the work practices and environment of the entire company) (HR survey q16j)

1-5 163 3.83 0.81

We have the data we need to make decisions (HR survey q16p)

1-5 161 3.42 0.86

Measure 2: Consistency 0.75

Looking across your entire company, please rate the level of consistency in behaviors and business processes across operating units (HR survey q1)

1-5 162 3.02 0.75

Regarding the first core activity of your company, the consistency within business unit (HR survey q9a)

1-5 154 3.73 0.97

Regarding the first core activity of your company, the consistency across functions (e.g., sales, finance, etc) (HR survey 9b)

1-5 137 3.32 1.02

Regarding the first core activity of your company, the consistency across geographies (HR survey q9c)

1-5 150 3.46 1.00

Effectiveness of IT in building consistent systems and processes for each operating unit (IT survey q13b)

1-5 101 3.50 0.89

Consistency of project prioritization and approval processes (IT survey q15a)

1-3 102 2.55 0.68

Measure 3: Business-Technology Link (BT Link) 0.57

Linkage of Business and IT strategy (IT survey q10) 1-4 118 2.70 0.91

Business Domain Knowledge of Existing IT Staff (IT survey q14b)

1-5 120 3.73 1.14

Business Unit Support for IT projects (IT survey q14e)

1-5 120 3.88 1.09

Employees from both business and IT working as peers (HR survey q16h)

1-5 180 3.57 0.98

Business executives understand IT (HR survey q16r) 1-5 178 3.10 1.02

IT executives understand the business (HR survey q16s)

1-5 179 3.64 0.93

Measure 4: Dissemination

Embedding many processes in technology (HR survey q16d)

1-5 177 3.25 .094

Strong ability to disseminate changes to business processes (HR survey q16m)

1-5 175 2.99 0.93

Measure 5: IT Governance

Consistency of IT project prioritization and approval process (IT survey 15a)

1-3 120 2.53 0.70

Involvement of Business with IT project 1-3 119 2.33 0.61

(IT survey 15b)

Managing demand (IT survey 15c) 1(no real demand management) – 3 (chargeback based on usage)

120 2.55 0.52

Tracking and managing the capture of IT project benefits (IT survey 15e)

1-3 120 1.64 0.68

Measure 6: Find

Effectiveness of 1) broadcasting employees’ work; 2) describing experience/expertise; 3) collaboration; 4) finding a colleague; 5) finding information/knowledge (HR survey 17a/b/c/d)

1-5 178; 178; 176; 177

2.86; 2.81; 3.31; 2.89

0.91; 0.97; 1.01; 1.08

Measure 7: Exploration (EXPR)

IT facilitates to create new products (IT survey 11a) 1-5 118 3.76 1.17

IT facilitates to enter new markets (IT survey 11b) 118 3.66 1.06

IT supports growth ambitions by delivering services or products that set us apart from competitors (IT survey 12c/HR survey 15c)

1-4 119; 172

2.49; 2.56

1.08; 1.00

IT plays a leading role in transforming our business (IT survey 12d/HR survey 15d)

1-4 119; 172

2.88; 3.00

1.16; 1.13

IT partnering with BIZ to develop new business capabilities supported by technology (IT survey 13f/HR survey 14e)

1-5 120; 177

3.25; 3.01

0.95; 1.06

Strong ability to make substantial/disruptive changes to business processes (HR survey 16l)

1-5 174 2.87 1.05

Measure 8: Exploit (EXPT)

Deliver year-over-year productivity (IT survey 11d) 1-4 117 4.07 1.06

Deliver basic technology services to the business at the lowest cost (IT survey 12a/HR survey 15a)

1-4 119; 177

2.75; 2.49

1.17; 1.25

IT improves the efficiency and cost of business operations (IT survey 12b/HR survey 15b)

1-4 120; 171

1.88; 1.87

0.75; 0.77

Providing basic services cost-effectively (IT survey 13a/HR survey 14a)

1-5 119; 179

4.24; 3.91

0.68; 0.80

Delivering new projects or enhancements on time and within budget (IT survey 13c/HR survey 14b)

1-5 119; 178

3.66; 3.17

0.87; 0.97

Reactively responding to business needs to improve existing systems or functions (IT survey 13d/HR survey 14c)

1-5 120; 177

3.65; 3.15

0.93; 0.86

Proactively engaging with business leaders to refine existing processes and systems (IT survey 13e/HR survey 14d)

1-5 120; 176

3.4; 3.03

0.86; 0.99

Stress operational excellence over innovation (HR survey 16f)

1-5 177 3.28 1.11

Strong ability to make incremental changes or improvements to business processes (HR survey 16k)

1-5 177 3.56 0.91

Measure 9: Centralized IT system

How many instances of the following systems are you running globally? (Number of Instances)

ERP/SCM (IT survey q8a) 1 (1); 2 (2-3); 3(4+)

100 1.47 0.88

CRM (IT survey q8b) 1 (1); 2 (2-3); 3(4+)

100 1.26 0.81

HR (IT survey q8c) 1 (1); 2 (2-3); 3(4+)

100 1.33 0.68

Financial System (IT survey q8d) 1 (1); 2 (2-3); 3(4+)

100 1.52 0.77

Level of centralization of IT resources on application development and maintenance

1-3 120 1.28 0.54

Measure 10: Importance of Typical Employee’s Education

The importance of educational background in making hiring decisions for the “typical” job (HR survey q4)

1-5 179 3.39 0.98

Measure 11: % of employees using IT

% of employees using PC/terminals/workstations (HR survey q7a)

% 179 77.4 26.4

% of employees using e-mails (HR survey q7b) % 179 73.9 28.6

Measure 11: IT investment

Ln(IT employee/total employee) Year = 2005 Year = 2006 Year = 2007 Year = 2008 Year = 2009

158; 158; 165; 116; 107

-4.10; -4.04; -3.78; -3.59; -2.87

0.81; 0.81; 0.91; 1.09; 1.10

Table 2. Correlation between Output and Data-Driven Decision-Making (DDD). The industry control is at 2-

digit NAICS level for manufacturing industries and 1-digit NAICS level for other industries. The years are

from 2005 to 2009. The number of firms was 165. Standard errors were clustered around firms.

Dependent Variable = Ln(Sales)

Ln(Material) 0.507*** 0.497*** 0.499*** 0.501***

(0.0449) (0.0446) (0.0446) (0.0454)

Ln(Capital) 0.120*** 0.120*** 0.120*** 0.121***

(0.0280) (0.0273) (0.0270) (0.0267)

Ln(IT Employee) 0.0823*** 0.0814*** 0.0809*** 0.0643***

(0.0225) (0.0231) (0.0228) (0.0233)

Ln(Non-IT Employee) 0.199*** 0.206*** 0.204*** 0.224***

(0.0316) (0.0322) (0.0325) (0.0328)

Data-driven decision-making (DDD) 0.0464** 0.0445** 0.0434**

(0.0205) (0.0204) (0.0204)

DDD x DDD -0.0123

(0.0159)

Importance of Typical Employee's Education 0.0374

(0.0247)

Ln(% of Employees using PC/e-mail) 0.0421

(0.0319)

Industry and Year Control Yes Yes Yes Yes

Constant 1.079*** 1.088*** 1.113*** 0.840***

(0.1910) (0.1840) (0.1860) (0.2530)

Observations 628 628 628 628

R-squared 0.918 0.92 0.92 0.922

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

Table 3. Correlation between Data and other survey variables

DDD CON BT DISS IT_gov Find EXPR EXPT NI EDU

Consistency

(CON) 0.53*** 1

BIZ-IT Link (BT) 0.39*** 0.35*** 1

Dissemination

(DISS) 0.37*** 0.40*** 0.49*** 1

Exploit (EXPT) 0.30*** 0.39*** 0.64*** 0.51*** 1

IT governance

(IT_gov) 0.28** 0.36*** 0.30*** 0.17* 0.27*** 1

Find 0.27*** 0.17* 0.28*** 0.41*** 0.23*** 0.079 1

Centralized IT 0.21*** 0.32*** 0.17*** 0.23*** 0.11 0.22** 0.22** 1

Exploration

(EXPR) 0.17** 0.24*** 0.59*** 0.41*** 0.62*** 0.16* 0.26*** 0.02 1

Importance of

Typical

Employee's

Education (EDU) 0.21*** 0.03 0.02 0.12 -0.007 0.054 0.20** -0.09 0.165* 1

Ln(% of

employees using

e-mail or PC) (PC) -0.07 -0.19** -0.02 -0.06 -0.09 -0.03 -0.08 0.005 0.029 0.27***

Ln(IT-

Employee/Total

Employee) 0.19** -0.004 0.11 0.05 -0.03 0.07 -0.09 -0.07 0.11 0.28***

Partial correlation, controlling industry at 2-digit NAICS. ***p<0.001, **p<0.05, *p<0.1. Test is against the null hypothesis that the correlation is zero.

Table 4. Robust check on the assumption that data-driven decision-making (DDD) is quasi-fixed over the last

5 years. Firms were categorized into 5 groups based on their response to a question on the change in the

consistency of business practices. The question was “To what extent do the following statements describe the

work practices and environment of your entire company; our business processes have become more

consistent over the past 3 years.” The choices were from 1 as “Describes not all” to 5 to “Completely

describes”. The first column is a model controlling for the group and the second not controlling for the

group.

Dependent Variable: Ln(Sales)

Ln(Material) 0.491*** 0.491***

(0.0460) (0.0460)

Ln(Capital) 0.131*** 0.130***

(0.0260) (0.0258)

Ln(IT Employee) 0.0603*** 0.0612***

(0.0223) (0.0230)

Ln(Non-IT Employee) 0.228*** 0.229***

(0.0313) (0.0325)

Data-driven decision-making (DDD) 0.0411** 0.0371*

(0.0198) (0.0199)

Importance of Typical Employee's Education 0.0365 0.0407*

(0.0243) (0.0244)

Ln(% of Employees using PC/e-mail) 0.0541** 0.0627**

(0.0256) (0.0258)

Constant 0.683*** 0.655***

(0.2250) (0.2300)

Group Control Yes No

Industry and Year Control Yes Yes

Observations 617 617

R-squared 0.926 0.924

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

Table 5: The correlation of Ln(CEO pay) with Data-driven decision-making (DDD). CEO pay in the first

three models was the total compensation including salary, bonus and value of option granted at the date of

grant as listed as tdc1 in the Compustat database. The standard errors were clustered around each firm.

CEO pay in the last three models is the sum of only salary and bonus. The period is from 2005 to 2009. The

survey was conducted in 2008 and the same value for “Data” constructed from the survey was applied to all 5

years.

Table 6: CEO pay and DDD with control for firms’ human capital. The average worker’s wage is at firm

level when data available and industry-average if not available.

Dependent Variable: Ln(CEO Pay) Ln(Salary+Bonus+Option) Ln(Salary+Bonus)

Data-driven decision-making (DDD) 0.157*** 0.118*** 0.0671* 0.0917*** 0.0710** 0.0603*

(0.0484) (0.0438) (0.0350) (0.0328) (0.0277) (0.0307)

Ln(Employee) 0.256*** 0.122***

(0.0447) (0.0254)

Ln(Sales) 0.454*** 0.232***

(0.0419) (0.0306)

Ln(Market Capitalization) 0.477*** 0.223***

(0.0545) (0.0295)

Constant 7.453*** 4.645*** 4.026*** 6.989*** 5.540*** 5.383***

(0.3200) (0.3760) (0.4800) (0.2550) (0.2950) (0.3030)

Observations 683 683 613 683 683 613

R-squared 0.247 0.371 0.474 0.148 0.2 0.217

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

Dependent Variable: Ln(CEO Pay)

Data-driven decision-making (DDD) 0.121*** 0.116** 0.115**

(0.0455) (0.0452) (0.0450)

DDD x DDD 0.0533 0.0590* 0.0608*

(0.0352) (0.0330) (0.0330)

Ln(Sales) 0.454*** 0.465*** 0.467***

(0.0445) (0.0464) (0.0472)

Importance of Typical Employee's Education 0.128*** 0.122**

(0.0490) (0.0486)

Ln(% of Employees using PC/e-mail) -0.0242 -0.0274

(0.1110) (0.1110)

Ln(Average Worker's Wage) 0.0527

(0.0503)

Constant 4.189*** 4.189*** 3.603***

(0.4030) (0.6940) (0.8630)

Observations 683 663 663

R-squared 0.337 0.357 0.358

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

5. References

Autor, D., L. Katz, et al. (1998). "Computing Inequality: Have Computers Changed the Labor Market?*." Quarterly Journal of Economics 113(4): 1169-1213.

Autor, D. H., L. F. Katz, et al. (1998). "Computing Inequality: Have Computers Changed the Labor Market?" Quarterly Journal of Economics 113(4): 1169-1213.

Baker, G. and B. Hall (2004). "CEO incentives and firm size." Journal of Labor Economics 22(4): 767-798.

Baker, G. P. and B. J. Hall (2004). "CEO Incentives and Firm Size." Journal of Labor Economics 22(4): 767-798.

Balconi, M. (2002). "Tacitness, codification of technological knowledge and the organisation of industry* 1." Research Policy 31(3): 357-379.

Bebchuk, L. and J. Fried (2005). "Pay Without Performance: Overview of the Issues." Journal of Applied Corporate Finance 17(4): 8-23.

Bebchuk, L., J. Fried, et al. (2002). "Managerial power and rent extraction in the design of executive compensation." University of Chicago Law Review 69: 751-846.

Bertrand, M. and S. Mullainathan (2001). "Are CEOS Rewarded for Luck? The Ones without Principals Are*." Quarterly Journal of Economics 116(3): 901-932.

Blackwell, D. (1953). "Equivalent comparison of experiments." Ann. Math. Statist. 24: 265-272.

Bresnahan, T., E. Brynjolfsson, et al. (2002). "Information Technology, Workplace Organization and the Demand for Skilled Labor: Firm-level Evidence." Quarterly Journal of Economics 111(1): 339-376.

Brynjolfsson, E. and L. Hitt (2000). "Beyond computation: Information technology, organizational transformation and business performance." The Journal of Economic Perspectives: 23-48.

Brynjolfsson, E., T. Malone, et al. (1994). "Does information technology lead to smaller firms?" Management Science: 1628-1644.

Brynjolfsson, E. and H. Mendelson (1993). "Information systems and the organization of modern enterprise." Journal of Organizational Computing 3(3): 245-255.

Collins, D. (2010). "Destroying the value of information." Journal of Management Excellence: The Value of Information(9).

Davenport, T. (2009). "How to design smart business experiments." Harvard Business Review.

Dow, J. and C. Raposo (2005). "CEO compensation, change, and corporate strategy." Journal of Finance: 2701-2727.

Frydman, C. (2005). Rising through the ranks: The evolution of the market for corporate executives, 1936-2003. Working Paper, Harvard University.

Gabaix, X. and A. Landier (2008). "Why has CEO pay increased so much?" Quarterly Journal of Economics: 49-99.

Garicano, L. (2000). "Hierarchies and the Organization of Knowledge in Production." Journal of Political Economy 108(5): 874-904.

Garicano, L. and E. Rossi-Hansberg (2006). "Organization and Inequality in a Knowledge Economy*." The Quarterly Journal of Economics 121(4): 1383-1435.

Gibbons, R. and L. Katz (1992). "Does Unmeasured Ability Explain Inter-Industry Wage Differentials?" Review of Economic Studies 59(July): 515-35.

Gurbaxani, V. and S. Whang (1991). "The impact of information systems on organizations and markets." Communications of the ACM 34(1): 59-73.

Hall, B. J. and K. J. Murphy (2003). "The Trouble with Stock Options." Journal of Economic Perspectives 17(3): 49-70.

Hannan, M. and J. Freeman (1984). "Structural inertia and organizational change." American sociological review 49(2): 149-164.

Hitt, L. and E. Brynjolfsson (1997). "Information technology and internal firm organization: an exploratory analysis." Journal of Management Information Systems 14(2): 101.

Holmstrom, B. and S. Kaplan (2001). "Corporate Governance and Merger Activity in the United States: Making Sense of the 1980s and 1990s (Digest Summary)." Journal of Economic Perspectives 15(2): 121-144.

Jensen, M. and K. Murphy (1990). "Performance pay and top-management incentives." Journal of Political Economy 98(2): 225.

Jensen, M. C. and W. H. Meckling (1976). "Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure." Journal of Financial Economics 3(4): 305-60.

Katz, L. and L. Summers (1989). "Industry Rents: Evidence and Implications." Brookings Papers on Economic Activity: Microeconomics: 209-75.

Kohavi, R., R. Longbotham, et al. (2009). "Controlled experiments on the web: survey and practical guide." Data mining and knowledge discovery 18(1): 140-181.

Krueger, A. and L. Summers (1987). Reflections on the Inter-Industry Wage Structure. Unemployment and the Structure of Labor Markets. K. Lang and J. Leonard. Oxford, Basil Blackwell.

Krueger, A. and L. Summers (1988). "Efficiency Wages and the Inter-Industry Wage Structure." Econometrica 56(March): 259-93.

Leavitt, H. and T. Whisler (1958). "Management in the 1980's." Harvard Business Review November-December: 41-48.

Malone, T., J. Yates, et al. (1987). "Electronic markets and electronic hierarchies." Communications of the ACM 30(6): 455.

Milgrom, P. and J. Roberts (1990). "The economics of modern manufacturing: technology, strategy, and organization." The American Economic Review: 511-528.

Piketty, T. and E. Saez (2003). "Income Inequality in The United States, 1913-1998*." Quarterly Journal of Economics 118(1): 1-39.

Pinsonneault, A. and K. Kraemer (1997). "Middle management downsizing: an empirical investigation of the impact of information technology." Management Science 43(5): 659-679.

Polanyi, M. (1958). Personal knowledge, Routledge London.

Rajan, R. and J. Wulf (2006). "The flattening firm: Evidence from panel data on the changing nature of corporate hierarchies." The Review of Economics and Statistics 88(4): 759-773.

Rosen, S. (1981). "The Economics of Superstars." American Economic Review 71(5): 845-857.

Rosenberg, N. (1982). Inside the black box: technology and economics, Cambridge Univ Pr.

Rule, J. and P. Attewell (1989). "What do computers do?" Social Problems: 225-241.

Tambe, P. and L. Hitt (2008). "Job Hopping, Knowledge Spillovers, and Regional Returns to Information Technology Investments." Working paper.

Tambe, P., L. Hitt, et al. (2009). "The Extroverted Firm: How External Information Practices Affect Productivity." working paper.

Tervio, M. (2008). "The difference that CEOs make: an assignment model approach." American Economic Review 98(3): 642-668.

Von Hippel, E. (1994). "" Sticky information" and the locus of problem solving: Implications for innovation." Management Science 40(4): 429-439.

Yermack, D. (1997). "Good timing: CEO stock option awards and company news announcements." Journal of Finance: 449-476.