Controlling Excel Chaos With Business Intelligence A White Paper by Diane Sklar
May 10, 2015
Controlling Excel ChaosWith Business Intelligence
A White Paper by Diane Sklar
Executive Summary
The Excel Challenge
Spreadsheet Chaos
Manual Labor Begets Errors
On the Spreadsheet Audit Trail
WebFOCUS Excel Sourcebooks
Setting up WebFOCUS Excel Sourcebooks
There Is a Solution
A Spreadsheet in Every Mailbox
The Corporate Excel Sourcebook
Deliver Advanced Functionality
Answering the Ad Hoc Question
Conclusion
1
2
2
2
2
4
13
4
4
5
5
6
11
Table of Contents
Business analysts develop Excel spreadsheets to assist with the day-to-day operational
decisions their jobs demand. Pleased with the autonomy and sophisticated analysis that Excel
supports, they share these innovations with colleagues who modify spreadsheet logic and
manually tack on data from their own information silos. Over time, rogue spreadsheets with data
from dubious sources is propogated throughout the organization. Executives find themselves
making decisions based on untraceable, questionable data. The enterprise is at a loss to audit
these numbers for themselves and for regulatory agencies.
Meanwhile, the IT division has complete, auditable, and backed-up operational system data that
is untapped by the Excel user community. IT executives are frustrated by the rampant spread of
unreliable information and their inability to leverage corporate data stores.
Information Builders has both push and pull strategies to serve up corporate data for use in
Excel spreadsheets. These strategies buttress Excel’s weaknesses as an enterprise business
analysis tool while allowing users to continue to operate in their preferred analysis environment.
1 Information Builders
Executive Summary
2 Controlling Excel Chaos With Business Intelligence
Spreadsheet Chaos
Spreadsheets were the standard model for doing financial analysis long before enterprises
automated accounting and finance functions. When Microsoft launched Excel, it mimicked the
spreadsheet layout and made the business analysts’ transition from paper to computer a
seamless one. Moreover, the easy-to-use, point-and-click interface running on the Windows
platform gave Excel a market-share victory over competitors like Lotus and VisiCalc.
Manual Labor Begets Errors
Excel was designed for the individual analyst working alone on a computer with a limited amount
of data. The original Excel model had a business analyst typing a small subset of corporate data
into a spreadsheet, cell by cell, and then manually manipulating that data to solve a business
problem. But manual data entry is an expensive, time-consuming misuse of the business
analyst’s time and error prone as well. Cell-by-cell data entry leads to simple mechanical errors
such as mistyping a number or pointing to the wrong cell. Graver errors can also occur when the
analyst makes errors in formulas because of bad keystrokes or bad logic.
When data is available in an existing spreadsheet, it can be cut and pasted via the clipboard or
merged via Excel functionality. This works best when the source and target ranges are formatted
identically. If they are not identical, data and formulas can inadvertently be corrupted.
Alternatively, ranges of data from an external Excel workbook can be referenced in place. The
effectiveness of this technique is dependent on the source file remaining unchanged.
Maintenance to spreadsheets chained in this way can be horrendous when there is no
centralized change management policy.
Ideally, data from corporate stores should be transferable directly to Excel spreadsheets, thereby
insuring that all business analysts start their spreadsheet ventures with the one true version of
enterprise data. But Microsoft’s vision of interoperability has remained focused on Excel and
other Microsoft desktop products. Outside connections must be established through ODBC or
DDE connections. Beyond these options, one must transform data to the text-based CSV format
and stage it on the Windows platform.
Figure 1: Manual spreadsheet generation is a breeding ground for errors.
The Excel Challenge
CorporateData
Manual Data/Formula Entry
ReportOutput
StyledAggregatedSpreadsheet
3 Information Builders
On the Spreadsheet Audit Trail
For many companies Excel has grown beyond a small-scale analysis tool. It is instead the main
vehicle for financial analysis in mission-critical departments, pushing Excel functionality beyond its
original intent. Excel has no built-in mechanism for separating the presentation, calculation, analy-
sis, and storage layers of an application. The onus of solid spreadsheet design is still left to the
spreadsheet developer. Of course, skill levels in Excel and in programming methodology vary widely
among analysts. In non-IT departments untrained spreadsheet programmers can apply resource-
intensive calculations at the wrong level, creating a performance bottleneck. Rigorous testing with
sufficiently varied data samples is rarely done at the departmental level. These are just some
examples of possible fallout from not applying IT methodology to the spreadsheet community.
Accurate and traceable financial information is critical to making sound business decisions that
move the enterprise forward. Auditable information is required outside the organization as well.
The 2002 Sarbanes-Oxley Act requires disclosure in annual reports of internal controls over
financial reporting. Spreadsheet errors can lead to expensive and embarrassing problems. The
European Spreadsheet Risk Interest Group analyzes and quantifies the cost of spreadsheet
errors worldwide. In the last six months alone, they have reported various situations, including:
■ A public auditor’s office in Minneapolis mistakenly reported the percentage change in unreserved
fund balances; an analyst set up the formula for the column in the spreadsheet, dividing the
difference between 2003 and 2004 balances by the 2004 total, instead of the 2003 figure
■ A Housing and Urban Development (HUD) audit revealed that a local housing authority had to
pay over $200,000 to cover expenses incurred when the authority overpaid some landlords
due to a data-entry error
■ In Nevada, a 2006 municipal budget was developed from a copy of the city’s 2005 budgeting
spreadsheet; in late 2005, a problem was revealed causing a $5 million deficit in the water
and sewer fund; while fixing the problem with the water and sewer budget, other errors were
uncovered and fixed
■ A well-known medical and consumer imaging company had to amend its third-quarter loss by
$9 million, announcing that the adjustment was needed because too many zeros were added
to an employee’s accrued severance on a spreadsheet; the company’s CFO characterized the
situation as “an internal control deficiency”
True robust error protection and auditing for Excel is generally left to third-party products. The
flourishing market of add-on tools bears witness to Excel’s weakness in this area. Auditing with
add-on tools becomes another training and standardization dilemma. Once again the enterprise
is dependent on the skill, prudence, and discipline of the individual analyst to properly deploy
audit tools.
Despite these problems, usage of Excel is not diminishing. It is still the undisputed giant among
business analysis tools. It can be found on every desktop in the enterprise. Its cost per seat is
negligible, keeping it the first choice for on-going enterprise application needs. How can the
organization protect itself from the rampant spread of uncontrolled and unsanctioned numbers
in Excel spreadsheets?
4 Controlling Excel Chaos With Business Intelligence
Setting Up WebFOCUS Excel Sourcebooks
Since Excel usage shows no sign of waning, IT’s best approach is to eliminate the resource-
intensive, error-prone data entry process in traditional Excel deployment. In most enterprises, IT
has already invested resources in creating a protected and scrubbed warehouse of summarized
financial results. When necessary, IT can also provide links to extract relevant transaction-level data.
A comprehensive study of all Excel user requirements can feed the design of a set of master
Excel workbooks supplying content to all Excel users. Workbook structure should be flexible,
allowing users to exploit any subset of workbooks, worksheets, or cell ranges. Ideally, data
should be delivered to these master workbooks already summarized and styled. It should also
be formatted for the specific version of Excel that users have on their desktops. Complex
formulas should be pre-set to avoid errors. Excel structures that are difficult to develop, such as
graphs and pivot tables, should be delivered analysis-ready.
Users will have different data latency requirements as well. Some workbooks may need to be
refreshed quarterly, other workbooks may require a monthly, weekly, daily or hourly feed.
There Is a Solution
WebFOCUS, Information Builders’ business intelligence suite of products, generates Excel
spreadsheets across all generations of Excel. This includes XML-based formats with
comprehensive Excel functionality as well as binary formats that optimize transfer speed.
All WebFOCUS Excel spreadsheets have their data summarized and formatted as a complete
spreadsheet on the WebFOCUS Reporting Server. The Reporting Server, typically located on the
same platform as the target data, summarizes data as per the request, minimizing the volume of
data transmitted and formatted on the desktop. By the time the spreadsheet arrives on the
user’s desktop it is ready to go, without wait-time for additional formatting.
Delivery of corporate spreadsheet data to Excel users can be accomplished by several
automated distribution strategies. For each distribution scenario, WebFOCUS ReportCaster
schedules jobs and extracts data from known, auditable corporate sources. It makes data
available at any interval, from every n years to every n minutes. The resulting extract can be
formatted in several ways:
■ As Excel spreadsheets, which are pushed to a distribution list of Excel users via e-mail
■ As a Corporate Excel Sourcebook(s) on a central server – users connect to the corporate
Excel sourcebook and pull relevant data into their own Excel workbooks by referencing
named ranges of cells
■ As database extracts on a central server with a parameter-driven user interface; users supply
parameter values that determine all content and stylistic characteristics for a target spread-
sheet to answer an ad hoc question
WebFOCUS Excel Sourcebooks
5 Information Builders
A Spreadsheet in Every Mailbox
When spreadsheet contents and format are uniform across a large population of users, a
ReportCaster job can e-mail Excel spreadsheets to a distribution list of user addresses. The job
is programmed once and tested rigorously by IT professional staff familiar with the corporate
files. The job is run once and output is collected for all interested parties.
E-mails with spreadsheet attachments are created for all entries on the distribution list. The
distribution list can be developed manually or derived from Microsoft Active Directory or an
LDAP directory. Each recipient on the distribution list can receive the entire report. Or, if the
report contents are private, the report can be burst so that recipients receive content only for the
sort group(s) relevant to them. For example, a company-wide profit and loss statement can be
burst by cost center so that managers receive the statements only for their own cost centers.
Figure 2: Bursting Excel spreadsheets saves system resources by running a job once for all users, but ensuresprivacy by distributing only the appropriate sections to each user.
The Corporate Excel Sourcebook
Sometimes spreadsheet content and format are highly customized and frequently change. In
such an environment, a suitable strategy is to provide foundation spreadsheet data in a generic
workbook or Corporate Excel Sourcebook. This workbook standardizes detail, summary, and
calculated amounts. It is created and stored on a universally accessible server by a ReportCaster
job that refreshes it as often as necessary.
Organization of WebFOCUS Corporate Excel Sourcebooks is extremely flexible. Data can be
structured on a single worksheet or multiple worksheets. If there are multiple worksheets, all
sheets can have identical format but different ranges of data. Using the profit and loss statement
example, a workbook could be organized with one sheet per cost center.
Corporate Directory(LDAP, Active Directory)
Global ExcelSpreadsheet
France
Spain
UK
Brazil
SpreadsheetDistribution
Schedule
Another multiple worksheet solution is to consolidate unrelated reports in a single workbook.
For example, one worksheet might contain inventory reports, the next sheet might contain
human resources reports, and so on. No connection between the worksheets is necessary.
To populate personal spreadsheets, users connect to the central server and reference the central
workbook in their local spreadsheets. References can pluck data from the entire workbook,
worksheet, or from a range of cells within a report. To facilitate selective data gathering, the
Corporate Excel Sourcebook can be designed with named cell ranges. Named ranges are put in
place when the WebFOCUS Excel job is set up. They shield users from needing to change their
personal spreadsheets when report size and position change in the Corporate Excel Source-
book. Named ranges can be set up for a sheet, report, column, row, or any rectangle of cells.
With Corporate Excel Sourcebooks, users each take exactly the data they want, no more and no
less. They can then style and manipulate that data in any way they please and the audit trail to
controlled corporate data is preserved.
Figure 3: A Corporate Excel Sourcebook contains corporate Excel data that all enterprise users can link to withtheir own spreadsheets.
Deliver Advanced Functionality
It is worth discussing the scope and variety of pre-formatted Excel features supported by
WebFOCUS. Maximum automated data manipulation means minimum chance for human error.
6 Controlling Excel Chaos With Business Intelligence
Employees
Purchasing Range
Orders
Customers
Sales
Inventory
123456789
A B C D E F
123456789
A B C D E F
Corporate Data
Payroll Sheet
Insurance Cost Range
7 Information Builders
WebFOCUS supports all Excel functionality, either directly or through the use of spreadsheet
templates. The specific mix of Excel capabilities pre-set by WebFOCUS extract and distribution
jobs is limited only by the standard versions of Excel in use. All features of every Excel release
are supported. Following is a summary of stylistic and analytic features available.
Summary LinesWebFOCUS generates report headings and footings, page headings and footings, and recap
lines on sort breaks, all of which can have a combination of text and numeric totals. Worksheet
and workbook names can also be automatically generated.
Styling and Conditional StylingWebFOCUS controls the font family, size, color, and style for each and every element of a
spreadsheet. Summary lines, column titles, column content, etc. can be styled separately.
Conditional styling is also supported so that data meeting defined business criteria is
highlighted by style or color.
Drill DownHypertext links to other spreadsheets, images, or HTML pages can be automatically embedded in
distributed Excel output. Links can be positioned in summary lines or within the data of the report.
Formulas and Calculation LogicComplex formulas can be implemented in the WebFOCUS extract job and passed to Excel. The
benefit is that these complex calculations are written and tested centrally by IT professionals
familiar with corporate data sources and with structured programming practices. The WebFOCUS
self-documenting code becomes the audit trail for Sarbanes Oxley and other compliance regulations.
Even relatively simple calculations such as growth rate and profitability can be incorrect when
factors are reversed. Consider the multiplicity of factors in some calculations typically imple-
mented in spreadsheets. In analyzing pay grades for example, human resources departments
often rate performance based on manager performance evaluation, absenteeism, years of
service, current salary within pay grade, and corporate guidelines for pay increases. If these
formulas are incorrectly calculated, legal issues could arise.
Insurance companies might base profitability projections for corporate health insurance
contracts on rates charged, historic dollar amount of claims, employee population age, and
numbers of dependents. All factors must be weighted and if a new factor is introduced, all
weights must be re-calculated.
8 Controlling Excel Chaos With Business Intelligence
Calculating the cost of goods sold and the value of inventory on hand can also involve an array
of factors. Inventory valuations typically involve a range of prices and costs for raw material,
labor, shipping, etc. at different points in time, periodic production volumes, and LIFO or FIFO
assumptions about the rotation of goods. A spreadsheet capturing these rules would quickly
become a dangerous breeding ground for error. The potential for error is dramatically reduced
when such calculations are done by a single, thoroughly tested and auditable module with
access to the correct data sources.
Figure 4: A typical WebFOCUS Excel spreadsheet has predefined calculations, styling, and drill-downs.
PivotTablesPivotTables make it possible to reveal trends in a data sample by sorting, filtering, adding key
elements, and charting. Despite a built-in Excel PivotTable Wizard, there are pitfalls to
developing PivotTables. While most report consumers manipulate PivotTables effectively, far
fewer users can readily produce sophisticated, styled PivotTables.
Pre-calculated formulas
Formatted headings and footings
Conditional styling when salesare greater than $100k
Drill down to other reports,spreadsheets, or graphs
9 Information Builders
With WebFOCUS, PivotTables can be developed by IT professionals and built into the extract job.
WebFOCUS PivotTable Developers work in a graphical user interface, building the initial view of
the table as well as the lists of cached and page control fields.
Figure 5: Most Excel users can manipulate PivotTables, but far fewer can design them.
TemplatesFor all spreadsheet functionality other than the features mentioned above, WebFOCUS can imple-
ment Excel functionality by leveraging templates. Additional functions might include print preview
functions or graphing. Undoubtedly, spreadsheets exist that represent a great deal of effort spent
in development of macros. If these macros have been tested and certified, they can be leveraged
in place without re-writing. They can be applied to WebFOCUS formatted data by use of templates.
Templates are skeleton workbooks containing any Excel functionality. During spreadsheet
creation, the WebFOCUS Reporting Server locates the template, merges data into a designated
sheet, and saves the results to a separate workbook file. The final spreadsheet has corporate
data inserted into a pre-existing user approved format. Templates not only leverage all existing
10 Controlling Excel Chaos With Business Intelligence
functionality of Excel, they also position WebFOCUS to leverage future functionality that
Microsoft will add to Excel.
Figure 6: WebFOCUS merges an Excel layout (template) with corporate data to provide the exact formattingand data manipulation required.
Figure 7: An Excel template before and after it is populated with data by WebFOCUS.
12%14%
5%
30%25%
CorporateData Excel
Templates
WebFOCUSPopulated
Spreadsheets
x x x x x xx x x x x xx x x x x xx x x x x xx x x x x xx x x x x xx x x
xxxxxxx xxxxxxxxxxxx xxxxxxxxxxxx xx
xxxxxxx xxxx
11 Information Builders
Answering the Ad Hoc Question
There will always be a need for new spreadsheets to address the latest business challenge. Late-
breaking situations that require new subsets of data can be tackled with an interactive ad hoc
reporting system. Such an application has a series of parameter selection screens that front-end
appropriate corporate data stores. All characteristics of a target Excel spreadsheet are defined
via the parameter selection screens. Which database fields and calculated fields are included,
what are the summarization rules of the report, the sort criteria, the formatting, and which
cascading stylesheet – these are all criteria picked by users from menus. The resulting output is
always a spreadsheet in the user’s preferred Excel format.
Figure 8: Answering the ad hoc question is handled with a nightly parameterized application that formatscustomized spreadsheet output.
Inventory
Sales
Orders
Corporate Data
A
B
C
▼
▼
▼
D
E
F
▼
▼
▼A
B
C
▼
▼
▼
D
E
F
▼
▼
▼A
B
C
▼
▼
▼
D
E
F
▼
▼
▼
Forms
Parameters:Sort
FilterAggregation
Styling
12 Controlling Excel Chaos With Business Intelligence
Figure 9: Detail of a parameterized form and the report it generates.
For example, the insurance company that historically computed health insurance rates based on
historic claims, population age, and number of dependents could add salary to the rate
calculation. Analysts want to model the effect of the new calculation on profitability of the
contract. Analysts can multi-select salary, along with the other rate calculation factors and derive
a new spreadsheet with factors including individual or aggregated salaries. This spreadsheet
could then be manipulated off-line in various what-if scenarios but the foundation data is
enterprise-certified.
13 Information Builders
Enterprises can take control of the burgeoning number of unsanctioned spreadsheets by
adopting a policy of centralized master spreadsheet generation. Excel is a powerful and
ubiquitous analytical tool that can make a business analyst self-sufficient, but it lacks strong
audit and error-checking capabilities. However, master spreadsheets drawn from the
company’s primary operational and warehoused data sources can provide an audit trail back
to elemental transactions.
WebFOCUS creates fully summarized, organized, and styled Excel spreadsheets with complex
formulas already in place, lowering the potential for keystroke and logic error. WebFOCUS-
generated spreadsheets can be pushed out to business analysts in e-mail or staged on a central
server where analysts fetch target data by hitching their own linked spreadsheets to the master
corporate spreadsheet. Furthermore, on the corporate Excel form, analysts can screen, sort, and
summarize samples of data to derive spreadsheet answers to ad hoc questions. With a little
analysis and the right WebFOCUS business intelligence tools, you can finally have a corporate
Excel spreadsheet strategy that works.
Conclusion
North AmericaUnited States■ Atlanta,* GA (770) 395-9913
■ Baltimore, MD Consulting: (703) 247-5565
■ Boston,* MA (781) 224-7660
■ Channels (800) 969-4636
■ Charlotte,* NC Consulting: (704) 494-2680
■ Chicago,* IL (630) 971-6700
■ Cincinnati,* OH (513) 891-2338
■ Cleveland,* OH (216) 520-1333
■ Dallas,* TX (972) 490-1300
■ Denver,* CO (303) 770-4440
■ Detroit,* MI (248) 743-3030
■ Federal Systems,* DC (703) 276-9006
■ Hartford, CT (860) 249-7229
■ Houston,* TX (713) 952-4800
■ Los Angeles,* CA (310) 615-0735
■ Metropolitan,* NY Sales: (212) 736-7928
Consulting: (212) 736-4433, ext. 4443
■ Minneapolis,* MN (651) 602-9100
■ New Jersey* (973) 593-0022
■ Orlando,* FL (407) 804-8000
■ Philadelphia,* PA (610) 940-0790
■ Pittsburgh, PA (412) 494-9699
■ St. Louis,* MO (636) 519-1411
■ San Jose,* CA (408) 453-7600
■ Seattle,* WA (206) 624-9055
■ Washington,* DC Sales: (703) 276-9006
Consulting: (703) 247-5565
CanadaInformation Builders (Canada) Inc.
■ Calgary (403) 538-5415
■ Ottawa (613) 233-0865
■ Montreal* (514) 334-0404
■ Toronto* (416) 364-2760
■ Vancouver* (604) 688-2499
MexicoInformation Builders Mexico
■ Mexico City 52-55-5062-0660
AustraliaInformation Builders Pty. Ltd.
■ Melbourne 61-3-9631-7900
■ Sydney 61-2-8223-0600
Europe■ Belgium Information Builders Belgium
Brussels 32-2-7430240
■ France Information Builders France S.A.
Paris 33-14-507-6600
■ Germany Information Builders (Deutschland)
Dusseldorf 49-211-522877-0
Eschborn 49-6196-77576-0
Munich 49-89-35489-0
Stuttgart 49-711-7287288-0
■ Netherlands Information Builders
(Netherlands) B.V.
Amsterdam 31-20-4563333
■ Portugal Information Builders Portugal
Lisbon 351-217-230-720
■ Spain Information Builders Iberica S.A.
Barcelona 34-93-344-32-70
Bilbao 34-94-425-72-24
Madrid 34-91-710-22-75
■ Switzerland Information Builders
Switzerland AG
Dietlikon 41-44-839-49-49
■ United Kingdom Information Builders
(UK) Ltd.
London 44-845-658-8484
Representatives■ Austria Raiffeisen Informatik Consulting GmbH
Vienna 43-12-1136-3870
■ Brazil InfoBuild Brazil Ltda.
São Paulo 55-11-3285-1050
■ China InfoBuild China, Inc.
Shanghai 86-21-50805432
■ Finland InfoBuild Oy
Espoo 358-207-580-843
■ Greece Applied Science
Athens 30-210-699-8225
■ Guatemala IDS de Centroamerica
Guatemala City 502-2361-0506
■ Gulf States Nesma Advanced Technologies
■ Bahrain ■ Kuwait ■ Oman
■ Qatar ■ Yemen ■ United Arab Emirates
Riyadh 96-1-465-6767
■ India Divas Offshore Software
Technologies Private Limited
Gurgaon 91-124-501-8801
■ Israel NESS A.T. Ltd.
Tel Aviv 972-3-5483638
■ Italy Selesta G C Applications S.P.A.
Genova 39-010-64201-224
Milan 39-02-2515181
Torino 39-011-5513-211
■ Japan K.K. Ashisuto
Osaka 81-6-6373-7113
Tokyo 81-3-5276-5863
■ Malaysia Elite Software Technology Sdn Bhd
Kuala Lumpur 60-3-21165682
■ Norway InfoBuild Norway
Oslo 47-23-10-02-80
■ Philippines
Beacon Frontline Solutions, Inc.
63-2-750-1972
■ Saudi Arabia Nesma Advanced Technology Co.
Riyadh 996-1-4656767
■ Singapore
Automatic Identification Technology Ltd.
65-6286-2922
■ South Africa Fujitsu Services (Pty.) Ltd.
Johannesburg 27-11-2335911
■ South Korea Unitech Infocom Co. Ltd.
Seoul 82-2-2026-3100
■ Sweden Cybernetics Business Solutions AB
Solna 46-7539900
■ Taiwan Galaxy Software Services
Taipei 886-2-2586-7890
■ Thailand Datapro Computer Systems Co. Ltd.
Bangkok 662-679-1927, ext. 200
■ Turkey Istanbul
Veripark 90-212-283-9123**
■ Venezuela InfoServices Consulting
Caracas 58-212-763-1653
Toll-Free Numbers■ Sales and Information
(800) 969-INFO
■ ISV, VAR, and SI Partner Information
(800) 969-4636
**Training facilities are located at these branches;additional locations are available.
**Authorized to sell iWay Software only.
Corporate Headquarters Two Penn Plaza, New York, NY 10121-2898 (212) 736-4433 Fax (212) 967-6406 DN4601444.0706
www.informationbuilders.com [email protected]
Canadian Headquarters 150 York St., Suite 1000, Toronto, ON M5H 3S5 (416) 364-2760 Fax (416) 364-6552 For International Inquiries +1(212) 736-4433
Copyright © 2006 by Information Builders, Inc. All rights reserved. [51]
All products and product names mentioned in this publication are trademarks or registered trademarks of their respective companies. Printed in the U.S.A.on recycled paper
Sales and Consulting Offices