PROCESS MINING 101 › system › files › resource-file › 2020-05...PROCESS MINING 101 Bridging Data and Process for Real-World Insights Process mining is a process analysis method

Page 1 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

PROCESS MINING 101

Bridging Data and Process for Real-World Insights

Process mining is a process analysis method that, in many ways, turns traditional business process management (BPM) on its head. Instead of analyzing processes with sticky notes, surveys, and stakeholder interviews, process mining uses real-life data to generate process visualizations. This not only enables a truly objective view of the current state but also facilitates ongoing monitoring and process improvement. In this whitepaper, you will learn the basics of process mining so you can evaluate the applicability of this approach for your own organization.

What is Process Mining? Process mining extracts information from event logs that are commonly available in enterprise systems such as CRM and ERP to create graphic depictions of how work was done (Figure 1).

Process Mining

Figure 1

SYSTEMS

Process mining can integrate large data sets from a wide array of systems including CRM, ERP, workflow, and ticketing systems. Process mining can also be applied to custom systems, legacy systems, spreadsheets, API, message logs, and much more. In a nutshell, any operational system that leaves a digital trail—or can be configured to do so—can be used for process mining.

Page 2 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

EVENT LOGS

An event log is a record of what happens when a task is performed. Event logs capture who did what, how long it took, and so on. For example, when a customer purchases a product at a store, the point of sale system creates an event log that shows what was purchased, when, and by whom. Depending on the system and how the organization configured it, this event log may capture many other details such as the cashier’s employee ID, method of payment, and so on. Process mining can integrate all of these data points, but at minimum the event log must contain:

» a case ID number: a unique identifier for the

specific instance of task execution;

» activity: well-defined steps or status changes in the

process; and

» timestamps: there must be a timestamp for each activity in order to analyze the event in

the correct order, and if you want to analyze activity durations, you will need a start and

complete timestamp for each activity.

While process mining typically involves large data sets, it’s important to note that there’s also no minimum amount of data required. Even a small number of cases can yield valuable insights if those cases are representative of the process. Note also that data may need to be cleaned (scrubbed of missing values, outliers, and so on), simplified (scrubbed of additional columns, notes, etc.) or merged before importing into the process mining tool. 1

PROCESS MINING

The user imports event log data (often in the form of .csv or .txt files) into the process mining tool and configures the columns to according to each requirement (case, activity, timestamps) and other data points of interest (e.g., department, product, process category).

VISUALIZATION

After the user has imported the data, the process mining tool automatically “discovers” and visually represents the underlying process. Users can filter and drill down into the results and, with the help of AI algorithms, pinpoint and examine variances (Figures 2 and 3). Most process mining tools allow for the user to export the results in an image (e.g., .pdf, .jpg) or dataset (e.g.,.csv, .xes, .mxml) file format.

Right now, I have to trust people about

whatever they say that they do, and

that’s not always accurate. Process

mining can help us create a more

accurate picture of what actually

happens and develop better KPIs.

- Ewa Boloczko,

Business Process Analyst, Sakata Seed

Page 3 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

Process Visualization and Variants in Signavio Process Mining Tool2

Figure 2

Drilling Down into Case Variant Data in Disco Process Mining Tool3

Figure 3

Page 4 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

Development of the Field Process mining is a relatively new field. Operational systems have long created event log data, but this was primarily used for IT security and performance-checking purposes. The exploration of applying event logs for process analysis developed in a mostly academic context until around 2010, when commercial process mining tools became more widely available and mature. In 2011, the Institute of Electrical and Electronic Engineers (IEEE) Task Force on Process Mining issued its seminal Process Mining Manifesto, which outlined guiding principles and a call to action to increase the maturity of process mining, and Dr. Wil van der Aalst published the first book on process mining, Process Mining: Data Science in Action.4

Since then, the field has exploded. A study conducted by ResearchandMarkets found that the market is expected to grow from $185.3 million (USD) in 2018 to $1.4 billion by 2023 and that digital transformation is the primary driver of this growth. Europe currently represents the largest market size, while North America shows the highest growth rate.5 A vast array of providers are creating process mining solutions, but Gartner estimates that about 25 vendors represent more than 95 percent of the total market.6

Types of Process Mining There are four main types of process mining (Figure 4).

Types of Process Mining

Figure 4

1. Process discovery—is when event logs are used to generate process models without additional information or tweaking.

2. Conformance checking—finds differences and commonalities between event logs and a process model. This can be used to determine if reality confirms to the model and vice versa.

Page 5 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

3. Process re-engineering—is improving, extending, or otherwise changing the process model based on event data. This can be used to adjust models to better reflect reality or enrich models with additional perspectives. This form of process mining is also called “enhancement” or “extension.”

4. Operational support—aims to directly influence the process by providing data-driven warnings, predictions, and/or recommendations.7

Benefits and Applications Process mining offers many benefits. First and foremost, of which is that it presents a truly objective view of the current-state process. Process mining also helps organizations harmonize disparate processes and systems, which in turn enables the process team to identify redundancies and pinpoint the most fruitful opportunities for improvement and automation. Additionally, process mining can be used post-improvement to identify new exceptions and monitor performance.

Process mining is hugely beneficial for digital transformation efforts. Process mining can help the organization:

» Prepare—identify and review all current processes prior to transformation.

» Plan—explore process changes necessary to achieve digital transformation goals.

» Optimize—detect process improvement opportunities in the context of digital

transformation.

Thus, process mining delivers valuable insights that help organizations make better decisions before and throughout digital transformation. Process mining helps decision makers define strategies and make investments based on data. This way, they will be less likely to be distracted by “shiny object” tech tools that won’t deliver value for your business. Process mining is also a valuable input for robotic process automation (RPA), which is a key piece of many organization’s digital transformation plans.

Organizations of all sizes, across industries and around the world, have used process mining to streamline, standardize, and optimize processes. A small sampling of examples shows the variety of applications.

» The School of Management at the City University of Seattle used process mining to

compare and accelerating the utilization of its e-learning platform.

» Toulouse Hospital used process mining to redesign and mutualize consulting services of

medical specialties.

» Bayer used process mining to identify inefficiency potentials and ensure process compliance

in procurement, sales, and logistics.

» Energy company Alliander used process mining to create a complete picture of its

purchasing process, leading to standardization and staff re-training.

» The Association of Certified Fraud Examiners used process mining to improve detection of

transaction fraud. 8

With process mining it will be possible to detect or diagnose problems based on facts and not on conjectures or intuitions.

- Pedro Robledo BPM Consultant, BPMteca.com

Page 6 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

Requirements Process mining requires process mining software and data from event logs (which include ID, timestamp, and process step). Fortunately, organizations have a wide range of options to choose from. There are several free/open source process mining toolkits available, such as PMLAB and ProM. Process mining functionality is also offered by commercial vendors such as Celonis, Fluxicon, ARIS, Signavio, and many more. Beyond the tool itself, process mining requires clean data with case IDs, activity, and timestamps.

Process mining also requires a combination of skillsets. Although process mining tools are designed to be user-friendly, the team responsible for process mining must also determine which processes to analyze, prepare the data, and act on the results. This requires the following roles:

» IT administrator—extracts data and helps the team understand data fields,

» data specialist—merges and re-formats data in preparation for process mining,

» data/process analyst—tests and fixes data quality,

» process owner/sponsor—provides leadership and support for the effort,

» domain expert—a subject matter expert to guide analysis and data validation,

» business analyst—evaluates improvement options from a business perspective

» project manager—keeps the process mining effort on track and aligned with business

needs,

» change manager—helps manage changes from the implementation of process mining and

its resulting improvements, and

» risk/privacy officer—ensures ethical use of data.

In some instances, the organization may have certain individuals that can fulfill multiple roles. However, most often a multi-disciplinary team is necessary for successful process mining.9

Methodology At the most basic level, process mining can be distilled into two steps. First, select the process you wish to mine. Second, capture the data for that process and feed it into the process mining tool. However, applying process mining to generate meaningful improvement opportunities requires a more intricate and thoughtful approach. Professor Santiago Aguirre recommends a four-stage methodology for process mining projects.

1. Define the Project. Define the problem that the organization aims to solve through process mining. Determine the scope and flow in the process in question. Note any gaps in process indicators and define project objectives/research questions.

2. Prepare Data. Locate and extract data from systems. Analyze quality and perform data cleaning as necessary. Merge data if needed to ensure all data sets include case IDs, activity, and timestamps.

3. Process Analysis. Feed data into the process mining tool to generate a visualization of the current-state process. Verify compliance, analyze process performance and findings, and identify the root cause of process problems.

4. Process Re-design. Identify and evaluate alternatives for improvement. Implement improvements and measure the results.10

Page 7 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

Challenges Perhaps the biggest challenge in process mining is the data itself. Extracting event data may require significant effort. For example, the organization may need to merge different data sets in order to create event logs with all the requisite information (case, activity, and timestamps. Organizations may also need to dig into legacy systems to find necessary data. Process mining tools do not clean the organization’s data. If the user feeds in messy, inaccurate, or otherwise “dirty” data into the process mining tool, the result will be messy and inaccurate.

Baselining can also be challenge. Without some understanding of the process and what it is designed to achieve, the end user won’t know which algorithm to apply or which variations to analyze. The user needs a baseline understanding of the history of the process as well. Process mining tools consider processes to be static. When a process changes over the course of analysis, this is called “concept drift.” Some process mining tools have concept drift detection algorithms, but these may not capture all forms of change.11

It’s also important to understand that process mining is not a silver bullet solution for all BPM needs. Process mining facilitates monitoring and reporting, but it is not a dashboard or reporting tool. Process mining provides insights for process improvement, but it does not automatically identify or implement improvements. Process mining is also not a replacement for process modeling or process simulation tools.12

Key Takeaways Process mining offers significant potential to improve the efficiency and accuracy of analysis for many processes. It’s not an all-in-one solution, but it is a highly valuable addition to any organization’s BPM toolbox. APQC expects adoption will increase as more vendors integrate process mining capabilities into their solutions. However, we recommend that organizations take a close look at their data quality before diving in. Without good data management, process mining will not deliver valuable insights—and may even lead organizations in the wrong direction. As with any data-based tool, garbage in = garbage out.

ABOUT APQC

APQC helps organizations work smarter, faster, and with greater confidence. It is the world’s foremost authority in benchmarking, best practices, process and performance improvement, and knowledge management. APQC’s unique structure as a member-based nonprofit makes it a differentiator in the marketplace. APQC partners with more than 500 member organizations worldwide in all industries. With more than 40 years of experience, APQC remains the world’s leader in transforming organizations. Visit us at https://www.apqc.org/, and learn how you can make best practices your practices.

1 “Data Requirements,” Process Mining Book 2.5. Retrieved May 7, 2020. 2 “Process Mining: Let Data Tell Your Real Story,” Signavio. Retrieved May 8, 2020. 3 “Hands-on Tutorial,” Process Mining Book 2.5. Retrieved May 7, 2020. 4 Aalst, Wil. (2018). “Process discovery from event data: Relating models and logs through abstractions.” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. e1244. and “Process Mining Manifesto” IEEE Task Force on Process Mining. Retrieved May 8, 2020.

Page 8 of 8

K010617 ©2020 APQC. ALL RIGHTS RESERVED

5 “Process Analytics Market by Process Mining Type (Process Discovery, Process Conformance & Process Enhancement), Deployment Type, Organization Size, Application (Business Process, It Process, & Customer Interaction) & Region - Global Forecast to 2023,” Research and Markets. June 2018. ID 4576970. 6 Marc Kerremans, “Market Guide for Process Mining,” Gartner. June 2019. 7 Aalst, Wil. (2018). “Process discovery from event data: Relating models and logs through abstractions.” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. e1244. 8 “Process Mining: A Database of Applications (2017 Edition).” HSPI Management Consulting. 9 Anne Rozinat, “Skills and Roles Needed for Your Process Mining Project.” Fluxicon. February 22, 2017. 10 Santiago Aguirre, “Methodology Process Mining” in Pedro Robledo, “Process Mining plays an essential role in Digital Transformation,” Medium. September 23, 2018. Retrieved May 7, 2020. 11 R'bigui, Hind & Cho, Chiwoon. (2017). The state-of-the-art of business process mining challenges. International Journal of Business Process Integration and Management. 8. 285. Retrieved May 8, 2020 via ResearchGate. 12 “What Process Mining is Not.” Process Mining Book 2.5. Retrieved May 8, 2020.