Data Warehousing and Business Intelligence for Plantation Companies

D A T A W A R E H O U S IN G A N D B U S I N E S S I N T E L L I G E N C E F O R P L A N T A T I O N -

B A S E D C O M P A N I E S

1 BACKGROUND

Malaysia has a well-established history in plantations. Internationally, the country is a major producer of palm oil and rubber products. Additionally, it is a significant producer of coconut, cocoa and fruit products. Some of the largest companies in Malaysia are plantation-based conglomerates, such as Sime Darby and Felda Global Ventures. Apart from these plantation giants, many other plantation companies in Malaysia are sizeable too, such as Tradewind Plantations, Tabung Haji Plantations, etc.

The plantation industry in Malaysia is well integrated vertically in upstream as well as downstream activities. Upstream, plantations grow oil palm, rubber, coconuts, cocoa and other products. These are processed by mills and factories to produce primary products such as crude palm oil, latex and rubber. Further

downstream, refineries, oleo-chemical plants and other types of manufacturing facilities produce value-added products such as cooking oil and oleo-chemicals.

Many plantation-based companies in Malaysia are vertically as well as horizontally integrated. Horizontally, many companies have established presence in the form of plantations and processing facilities in many parts of the country as well as overseas.

As such, the plantation industry is characterized by a broad range of activities. A significant challenge faced by the Management of plantation-based companies is the need to integrate and coordinate related activities in an effective and cost-efficient manner across wide geographical areas.

2 CHALLENGES

Similar to management challenges of any other businesses, information plays a crucial role in the plantation industry. Key information needed by the management of plantation-based companies include

Production data - to monitor productivity and for booking sales

Cost data - to monitor and manage costs

Operational data (e.g..maintenance work carried out) - to monitor maintenance and upkeep activities

Logistics data - to monitor inventories and deliveries

Sales data - to monitor prices against quantities booked and shipment requirements

In view of the diversity in activities, many different types of information systems are used to manage plantation-based companies. Common ones include

Plantation management system - manages labour, contracts, collections, upkeep and maintenance, vehicles and inventories in plantations

Oil mill management systems - manages the operations of oil mills including production efficiency and receptions/ deliveries, commonly incorporating weighbridge and SCADA systems

Palm oil refinery systems - manages the operations of palm oil refineries including process control, inventory, consumptions, outputs and logistics

Apart from the industry specific information systems mentioned above, other common information systems such as accounting, human resources and payroll are also used.

Data integration and sharing across such a number of systems remains an unresolved need in many plantation-based companies. This is still commonly satisfied through manual means, employing many staffs, long processing cycles and high costs characterized by error prone manual processes. There are ample prospects to reduce costs and increase efficiency in this area.

Challenges that need to be overcome include

Poor data communication infra-structure - especially faced by plantations in remote locations, these are typically resolved by subscribing to satellite communications

Technology differences - differences in data processing technologies make process and data integration a complex affair

Data standards differences - differences in data standards adhered by different systems (e.g. reference codes, data formats)

Fortunately, technologies to overcome these challenges are widely available and have proven themselves in resolving many similar problems.

3 DATA WAREHOUSING

In this section, we describe the solution to resolving data integration challenges faced by the plantation-based companies using data warehousing technologies.

3.1 WHAT IS DATA WAREHOUSING

Data Warehousing, Business Intelligence, and Business Analytics are terms commonly used to describe information systems that integrate data from multiple data sources and combine them to provide management with a unified view of the enterprise. This unified view is commonly referred to as the single version of truth.

Strictly and technically speaking, the terms Data Warehouse, Business Intelligence, Business Analytics refer to certain specific components used to achieve the overall objective of providing the single version of truth. When technical people such as software engineers use such terms, they typically refer to these specific components rather than the overall solution. However, these terms are more commonly used by business users in a loose sense, vaguely referring them to the entire system than specific components.

3.2 COMPONENTS OF DATA WAREHOUSE

A data warehousing system consists of the following components:

3.2.1 DATA SOURCES

Data sources refer to those systems that provide data to be integrated by the data warehouse. For a plantation-based conglomerate, such data sources may comprise of

the plantation system

the oil mill/ rubber factory system

the logistics/ weighbridge system, if these are kept separate from the plantation system

the head office/ regional office accounting system

the human resource and payroll system

the oil refinery system

other systems used by downstream manufacturing interests, such as MRPs (Manufacturing Resource Planning systems) and ERPs (Enterprise Resource Planning systems)

Data sources and normally purchased and implemented independently of the data warehouse. In fact, many plantation-based businesses invest in these data sources first, many years before they invest in a data warehouse.

3.2.2 EXTRACTION, TRANSFORMATION AND LOADING SYSTEMS (ETL)

ETL systems refer to subsystems that are normally designed and built during the implementation of the data warehouse itself. However, strictly speaking, the ETL systems are separate from the data warehouse. They

may be supplied by the data warehousing system itself or they may be third party tools that are purchased from another party than those providing the data warehousing solution.

The role of ETL systems is to

extract relevant data from the source systems

transform the data into suitable formats for unification by the daa warehouse

load the data into the data warehouse

ETL systems typically use Data Staging areas to perform these functions, although in simple implementations, the Data Staging area may not be necessary. The Data Staging area is used to store data temporarily while it is being tranformed for loading into the data warehouse.

Some data warehousing tools claim that they do not need the ETL operations since they can accept data in the raw int the data warehouse. These claims are typically true for simple data warehousing requirements, but the absence of a proper and specialised ETL tool can hamper the data integration tasks complex environments.

3.2.3 DATA WAREHOUSE

If the term is used in a strict, technical sense, Data Warehouse refers to a specific component in the overall solution. A data warehouse is a database designed and developed to hold a unified view of data collected from the multiple data sources.

There are many schools of thought on how data warehouses are to be designed, ranging from proponents of a completely normalised databases to proponents of denormalised databases. (Normalisation of a database refers to how the data warehouse should be designed. There are many degrees of normalisation recognised by the industry. The terms Star Schemas and Snow Flake Schemas are also commonly used to describe the varying degrees of normalisation employed). Although there are many different opinions on the method of implementation, best practices have emerged in the data warehousing industry recognising the suitability of different components for different needs and requirements.

The ETL processes described in the previous section take into account the design of the data warehouse, which dictates how data is to be transformed from the data sources in order to be loaded into the data warehouse in an integrated manner with one another.

3.2.4 OLAP CUBES

OLAP or Online Analytical Processing refers to the design of databases that are special for data analysis purposes. Originally, most databases are design to store data supporting transaction processing functions. These databases are called OLTP or Online Transaction Processing databases. Although OLTP databases can fulfill data analysis requirements, they are not efficient in performing analysis on large amount of data. OLAP databases are specialised for these analysis purposes. They store data at multiple levels of summarization, which can be precalculated, in a way that provides efficient data slicing-and-dicing functions. Slice-and-dice refers to the ability to view a set of data from multiple angles and at different levels of summarization. Other commonly used terms that refer to the capabilities made efficient by OLAP databases are drill-down, drill-up

and drill-through. These refer to the ability to see the same data at different levels of summarization and to link to related data seen from a different perspective.

OLAP databases are commonly referred to as OLAP cubes because the Rubik Cube provides a suitable (although technically inaccurate) visual representation of how users can manipulate data stored in an OLAP database.

Data from the data warehouse is used to build these OLAP cubes at suitable intervals. What is a suitable interval depends on the users of the data. Update intervals and regularity can range from long periods such as annually to very short periods such as immediately.

Some Business Intelligence tools are capab;e of providing data slice-and-dice capabilties without using OLAP databases. These are typically what is known as in- memory OLAP processors. Although these tools are relatively easy to develop on, there is a limit to the amount of data and the number of users they can support without the use of dedicated OLAP databases. OLAP databases also provide a greater degree of configurability and control compared to these simple tools.

3.2.5 BUSINESS INTELLIGENCE

So far, all the components described above are not visible to the end user. The ETL tools and the databases are used by developers and systems administrators to process the data and to store it. In order to use the data, end users need another set of tools to present the data from the databases for their consumption. The set of tools that present the data to end users are commonly known as Business Intelligence tools.

These tools typically provide ways and means to present the data visually, using charts and dashboards. They are also interactive, meaning that users can click on them to slice-and-dice the data or to drill-down, drill-up or drill-through. Furthermore, they allow users to select filter the data to show only what they want to see.

Business Intelligence tools also typically provide means for users to perform their own ad-hoc enquiry and to present the results of these enquiries as tables of data or as graphs. Some Business Intelligence tools also support the presentation of Key Performance Indicators in performance dashboards, and the use of methodologies such as the Balanced Scirecard.

Many Business Intelligence tools are web-based, meaning that they are installed on the server but may be accessed and used from any part of the world with a connection to the server. Other Business Intelligence tools are incorporated as add-in modules to popular office software such as Microsoft Excel, enabling the data to be used and further manipulated in spreadsheets.

3.2.6 REPORTS

Business Intelligence tools provide flexibility of access to data for analysis purposes. However, the presentation of data on Business Intelligence tools are normally unsuitable for printed reports. Users typically have pre-formatted reports that are printed and distributed at regular intervals to management before they embark on data warehousing projects. Even with the implementation of data warehouses which gives management the ability to analyse their data, the typical management team normally wants to use the printed reports that they are used to and comfortable with.

This need is typically met through the use of reporting tools to present data that is more in line with the printed report format.

3.3 BENEFITS OF DATA WAREHOUSES

Synchronising activities in a large plantation-based group is a major activity undertaken by many staff members. Losses due to uncoordinated actions between the plantation department, the marketing department, the manufacturing facilities and the logistics department can erode any hard earned profit margin in the competitive plantation industry.

Data warehousing technologies promise to deliver the ability to combine and share information within a plantation-based group in a cost-effective manner. Oft-mentioned benefits of having a data warehouse are

Access to a single version of truth by integrating data about various aspects of the business

Reduce cost, time and error of integrating/processing data and producing reports to summarise the state of the business

Keep an archive of data in a form suitable for analysis and trend study

Improve decision making and business monitoring

Develop informed business plans on the strategic and tactical levels, and monitor the effectiveness of their implementations

Obviously, the benefit of having a data warehouse is the ability to access a single version of truth about one's plantation-based business. However, the construction of a total, enterprise-wide data warehouse is a complex, costly and time consuming exercise. The rewards of such a big and major exercise may be worth it, but it will not be realised immediately.

A more practical way of realising the benefits of data warehouses is to develop the data warehouse incrementally by prioritising and integrating data on pain spots where the integration and ability to analyse data is needed most immediately. The gradual and step-by-step implementation of such a data warehouse is focused on fulfilling the needs of that each paticular area first. These smaller scale data warehouses are

normally known as data marts. Data marts provide quick wins and higher ROI on data warehousing investments than time consuming, enterprise wide data warehouses.

Different plantation-based businesses have different priority areas. Some pf the areas that a plantation-based business may wish to prioritise for data marts include

Automatically collect and present daily harvesting data from a large number of plantations so that the Marketing Department know how much product to commit to sales

Analyse sales contracts data to determine whether sales and pricing practices can be improved and who are the best customers to focus on

Monitor upkeep and maintenance activities to ensure that plantations are maintained at an optimal level to support sustainable production

Monitor cost of maintenance and upkeep vis-a-vis on the ground conditions and level of activities carried out

Monitor efficiency of production and extraction/ conversion rates achieved and determining factors influencing them

Monitor cost of production on a continual basis

Monitor use of labour and their cost

Monitor purchasing activities, material consumption and optimal inventory levels

Once the infrastructure costs of establishing a data warehouse had been committed, the incremental cost of developing data marts is easily recovered from benefits gained from insights achieved and cost savings from reduction of manual labour in processing the data.

CONCLUSION

Data warehousing promises productive gains for managing data consistency and integration issues in the plantation industry with its complex, multiple data sources and systems. However, the best approach for the industry is to prioritize areas where data need to be integrated and presented faster for monitoring, analysis and action rather than to take a big bang, do everything at once approach.