ASWEC 2008Slide 1 Construction by Configuration: An opportunity for SE research Prof. Ian Sommerville St Andrews University Scotland.

ASWEC 2008 Slide 1

Construction by Configuration: An opportunity for SE research

Prof. Ian SommervilleSt Andrews UniversityScotland

ASWEC 2008 Slide 2

St Andrews

• Small Scottish town, on the north-east coast of the UK

• Home of golf• Scotland’s oldest

university (founded in 1413)

• Small university focusing on research and teaching excellence

ASWEC 2008 Slide 3

Software engineering 2008• Over the past 10 years or so, there has been a

radical shift in development strategy for large-scale organisational information systems

• Instead of developing software by programming in a conventional language, the primary development mode is now based around the reuse of off-the-shelf systems such as ERP systems or generic systems (COTS) for a specific application area

• Existing approaches to software engineering have to evolve to take into account this approach to systems engineering

ASWEC 2008 Slide 4

Commercial Off-the-Shelf Systems (COTS)

• COTS-based reuse relies on adopting a generic system that has been developed to deliver some business function and adapting that systems to the needs of a particular organisation

• Increasingly, new systems are developed by integrating several existing COTS, sometimes with different owners. Systems are becoming systems of systems

ASWEC 2008 Slide 5

COTS-based reuse• COTS-based reuse is based around systems

for a specific application domain (e.g. patient information system in healthcare).

• This incorporates generic functionality and abstractions from the domain which are configured for each specific customer. COTS are designed for configuration.

• This can be a (very) long-term process- The current Spanish en-route air traffic

control system is being reconfigured for use in the UK. The process is anticipated to take 6 years.

ASWEC 2008 Slide 6

Construction by configuration (C-b-C)

• COTS and ERP systems include a set of reusable abstractions that represent an application domain

• The reusable abstractions have to be configured to adapt them to their local circumstances of use

• This can range from simple parameter setting through the definition of business rules

• Sometimes, configuration may require new, special purpose component development.

• C-b-C also may also include configuring the process as well as configuring the software that is the way people work has to change to meet the model assumed by the COTS system

ASWEC 2008 Slide 7

Configuration• I use the term ‘configuration’ to cover both

customisation and adapting software to a specific execution environment

• Adapting the system to reflect:- The specific needs of a customer (e.g. a hospital);

- The requirements of a user or group of users (e.g. maternity unit or physiotherapy or both);

- Interaction with other systems;

- The characteristics of the platform on which the software executes.

ASWEC 2008 Slide 8

Configuration activities

• Selecting required system modules • Defining a data model• Defining business rules• Defining workflows• Defining external interactions• Defining the user interface• Defining reports produced• Setting platform parameters• Data re-engineering• Re-defining business processes

ASWEC 2008 Slide 9

A patient management system

• We have followed the deployment of a patient management system (PIMS) for mental healthcare in Edinburgh, Scotland

• Based around a generic COTS-package developed for hospitals in England

• This was designed to be adapted for supporting different kinds of clinic including mental healthcare

• Scotland has its own legal system and laws and healthcare is a devolved responsibility. The Scottish government sets its own targets and priorities for healthcare

ASWEC 2008 Slide 10

Procurement issues• A major influence in choosing the particular

COTS system was that it offered the opportunity for hospital managers (rather than clinical staff) to control how information was recorded

• The government placed a tight deadline on hospitals for reporting against a set of targets

• There was little time to carry out a detailed comparison of alternatives and this system had already been successfully deployed elsewhere

ASWEC 2008 Slide 11

Software engineering

• Configuring the data model required for a particular set of clinics

• Configuring the menus for the particular type of clinic and the patient information that had to be recorded

• Configuring the reports to be generated by the system

• Configuring the rules that should be applied to the system data

ASWEC 2008 Slide 12

Issues and problems• Objectives conflict

- Management objective - provide reports against a set of headings defined by the Executive

- Clinical objectives - record patient information according to clinical classification

• Invalid configuration assumptions- The language for defining the rules of the system

was not expressive enough to cover the requirements of Scots law regarding the forced detention of patients who were a danger to themselves or others

• Failure to configure the process- Local process differences meant that different clinics

recorded different information about patients

ASWEC 2008 Slide 13

The C-b-C process for COTS

• There is often no clear distinction between specification, design and development

• Systems are rarely completely configured before being put into use. The configuration process continues as the system is integrated with operational processes

• There can be extensive (uncontrolled) “user” configuration of the system

• The expectations of system stakeholders have to be configured to reflect the realities of he system

ASWEC 2008 Slide 14

Process characteristics• Two-stage system requirements process

- Identify general business requirements to chose a reusable system;

- Identify specific requirements from the settings where the system will be used.

• Business process redesign

- The business processes have to be redesigned to r4flect the constraints of the selected system

• Co-design of process and software- May be more active stakeholder involvement in the

development process.• Limited testing - often post-deployment’ testing to

iron out problems• Good practices such as configuration management,

reviews, etc. are very difficult to implement

ASWEC 2008 Slide 15

Choosing a COTS system• The decision on which COTS system to use is

rarely a transparent process, based on a detailed analysis of the requirements in a specific setting

• Issues that affect the decision include:- Political issues;

- Platform issues;

- Cost and schedule issues;

- Availability of expertise;

- Prejudice!

ASWEC 2008 Slide 16

Co-design

• For C-by-C, separating requirements and development/deployment is unlikely to be successful

• Requirements compromises are essential because the stakeholder’s real needs need to be matched with what the system actually provides

• Configurability makes co-design, with stakeholder involvement in the design process, much easier

- However, users may misunderstand the (lack of) system flexibility

ASWEC 2008 Slide 17

System testing

• Testing is a particular problem for COTS-based systems

• Systems are not designed for running automated test suites

• There is no specification that can serve as a basis for deriving tests

• Failures are often not observable by testers as they are failures in process support that can only be recognised by users

- Problems that arise are often a consequence of interactions between the process of use and the system rather than system failure.

ASWEC 2008 Slide 18

System evolution• Handover process from consultants to local IT

staff. Change of system ‘ownership’- Limited expertise with system - may be managers

rather than software engineers

• Problems of change management exacerbated because of the system configurability

- Increasing system entropy as local configuration changes are implemented

• Evolution of underlying COTS system outside control of system owners

ASWEC 2008 Slide 19

Configuration problems

• Understanding the configuration possibilities- Knowing what can be configured is not easy,

especially if more than one product is included in the system

• Understanding how to configure a system- Typically, configuration requires both business

knowledge and technical knowledge

• Predicting the consequences of configuration decisions

- It is often difficult to understand how changing a configuration will affect the use and behaviour of a system while it is in operation

• Understanding how a system is configured

ASWEC 2008 Slide 20

Multiple configuration possibilities

• Ways in which MS Word can be configured (that I’m aware of)

- Preferences screen- Customisation screen- Organiser screen- Definition of templates- Definition of styles- Definition of macros- Inclusion of add-ins (e.g. Endnote)

ASWEC 2008 Slide 21

Understanding how to configure

ASWEC 2008 Slide 22

Configuration predictability• If you change a program, it is generally

possible to hypothesise how this will affect the behaviour of the executing system

• When you change a configuration, the relationship is less obvious

• Change becomes a process of ad hoc experimentation. Gurus evolve who can suggest what to do but who can’t explain why it should be done that way

ASWEC 2008 Slide 23

Understanding the configuration

• Once a system has been configured, how can others understand the configuration.

• Requirements/design/code traceability is difficult enough in conventional systems but much harder in COTS where:

- Requirements are not properly documented as a result of the co-design process;

- Understanding requires knowledge of the COTS system + knowledge of the configuration.

• Change costing and impact analysis is difficult- E.g. in the healthcare system we looked at, it was

originally estimated that changing menus would take a few days. It ended up taking several months.

ASWEC 2008 Slide 24

Research challenges

• Design for configuration- Design principles for configurability

- Configuration visibility

- Configuration description

• Configuration engineering- Knowledge management for system engineering:

- System modelling

- Methods and tools for COTS-based system testing

- Adapt supporting software engineering processes for C-by-C

ASWEC 2008 Slide 25

Principles for configurable design

• We need a coherent set of tested principles in this area to guide design and implementation

• Possible examples of principles- Principle of visibility - make configurations explicit?

- Principle of low coupling - reduce dependencies across configurations?

- Principle of scalability - separate configuration of system deployment from configuration of functionality?

- Principle of localisation - localise volatile configuration entities?

• The diversity of different approaches to C-b-C mean that identifying unifying principles across different systems is very difficult

ASWEC 2008 Slide 26

Configuration visualisation• In most current systems, configurations are

opaque - it is difficult or impossible to view the configuration as a whole, at different levels of abstraction or to navigate through the configuration space

• There is a need for tools that allow engineers to see the ‘configuration state’ of a system and to explore dependencies across that state

- What configuration information is most useful to system developers?

- What is the best way to present configuration views to system developers?

ASWEC 2008 Slide 27

Configuration description• Configuration policies

- There is sometimes a wide gap between organisational policy (especially, security policy) and the realisation of that policy as a configuration

• Configuration compilation- Methods and tools are needed to allow higher level

configuration specification and the automatic generation of a configuration from that specification

ASWEC 2008 Slide 28

Knowledge management• Configuration engineering requires a close

collaboration between different classes of system user, business analysts and system developers

• Our experience has been that major problems arise because of failure to communicate across and within the different groups in an organisation

• Better knowledge management support is required to capture, classify and share knowledge about assumptions, problems, the organisation and the system itself

ASWEC 2008 Slide 29

Configuration modelling• Modelling notations such as the UML are focused

on developing object-oriented models of the system

• While some UML models (such as Use-Case models) can be used for configuration engineering, the majority of the UML models are either inappropriate or cannot be applied in a natural way when configuring a generic system

• What domain-oriented modelling techniques are most useful, how can they be used and adapted to support understanding and analysis of systems being built using CbyC?

ASWEC 2008 Slide 30

System testing• Workflow testing

- How can workflow models built into the system be tested against the reality of work?

• Test management- How can test coverage be assessed and improved?

- How can regression testing be supported?

• Test-first development- Is this a valid approach for systems developed using

CbC?

ASWEC 2008 Slide 31

Supporting processes• Users may be partly responsible for parts of

the configuration process. This causes major problems with supporting processes

- Configuration management

• Existing tools are oriented to source code

- Change management

• Can users be stopped from making ad hoc changes? How do users find out about changes?

- Quality management

• What does a ‘high-quality’ configuration mean?

ASWEC 2008 Slide 32

Conclusions• Construction by configuration is a reality for

modern business software development

• Political, technical and business factors mean that this approach will be increasingly used for all types of system

• Our current ‘ad hoc’ approaches are not good enough - we need to adapt software engineering methods and techniques for this approach.