-
The Executive Guide to Slashing Software Development Costs
How to reduce software development costs by 80%.
Utopia? No just the modernisation of
out-dated software development methods.
An eBook by
Ken Galbraith
A no-holds-barred look at current IT practices and the impact
they have on
business
By k.g
-
Copyright 2013 by K M Galbraith. All rights reserved.
This eBook may be downloaded and circulated in its published
form.
No extracts may be used without the authors permission.
Disclaimer:
This book is based on the personal experiences of the
author.
Outcomes and productivity gains may vary in other
organisations.
Experiences relating to Circatec clients or other parties have
been changed to the extent
required to avoid identifying the other party.
All research material is from publically available sources.
Trademarks:
CA, CA Plex and Plex are trademarks of CA Technologies.
XSOL InOrder is a trademark of XSOL Limited.
Additional copies:
Copies of this eBook can be obtained from Circatecs web
site:
www.circatec.com.au
-
Contents
Introduction 1
Software excellence 2
The current state 3
The traditional software development life cycle (SDLC) 3
The Agile software development life cycle (SDLC) 4
Over reliance on methodology 5
Cant I just send it offshore? 7
The fundamental flaw 7
There is a better way 8
Automated software development tools 8
Application development 8
Rules Engines 10
Benefits of automated software development 11
Managing automated development projects 13
Requirements gathering 13
Architecture and design 14
User interface and patterns 15
Development teams 16
Planning automated software development 17
The benefits are enormous 18
It doesnt stop here 19
The larger the development the greater the savings 19
Compelling case for change 20
How corporate IT resists change 21
Common practices used to maintain control 22
I can do it just as quickly by hand! 24
The CEOs toolkit 24
Bringing about change 25
Structure 25
Existing systems 26
What about work-in-progress? 26
Resistance to change 26
Be prepared to start again 27
What will it take? 28
Change will occur when change is demanded. 28
About the author 29
-
Page 1
Introduction Fundamentally the way software is developed and
delivered hasnt changed in 40 years.
Over the last ten to fifteen years tools and methodologies to
dramatically streamline software
development have been maturing in function and
sophistication.
And yet the software industry has been able to resist change. It
is reasonable to say that no
other industry has been able to resist change for so long and
get away with it.
The impact on an organisations ability to deliver, adapt and
take advantage of the rapidly
changing commercial and legislative environment is severe. In
many instances IT dictates
corporate agility and speed to market.
While boards and CEOs require system agility and quick response
to changing requirements,
their IT department often remains inflexible and resistant to
change.
IT departments are empowered and trusted by their organisation
to provide software and
technology solutions for the benefit of the organisation. Far
too often they fail this trust by
blocking new ways and technologies, often using dismissive
statements that, if challenged,
cannot be justified mostly however, they go unchallenged.
Too often Corporate IT is making key and fundamental decisions
that have a significant
impact on the business and there is no mechanism in place to
hold them to account. The
business managers most affected are seldom engaged in these
decisions and are, therefore,
unaware of the alternative solutions available to them.
In 2005 Gartner Group (a leading technology trend forecaster)
forecast that 40% of IT job roles
as they existed then would be lost to automation. They went on
to forecast that more
American IT jobs would be lost to automation than to
outsourcing. The most significant
forecast was that between 2005 and 2010 the productivity of IT
workers would increase by a
factor of 10.
Today, some 8 years later, the uptake of automation tools and
agile methodologies has
effectively been ignored (the generous view) or blocked (in my
experience) by the broader IT
community.
The impact is much more than just high cost and long delivery
cycles for software
development, there is a very significant impact on operating
costs when systems are out of
step with organisational needs.
This book looks at the current state of software development and
then explores commercially
available and proven alternatives before discussing how to bring
about significant change.
Let me clear here. The change I am talking about is a
fundamental shift in the way we design,
develop and deliver software applications.
If we keep doing the same things we will continue to get the
same results.
If we want to change the outcome we must change how things are
done.
-
Page 2
Software excellence We need to understand what constitutes
excellent enterprise software in order to objectively
look at the tools and methodologies used to develop an
enterprise application.
User experience The software must be easy to use and enhance the
users undertaking
of their everyday job functions.
This means the software must be intuitive to use and reflective
of the
enterprise business processes.
In high work load environments the software is so well aligned
to the
process that user interaction with the application is highly
efficient.
Value add The software must add value to the organisation by
providing a more
efficient workplace, eliminating error and waste, enhancing
the
customer experience, reducing corporate or financial risk, and
other
tangible organisational benefits.
Flexible The ability to adapt to changing organisational demands
is critical to
the long term value of the software.
Reliable The software must be available and run consistently day
after day.
Other than infrastructure problems the software availability
should be
100%. Transactional processing must be so consistent that
once
audited subsequent transactions are accepted as correct.
Secure Security in this context is related to the validation of
transactions to
ensure data and transactional accuracy. Protections around
the
maintenance of data to prevent malicious transactions or
accidental
data errors. Incoming electronic data files must pass
through
validation filters before being posted to the database.
Cost effective The cost of development must pass a business
test. Implementation of
the software must be efficient and the ongoing cost of
maintenance,
including changes and enhancements, must be relative to the
value
the software is delivering to the organisation.
Scalable The ability of the organisation to grow and/or expand
its products and
services must not be hampered by limitations in the softwares
ability
to process the increased volumes of transactions.
The tools, standards and methodologies adopted by corporate IT
and their service providers
must be totally aligned to the delivery software excellence as
judged by the organisation
they serve.
The sole measure of successful software is the value it brings
to the organisation.
-
Page 3
The current state The traditional software development life
cycle (SDLC)
The waterfall approach to software development has been around
since the early 1970s and is
still widely used today. It is a sequential design and
development methodology as shown in
this flow chart.
The waterfall model assumes each stage is complete before
proceeding to the next.
RequirementsAnalysis &
specificationDesign
Architecture & components
ImplementationConstruct
components
VerificationTest & install
MaintenanceModifications &
defects
Because Waterfall is sequential it is impossible to go back. The
assumption is that the
requirements gathering is an accurate and complete reflection of
the requirement (which it
seldom is) and the design is an accurate reflection of the
requirement.
The advantages of Waterfall are:
It enforces discipline, every stage has a start and end and
progress can be monitored
through the use of milestones.
There is considerable documentation which minimises the impact
of staff turnover
on the project.
The client (external or internal business unit) knows what to
expect. The detailed
specifications describe what the software will do on
completion.
The disadvantages of Waterfall are:
Change of requirements is difficult to manage due to the large
amount of
documentation.
Once a stage has been completed it is difficult to go back.
If a specification error or change is required the project must
go back to the
beginning.
Bugs and design flaws arent identified until after the entire
application has been
written.
-
Page 4
The disconnect
As waterfall moves from one phase to another it engages
different disciplines and skills. Each
with their own language and terms of reference.
This creates a problem as the language moves from end user
terminology to technical or
system terminology, compounded by the separation of development
from the client.
What often happens is:
Clients domain expert talks to
business analyst
Business analyst develops
requirements specification
Systems analyst interprets
requirements into development specification
Development specification goes
to programmers to write the software
Business analyst talks to systems
analyst
Client doesnt fully accept the software and the loop starts
again!
Developed software released
to the client for acceptance
Returns the specification to the client for approval
The developed software is the Programmers interpretation of the
System Analysts
interpretation of the Business Analysts interpretation of the
Clients requirements
The client is now out of the loop!
Programmers are now completely isolated from the client
Highly technical document that would be difficult for the client
to understand
Programmers interpret the
development spec into program code
The Agile software development life cycle (SDLC)
To overcome the shortcomings of Waterfall and its variants,
Agile methodologies started to
emerge in the 1990s.
The objective of Agile is to replace the sequential methodology
of Waterfall with an iterative
methodology using small cross functional teams rapidly
delivering small, useful pieces of
software.
These deliveries happen at regular intervals enabling user
testing and evaluation throughout
the development cycle. By developing small pieces of software,
changes can easily be
accommodated.
As delivery is an iterative process client cooperation and
feedback is encouraged. Agile
supports the use of prototyping or modelling where appropriate
to ensure the delivered
software meets the clients requirements.
A key aspect of Agile is the minimisation of documentation.
Requirement specifications are
just enough to communicate the requirement to the development
team. This may be as
little as notes and flows on a whiteboard or a spreadsheet
demonstrating financial
transactions and postings.
-
Page 5
While this may be disturbing to some, if we accept the only
measure of successful software is
the benefit it brings on completion then we must also accept
that all documentation used
during development is obsolete the moment the software is
accepted and put into
production.
Each iteration
Iteration test & accept
Iteration release
Agile planningRequirements
Define
Develop
Work breakdownPriorities / dependencies
Iteration schedule
Progressive testing enables early acceptance
of the completed
Just enough documentation for the development team to
build the iteration
User engagement in assessing the iteration or
prototype prior to relaease Feedback
Agile will deliver significant productivity gains, however it
will only reduce the time and cost of
software design and specification. This may equate to savings in
the 10-20% range.
You can be too Agile
The problem with Agile is that it can easily be taken too
literally. In a development where a
single team builds a function or even the entire application,
Agile can deliver significant
productivity gains.
In a larger development with multiple teams working to produce a
single application, an
overriding architectural design and development standard must be
in place. Without this the
teams will spend unnecessary time rebuilding prototypes, or
reworking early developments,
in order to provide a single cohesive application.
Ensuring the design work has taken place will ensure the
development delivers the benefits
available in Agile.
Over reliance on methodology
We cannot discuss software development without a critical
discussion on project
management methodologies.
The vast majority of software projects follow a highly
structured project management
methodology based on the waterfall approach to software
development.
It is worth noting that many highly structured methodologies are
designed for, or evolved
from, construction, engineering, mining, etc. where it is
entirely appropriate to have rigid
controls around documentation, certification and change
management approval. Whether all
these controls are appropriate in software development is
questionable.
-
Page 6
Question: Do you need the same approval process to add a field
to a screen and underlying
data table as you do to authorise an engineering change on a
bridge? The exponents of these
methodologies say yes, whereas reality says no.
I have some specific issues with highly structured project
management methodologies in
software development.
They perpetrate the disconnect between the business unit and the
development
team that is inherent in Waterfall.
The methodology becomes the purpose, not the outcome the timely
delivery of
usable and cost effective software.
They add a huge amount of unnecessary overhead, cost and delay
to a software
development project.
Any prescriptive methodology enables low competency operatives
to hide behind
the methodology.
Make no mistake no methodology will guarantee a successful
outcome. However the more
structured and bureaucratic it is, the more certain you can be
that any failure will be
expensive, well documented and a long time in the making.
Flexibility is the key
Very few methodologies are absolutely prescriptive. The vast
majority, including Prince2 and
APM BoK are frameworks to be tailored to a specific project.
In the right hands, they provide a valuable and practical
framework for managing a project.
I have worked with some extremely capable project managers. They
all have common traits,
in that they are people managers and they are focussed on the
destination not the journey.
Plus they have domain knowledge either of the industry they are
working in or the
technology they are implementing (which implies a degree of
industry knowledge as well).
Problems arise with highly structured methodologies when the
project manager is a
practitioner of the methodology. Instead of guiding, mentoring
and smoothing troubled
waters to get an outcome, they focus on the methodology
demanding strict adherence,
regardless of practicality or relevance.
This can also occur when a service provider has been retained to
provide project management
resources. They need strict adherence to the methodology
because:
They sold the complete project management package to their
client.
It enables them to put in methodology practitioners instead of
domain experts.
They can forecast their fee revenue.
If you are going to outsource project management, ensure you are
getting a project manager
not a methodology practitioner.
-
Page 7
Cant I just send it offshore?
I have yet to meet anyone who has had a good experience with the
offshore development of
an enterprise application.
Generally the problems have nothing to do with competence. The
problems usually come
down to two issues.
Communication of requirements.
Requirements must be very precise because of the geographical
and time zone separation
between the business analysts and the developers.
I have already demonstrated the disconnect and lack of precision
inherent in the
conversion of business requirements into programming
specifications and then into
delivered software. This disconnect is exacerbated with offshore
development.
Lack of domain knowledge.
Inherent in most specifications is an assumption that the
programmer understands the
domain. For example, in specifying financial transactions, a
system analyst may assume
the programmer knows the transaction must be validated and
doesnt even think to
specify it.
The absence of domain knowledge anywhere in the development
cycle will lead to
misunderstanding, errors and exclusions.
In a traditional waterfall development there are inevitably
disagreements during testing when
the analyst rejects a function and the programmer argues they
programmed to the
specification. The frequency and severity of these disagreements
and the associated rework
will increase significantly in offshore development.
Unless you are very lucky, any savings in development charge
rates will be consumed in travel,
communication, rework and dispute resolution.
The fundamental flaw
Regardless of methodology the fundamental flaw is the lack of
automation in the
development of application software.
A typical software development department consists of a team of
programmers writing code
just as they did 40 years ago. Sure, the programming languages
have changed but its like
giving a carpenter a nail gun. Hes a little quicker but hes
still driving in nails.
While Agile may deliver savings in the 10-20% range, it is a
fraction of the savings to be made
by automating software development.
-
Page 8
There is a better way We see how the dependence on the manual
development of software, regardless of
development methodology, fails to meet the cost restraints and
agility demanded by modern
business.
There is a much better way to develop enterprise software
applications that will deliver:
Cost reductions of 70 80%.
Extremely flexible software solutions.
A high level of client engagement in the development cycle.
A significant reduction in the elapsed time to specify, develop
and implement a
solution.
Bug free software.
Massive reductions in the size of your IT department.
Automated software development tools will deliver this and much
more.
Automated software development tools
Typical characteristics of automated software development tools
are:
A structured modelling framework that enforces standards and
consistency.
Definable patterns that establish consistent user interfaces and
behaviours.
Inherent functions to speed up model building.
An extremely high level of component reuse.
The automatic generation of program code.
There are two groups of tools that between them will virtually
eliminate the need for any
manual program development:
Application development tools.
Rules and decision engines.
The strength of these tools is the ability of the developer to
work directly with a domain
expert, creating a model of the requirement and generating the
software code without the
use of manual programming.
The structure, models and patterns that characterise automated
software development tools
enforces the standards and consistency essential to deliver
software excellence.
Application development
Circatecs requirement was for a toolset that enabled us to build
commercial grade enterprise
applications. This means large server based applications
delivered via an internal network,
remote desktop or via a web browser without the need for a
secondary web based layer.
As a software developer we were also reluctant to limit
ourselves to a single platform and
database.
-
Page 9
There are a number of excellent automated application builders
on the market but CA Plex
was the only automation tool we found that met our broad
criteria.
Plex originated in the 1990s as IBMs Synon product. Computer
Associates acquired it some
years ago and it has continued to evolve into what is arguably
one of the best tools currently
available. Plex applications have been proven in very large
enterprise systems so we know
they are scalable.
CA Plex is multi-platform. It can generate C#, C++, Java or RPG
III & IV code to run on Sequel
Server, Oracle or DBII databases.
Plex provides enormous productivity gains, enabling us to build
applications in 15 20% of the
time it would take to manually program the application.
Some of the key aspects of Plex are:
Patterns.
There are two classes of patterns in Plex, the user interface
(UI) and process patterns.
The UI patterns provide a consistent user interface across the
entire application. Typically
we would have three or four patterns for use in specific
functions, for example:
High volume data entry may be a single screen so all the fields
pertaining to the
function are displayed in one large panel.
The enquiry screen however may reflect the needs of a call
centre and display core
information with links to every other related area of the
system.
Functional patterns will define a regularly occurring pattern.
For example, in a financial
system the standard process may be to enter and save a
transaction, have another staff
member confirm the transaction and, on confirmation update the
underlying ledgers. By
building this as a pattern we enforce the process and where the
confirmation cycle isnt
required it is turned off.
The point of patterns is that they enforce consistency across
the entire system but they
also save a huge amount of development time. When a developer is
building a function
the first thing they do is select the pattern and the user
interface is immediately inherited.
Data base generation.
The database is generated from the model. Developers do not have
to build a database
before they build the application.
This is why, in a Plex world, we can generate the application to
run on SQL, Oracle or
DBII. We simply tell Plex what database to compile when we run
the model.
Models.
To call or initiate another program its as simple as naming the
application in a link in the
model and Plex generates the physical links.
Most business environments have situations where exceptions
trigger a different process.
In the Plex model the developer simply defines the If / Then
statement.
The whole point of these examples is to illustrate how quickly
complex or repetitive
programming tasks can be defined in a model without any manual
programming.
-
Page 10
Plex generates the program code in the designated language and
compiles the designated
database.
Because the code is machine generated there are no bugs, so
testing is limited to ensuring
that the model is a true reflection of the requirement. If there
are design errors or functional
gaps the changes are made in the model and a fresh set of code
is generated. This ensures the
software code is always clean and free of the redundant code
that builds up in manually
programmed software.
Rules Engines
A rules engine provides a graphical model in which to define the
business rules. Having
defined the rules flow, the formula for each branch or decision
node are defined and the
model tested.
All good rules engines will have inbuilt test capabilities and
will be self documenting.
The rules engine will automatically generate the model into
program code to be deployed into
the host environment.
In an enterprise environment, rules engines are generally
deployed to:
Add functionality that is difficult or expensive to develop in
the host application. Rules
engines can play a valuable role in extending the life of legacy
software by adding
functionality that would be prohibitive to develop on an
out-dated platform.
Encapsulate legacy products. Wealth management, superannuation
and insurance often
have closed products they are legally bound to maintain until
there are no more clients in
the product. A rules engine is ideal to define all the product
rules and calculations into a
platform independent rules engine that can be mapped to a host
system. For example a
pension scheme may take member data from the host, apply the
pension scheme rules
and return the pension payment to the host to process the cheque
or EFT. If the fund
migrates to a new administration system, the rules engine
remains unchanged and is
mapped to the new system.
Data validation and rectification is a perfect domain for a
rules engine. Far too often
organisations rely on scripting languages or spreadsheets both
of which are time
consuming to develop, difficult to test and impossible to
guarantee their ongoing
integrity. The version control of a good rules engine, combined
with the fact that the
generated code is maintained by changing and regenerating the
model, provides
certainty over the rules and calculations being applied to the
data.
As with application builders such as Plex, a rules engine does
not require any programming to
create even very complex applications. Because they generally
access and/or update a host
system, there is a data mapping exercise in the deployment of a
rules engine. This is a
technical role, which in most cases it can be carried out by one
of your development team or a
suitably qualified analyst.
-
Page 11
This is a conceptual model of a Defined Benefit Superannuation
Scheme illustrating how
relatively easy it is to graphically define a very complex set
of rules.
Benefits of automated software development
The benefits of automated software development are
significant.
Model based development tools enforce consistency and structure
leading to a quality
outcome.
Testing is substantially reduced in a model/pattern
environment.
If the model or pattern works once, it works consistently
everywhere its applied.
If the model isnt changed, there is no need to retest the whole
application if a
change is made elsewhere.
Productivity gains are enormous. The more complex the
development the greater the
productivity gain.
Implementation is simplified as a result of:
The consistent behaviour of model based software.
The user engagement in the requirements gathering.
The solution is based on business processes.
The software is bug free.
The model generates a new set of program code every time its
deployed.
The code is machine generated so there are no program
errors.
Any software faults will be in the logic of the model.
The model is corrected and a fresh set of code is generated.
-
Page 12
On-going cost of ownership is reduced substantially:
If the business requirement changes simply change the model and
redeploy.
Software compiled by the models is well structured, consistent
and stable.
Every time a model is changed and deployed the software code is
totally recompiled
so there is no build up of old code from previous changes.
-
Page 13
Managing automated development projects Requirements
gathering
We are translating business requirements directly into modelling
tools, so we arent restrained
by software development considerations when gathering functional
requirements.
We are, therefore, free to define business requirements from a
process improvement
perspective. This approach has many benefits:
Its an opportunity to completely review existing practices,
identifying opportunities
for process improvement.
The revised business processes become the specification for the
new application.
Users have been engaged in defining business processes not the
system definition.
When the developed software is delivered, it is seen as
facilitating the process
change users have already identified not the cause of
change.
This last point is very important as user resistance can be a
major barrier to a successful
implementation.
Business process modelling
There are many process modelling tools on the market. The
problem with many of them is
that they are far too technical for the business user to relate
to. A lot of process modelling
standards have emerged from IT and are related to software
design, rather than business
process improvement.
I favour tools and modelling techniques that reflect the user
environment and that the user
can relate to. There is nothing more powerful than having the
process model up on a screen
and working with the users to highlight problems with current
practices and opportunities for
improvement.
At Circatec we use two tools Microsoft Visio and XSOL InOrder
from XSOL in Auckland, New
Zealand.
XSOL enables us to model business processes in a graphical
format, within a structured model
that reflects the organisation not the software or database
design. This is important if we
return to primary measure of successful software the value it
brings to the organisation.
A big advantage of the XSOL structure is that a process must be
completed in the model, it
cant be left as a to be resolved later. I have used this feature
many times in workshops,
demonstrating to participants that a process as weve mapped it
doesnt work, forcing
continued discussion and resolution of process conflicts.
XSOL also captures notes in user defined categories. As we work
through the process
modelling we capture process improvement notes for the
implementation, developer notes
for development team and legislative or compliance notes to
ensure the developed
applications are technically and legislatively correct.
So when we start building the Plex models or rules engines we
know they are based on clearly
defined and agreed business processes and requirements.
-
Page 14
Architecture and design
As with any development, design is critical, however it is very
different to the detailed
architectural design required for a hand coded system.
In model based development we know the patterns in the automated
software development
tool will provide a consistent architecture and will generate
the database from the model.
Therefore we only need to determine the modules or functional
groups and their
relationships.
With the model having facilities to call other programs, we only
need to consider what
functions are core and what functions will be external modules
or rules engines. The following
schematic illustrated the conceptual design of a superannuation
administration system. Core
functions in this instance are considered to be 90% stable with
incremental enhancement over
the life of the application. External modules are more volatile.
They will be fund specific
functions or subject to legislative change.
Database
Core Functionality
Fund specific functions
Fund specific variables
Legislatively sensitive functions
Ele
ctro
nic
File
Man
ager
Document management
Workflow processes & alerts
User Interface
Browser
Workflow
Data validation and rules layer
In this instance there are supporting designs for each element
of this high level model. The
graphical designs guide the development of the functional models
and subsequently the
entire application. The designs have been developed in
consultation with domain experts and
developers. It can be printed on a large sheet of paper and put
up on the wall.
This is not just an architectural guide, it is used to structure
the work breakdown and work
allocations to support the modular approach.
-
Page 15
User interface and patterns
What is this application to be used for, therefore what are the
key requirements for the user
interface?
The user interface determines the user experience and therefore
user acceptance of the
system and user efficiency.
I am past letting developers determine the UI design. So before
any development begins, we
determine the UI requirements for each user group within the
organisation. For example:
The manual entry of large volumes of data from incoming
documents calls for a
single screen laid out to reflect the format of the document. It
needs to flow from one
field to the next in the same flow as the document and it needs
appropriate
validations to prevent data entry errors.
A call centre needs fast access to the callers details to verify
the callers identity, then
seamless access to every part of the system needed to answer
queries or process
instructions.
Once these requirements are established the UI patterns are
designed. Commonality and
reuse is the key to good pattern design, determining how many
patterns are required and how
each will behave.
Here is an example of a UI pattern design. Totally graphical and
created with a drawing tool in
a matter of hours. In this particular application five patterns
were drawn up in a little over a
day.
This screen, its functions and logic will be reused wherever an
enquiry screen is required. In
this example it will be used for all member, employer, planner
and other external party
enquiries in a superannuation administration system.
-
Page 16
The buttons include enforced checking and verification of data
entry and modifications, a link
to the document repository to retrieve documents specific to the
subject of the enquiry and
tabs to every system function relating to the enquiry.
Once the pattern is built it is reused in every function
requiring this style of pattern. The
developer does not have to develop every screen in the system,
they take the pattern and
build the function model within the pattern, suppressing
anything in the pattern not required
for a specific enquiry.
Development teams
Small is wonderful when it comes to automated software
development teams.
In a Plex environment one business analyst (domain expert) to
two Plex developers is
regarded as a good balance. In complex functions it can be one
to one for a period.
Whiteboards, pin boards and electronic boards are the key to
effective agile development. The
analyst communicates the requirement to the team and other
stakeholders in the most
effective manner. This can be as simple as notes and flows
captured on the electronic board to
a spreadsheet replicating financial transactions and postings.
We seldom replicate source
documents theres no value in that. For example the Australian
Taxation Office has a
document for every form of federal taxation. The document
contains the tax rules, working
examples and software developer notes. Whether its a Plex model
or a rules engine this is all
the information the development team needs.
As the business analyst is part of each development team,
questions are answered as they
arise. The BA also does iteration testing so design errors are
fixed in the model as they are
found. In well functioning teams the interaction between the BA
and the model developers is
so dynamic that potential omissions or misunderstandings are
rectified before the first
prototype is generated.
Building a rules engine is even more dynamic. It is often a one
to one ratio with the domain
expert talking directly to rules engine builder. The rules and
calculations can be tested within
the model so by the time the model is finished it has been
tested and validated, ready to
deploy.
A major advantage of these small, dynamic teams is the ease with
which the client can be
engaged. A three way meeting of client, analyst and developer
can add enormous value and
impetus to the development of a functional model.
Models can be refined and code regenerated as often as required
to prove the application
Developer models the requirement
Direct &unambiguous communication between experts
Prototypes are developed and refined until the
model accurately reflects the requirement
The model is finalised and the
software is generated
automatically
Domain expert talks to the developer
Developer models the requirement
Deployed Application
The lines of communication are short and direct, eliminating the
ambiguity and disconnects that are inherent in the traditional
SDLC.
-
Page 17
Planning automated software development
By now the relative simplicity of the methodologies surrounding
automated software
development is apparent. The same applies to the planning and
management of the
development. By breaking the development into small components,
development is
production, not a project.
Whats the difference?
A project allocates resources to task. It assumes a relatively
elastic supply of resources, hence
the tendency to throw large numbers of programmers and
contractors at a development, with
the resulting well documented failure rates.
Production, on the other hand, allocates tasks to resources. It
assumes relatively fixed
production resources, placing the emphasis on effective use of
resources through work
allocation planning and bottleneck management.
If we accept that domain knowledge is vital in the development
of enterprise applications this
is the correct view of our software development environment.
As in any production environment there is selective outsourcing
or sub-contracting to manage
demand peaks or skill shortages.
In an automated software development environment this approach
is extremely effective
when:
i) Development tasks are broken down into the smallest practical
blocks of work, ideally 1 5 day blocks.
ii) A block of work is specific to a skill, for example
analysis, development, testing, etc.
iii) Work is allocated to each team based on both availability
and domain knowledge. A production planning board shows to whom
each task is allocated to and how long they have
to complete it. The visual planning has significant
benefits:
Who owns the work at any point in the production plan.
What the forward workload is by person and job.
The impact of being late.
The interleaving of jobs for multiple projects or clients is
also highly visible and gets away
from the concept of individual projects vying for shared
resources.
The team leader or production manager knows exactly the state of
every job at a glance and
can take action immediately to alleviate problems. So too does
every team member.
The planning board time scale is days, so every day the whole
team can see exactly where
every job is up to.
A longer term view is easily set up a maintained in Excel,
printed on A3 and posted on the wall.
Now there is a highly visual, easily maintained view of the
immediate and long term work load
and work allocations with an absolute minimum of management
overhead.
Once again the emphasis is on visibility and simplicity without
sacrificing accuracy in any way what-so-ever.
-
Page 18
The benefits are enormous We benchmarked a recent Circatec
development of a superannuation and investment fund
administration system using function point analysis, an industry
standard measure of
software functionality.
The benchmark we used to cost a traditional SDLC was the costing
model the University of
Southern California, Center for Systems and Software Engineering
refers to as the
Constructive Cost Modell methodology COCOMOII. (You can access
this model using:
http://diana.nps.edu/~madachy/tools/COCOMOII.php)
I have used a monthly cost of $22,200 being 18.5 average working
days a month at $1,200 a
day, calculated as the average internal costs of a development
team and management
structure plus employment costs and overhead.
The model gave these results:
Work effort in person months
Development cost
Benchmark 598 $13,275,600
Circatec build 97 $2,153,400
Ratio 16%
This costing model shows a cost saving of $11.12 million using
automated software
development instead of the traditional SDLC.
Furthermore:
The function point count is 2898 which is at the upper end of
size and complexity.
At no stage did we have more than four people working on this
development.
This comparison used cost criteria reflective of a well run
project.
There is ample literature available to support the view that a
development of this size
following a conventional SDLC has an extremely high probability
of failure.
So lets change the costing criteria to be more reflective of a
large development team with a
high number of contractors, limited domain knowledge and
increasing time constraints.
Drivers Benchmark Large team Project under
pressure
Team Cohesion High Low Very low
Analyst capability High Nominal Nominal
Programmer capability High Nominal Nominal
Personnel continuity High Low Very low
Application experience High Low Very low
Platform experience High Low Low
Language & toolset experience High Nominal Nominal
Time constraint High Very High Very High
Use of software tools High Low Low
Person months 598 2,706 3,845
Cost $13.3 million $60.1 million $85.4 million
The result is a staggering blow out in cost and development
effort.
-
Page 19
Unbelievable? Not really. Regardless of what industry you are in
Im sure you know of at least
one project that has gone this way. If not, look at some high
profile public projects, theres at
least one in every city.
Suddenly 16% of benchmark using automated software development
is now 2-3% of the cost
of a poorly performing conventional SDLC. The chances of a blow
out of this proportion in an
agile, model based, automated software development are
practically nil.
It doesnt stop here
The whole structure of the IT department changes in an automated
software development
environment.
Project managers Arent required for any aspect of the software
development cycle. A
reduced role in requirements gathering and post development
implementation.
Software architects On a large or complex development an
architect may add value in the
initial design phase. This is a short term contract position at
the most.
Data modellers Not required as a dedicated role, providing the
developers have a
good understanding of data structures and database
conventions
which they will if the right people have been recruited.
Testers Testing is incremental and an integral function of the
BA / developer
team. There is value in having a small pre-release testing
function
within the development group.
Systems support Business focussed requirements gathering, client
engagement during
the development cycle and ease of implementation reduces the
client support roles.
The larger the development the greater the savings
The larger and more complex the requirement, the greater the
productivity gains and the gap
between the traditional SDLC and automated software
development.
Waterfall SDLC
Agile SDLC
Automated software development
Size & complexity of development
Co
st
-
Page 20
There is also much greater certainty in automated software
development. The teams are
smaller, therefore you can be more selective who you engage.
The agile methodology suggested earlier requires design and work
breakdown phases before
commencing any development work. These will provide a reliable
estimate of work plus small
development iterations for cost and progress monitoring.
Compelling case for change
The cost savings are compelling on their own.
Plus the added benefits of system agility, speed of change and
greater engagement of clients
in the development cycle.
The business case for a major change to corporate IT is
compelling.
-
Page 21
How corporate IT resists change Theres a compelling business
case for automated software development and agile
methodologies and yet the uptake is minimal. To add insult to
injury this is not new
technology. Most automated software development tools have been
commercially available
for ten to fifteen years!
Before we consider a significant change to corporate IT we need
to identify the barriers to
change. Yes many of the usual barriers to change exist but there
are some unique issues to
address in bringing about wholesale change to corporate IT.
Some of the fundamental problems are:
The very reason many middle and senior IT management roles exist
is to manage the
existing IT structure and workforce.
The powerbase and kudos of senior developers, system architects,
data modellers and
other specialists relies on preserving the importance of their
role and discipline.
For many consulting firms their entire fee base is built on
large, highly structured projects
and the supply of large number of IT contractors. Theres not a
lot of revenue in
automated software development projects!
There is a whole industry offering training and accreditation
particularly Prince2, APM
BoK and, to a lesser extent, Agile.
The market is crowded with consultants and contractors needing
clients to buy the whole
package to ensure a reasonable period of employment.
So there is a whole industry built around the status quo and
they are very effective at blocking
the uptake of automated software development tools and efficient
methodologies.
Unfortunately they are often the people business executives turn
to for advice and they fail
the trust placed in them.
Let me give you just two examples. I have amended some details
to avoid identifying the
organisations, however many readers will identify with
these.
1. An organisation has a number of legacy financial products
that are largely rules based.
They want to move them off several old platforms that are
becoming more difficult and
expensive to maintain. The internal cost to redevelop these
products to run on a common
platform was in excess of $2 million dollars, which could not be
cost justified.
A rules engine vendor analysed the rules and estimated it would
cost between $2-300,000
to develop all the legacy products into three or four rules
engines. The vendor was willing
to provide a fixed price quotation and requested access to all
the product rules. A systems
architect blocked any further analysis on the basis that we dont
want another
technology on site.
So the vendor was not able to speak to the business unit or
obtain any further information
to formalise a quotation nor did the business unit know there
was a viable option on
offer.
-
Page 22
From a commercial perspective this one person was able to make a
totally unreasonable
and unsupportable decision, effectively forcing the organisation
to continue maintaining
legacy products on legacy systems, whilst continuing to incur
the increasing cost of
maintenance.
2. A company was developing a web based financial planning tool.
A rules engine vendor
offered to build all the financial calculators for a little over
$100,000 and 3 months
duration. A web developer assured the company they could program
the calculators for
$75,000. Two years and many dollars later the product wasnt
complete and the window
of opportunity had closed.
These examples illustrate the difficulty vendors of automated
software development
tools face. The people they have to sell to are the very people
who dont want the tools in
the organisation.
The above examples occurred because corporate IT controlled the
information flow into the
respective organisations.
Common practices used to maintain control
Inappropriate use of standards
No one denies the importance of standards but they must be the
right standards for the
right reasons.
There are many technology standards that are critical,
especially in system to system
communication or the electronic transfer of data between
organisations.
However many standards and methodologies have evolved from
professional bodies or
special interest groups. Whilst they may add value in some
circumstances they are not
commercial imperatives and are certainly not set in stone.
For example Wikipedia lists::
Data modelling 10 techniques and notation standards.
Process modelling 6 process modelling techniques and 5 languages
for BPM
(Business Process Modelling).
Software development - 2 methodologies, SDLC and Agile with 5
common SDLC
development models and 4 common Agile development models.
Project management 10 common methodologies.
Why on earth does any organisation want or need to base the
selection or development of
business solutions around any one of the myriad of standards and
standard
methodologies.
In far too many cases standards are mandated by the IT group
because:
Its the sandpit the technologists want to play in.
The standard is used to block more efficient options.
The only justification for mandating a technology or methodology
standard in an organisation
is to genuinely protect or add value to the organisations IT
systems.
-
Page 23
So it is absolutely critical that standards are vigorously
challenged to ensure they are aligned
with the enterprise core business and core enterprise systems.
Otherwise they become a tool
for technologists to block unwelcome automated development tools
and efficient
methodologies.
Request for Proposal or Tender
I have seen too many tenders where the technology and
methodology sections are as large, or
larger, than the business requirements.
These documents are extremely detrimental to the organisation
for several reasons:
1. They shift the focus from a business solution to a technology
solution.
2. They shift control from the business units to IT, enabling IT
to protect the status quo.
3. They often block innovative solutions to business
problems.
For many years Circatec was actively engaged in the
specification and selection of medium to
large ERP systems. In all that time I refused to specify the
underlying technology on the basis
that we were looking for the best possible solution to a
business requirement. Having found
the best solution then wed look at the technological issues.
If your organisation is looking for a solution to a business
problem then dont let your IT
department write or run the RFP/RFT process. Only allow them to
specify technical
requirements that are absolutely essential and justified from an
enterprise perspective. When
youve identified the functionality you need for the enterprise
then engage IT to make it
happen.
Do not underestimate how important this is. A number of vendors
of automated software
development tools and/or services are reluctant to respond to a
tender run by an internal IT
department, especially when the technology and methodology
sections are deliberately
prescriptive.
Circatec recently declined to bid on a development because the
added cost of meeting the
specified implementation and technology standards exceeded the
cost to develop the
solution using automation tools and agile methodologies. We know
the project didnt proceed
because it couldnt be cost justified on responses from complying
vendors. We also know
that we could have provided a solution within a cost justifiable
budget if we had been allowed
to submit a non-complying bid.
Size of the service provider
The average application build using automated software
development is approximately
15-20% of the work effort of a standard SDLC build. Rules
engines are even faster, often
just 10-15% of the build time.
Agile methodologies and business oriented process modelling
tools significantly reduce
the requirements gathering and process improvement
activities.
Rules engines can cut months out of data conversion and
rectification.
It follows that any service provider using or supporting these
tools and techniques will be
small. And so they should be, otherwise they are just another
firm looking for billable hours to
feed a large team.
-
Page 24
However size is often used against the service provider:
They arent big enough to support our needs.
They are too small to really understand our requirements.
Our preferred supplier has 200 staff so they are better placed
to handle our needs.
(Code for they use the same inefficient practices we do).
The last thing your IT group wants is a small, agile, highly
effective organisation delivering
more product in a fraction of the time it takes in-house.
I can do it just as quickly by hand!
If I had a dollar for every time Ive heard this Id be a rich
man!
It may be true at a micro level but what the programmer is
really saying is:
I can knock up a piece of untested, undocumented code which Ill
imbed in the existing
code with no notation. No one will know its there or what it
does, perpetrating my role as
an indispensable source of knowledge.
The same is true with data validation, rectification or
conversion. Programmers will claim they
can quickly write a script to carry out a data management
function. What they dont say is that
scripts have about as much integrity as a spreadsheet in that
they are difficult to validate and
have no change control.
Micro level comparisons quickly become irrelevant. Other than
patching an existing program
there are very few micro developments. If we follow the
scripting example a little further what
we will find is that the single script quickly becomes 100
scripts. New scripts are written to
replace early scripts and the library of scripts grows, as does
the risk of running an obsolete or
inaccurate script.
The CEOs toolkit
Cutting through the screen of misinformation is relatively easy
using what I call the CEOs
toolkit:
Why
So what
Prove it
The conversation is:
In plain business language, and in the context of our business
needs, tell me:
Why: are we doing it this way?
Why: is that important?
So what: if what you say does or doesnt happen?
Prove it: with evidence and a business reason to support your
opinion.
You may have to keep asking why to get to the real reason. When
you do then you can assess
whether there is a business case to support the IT position or
not.
Organisations can no longer afford or justify self indulgence by
their IT departments and project management offices.
-
Page 25
Bringing about change Changing the structure and culture of your
IT department will not be easy. It will have to be
driven from the very top Board, CEO and senior management.
This scale of restructure requires selective redundancies, a
potential HR nightmare. The
management team driving the restructures will need to know they
have a board mandate and
CEO support for the changes they are to make. They may need
expert industrial law advice
and HR guidance.
It is a daunting but commercially imperative task, and its very
achievable.
Your CIO may actually be your greatest ally. Many CIOs feel they
are caught in the middle
between the pressure from the business units for higher levels
of IT service and an inflexible IT
workforce. They know whats ahead of them if they try to
implement change without
organisational support. On the other hand if your CIO is part of
the problem this is your first
replacement.
Through recruitment I know there are many experienced developers
and business analysts
excited by the potential of automated software development.
Small teams attract and
stimulate good quality people. Competent people thrive in an
environment that values their
knowledge and skills. You may already have your automated
software development teams on
your staff.
The key is attitude. A developer or business analyst can lack
some specific experience if their
attitude is a willingness to learn, to take on new methods and
challenges and add value to a
small, self motivated team. This is one of the key criteria in
determining which members of
the existing IT group will transition to the automated software
development group.
Structure
An agile automated development environment has a very flat
management structure.
A small number of largely self-managed teams (3-5 members) will
generate a tremendous
amount of high quality software. Depending on the number of
teams a single development
manager with possibly a coordinator to handle work allocations,
time recording, etc. will be
the total management structure.
There is no need for a project management office (PMO) in an
automated software
development environment. There will, of course, be other
corporate technology programs to
manage, however if your PMO sees itself as a control centre,
rather than a facilitator,
dismantle it and start again, otherwise they will just be a
barrier to change. Hire project
managers not methodology practitioners.
It is questionable whether there is any need for full time
software architects and data
modellers. These are roles that can be outsourced at the
commencement of a major
development to define the design and structural elements of the
application. The automated
software development tools will enable the teams to adhere to
these design principles, and
there will be the skills and understanding in the development
teams to apply the design
guidelines.
IT infrastructure, network support and end user help desk
functions will remain largely
unchanged.
-
Page 26
Existing systems
Existing systems, especially internally developed and/or
maintained, will require continuity of
support. Consider setting up a separate product support group,
however it will need strong
management to contain the maintenance of existing systems. New
functionality should be
developed by the automation teams.
Alternatively, if enough of the old IT group have been
transitioned, maintenance can be
provided by the new group. The advantage of this is that once
developers have absorbed
themselves in automated software development tools and rules
engines they wont want to
do more than necessary in the legacy system.
What about work-in-progress?
If a development is in its early stages, and is more than a few
weeks work, then consider
redeveloping it using automated software development. As a
general rule any development
that hasnt past the 50% complete point will deliver cost savings
by starting again in an
automated environment.
If a development has passed the 50% complete stage, is on time
and budget, and the
development team is still intact, then it should be completed by
the current team.
If a development is over time and budget and there is no
confidence in the expected
completion date and cost then it should be stopped and
reassessed. There is every likelihood
an automated software development team will redevelop it faster,
more cheaply and to a
higher quality than persevering with it. Quality is often the
first casualty of troubled
development projects and must be a factor in the decision to
start again.
Resistance to change
Like any form of workplace automation this is a change
management project with little room
for compromise.
The resistors must be identified and removed if they wont adopt
the new methods. Weve all
met the passive resistors who agree to change in meetings but
wont change. There is always
an excuse and in the meantime they are gathering supporters and
the whole rollout is
jeopardised. Fortunately with automated software development
they are easily identified
they are either using the tools or they arent.
There is a small but vocal group of IT professionals who can be
openly scornful and
disrespectful of anyone with a different perspective to theirs.
I call it technological arrogance
and these are some of the most difficult people to manage when
implementing change on
this scale, especially when the change diminishes their role and
importance in the
organisation.
They can be openly hostile and dismissive and their greatest
weapon is to resort to techno-
speak, running out a string of indecipherable reasons why
whatever youve just suggested
wont work. Often their claims are facetious, exaggerated or just
plain wrong. Fortunately
they do tend to make their presence known.
No organisation is obliged to provide technologists or
methodology practitioners their own
personal sandpits. If someone wont play in the corporate sandpit
for the good of the
organisation there is no place for them in small, agile, self
motivated teams.
-
Page 27
Be prepared to start again
I have experienced first hand the total block of model based and
automated software
development tools. Its as if the group psyche is total
commitment to the status quo, they
genuinely cant see anything wrong with their work practices and
methodologies.
When this happens an organisation must accept that they may have
to shut down a whole IT
section and/or PMO and start again.
This may seem drastic, but in these circumstances it will be
more effective to start from a zero
base, recruiting the roles and skills needed for an agile,
automated environment.
Despite a genuine concern around the loss of knowledge of
existing systems, losing blockages
to change in a well managed manner is infinitely more productive
than keeping these people
because of their perceived knowledge. When theyre gone its often
surprising how much
knowledge is retained in the organisation.
The savings and other tangible benefits are too great to
compromise or delay the restructure.
-
Page 28
What will it take? Ive been critical of corporate ITs resistance
to change, but in all fairness in many cases they
are doing, and delivering, what is asked of them.
There will be no groundswell of change from within the IT
industry.
It will have to come from boards, CEOs and senior
executives.
As they look at what their organisation has spent and is being
asked to spend on IT systems,
particularly the development and maintenance of software
applications.
Then, knowing there are alternatives, they are entitled to get
frustrated, even angry, at the
impact on the revenue they work so hard to earn and the profit
they are expected to deliver,
they will start to demand change.
These are mainstream alternatives
CA Technologies, the developer of CA Plex, is one of the largest
independent
software companies in the world. (Wikipedia)
IBM reduced development costs by $300 million a year by adopting
Agile
methodologies. (Google How IBM saved $300 million)
Gartner Group is the leading technology analyst and trend
forecaster. A positive
Gartner review can have a significant impact on the market
acceptance of a software
product or technology.
None of these are small niche players!
Change will occur when change is demanded.
When senior management will not accept multi-million dollar
projects blowing out five or
ten fold.
When IT budgets are reduced and greater output is expected.
When business units demand, and expect to receive, rapid
delivery of solutions.
When boards demand greater value for their technology spend.
Only then will agile, automated and delivery focussed IT groups
emerge.
And emerge they can. There is no practical barriers to the use
of model based, automated
software development tools in business applications including
browser and smart phone
apps.
The 40% reduction in IT jobs forecast by the Gartner Group in
2005 is now, and was then,
attainable.
It only requires organisations to insist on change and empower
their senior executives to
make it happen.
-
Page 29
About the author Ken Galbraith is the founder of Circatec Pty
Ltd, a software development and consulting
company based in Melbourne, Australia.
He has been in the software industry for nearly 30 years. During
that time he has been an
owner and/or director of three software development and
consulting businesses.
For fifteen years during the 1980s/90s Ken was involved in
enterprise (ERP) systems for
manufacturing and logistics. This was a period of radical change
in manufacturing globally.
ERP systems and associated process change and consulting
services were an integral
component of the change.
In 1996 Ken formed the INL Consulting Group, specifying,
selecting and implementing ERP
systems for a number of medium to large manufacturing and
logistics companies. During this
period he personally started working in superannuation and
wealth management.
In 2000 INL Consulting became Circatec. The business focus
shifted from consulting to the
provision of technology and software solutions.
By 2005 Circatec was using model based development tools in
business process automation
and rules based business solutions. Services expanded into the
development of large and
complex application software. Through this Ken gained enormous
experience in the power of
these tools.
Circatec was also marketing the toolsets and supporting
services. An exercise that provided
first hand experience in the ability of corporate IT to block
the adoption of automated
software development tools and efficient development
methodologies.
Today Ken, and Circatec, remains committed to changing the
software development and
delivery paradigm by working with senior management to bring
about a cultural shift in
corporate IT.
You can contact Ken Galbraith through the Circatec website or by
email.
Web: www.circatec.com.au
Email: [email protected]
Additional copies of this eBook can be obtained from the
Circatec website.