YOU ARE DOWNLOADING DOCUMENT

Please tick the box to continue:

Transcript
  • The Executive Guide to Slashing Software Development Costs

    How to reduce software development costs by 80%.

    Utopia? No just the modernisation of

    out-dated software development methods.

    An eBook by

    Ken Galbraith

    A no-holds-barred look at current IT practices and the impact they have on

    business

    By k.g

  • Copyright 2013 by K M Galbraith. All rights reserved.

    This eBook may be downloaded and circulated in its published form.

    No extracts may be used without the authors permission.

    Disclaimer:

    This book is based on the personal experiences of the author.

    Outcomes and productivity gains may vary in other organisations.

    Experiences relating to Circatec clients or other parties have been changed to the extent

    required to avoid identifying the other party.

    All research material is from publically available sources.

    Trademarks:

    CA, CA Plex and Plex are trademarks of CA Technologies.

    XSOL InOrder is a trademark of XSOL Limited.

    Additional copies:

    Copies of this eBook can be obtained from Circatecs web site:

    www.circatec.com.au

  • Contents

    Introduction 1

    Software excellence 2

    The current state 3

    The traditional software development life cycle (SDLC) 3

    The Agile software development life cycle (SDLC) 4

    Over reliance on methodology 5

    Cant I just send it offshore? 7

    The fundamental flaw 7

    There is a better way 8

    Automated software development tools 8

    Application development 8

    Rules Engines 10

    Benefits of automated software development 11

    Managing automated development projects 13

    Requirements gathering 13

    Architecture and design 14

    User interface and patterns 15

    Development teams 16

    Planning automated software development 17

    The benefits are enormous 18

    It doesnt stop here 19

    The larger the development the greater the savings 19

    Compelling case for change 20

    How corporate IT resists change 21

    Common practices used to maintain control 22

    I can do it just as quickly by hand! 24

    The CEOs toolkit 24

    Bringing about change 25

    Structure 25

    Existing systems 26

    What about work-in-progress? 26

    Resistance to change 26

    Be prepared to start again 27

    What will it take? 28

    Change will occur when change is demanded. 28

    About the author 29

  • Page 1

    Introduction Fundamentally the way software is developed and delivered hasnt changed in 40 years.

    Over the last ten to fifteen years tools and methodologies to dramatically streamline software

    development have been maturing in function and sophistication.

    And yet the software industry has been able to resist change. It is reasonable to say that no

    other industry has been able to resist change for so long and get away with it.

    The impact on an organisations ability to deliver, adapt and take advantage of the rapidly

    changing commercial and legislative environment is severe. In many instances IT dictates

    corporate agility and speed to market.

    While boards and CEOs require system agility and quick response to changing requirements,

    their IT department often remains inflexible and resistant to change.

    IT departments are empowered and trusted by their organisation to provide software and

    technology solutions for the benefit of the organisation. Far too often they fail this trust by

    blocking new ways and technologies, often using dismissive statements that, if challenged,

    cannot be justified mostly however, they go unchallenged.

    Too often Corporate IT is making key and fundamental decisions that have a significant

    impact on the business and there is no mechanism in place to hold them to account. The

    business managers most affected are seldom engaged in these decisions and are, therefore,

    unaware of the alternative solutions available to them.

    In 2005 Gartner Group (a leading technology trend forecaster) forecast that 40% of IT job roles

    as they existed then would be lost to automation. They went on to forecast that more

    American IT jobs would be lost to automation than to outsourcing. The most significant

    forecast was that between 2005 and 2010 the productivity of IT workers would increase by a

    factor of 10.

    Today, some 8 years later, the uptake of automation tools and agile methodologies has

    effectively been ignored (the generous view) or blocked (in my experience) by the broader IT

    community.

    The impact is much more than just high cost and long delivery cycles for software

    development, there is a very significant impact on operating costs when systems are out of

    step with organisational needs.

    This book looks at the current state of software development and then explores commercially

    available and proven alternatives before discussing how to bring about significant change.

    Let me clear here. The change I am talking about is a fundamental shift in the way we design,

    develop and deliver software applications.

    If we keep doing the same things we will continue to get the same results.

    If we want to change the outcome we must change how things are done.

  • Page 2

    Software excellence We need to understand what constitutes excellent enterprise software in order to objectively

    look at the tools and methodologies used to develop an enterprise application.

    User experience The software must be easy to use and enhance the users undertaking

    of their everyday job functions.

    This means the software must be intuitive to use and reflective of the

    enterprise business processes.

    In high work load environments the software is so well aligned to the

    process that user interaction with the application is highly efficient.

    Value add The software must add value to the organisation by providing a more

    efficient workplace, eliminating error and waste, enhancing the

    customer experience, reducing corporate or financial risk, and other

    tangible organisational benefits.

    Flexible The ability to adapt to changing organisational demands is critical to

    the long term value of the software.

    Reliable The software must be available and run consistently day after day.

    Other than infrastructure problems the software availability should be

    100%. Transactional processing must be so consistent that once

    audited subsequent transactions are accepted as correct.

    Secure Security in this context is related to the validation of transactions to

    ensure data and transactional accuracy. Protections around the

    maintenance of data to prevent malicious transactions or accidental

    data errors. Incoming electronic data files must pass through

    validation filters before being posted to the database.

    Cost effective The cost of development must pass a business test. Implementation of

    the software must be efficient and the ongoing cost of maintenance,

    including changes and enhancements, must be relative to the value

    the software is delivering to the organisation.

    Scalable The ability of the organisation to grow and/or expand its products and

    services must not be hampered by limitations in the softwares ability

    to process the increased volumes of transactions.

    The tools, standards and methodologies adopted by corporate IT and their service providers

    must be totally aligned to the delivery software excellence as judged by the organisation

    they serve.

    The sole measure of successful software is the value it brings to the organisation.

  • Page 3

    The current state The traditional software development life cycle (SDLC)

    The waterfall approach to software development has been around since the early 1970s and is

    still widely used today. It is a sequential design and development methodology as shown in

    this flow chart.

    The waterfall model assumes each stage is complete before proceeding to the next.

    RequirementsAnalysis &

    specificationDesign

    Architecture & components

    ImplementationConstruct

    components

    VerificationTest & install

    MaintenanceModifications &

    defects

    Because Waterfall is sequential it is impossible to go back. The assumption is that the

    requirements gathering is an accurate and complete reflection of the requirement (which it

    seldom is) and the design is an accurate reflection of the requirement.

    The advantages of Waterfall are:

    It enforces discipline, every stage has a start and end and progress can be monitored

    through the use of milestones.

    There is considerable documentation which minimises the impact of staff turnover

    on the project.

    The client (external or internal business unit) knows what to expect. The detailed

    specifications describe what the software will do on completion.

    The disadvantages of Waterfall are:

    Change of requirements is difficult to manage due to the large amount of

    documentation.

    Once a stage has been completed it is difficult to go back.

    If a specification error or change is required the project must go back to the

    beginning.

    Bugs and design flaws arent identified until after the entire application has been

    written.

  • Page 4

    The disconnect

    As waterfall moves from one phase to another it engages different disciplines and skills. Each

    with their own language and terms of reference.

    This creates a problem as the language moves from end user terminology to technical or

    system terminology, compounded by the separation of development from the client.

    What often happens is:

    Clients domain expert talks to

    business analyst

    Business analyst develops

    requirements specification

    Systems analyst interprets

    requirements into development specification

    Development specification goes

    to programmers to write the software

    Business analyst talks to systems

    analyst

    Client doesnt fully accept the software and the loop starts

    again!

    Developed software released

    to the client for acceptance

    Returns the specification to the client for approval

    The developed software is the Programmers interpretation of the System Analysts

    interpretation of the Business Analysts interpretation of the Clients requirements

    The client is now out of the loop!

    Programmers are now completely isolated from the client

    Highly technical document that would be difficult for the client to understand

    Programmers interpret the

    development spec into program code

    The Agile software development life cycle (SDLC)

    To overcome the shortcomings of Waterfall and its variants, Agile methodologies started to

    emerge in the 1990s.

    The objective of Agile is to replace the sequential methodology of Waterfall with an iterative

    methodology using small cross functional teams rapidly delivering small, useful pieces of

    software.

    These deliveries happen at regular intervals enabling user testing and evaluation throughout

    the development cycle. By developing small pieces of software, changes can easily be

    accommodated.

    As delivery is an iterative process client cooperation and feedback is encouraged. Agile

    supports the use of prototyping or modelling where appropriate to ensure the delivered

    software meets the clients requirements.

    A key aspect of Agile is the minimisation of documentation. Requirement specifications are

    just enough to communicate the requirement to the development team. This may be as

    little as notes and flows on a whiteboard or a spreadsheet demonstrating financial

    transactions and postings.

  • Page 5

    While this may be disturbing to some, if we accept the only measure of successful software is

    the benefit it brings on completion then we must also accept that all documentation used

    during development is obsolete the moment the software is accepted and put into

    production.

    Each iteration

    Iteration test & accept

    Iteration release

    Agile planningRequirements

    Define

    Develop

    Work breakdownPriorities / dependencies

    Iteration schedule

    Progressive testing enables early acceptance

    of the completed

    Just enough documentation for the development team to

    build the iteration

    User engagement in assessing the iteration or

    prototype prior to relaease Feedback

    Agile will deliver significant productivity gains, however it will only reduce the time and cost of

    software design and specification. This may equate to savings in the 10-20% range.

    You can be too Agile

    The problem with Agile is that it can easily be taken too literally. In a development where a

    single team builds a function or even the entire application, Agile can deliver significant

    productivity gains.

    In a larger development with multiple teams working to produce a single application, an

    overriding architectural design and development standard must be in place. Without this the

    teams will spend unnecessary time rebuilding prototypes, or reworking early developments,

    in order to provide a single cohesive application.

    Ensuring the design work has taken place will ensure the development delivers the benefits

    available in Agile.

    Over reliance on methodology

    We cannot discuss software development without a critical discussion on project

    management methodologies.

    The vast majority of software projects follow a highly structured project management

    methodology based on the waterfall approach to software development.

    It is worth noting that many highly structured methodologies are designed for, or evolved

    from, construction, engineering, mining, etc. where it is entirely appropriate to have rigid

    controls around documentation, certification and change management approval. Whether all

    these controls are appropriate in software development is questionable.

  • Page 6

    Question: Do you need the same approval process to add a field to a screen and underlying

    data table as you do to authorise an engineering change on a bridge? The exponents of these

    methodologies say yes, whereas reality says no.

    I have some specific issues with highly structured project management methodologies in

    software development.

    They perpetrate the disconnect between the business unit and the development

    team that is inherent in Waterfall.

    The methodology becomes the purpose, not the outcome the timely delivery of

    usable and cost effective software.

    They add a huge amount of unnecessary overhead, cost and delay to a software

    development project.

    Any prescriptive methodology enables low competency operatives to hide behind

    the methodology.

    Make no mistake no methodology will guarantee a successful outcome. However the more

    structured and bureaucratic it is, the more certain you can be that any failure will be

    expensive, well documented and a long time in the making.

    Flexibility is the key

    Very few methodologies are absolutely prescriptive. The vast majority, including Prince2 and

    APM BoK are frameworks to be tailored to a specific project.

    In the right hands, they provide a valuable and practical framework for managing a project.

    I have worked with some extremely capable project managers. They all have common traits,

    in that they are people managers and they are focussed on the destination not the journey.

    Plus they have domain knowledge either of the industry they are working in or the

    technology they are implementing (which implies a degree of industry knowledge as well).

    Problems arise with highly structured methodologies when the project manager is a

    practitioner of the methodology. Instead of guiding, mentoring and smoothing troubled

    waters to get an outcome, they focus on the methodology demanding strict adherence,

    regardless of practicality or relevance.

    This can also occur when a service provider has been retained to provide project management

    resources. They need strict adherence to the methodology because:

    They sold the complete project management package to their client.

    It enables them to put in methodology practitioners instead of domain experts.

    They can forecast their fee revenue.

    If you are going to outsource project management, ensure you are getting a project manager

    not a methodology practitioner.

  • Page 7

    Cant I just send it offshore?

    I have yet to meet anyone who has had a good experience with the offshore development of

    an enterprise application.

    Generally the problems have nothing to do with competence. The problems usually come

    down to two issues.

    Communication of requirements.

    Requirements must be very precise because of the geographical and time zone separation

    between the business analysts and the developers.

    I have already demonstrated the disconnect and lack of precision inherent in the

    conversion of business requirements into programming specifications and then into

    delivered software. This disconnect is exacerbated with offshore development.

    Lack of domain knowledge.

    Inherent in most specifications is an assumption that the programmer understands the

    domain. For example, in specifying financial transactions, a system analyst may assume

    the programmer knows the transaction must be validated and doesnt even think to

    specify it.

    The absence of domain knowledge anywhere in the development cycle will lead to

    misunderstanding, errors and exclusions.

    In a traditional waterfall development there are inevitably disagreements during testing when

    the analyst rejects a function and the programmer argues they programmed to the

    specification. The frequency and severity of these disagreements and the associated rework

    will increase significantly in offshore development.

    Unless you are very lucky, any savings in development charge rates will be consumed in travel,

    communication, rework and dispute resolution.

    The fundamental flaw

    Regardless of methodology the fundamental flaw is the lack of automation in the

    development of application software.

    A typical software development department consists of a team of programmers writing code

    just as they did 40 years ago. Sure, the programming languages have changed but its like

    giving a carpenter a nail gun. Hes a little quicker but hes still driving in nails.

    While Agile may deliver savings in the 10-20% range, it is a fraction of the savings to be made

    by automating software development.

  • Page 8

    There is a better way We see how the dependence on the manual development of software, regardless of

    development methodology, fails to meet the cost restraints and agility demanded by modern

    business.

    There is a much better way to develop enterprise software applications that will deliver:

    Cost reductions of 70 80%.

    Extremely flexible software solutions.

    A high level of client engagement in the development cycle.

    A significant reduction in the elapsed time to specify, develop and implement a

    solution.

    Bug free software.

    Massive reductions in the size of your IT department.

    Automated software development tools will deliver this and much more.

    Automated software development tools

    Typical characteristics of automated software development tools are:

    A structured modelling framework that enforces standards and consistency.

    Definable patterns that establish consistent user interfaces and behaviours.

    Inherent functions to speed up model building.

    An extremely high level of component reuse.

    The automatic generation of program code.

    There are two groups of tools that between them will virtually eliminate the need for any

    manual program development:

    Application development tools.

    Rules and decision engines.

    The strength of these tools is the ability of the developer to work directly with a domain

    expert, creating a model of the requirement and generating the software code without the

    use of manual programming.

    The structure, models and patterns that characterise automated software development tools

    enforces the standards and consistency essential to deliver software excellence.

    Application development

    Circatecs requirement was for a toolset that enabled us to build commercial grade enterprise

    applications. This means large server based applications delivered via an internal network,

    remote desktop or via a web browser without the need for a secondary web based layer.

    As a software developer we were also reluctant to limit ourselves to a single platform and

    database.

  • Page 9

    There are a number of excellent automated application builders on the market but CA Plex

    was the only automation tool we found that met our broad criteria.

    Plex originated in the 1990s as IBMs Synon product. Computer Associates acquired it some

    years ago and it has continued to evolve into what is arguably one of the best tools currently

    available. Plex applications have been proven in very large enterprise systems so we know

    they are scalable.

    CA Plex is multi-platform. It can generate C#, C++, Java or RPG III & IV code to run on Sequel

    Server, Oracle or DBII databases.

    Plex provides enormous productivity gains, enabling us to build applications in 15 20% of the

    time it would take to manually program the application.

    Some of the key aspects of Plex are:

    Patterns.

    There are two classes of patterns in Plex, the user interface (UI) and process patterns.

    The UI patterns provide a consistent user interface across the entire application. Typically

    we would have three or four patterns for use in specific functions, for example:

    High volume data entry may be a single screen so all the fields pertaining to the

    function are displayed in one large panel.

    The enquiry screen however may reflect the needs of a call centre and display core

    information with links to every other related area of the system.

    Functional patterns will define a regularly occurring pattern. For example, in a financial

    system the standard process may be to enter and save a transaction, have another staff

    member confirm the transaction and, on confirmation update the underlying ledgers. By

    building this as a pattern we enforce the process and where the confirmation cycle isnt

    required it is turned off.

    The point of patterns is that they enforce consistency across the entire system but they

    also save a huge amount of development time. When a developer is building a function

    the first thing they do is select the pattern and the user interface is immediately inherited.

    Data base generation.

    The database is generated from the model. Developers do not have to build a database

    before they build the application.

    This is why, in a Plex world, we can generate the application to run on SQL, Oracle or

    DBII. We simply tell Plex what database to compile when we run the model.

    Models.

    To call or initiate another program its as simple as naming the application in a link in the

    model and Plex generates the physical links.

    Most business environments have situations where exceptions trigger a different process.

    In the Plex model the developer simply defines the If / Then statement.

    The whole point of these examples is to illustrate how quickly complex or repetitive

    programming tasks can be defined in a model without any manual programming.

  • Page 10

    Plex generates the program code in the designated language and compiles the designated

    database.

    Because the code is machine generated there are no bugs, so testing is limited to ensuring

    that the model is a true reflection of the requirement. If there are design errors or functional

    gaps the changes are made in the model and a fresh set of code is generated. This ensures the

    software code is always clean and free of the redundant code that builds up in manually

    programmed software.

    Rules Engines

    A rules engine provides a graphical model in which to define the business rules. Having

    defined the rules flow, the formula for each branch or decision node are defined and the

    model tested.

    All good rules engines will have inbuilt test capabilities and will be self documenting.

    The rules engine will automatically generate the model into program code to be deployed into

    the host environment.

    In an enterprise environment, rules engines are generally deployed to:

    Add functionality that is difficult or expensive to develop in the host application. Rules

    engines can play a valuable role in extending the life of legacy software by adding

    functionality that would be prohibitive to develop on an out-dated platform.

    Encapsulate legacy products. Wealth management, superannuation and insurance often

    have closed products they are legally bound to maintain until there are no more clients in

    the product. A rules engine is ideal to define all the product rules and calculations into a

    platform independent rules engine that can be mapped to a host system. For example a

    pension scheme may take member data from the host, apply the pension scheme rules

    and return the pension payment to the host to process the cheque or EFT. If the fund

    migrates to a new administration system, the rules engine remains unchanged and is

    mapped to the new system.

    Data validation and rectification is a perfect domain for a rules engine. Far too often

    organisations rely on scripting languages or spreadsheets both of which are time

    consuming to develop, difficult to test and impossible to guarantee their ongoing

    integrity. The version control of a good rules engine, combined with the fact that the

    generated code is maintained by changing and regenerating the model, provides

    certainty over the rules and calculations being applied to the data.

    As with application builders such as Plex, a rules engine does not require any programming to

    create even very complex applications. Because they generally access and/or update a host

    system, there is a data mapping exercise in the deployment of a rules engine. This is a

    technical role, which in most cases it can be carried out by one of your development team or a

    suitably qualified analyst.

  • Page 11

    This is a conceptual model of a Defined Benefit Superannuation Scheme illustrating how

    relatively easy it is to graphically define a very complex set of rules.

    Benefits of automated software development

    The benefits of automated software development are significant.

    Model based development tools enforce consistency and structure leading to a quality

    outcome.

    Testing is substantially reduced in a model/pattern environment.

    If the model or pattern works once, it works consistently everywhere its applied.

    If the model isnt changed, there is no need to retest the whole application if a

    change is made elsewhere.

    Productivity gains are enormous. The more complex the development the greater the

    productivity gain.

    Implementation is simplified as a result of:

    The consistent behaviour of model based software.

    The user engagement in the requirements gathering.

    The solution is based on business processes.

    The software is bug free.

    The model generates a new set of program code every time its deployed.

    The code is machine generated so there are no program errors.

    Any software faults will be in the logic of the model.

    The model is corrected and a fresh set of code is generated.

  • Page 12

    On-going cost of ownership is reduced substantially:

    If the business requirement changes simply change the model and redeploy.

    Software compiled by the models is well structured, consistent and stable.

    Every time a model is changed and deployed the software code is totally recompiled

    so there is no build up of old code from previous changes.

  • Page 13

    Managing automated development projects Requirements gathering

    We are translating business requirements directly into modelling tools, so we arent restrained

    by software development considerations when gathering functional requirements.

    We are, therefore, free to define business requirements from a process improvement

    perspective. This approach has many benefits:

    Its an opportunity to completely review existing practices, identifying opportunities

    for process improvement.

    The revised business processes become the specification for the new application.

    Users have been engaged in defining business processes not the system definition.

    When the developed software is delivered, it is seen as facilitating the process

    change users have already identified not the cause of change.

    This last point is very important as user resistance can be a major barrier to a successful

    implementation.

    Business process modelling

    There are many process modelling tools on the market. The problem with many of them is

    that they are far too technical for the business user to relate to. A lot of process modelling

    standards have emerged from IT and are related to software design, rather than business

    process improvement.

    I favour tools and modelling techniques that reflect the user environment and that the user

    can relate to. There is nothing more powerful than having the process model up on a screen

    and working with the users to highlight problems with current practices and opportunities for

    improvement.

    At Circatec we use two tools Microsoft Visio and XSOL InOrder from XSOL in Auckland, New

    Zealand.

    XSOL enables us to model business processes in a graphical format, within a structured model

    that reflects the organisation not the software or database design. This is important if we

    return to primary measure of successful software the value it brings to the organisation.

    A big advantage of the XSOL structure is that a process must be completed in the model, it

    cant be left as a to be resolved later. I have used this feature many times in workshops,

    demonstrating to participants that a process as weve mapped it doesnt work, forcing

    continued discussion and resolution of process conflicts.

    XSOL also captures notes in user defined categories. As we work through the process

    modelling we capture process improvement notes for the implementation, developer notes

    for development team and legislative or compliance notes to ensure the developed

    applications are technically and legislatively correct.

    So when we start building the Plex models or rules engines we know they are based on clearly

    defined and agreed business processes and requirements.

  • Page 14

    Architecture and design

    As with any development, design is critical, however it is very different to the detailed

    architectural design required for a hand coded system.

    In model based development we know the patterns in the automated software development

    tool will provide a consistent architecture and will generate the database from the model.

    Therefore we only need to determine the modules or functional groups and their

    relationships.

    With the model having facilities to call other programs, we only need to consider what

    functions are core and what functions will be external modules or rules engines. The following

    schematic illustrated the conceptual design of a superannuation administration system. Core

    functions in this instance are considered to be 90% stable with incremental enhancement over

    the life of the application. External modules are more volatile. They will be fund specific

    functions or subject to legislative change.

    Database

    Core Functionality

    Fund specific functions

    Fund specific variables

    Legislatively sensitive functions

    Ele

    ctro

    nic

    File

    Man

    ager

    Document management

    Workflow processes & alerts

    User Interface

    Browser

    Workflow

    Data validation and rules layer

    In this instance there are supporting designs for each element of this high level model. The

    graphical designs guide the development of the functional models and subsequently the

    entire application. The designs have been developed in consultation with domain experts and

    developers. It can be printed on a large sheet of paper and put up on the wall.

    This is not just an architectural guide, it is used to structure the work breakdown and work

    allocations to support the modular approach.

  • Page 15

    User interface and patterns

    What is this application to be used for, therefore what are the key requirements for the user

    interface?

    The user interface determines the user experience and therefore user acceptance of the

    system and user efficiency.

    I am past letting developers determine the UI design. So before any development begins, we

    determine the UI requirements for each user group within the organisation. For example:

    The manual entry of large volumes of data from incoming documents calls for a

    single screen laid out to reflect the format of the document. It needs to flow from one

    field to the next in the same flow as the document and it needs appropriate

    validations to prevent data entry errors.

    A call centre needs fast access to the callers details to verify the callers identity, then

    seamless access to every part of the system needed to answer queries or process

    instructions.

    Once these requirements are established the UI patterns are designed. Commonality and

    reuse is the key to good pattern design, determining how many patterns are required and how

    each will behave.

    Here is an example of a UI pattern design. Totally graphical and created with a drawing tool in

    a matter of hours. In this particular application five patterns were drawn up in a little over a

    day.

    This screen, its functions and logic will be reused wherever an enquiry screen is required. In

    this example it will be used for all member, employer, planner and other external party

    enquiries in a superannuation administration system.

  • Page 16

    The buttons include enforced checking and verification of data entry and modifications, a link

    to the document repository to retrieve documents specific to the subject of the enquiry and

    tabs to every system function relating to the enquiry.

    Once the pattern is built it is reused in every function requiring this style of pattern. The

    developer does not have to develop every screen in the system, they take the pattern and

    build the function model within the pattern, suppressing anything in the pattern not required

    for a specific enquiry.

    Development teams

    Small is wonderful when it comes to automated software development teams.

    In a Plex environment one business analyst (domain expert) to two Plex developers is

    regarded as a good balance. In complex functions it can be one to one for a period.

    Whiteboards, pin boards and electronic boards are the key to effective agile development. The

    analyst communicates the requirement to the team and other stakeholders in the most

    effective manner. This can be as simple as notes and flows captured on the electronic board to

    a spreadsheet replicating financial transactions and postings. We seldom replicate source

    documents theres no value in that. For example the Australian Taxation Office has a

    document for every form of federal taxation. The document contains the tax rules, working

    examples and software developer notes. Whether its a Plex model or a rules engine this is all

    the information the development team needs.

    As the business analyst is part of each development team, questions are answered as they

    arise. The BA also does iteration testing so design errors are fixed in the model as they are

    found. In well functioning teams the interaction between the BA and the model developers is

    so dynamic that potential omissions or misunderstandings are rectified before the first

    prototype is generated.

    Building a rules engine is even more dynamic. It is often a one to one ratio with the domain

    expert talking directly to rules engine builder. The rules and calculations can be tested within

    the model so by the time the model is finished it has been tested and validated, ready to

    deploy.

    A major advantage of these small, dynamic teams is the ease with which the client can be

    engaged. A three way meeting of client, analyst and developer can add enormous value and

    impetus to the development of a functional model.

    Models can be refined and code regenerated as often as required to prove the application

    Developer models the requirement

    Direct &unambiguous communication between experts

    Prototypes are developed and refined until the

    model accurately reflects the requirement

    The model is finalised and the

    software is generated

    automatically

    Domain expert talks to the developer

    Developer models the requirement

    Deployed Application

    The lines of communication are short and direct, eliminating the ambiguity and disconnects that are inherent in the traditional SDLC.

  • Page 17

    Planning automated software development

    By now the relative simplicity of the methodologies surrounding automated software

    development is apparent. The same applies to the planning and management of the

    development. By breaking the development into small components, development is

    production, not a project.

    Whats the difference?

    A project allocates resources to task. It assumes a relatively elastic supply of resources, hence

    the tendency to throw large numbers of programmers and contractors at a development, with

    the resulting well documented failure rates.

    Production, on the other hand, allocates tasks to resources. It assumes relatively fixed

    production resources, placing the emphasis on effective use of resources through work

    allocation planning and bottleneck management.

    If we accept that domain knowledge is vital in the development of enterprise applications this

    is the correct view of our software development environment.

    As in any production environment there is selective outsourcing or sub-contracting to manage

    demand peaks or skill shortages.

    In an automated software development environment this approach is extremely effective

    when:

    i) Development tasks are broken down into the smallest practical blocks of work, ideally 1 5 day blocks.

    ii) A block of work is specific to a skill, for example analysis, development, testing, etc.

    iii) Work is allocated to each team based on both availability and domain knowledge. A production planning board shows to whom each task is allocated to and how long they have

    to complete it. The visual planning has significant benefits:

    Who owns the work at any point in the production plan.

    What the forward workload is by person and job.

    The impact of being late.

    The interleaving of jobs for multiple projects or clients is also highly visible and gets away

    from the concept of individual projects vying for shared resources.

    The team leader or production manager knows exactly the state of every job at a glance and

    can take action immediately to alleviate problems. So too does every team member.

    The planning board time scale is days, so every day the whole team can see exactly where

    every job is up to.

    A longer term view is easily set up a maintained in Excel, printed on A3 and posted on the wall.

    Now there is a highly visual, easily maintained view of the immediate and long term work load

    and work allocations with an absolute minimum of management overhead.

    Once again the emphasis is on visibility and simplicity without sacrificing accuracy in any way what-so-ever.

  • Page 18

    The benefits are enormous We benchmarked a recent Circatec development of a superannuation and investment fund

    administration system using function point analysis, an industry standard measure of

    software functionality.

    The benchmark we used to cost a traditional SDLC was the costing model the University of

    Southern California, Center for Systems and Software Engineering refers to as the

    Constructive Cost Modell methodology COCOMOII. (You can access this model using:

    http://diana.nps.edu/~madachy/tools/COCOMOII.php)

    I have used a monthly cost of $22,200 being 18.5 average working days a month at $1,200 a

    day, calculated as the average internal costs of a development team and management

    structure plus employment costs and overhead.

    The model gave these results:

    Work effort in person months

    Development cost

    Benchmark 598 $13,275,600

    Circatec build 97 $2,153,400

    Ratio 16%

    This costing model shows a cost saving of $11.12 million using automated software

    development instead of the traditional SDLC.

    Furthermore:

    The function point count is 2898 which is at the upper end of size and complexity.

    At no stage did we have more than four people working on this development.

    This comparison used cost criteria reflective of a well run project.

    There is ample literature available to support the view that a development of this size

    following a conventional SDLC has an extremely high probability of failure.

    So lets change the costing criteria to be more reflective of a large development team with a

    high number of contractors, limited domain knowledge and increasing time constraints.

    Drivers Benchmark Large team Project under

    pressure

    Team Cohesion High Low Very low

    Analyst capability High Nominal Nominal

    Programmer capability High Nominal Nominal

    Personnel continuity High Low Very low

    Application experience High Low Very low

    Platform experience High Low Low

    Language & toolset experience High Nominal Nominal

    Time constraint High Very High Very High

    Use of software tools High Low Low

    Person months 598 2,706 3,845

    Cost $13.3 million $60.1 million $85.4 million

    The result is a staggering blow out in cost and development effort.

  • Page 19

    Unbelievable? Not really. Regardless of what industry you are in Im sure you know of at least

    one project that has gone this way. If not, look at some high profile public projects, theres at

    least one in every city.

    Suddenly 16% of benchmark using automated software development is now 2-3% of the cost

    of a poorly performing conventional SDLC. The chances of a blow out of this proportion in an

    agile, model based, automated software development are practically nil.

    It doesnt stop here

    The whole structure of the IT department changes in an automated software development

    environment.

    Project managers Arent required for any aspect of the software development cycle. A

    reduced role in requirements gathering and post development

    implementation.

    Software architects On a large or complex development an architect may add value in the

    initial design phase. This is a short term contract position at the most.

    Data modellers Not required as a dedicated role, providing the developers have a

    good understanding of data structures and database conventions

    which they will if the right people have been recruited.

    Testers Testing is incremental and an integral function of the BA / developer

    team. There is value in having a small pre-release testing function

    within the development group.

    Systems support Business focussed requirements gathering, client engagement during

    the development cycle and ease of implementation reduces the

    client support roles.

    The larger the development the greater the savings

    The larger and more complex the requirement, the greater the productivity gains and the gap

    between the traditional SDLC and automated software development.

    Waterfall SDLC

    Agile SDLC

    Automated software development

    Size & complexity of development

    Co

    st

  • Page 20

    There is also much greater certainty in automated software development. The teams are

    smaller, therefore you can be more selective who you engage.

    The agile methodology suggested earlier requires design and work breakdown phases before

    commencing any development work. These will provide a reliable estimate of work plus small

    development iterations for cost and progress monitoring.

    Compelling case for change

    The cost savings are compelling on their own.

    Plus the added benefits of system agility, speed of change and greater engagement of clients

    in the development cycle.

    The business case for a major change to corporate IT is compelling.

  • Page 21

    How corporate IT resists change Theres a compelling business case for automated software development and agile

    methodologies and yet the uptake is minimal. To add insult to injury this is not new

    technology. Most automated software development tools have been commercially available

    for ten to fifteen years!

    Before we consider a significant change to corporate IT we need to identify the barriers to

    change. Yes many of the usual barriers to change exist but there are some unique issues to

    address in bringing about wholesale change to corporate IT.

    Some of the fundamental problems are:

    The very reason many middle and senior IT management roles exist is to manage the

    existing IT structure and workforce.

    The powerbase and kudos of senior developers, system architects, data modellers and

    other specialists relies on preserving the importance of their role and discipline.

    For many consulting firms their entire fee base is built on large, highly structured projects

    and the supply of large number of IT contractors. Theres not a lot of revenue in

    automated software development projects!

    There is a whole industry offering training and accreditation particularly Prince2, APM

    BoK and, to a lesser extent, Agile.

    The market is crowded with consultants and contractors needing clients to buy the whole

    package to ensure a reasonable period of employment.

    So there is a whole industry built around the status quo and they are very effective at blocking

    the uptake of automated software development tools and efficient methodologies.

    Unfortunately they are often the people business executives turn to for advice and they fail

    the trust placed in them.

    Let me give you just two examples. I have amended some details to avoid identifying the

    organisations, however many readers will identify with these.

    1. An organisation has a number of legacy financial products that are largely rules based.

    They want to move them off several old platforms that are becoming more difficult and

    expensive to maintain. The internal cost to redevelop these products to run on a common

    platform was in excess of $2 million dollars, which could not be cost justified.

    A rules engine vendor analysed the rules and estimated it would cost between $2-300,000

    to develop all the legacy products into three or four rules engines. The vendor was willing

    to provide a fixed price quotation and requested access to all the product rules. A systems

    architect blocked any further analysis on the basis that we dont want another

    technology on site.

    So the vendor was not able to speak to the business unit or obtain any further information

    to formalise a quotation nor did the business unit know there was a viable option on

    offer.

  • Page 22

    From a commercial perspective this one person was able to make a totally unreasonable

    and unsupportable decision, effectively forcing the organisation to continue maintaining

    legacy products on legacy systems, whilst continuing to incur the increasing cost of

    maintenance.

    2. A company was developing a web based financial planning tool. A rules engine vendor

    offered to build all the financial calculators for a little over $100,000 and 3 months

    duration. A web developer assured the company they could program the calculators for

    $75,000. Two years and many dollars later the product wasnt complete and the window

    of opportunity had closed.

    These examples illustrate the difficulty vendors of automated software development

    tools face. The people they have to sell to are the very people who dont want the tools in

    the organisation.

    The above examples occurred because corporate IT controlled the information flow into the

    respective organisations.

    Common practices used to maintain control

    Inappropriate use of standards

    No one denies the importance of standards but they must be the right standards for the

    right reasons.

    There are many technology standards that are critical, especially in system to system

    communication or the electronic transfer of data between organisations.

    However many standards and methodologies have evolved from professional bodies or

    special interest groups. Whilst they may add value in some circumstances they are not

    commercial imperatives and are certainly not set in stone.

    For example Wikipedia lists::

    Data modelling 10 techniques and notation standards.

    Process modelling 6 process modelling techniques and 5 languages for BPM

    (Business Process Modelling).

    Software development - 2 methodologies, SDLC and Agile with 5 common SDLC

    development models and 4 common Agile development models.

    Project management 10 common methodologies.

    Why on earth does any organisation want or need to base the selection or development of

    business solutions around any one of the myriad of standards and standard

    methodologies.

    In far too many cases standards are mandated by the IT group because:

    Its the sandpit the technologists want to play in.

    The standard is used to block more efficient options.

    The only justification for mandating a technology or methodology standard in an organisation

    is to genuinely protect or add value to the organisations IT systems.

  • Page 23

    So it is absolutely critical that standards are vigorously challenged to ensure they are aligned

    with the enterprise core business and core enterprise systems. Otherwise they become a tool

    for technologists to block unwelcome automated development tools and efficient

    methodologies.

    Request for Proposal or Tender

    I have seen too many tenders where the technology and methodology sections are as large, or

    larger, than the business requirements.

    These documents are extremely detrimental to the organisation for several reasons:

    1. They shift the focus from a business solution to a technology solution.

    2. They shift control from the business units to IT, enabling IT to protect the status quo.

    3. They often block innovative solutions to business problems.

    For many years Circatec was actively engaged in the specification and selection of medium to

    large ERP systems. In all that time I refused to specify the underlying technology on the basis

    that we were looking for the best possible solution to a business requirement. Having found

    the best solution then wed look at the technological issues.

    If your organisation is looking for a solution to a business problem then dont let your IT

    department write or run the RFP/RFT process. Only allow them to specify technical

    requirements that are absolutely essential and justified from an enterprise perspective. When

    youve identified the functionality you need for the enterprise then engage IT to make it

    happen.

    Do not underestimate how important this is. A number of vendors of automated software

    development tools and/or services are reluctant to respond to a tender run by an internal IT

    department, especially when the technology and methodology sections are deliberately

    prescriptive.

    Circatec recently declined to bid on a development because the added cost of meeting the

    specified implementation and technology standards exceeded the cost to develop the

    solution using automation tools and agile methodologies. We know the project didnt proceed

    because it couldnt be cost justified on responses from complying vendors. We also know

    that we could have provided a solution within a cost justifiable budget if we had been allowed

    to submit a non-complying bid.

    Size of the service provider

    The average application build using automated software development is approximately

    15-20% of the work effort of a standard SDLC build. Rules engines are even faster, often

    just 10-15% of the build time.

    Agile methodologies and business oriented process modelling tools significantly reduce

    the requirements gathering and process improvement activities.

    Rules engines can cut months out of data conversion and rectification.

    It follows that any service provider using or supporting these tools and techniques will be

    small. And so they should be, otherwise they are just another firm looking for billable hours to

    feed a large team.

  • Page 24

    However size is often used against the service provider:

    They arent big enough to support our needs.

    They are too small to really understand our requirements.

    Our preferred supplier has 200 staff so they are better placed to handle our needs.

    (Code for they use the same inefficient practices we do).

    The last thing your IT group wants is a small, agile, highly effective organisation delivering

    more product in a fraction of the time it takes in-house.

    I can do it just as quickly by hand!

    If I had a dollar for every time Ive heard this Id be a rich man!

    It may be true at a micro level but what the programmer is really saying is:

    I can knock up a piece of untested, undocumented code which Ill imbed in the existing

    code with no notation. No one will know its there or what it does, perpetrating my role as

    an indispensable source of knowledge.

    The same is true with data validation, rectification or conversion. Programmers will claim they

    can quickly write a script to carry out a data management function. What they dont say is that

    scripts have about as much integrity as a spreadsheet in that they are difficult to validate and

    have no change control.

    Micro level comparisons quickly become irrelevant. Other than patching an existing program

    there are very few micro developments. If we follow the scripting example a little further what

    we will find is that the single script quickly becomes 100 scripts. New scripts are written to

    replace early scripts and the library of scripts grows, as does the risk of running an obsolete or

    inaccurate script.

    The CEOs toolkit

    Cutting through the screen of misinformation is relatively easy using what I call the CEOs

    toolkit:

    Why

    So what

    Prove it

    The conversation is:

    In plain business language, and in the context of our business needs, tell me:

    Why: are we doing it this way?

    Why: is that important?

    So what: if what you say does or doesnt happen?

    Prove it: with evidence and a business reason to support your opinion.

    You may have to keep asking why to get to the real reason. When you do then you can assess

    whether there is a business case to support the IT position or not.

    Organisations can no longer afford or justify self indulgence by their IT departments and project management offices.

  • Page 25

    Bringing about change Changing the structure and culture of your IT department will not be easy. It will have to be

    driven from the very top Board, CEO and senior management.

    This scale of restructure requires selective redundancies, a potential HR nightmare. The

    management team driving the restructures will need to know they have a board mandate and

    CEO support for the changes they are to make. They may need expert industrial law advice

    and HR guidance.

    It is a daunting but commercially imperative task, and its very achievable.

    Your CIO may actually be your greatest ally. Many CIOs feel they are caught in the middle

    between the pressure from the business units for higher levels of IT service and an inflexible IT

    workforce. They know whats ahead of them if they try to implement change without

    organisational support. On the other hand if your CIO is part of the problem this is your first

    replacement.

    Through recruitment I know there are many experienced developers and business analysts

    excited by the potential of automated software development. Small teams attract and

    stimulate good quality people. Competent people thrive in an environment that values their

    knowledge and skills. You may already have your automated software development teams on

    your staff.

    The key is attitude. A developer or business analyst can lack some specific experience if their

    attitude is a willingness to learn, to take on new methods and challenges and add value to a

    small, self motivated team. This is one of the key criteria in determining which members of

    the existing IT group will transition to the automated software development group.

    Structure

    An agile automated development environment has a very flat management structure.

    A small number of largely self-managed teams (3-5 members) will generate a tremendous

    amount of high quality software. Depending on the number of teams a single development

    manager with possibly a coordinator to handle work allocations, time recording, etc. will be

    the total management structure.

    There is no need for a project management office (PMO) in an automated software

    development environment. There will, of course, be other corporate technology programs to

    manage, however if your PMO sees itself as a control centre, rather than a facilitator,

    dismantle it and start again, otherwise they will just be a barrier to change. Hire project

    managers not methodology practitioners.

    It is questionable whether there is any need for full time software architects and data

    modellers. These are roles that can be outsourced at the commencement of a major

    development to define the design and structural elements of the application. The automated

    software development tools will enable the teams to adhere to these design principles, and

    there will be the skills and understanding in the development teams to apply the design

    guidelines.

    IT infrastructure, network support and end user help desk functions will remain largely

    unchanged.

  • Page 26

    Existing systems

    Existing systems, especially internally developed and/or maintained, will require continuity of

    support. Consider setting up a separate product support group, however it will need strong

    management to contain the maintenance of existing systems. New functionality should be

    developed by the automation teams.

    Alternatively, if enough of the old IT group have been transitioned, maintenance can be

    provided by the new group. The advantage of this is that once developers have absorbed

    themselves in automated software development tools and rules engines they wont want to

    do more than necessary in the legacy system.

    What about work-in-progress?

    If a development is in its early stages, and is more than a few weeks work, then consider

    redeveloping it using automated software development. As a general rule any development

    that hasnt past the 50% complete point will deliver cost savings by starting again in an

    automated environment.

    If a development has passed the 50% complete stage, is on time and budget, and the

    development team is still intact, then it should be completed by the current team.

    If a development is over time and budget and there is no confidence in the expected

    completion date and cost then it should be stopped and reassessed. There is every likelihood

    an automated software development team will redevelop it faster, more cheaply and to a

    higher quality than persevering with it. Quality is often the first casualty of troubled

    development projects and must be a factor in the decision to start again.

    Resistance to change

    Like any form of workplace automation this is a change management project with little room

    for compromise.

    The resistors must be identified and removed if they wont adopt the new methods. Weve all

    met the passive resistors who agree to change in meetings but wont change. There is always

    an excuse and in the meantime they are gathering supporters and the whole rollout is

    jeopardised. Fortunately with automated software development they are easily identified

    they are either using the tools or they arent.

    There is a small but vocal group of IT professionals who can be openly scornful and

    disrespectful of anyone with a different perspective to theirs. I call it technological arrogance

    and these are some of the most difficult people to manage when implementing change on

    this scale, especially when the change diminishes their role and importance in the

    organisation.

    They can be openly hostile and dismissive and their greatest weapon is to resort to techno-

    speak, running out a string of indecipherable reasons why whatever youve just suggested

    wont work. Often their claims are facetious, exaggerated or just plain wrong. Fortunately

    they do tend to make their presence known.

    No organisation is obliged to provide technologists or methodology practitioners their own

    personal sandpits. If someone wont play in the corporate sandpit for the good of the

    organisation there is no place for them in small, agile, self motivated teams.

  • Page 27

    Be prepared to start again

    I have experienced first hand the total block of model based and automated software

    development tools. Its as if the group psyche is total commitment to the status quo, they

    genuinely cant see anything wrong with their work practices and methodologies.

    When this happens an organisation must accept that they may have to shut down a whole IT

    section and/or PMO and start again.

    This may seem drastic, but in these circumstances it will be more effective to start from a zero

    base, recruiting the roles and skills needed for an agile, automated environment.

    Despite a genuine concern around the loss of knowledge of existing systems, losing blockages

    to change in a well managed manner is infinitely more productive than keeping these people

    because of their perceived knowledge. When theyre gone its often surprising how much

    knowledge is retained in the organisation.

    The savings and other tangible benefits are too great to compromise or delay the restructure.

  • Page 28

    What will it take? Ive been critical of corporate ITs resistance to change, but in all fairness in many cases they

    are doing, and delivering, what is asked of them.

    There will be no groundswell of change from within the IT industry.

    It will have to come from boards, CEOs and senior executives.

    As they look at what their organisation has spent and is being asked to spend on IT systems,

    particularly the development and maintenance of software applications.

    Then, knowing there are alternatives, they are entitled to get frustrated, even angry, at the

    impact on the revenue they work so hard to earn and the profit they are expected to deliver,

    they will start to demand change.

    These are mainstream alternatives

    CA Technologies, the developer of CA Plex, is one of the largest independent

    software companies in the world. (Wikipedia)

    IBM reduced development costs by $300 million a year by adopting Agile

    methodologies. (Google How IBM saved $300 million)

    Gartner Group is the leading technology analyst and trend forecaster. A positive

    Gartner review can have a significant impact on the market acceptance of a software

    product or technology.

    None of these are small niche players!

    Change will occur when change is demanded.

    When senior management will not accept multi-million dollar projects blowing out five or

    ten fold.

    When IT budgets are reduced and greater output is expected.

    When business units demand, and expect to receive, rapid delivery of solutions.

    When boards demand greater value for their technology spend.

    Only then will agile, automated and delivery focussed IT groups emerge.

    And emerge they can. There is no practical barriers to the use of model based, automated

    software development tools in business applications including browser and smart phone

    apps.

    The 40% reduction in IT jobs forecast by the Gartner Group in 2005 is now, and was then,

    attainable.

    It only requires organisations to insist on change and empower their senior executives to

    make it happen.

  • Page 29

    About the author Ken Galbraith is the founder of Circatec Pty Ltd, a software development and consulting

    company based in Melbourne, Australia.

    He has been in the software industry for nearly 30 years. During that time he has been an

    owner and/or director of three software development and consulting businesses.

    For fifteen years during the 1980s/90s Ken was involved in enterprise (ERP) systems for

    manufacturing and logistics. This was a period of radical change in manufacturing globally.

    ERP systems and associated process change and consulting services were an integral

    component of the change.

    In 1996 Ken formed the INL Consulting Group, specifying, selecting and implementing ERP

    systems for a number of medium to large manufacturing and logistics companies. During this

    period he personally started working in superannuation and wealth management.

    In 2000 INL Consulting became Circatec. The business focus shifted from consulting to the

    provision of technology and software solutions.

    By 2005 Circatec was using model based development tools in business process automation

    and rules based business solutions. Services expanded into the development of large and

    complex application software. Through this Ken gained enormous experience in the power of

    these tools.

    Circatec was also marketing the toolsets and supporting services. An exercise that provided

    first hand experience in the ability of corporate IT to block the adoption of automated

    software development tools and efficient development methodologies.

    Today Ken, and Circatec, remains committed to changing the software development and

    delivery paradigm by working with senior management to bring about a cultural shift in

    corporate IT.

    You can contact Ken Galbraith through the Circatec website or by email.

    Web: www.circatec.com.au

    Email: [email protected]

    Additional copies of this eBook can be obtained from the Circatec website.


Related Documents