Code generation tools

Code generation tools

Galit Keret

Moti Zviling

code generation tools 2

Contents Why is code generation technology required? What is it and what kind of a solution it offers?

CGN – Code generation network Features of a good code generation tool Related terminology

Active and passive code generation RAD and CASE

Code generation products How to find the right tool for your needs? Code generation models

Code samples for each model Partial Products list for each model

So why is it not vastly used? Demo

Keret


Sometimes it’s a bummer to be a software engineer…

Ever felt like this?

Writing thousands and thousands of tedious code, your deadline is impossible to reach (as always), and in the end you found yourself facing dozens of bugs and worse than that - you discover your application’s performance is poor…


Some scary facts about software world

(from OOD course) 84% of software projects are not on time 31% of software projects never complete ~60% of completed code is never used ~200 Billion $ a year lost to software bugs

Let’s say this in words: Most software is buggy, unstable and insecure A lot of software is totally unusable

Introducing a new concept: Code generation tools.


What is “Code generation ?”

Strictly speaking: Code generation is the technique of building code not manually but using other programs.

You simply need to define a specification of abstract requirements to the tool. Using templates, it will then build one or more output files based on your requirements .

Saves you the need to write the tedious parts or parts that are not in the areas of your expertise.


CGN – Code generation network

Covers all aspects of code generation:

Includes interviews with engineers using and building code generators as well as articles, tutorials and example code.

Holds an alphabetically database of generators and a list of books about code generation.


Features of a good code generation tool

Quality - Generated code always increases in quality over time because as bugs or shortcomings are found they can be fixed in the code templates and applied across the code base.

Consistency - Generated APIs are consistent in class structure, variable naming and signatures.

Productivity – Generators build code in a fraction of the time that it takes an

engineer to produce the equivalent amount of code. Generators allow the engineer to offload grunt-work so that

they can concentrate on tasks that require more creative solutions.

Save testing time – don’t need to test the generated code. Abstraction - Generators provide a layer of abstraction between

the design and the code base. The definition files used by these generators hold the schema and business rules for your application in a declarative form.


Related terminology:Active and passive code generators

There are two types of code generators: Active generators - intended to be run multiple

times on the same output code as the design of the input or generator changes.

Passive generators - create the initial code, and then it is the responsibility of the developer to maintain it (example: a wizard).

Although both types will provide an initial boost to productivity, active generation is superior , because in this type of code generation the code is maintained and bugs in the templates can be rolled out across the whole code base.


Active and passive code generation workflow

The standard workflow in software engineering:edit-compile-test.

The workflow cycle of a passive generator:run the generator, then follow the edit-compile-test cycle repeatedly on the code created by the generator.

The workflow cycle of an active generator:run the generator, then compile, edit and test the output. Found a problem in the generated code?alter the templates or input of the generator and re-generate. Found a problem in the hand-written code portion? follow the standard edit-compile-test workflow.


Related terminology:RAD and CASE

CASE - computer assisted software engineering Similar to code generation tools. The analyst could define the business logic using

visual tools and a generator would build all of the application code.

RAD – Rapid application development An approach to building computer systems which

combines CG/CASE tools and techniques, user-driven prototyping, and stringent project delivery time limits.

The typical RAD project delivers a fully functional computer system in just a few days!


CASE v. Code generation tools

In both methods a generator build an application from an abstract description.

The difference is in the approach:

CASE tools attempt to replace trained engineers with tools that do the same work.

CG tools are built by engineers for themselves – to make their life easier.

Both tools can end up in the same place!


Some facts about code generation products

There are almost 200 code generation tools in the code generation network, and this list is far from being complete.Yet only few engineers know about code generators and fewer still have used them.CGT claims that the reason for this is lack of advertisement and clear information about it.

The most common uses of code generators are database access and UI generators.

There are more generators written for Java users than all of the other technologies combined.


Finding the right code generatorfor your needs

The code generation decision tree will help you look for the right generation tool for your project.

If none of the generators listed address your needs you can use one of the ‘custom code generators’ which supports scripting in a variety of languages.

Failing that, you can always write your own generator…(just be sure to first read the DO IT YOURSELF section at the Code generation network)


Code generation models

1. Code munger

2. Inline code expander

3. Mixed code generator

4. Partial class generator

5. Tier generator

6. Domain language


Code Munger Model

The most common form of code generators The code munger reads the source file,

analyzes it, and creates one or more output files.

Processing Flow:


A code munger model example –a translator

A translator takes a code written in some programming language and translates it to a code written in some other language.

The input and output of a ‘C to Java’ translator will look like this:

Inputchar c;

int x;

int foo(int y, char d)

{

int i;

for (i=0; i<3; i=i+1)

{ x=3; y=5; }

}

outputclass Wrapper {

char c;

int x;

int foo(int y, char d)

{

int i;

for (i = 0; i < 3; i = i + 1)

{ x = 3; y = 5; }

}

}


Some Code Munger Model products

JavaDoc: reads and parses the comments within a Java source code file, then creates HTML documentation. very easy to learn and use.

ANTLR: helps to build programming language translator. Generates recognizers in Java, C++, C#, and soon Python.

C2J: a C/C++ to Java translator.The generated code is not understandable!

Jazillian: a C to Java translator.The generated code is natural (as if it was written by hand), but there are some minor translation bugs.(presented in demo)


Inline Code Expander Model - overview

It starts with designing a new language. Usually this new language is an existing language with some syntax extensions. The inline code expander is then used to turn this language into production code in a high level language.

Processing Flow: The input is a source file for a new

language. The output is a source file in a high

level language suitable for

compilation.


An Inline Code Expander example:

C with embedded SQL

Embedded SQL is a method of combining the computing power of a high-level language like Java or C/C++ and the database manipulation capabilities of SQL. It allows you to execute any SQL statement from an application program.

In the example code, SQL is embedded into C source code files using special markup. These hybrid C files are given their own extension, such as '.sqlc‘:

// main.sqlc

int main( int argc, char *argv[] ) {

< SQL select * from users > return 0;

}


An Inline Code Expander example - continued

After running the generator the corresponding created C file will look like this:

#include "db.h“

int main( int argc, char *argv[] ) {

DBHandle *db_handle = db_connect(); DBQueryDara *db_data = db_db_query( db_handle, "select * from users" ); for( int record = 0; record < db_data->length; record++ )

{ // fetch and process data } return 0;

}


Some Inline Code Expander products

Pro*C (Oracle) - Allows embedding of SQL into C source code. It takes some time to learn the

Pro*C syntax, which is not intuitive, and it’s difficult to use in complex operations and also the generated code is not object oriented. Only 2 links in Google refer to this technology which means it’s not a success story.

SQLJ (Oracle) - Allows embedding of SQL into Java source code. a more intuitive syntax and some what more object-oriented approach to the Pro*C way of embedding SQL statements in programs, but still the recommendation is to use it only when the data being mapped is quite simple (don’t use it in case of multiple mappings of Java objects to tables and vice versa)


Mixed Code Generator Model

The same processing flow as the Inline Code Expander, except that the input file is a real source file that can be compiled and run.

This technique can be used for a variety of purposes: Building the user interface glue code to attach

variables to user interface controls. Building test case code from test case data

stored in comments. Creating database access code from access

specification stored in comments.


Mixed Code Generator : example

Mixed Code Generator

After running the generator it will fill the space between the comments with an implementation of the requested SQL statement (it will look like the output showed in the previous example). The comments are preserved so that if the generator is run again, the same replacement can be done.

#include "db.h"int main( int argc, char *argv[] )

{

// SQL: select * from users

// SQL end

return 0; }

an example input file for the SQL handling mixed code generator:


Mixed Code Generators v. Inline Code Expanders

a major disadvantage of an Inline Code Expander is that the debugging is done against the output file. This means that you need to integrate any changes you make to the output code back into the input file after debugging is finished.

In mixed code generation model the source code can be debugged directly (the input and output files are the same). They can also integrate nicely into IDEs as external tools.


Mixed code generators v. Other generators

Most generators build or manage entire files or even generate a whole application tier.

This creates a barrier to entry of using code generators on existing products which don't take well the fact of having large sections of code replaced completely.

Mixed code generation tools provide genuine aid in developing both new and existing applications, by enabling code generation for multiple small code fragments within a single file or distributed throughout multiple files.


Some Mixed Code Generator products

Codify - A code generator that integrates into the Visual Studio .NET framework which can generate multiple code fragments within a single file.

For example: can generate the code for

get/set methods of class properties.


Partial Class Generator Model

A partial class generator builds a set of base classes that are used as a platform to build the production classes.

The base classes are designed to do the majority of the low level work of the completed class, leaving the derived class free to override specific behaviors on a case-by-case basis.


Partial Class Generator – processing flow

The generator reads a definition file, then using a set of templates,

builds one or more base classes.

These base classes are then augmented with derived classes

All of the classes are then compiled together to build the final

product.


Partial class generator – database access classes

An typical use of a partial class generator is to build database access classes.

The generator builds the basic persistence code for each class and its fields. The derived classes are responsible for any custom behavior.


Some Partial Class Generator products

Expert Coder – a toolkit that supports the creation of code generators

Codegen - an open source (LGPL), UML Class Diagram to C#, Java and VB.NET code generation framework.


Tier Generator Model

A tier generator builds and maintains an entire tier within an application.

Processing flow:

The tier generator takes an abstract definition file and using a set of templates builds one or more output files that constitute all of the functionality for an entire tier of the application.


Example of a tier generation tool

a good database/web application is built as a n-tiers application (e.g., database access, business logic and user interface).

Most of the tier generators builds the database access tier or the user interface tier of such applications.

There are so many CG tools that generate the database access tier for database applications that they even got a special name:O/R mapping tools (“Object to Relational mapping”)


Tier generation example (continued)

O/R mapping tool - a code generation tool that connects to your database and reads its schema, then lets you map objects to database tables and views, specify insert, update, load, and delete operations, queries and stored procedure calls, as methods to these objects.

It also lets you define the relationships between objects (one-to-one, one-to-many, etc.) based on relationships between tables in the database.

It then generates fully working persistence objects code for you.



An O/R mapping tool processing flow:



Below is an example of code a user will write, using an O/R mapping tool:

try {

Employees objInfo = new Employees;()

EmployeesFactory objFactory = new EmployeesFactory;()

objInfo.EmployeeID = EmployeeID;objFactory.Load(objInfo); // code here to use the “objInfo” object } catch(Exception ex) {

// code here to handle the exceptionreturn;

}


Some Tier generation products

Tier developper – an O/R mapping tool that runs on.NET platform. Very easy to learn and use. Excellent demos are available on site.

XLInesoft ASPRunner Professional – Creates a set of ASP pages to access and modify MS Access, MS SQL, Oracle and other databases. Also very easy to learn and use but has less configuration options and generates less languages than ‘Tier developper’. (presented in demo)

BrightSword™ Designer – build database-driven web applications in ASP, ASP.NET, JSP and PHP.Not user friendly as ASPRunner and the trial version crashes.


Tier generators Vs. Partial class generators

Tier generators are harder to write: Tier generator maintains all of the tier’s code, including all edge

cases. Partial class generator follows the 80/20 rule: it handles 80% of the

standard cases, and the rest 20% is handled by customized derived classes.

But tier generators are better: Better business rules abstraction Easy to port the application to other platforms

The result: generators often start as partial class generators. As the code solidifies and the problem domain becomes better understood the 20% become smaller and smaller until the whole tier is auto-generated.


A domain specific language model

The tool provides the user with a new language that has types, syntax and operations that map directly to the concepts of the user’s domain.

Examples: Mathematica and Matlab These languages make it easy for the scientist to represent constructs that are difficult to code in general purpose languages, such as multi-dimensional matrix math.

Another example, although not as clean as the pervious example because it obligates the user to describe her domain in terms of classes and relationships between classes is MDA.


MDA - Model-Driven Architecture

MDA is a set of related standards specified by the Object Management Group (OMG).

These standards: UML, XMI, MOF, OCL, CWM are used to turn a model written in Unified Modeling Language [UML] to engineering artifacts in the form of source code, or documentation.

MDA is targeted mainly at the professional production languages: Java, C#, and C++.


MDA : the 3 tiers model

The MDA 3-tiers model for code generation:

How is it done? Using XMI

XMI is an XML based export standard for UML models. The platform-independent UML model is maintained in the Meta

Object Facility (MOF) repository and exported via XMI to the generator.

The generator then reads the XMI and applies transformations to it to create the platform-specific model. This model is then used as input to a set of templates which build the output code.

Platform independent model (PIM)

Platform specific model (PSM)

Templates Code


MDA processing flow


MDA processing flow and the 3 tiers model


Some MDA products

IBM Rational Rose - a popular UML to C/C++ generator. Undo/redo operations on the UML diagrams are not supported.

BridgePoint – a UML to C/C++ generator. A good product but no free trial version.

UModel by Altova – a UML to Java and vice versa generator. Very easy to learn and use, supports undo/redo operations

(presented in demo)

BridgePoint


If it’s so great than why aren’t we using it?

8 traditional arguments against using code generation:

Why not just use cut and paste? Code generation doesn’t work on existing code bases. The quality of the output code is bad. Generators are too complex. Generators are great in the beginning, but they are never

maintained (Engineers will ignore the 'do not edit' comments).

My application isn't complex enough for generation. My application is too complex for generation. Code generation will eat my job .


Summary of tested products

Product

LinkModel

Application

Type

Target

Language

Bottom

Line

Jazillian Code Munger

a C to Java translator

Java Some minor translation bugs

C2J Code Munger

a C to Java translator

Java Generates strange code!

JavaDoc Code Munger

creates HTML documentation

for Java APIsJava

Pro*C (Oracle)

Inline Code Expander

DB access code:

embedding of SQL into C

C

The syntax is not intuitive (much difficult to learn than SQLJ)

Legend: Recommended Worth trying don’t try


Products summary (continued)Product

LinkModel

Application

Type

Target

Language

Bottom

Line

SQLJ (Oracle)

Inline Code Expander

DB access code:

Embedding of SQL into Java

Javagood only when the data being mapped is quite simple

Codify Mixed Code Generator

Any code.

Integrates into Visual Studio .NET

C++ C# Java

Built in templates are for get/set methods only. Write other templates you’ll need by using the templates editor

Codegen Partial class generator

Any code.

UML to C#, Java and VB.NET source code

C# Java VB.NETAn open source

so worth trying but was not tested by us

Tier developper

Tier generator

DB access + UI code:

an O/R mapping tool that runs on .NET platform.

VB.NET SQL ASP.NET Visual Basic C# C++


Products summary (continued)

Product

LinkModel

Application

Type

Target

Language

Bottom

Line

ASPRunner

Professional

Tier generator


build database-driven

web applications

ASP ASP.NET Supports only ASP output

BrightSword™ Designer

Tier generator


build database-driven

web applications

VB.NET PHP ASP.NET ASP JSP C#

Trial version

crashes

IBM Rational Rose

MDA

generator

Any code:

UML to C/C++ generator

C/C++ No undo/ redo operations

UModel by Altova

MDA

generator

Any code:

UML to Java generator

Java


Demo Part


Types of code generation

A reminder:Code generators are programs that automatically generate high level code

(e.g. C, C++, C#, Java, Perl, Python, Ruby, etc.).

List of code generation applications: http://www.codegeneration.net/generators.php


Code generation – decision tree


ASPRunnerPro


ASPRunnerPro - description

ASPrunnerPro is a database management tool that provides easy access and manipulation possibilities for any database on the Web. Designed to suit all users from beginners to experienced developers, ASPRunnerPro creates Active Server Pages (ASP) enabling users to search, edit, delete and add data to the Oracle, SQL Server, MS Access, DB2, or MySQL databases.

Sample generated ASP pages: http://www.xlinesoft.com/asprunnerpro/screenshots.htm

Download a trial version from:http://www.xlinesoft.com/asprunnerpro/download.htm


Product Features: Easy to use wizard-like interface. Generates 100% pure ASP code. Numerous search modes. Add, view, edit, copy and delete pages. Built-in FTP Client to upload ASP

pages to the Web server. User self-register page, password reminder,

change password pages. Multilingual templates. Ability to choose language while logging

in. Creates password protected ASP pages. A very simple installation.

ASPRunnerPro - features

The ASPRunner wizard


Jazillian – C to java translator

http://www.jazillian.com/trial.html


Jazillian – Description

Jazillian takes the C source files that you provide and applies a series of transformation rules to convert various C constructs and patterns into their Java equivalents.


Jazillian – Example1

Example 1: "Hello, World" The classic C program: main(char *argv[], int argc) { printf("Hello, world!\n"); }

Becomes the classic Java program: public class Hello { public static void main(String[] args)

{ System.out.println("Hello, world!"); } }

In this case, Jazillian applied the following "rules": Assuming the file was called "hello.c", The Java file "Hello.java" was created. The signature of the main() function was changed. The printf() call was changed to a System.out.println() call. The function was enclosed inside a new "Hello" class. The key to Jazillian is that it produces not just correct, but reasonable Java

code. Each of the three "rules" applied here illustrates this.


UModel-Altova - description

UModel™ 2005 is an affordable UML modeling application with a rich visual interface and superior usability features to help level the UML learning curve, and includes many high-end functions to empower users with the most practical aspects of the UML 2.0 specification.


UModel-Altova - features

UModel™ 2005 supports: ·context sensitive entry helpers ·syntax coloring in diagrams ·customizable design elements ·unlimited Undo and Redo ·sophisticated Java code generation from models ·reverse engineering of existing Java source code ·complete round-trip processing allowing code, and model merging

These capabilities allow developers, including those new to software modeling, to quickly leverage UML to enhance productivity and maximize their results.


Elevator Problem demo

Problem description: control elevators in a building with m floors.

Constrains: Each elevator has a set of m buttons Each button is illuminated when pressed

(moving to the desired floor) Illumination is canceled when elevator visits the

corresponding floor.


Elevator Problem (continued)

Each floor (except first and top) has 2 buttons: request up-elevator, request down-elevator

Buttons are illuminated when pressed Illumination is canceled when elevator visits the

floor and then moves in the desired direction. When elevator has no request, it remains at its

current floor with its doors closed.


Elevator Problem (continued)

Analysis in UML (unified modeling language) UML: specifies semantics and notation but no

process is currently defined. We will show:

Use case diagrams Class diagrams Collaboration diagrams (sequence and state diagrams will not be shown) Generating code from the above diagrams


Use Case Diagrams

Generalized description of how a system will be used Provided an overview of the intended functionality of the system Understandable by beginners and professionals Elevator scenario is extracted as follows: 1. Passenger pressed floor button 2. Elevator system detects floor button pressed 3. Elevator moves to the floor 4. Elevator doors open 5. Passenger gets in and presses elevator button 6. Elevator doors closes 7. Elevator moves to required floor 8. Elevator doors open 9. Passenger gets out 10. Elevator doors closes


Class diagrams

Static structure of the object Internal structure Relationships among objects


More Tools in a Nutshell


Camino – Tier generation


Camino – Tier generation

Integrated Support for Struts-Tiles Camino allows you to manage multiple Tiles definition XML

files for multiple Struts sub-apps/modules. Document Generation Wizard

With a push of button, you can easily create design documents of your web-application in HTML.

Based on the configurations of the applications and the annotations you added, Camino automatically generates the index pages, hyperlinks for all the cross-references, storyboard images and description pages.

Application Model Verifier Catch hard-to-find mistakes and verify referential integrity of

the configurations of your applications as early as possible. Camino Pro verifies all of your code artifacts and notifies you

errors and potential errors.


Camino – Integrated Support for Struts-Tiles


Camino – Document Generation Wizard


Camino – Application Model Verifier


AndroMDA – Tier generation

Target – any code Targets – JAVA Description: reads a UML model (exported

as XML) and creates Java code. Home page: http://www.andromda.org/


Albatross – Tier generation

Target – any code Targets – Asp.net, C#, SQL, VB.net Description: Build wizards to generate the code. Albatross has a script language to generate code. Input formats: Source Code, XML, Custom Allowed customization: Code Generation

templates, Type of files generated, Input format, Output format, Input processing.

Home page: Http://www.thecodegenerator.com


Bridge Point – Tier generation

Target – Full web application Implementation language - C Targets: C & C++ Description: An MDA model compiler that

builds C and C++ from UML models.

Home page: http://www.projtech.com


Modelisoft – Tier generation

Target – Full web application Implementation language - C Targets: Database,C# Description: Able to make design

decisions controlled by UML stereotypes. In conjunction with the Modelisoft AutoGUI library, it can produce a Windows application allowing browsing the model

Home page: http://www.modelisoft.com


JavaDoc – a code munger example

The most common form of code generators. A code munger processes one or more

source code files and generates some number of corresponding output files. JavaDoc is an example of a code munger. JavaDoc reads and parses the comments within a Java source code file, then creates HTML documentation from the comments using a set of templates.


The processing flow of JavaDoc Code munging:

The Java files come into the generator, the comments are parsedand the output HTML files are created using a set of templates


Article links

Many ideas and phrases included in this presentation and in the web site link were originated in the following articles/links:

Code Generator Models FAQs about code generations One page guide to code generators MDA generation Generating Java and XML Using StringTemplate Language Translation Using ANTLR and

StringTemplate Tool libraries for a code generation tool developer Top 10 Must Have Features in O/R Mapping Tools


Thank You

The END!Questions ?

Code generation tools

Documents

aspects of code generation

code generation toolsfeatures

code generation toolswhat

code generation technology

output code

initial code

example code

code base