JAB14 Honey I shrunk the code

Honey, I shrunk the code !

JandBeyond 2014, Königstein, 30 May 2014

2004 - 20142004 - 2014

Hi ! :-)

2

Beat Joomler since 12 years (ok, Mambo dinosaur ;-) Top-20 Joomla 1.0.15 and 3.2.0 contributor

(“App-Store” install from web + JUX team) Membre Joomla Security JSST and JBS ‘beat’ on forums, otherwise in Switzerland PhD Swiss Federal Institute of Tech. & Bell Labs Datacoms, Internet, Sécurité & web-apps “Make it as simple as possible, but no less”

I love open-source and work with the community

30.05.14 © Joomlapolis.com 2004-2014 3

Today…

Why are…● Open-source…● Software System Architecture and Design…● Abstraction, Consistency, DRY…● Evolution of Software Engineering…● Methodology and specifcations before implementation…

…so important ?● 2014: Introducing AHA!WOW?

● Features, Examples from CB 2.0 and CBSubs GPL 3.0

● Conclusions

3

30.05.14 © Joomlapolis.com 2004-2014 4

Why Open-source ?

● Freedom 0. run for any purposes

● Freedom 1. study & change to your wish

● Freedom 2. redistribute to help your neighbor

● Freedom. 3. redistribute modifed copies

● You control your computer. Not the other way around!● You own and control your own data: Open-data

● The SaaS problem● Google Reader, Posterous, Google Wave, …

● The free SaaS problem● next: Google Mail ???

30.05.14 © Joomlapolis.com 2004-2014 5

Why Software System Architecture design ?

● Good Architecture brings:● Beauty● Functionality● Consistency● Durability● Happyness !

● Lack of architecture results in:● The reverse (unless you are very lucky !)

● Good architects are hard to f nd and hard to keep.

30.05.14 © Joomlapolis.com 2004-2014 6


● In software good architecture translates in:● Clean vs messy code● Re-usability of code and libraries vs dedicated purpose● Consistent generic API vs unconsistent dedicated API● No copy-paste (DRY) vs copy-pasted redundant code● Aspect-oriented code vs mixed-aspects code● Independant parts vs Tightly-coupled inter-dependant parts

30.05.14 © Joomlapolis.com 2004-2014 7


● Easy ?● Yes, in theory!● In practice, no! :

● History● Can't rewrite all all the time● Need for backwards-compatibility● Lack of time● Large changes and refactorings needed● To complicated to review (thus to be committed)

● But doable!● Example: Observers and Mapper Pattern I introduced for Tags● Used without changes for Content History !

30.05.14 © Joomlapolis.com 2004-2014 8

Why Abstraction, Consistency, DRY ?

● Abstraction is boring● Requires understanding benefts of modelisation● But allows reuse in diferent, unexpected situations

● Consistency is not easy● Requires understanding of the existing and examples● But is a big win for the user experience and new features

● DRY is more work at the frst copy● Hard to understand for graduates of the

“Copy-Paste University”● But brings maintenability and code-reduction

● Less code, Simpler code, Easier to maintain.

30.05.14 © Joomlapolis.com 2004-2014 9

Evolution of Software Engineering

● Pionneering eras

● Permanent Software crisis since the seventies:● Budgets, schedule, productivity, quality, security● Reaching limits of current paradigms and architectures● Reaching limits of current software methodology

● 80's and 90's: No silver bullet● 80's: Software maintenance = 2x development cost● 90's: Costs raised +30%● 95 survey: 50% of operational software not considered as successful● Average software project overshoots by 50%

● 75% of large software are unused or do not meet specs.

30.05.14 © Joomlapolis.com 2004-2014 10


● Houston, we still have a problem:

A 10+ Million $ software project has100% chances to:

● Run late● Be over budget● Be unproductive● Unusable● Have quality issues● Have security (sometimes lethal) issues

IF it even succeeds one day !

30.05.14 © Joomlapolis.com 2004-2014 11


● Many trials to circumvent the crisis:● Methods:

● Structured● Object-oriented● Unit-tests● Output-tests

● Tools (IDE, lint, testing)● Discipline and professionalism● Formal methods● Processes

● No Silver Bullet yet found!

● And there is no single solution to the crisis.

30.05.14 © Joomlapolis.com 2004-2014 12


● Current trends:● Aspects (Aspect Oriented Programming)● Entity – Relation - Processor● Hierarchical Model – View - Controller● Model-driven design

● Model transformation● Code generation

● Dependency Injection Containers● Agile● Extreme Programming● Lightweight coding● Formal methods● Code-generators

30.05.14 © Joomlapolis.com 2004-2014 13


● Next trends: (my own forecast)

● Formal Description Languages● Specifcations-driven interpreters and code-generators● Automated Dependency Injection Containers● Lightweight coding● Domain-specifc Languages

● And in Web User Experience:● More natural interactions● Simpler Uis, but more powerful● Workfow-oriented Uis● Mobile frst

30.05.14 © Joomlapolis.com 2004-2014 14

Why Methodology and specs before implementation ?

● Well…● It's like building a house without drawings !

● Will it hold tight ?● Will workers be happy to frst build then go through a design

review process ?● Will it:

● Look beautiful ?● Be functional ?● Be consistent ?● Durable ?

● Will its users be happy ?

● With careful planning and specs: Yes !

30.05.14 © Joomlapolis.com 2004-2014 15

Enough with problems!… But is there a solution ?

● Remember: There is no silver bullet…

● But we can do much, much, much, much better !

● How ?

● Easy:

1. Specify functionality

2. Review specifcation

3. Design software

4. Review design to respect specs

3. Implement according to design

4. Verify that implementation respects design

● Is that easy ?

● Does it guarantee that implementation respects specs ?

Mor

e

Traditional design● Specify● Design● Implement

(in MVC)

16© Joomlapolis.com 2004-2014

…Unrelated !

30.05.14 © Joomlapolis – 2004 - 2014 17

AHAWOW

Mor

e

Community Builder 2.0 5 Years of research

at Joomlapolis Labs!


Related !

2012-05-20 19© Joomlapolis.com 2004-2014

NO Implementation!

No <?PHP ?> code!

No <script /> code!No SQL queries!

Look Ma !

INT

RO

DU

CIN

G…

AHA?

© Joomlapolis.com 2004-2014 20

AutomatcHomogenousApplicatons

WebOptmizedWorkfows? !

WOW!™

AHA? …Compare… WOW!Classical (H)MVC:

Model+View+ Controller implementation

+ Helpers and Javascript helpers Not hierarchical Not agonistic (PHP) “COPY-PASTE” code Lot of layouting Not formal

AHA?WOW!™ MVC:

Specifcation only

Hierarchical – MVC

Agonistic (portable)

DRY, no specifc code(or very little)

Formal


AH

A?W

OA

!

So……?…The 2’ Example:

Currency rates browser


Mor

e

❶ Model:

<table name="#__currencies" class="Currency">

<columns>

<column name="id" type="sql:int(11)" label="Currency Id" />

<column name="base_currency" type="sql:varchar(3)" label="Base currency" />

<column name="currency" type="sql:varchar(3)" label="Target currency" />

<column name="rate" type="sql:foat" null="true" label="Exchange rate" />

</columns>

</table>


This installs / upgrades database !

Mor

e

❷ Controller:

<action name="show_currencies" request="view" requestparams="showcur" >

<showview view="showcurrencies" mode="show">

<data name="currencies" table="#__currencies">

</showview>

</action>


Controller links Model with View

Mor

e

❸ View:

<view name="showcurrencies" type="params" ui="admin" label="Currencies">

<tablebrowser name="currenciesbrowser" label="Currency exchange rates">

<listfelds>

<rows type="standard" height="auto" limit="15" >

<feld name="" type="rownumber" width="20px" align="right" label="#" />

<feld name="id" type="primarycheckbox" width="20px" />

<feld name="currency" type="string" width="10%" label="Currency" />

<feld name="base_currency" type="string" width="30%" label="Base currency" />

<feld name="rate" type="foat" width="30%" label="Exchange rate" />

<feld name="id" type="int" width="20%" label="Id" />

</rows>

</listfelds>

</tablebrowser>

</view>


Views defne Items to display & Layout

TH

E R

ES

ULT

The result:



CBSubs GPL 3.0 uses already

AHA?WOW!™CB 2.0 now too !

CBSubs Administration: 0 lines of specifc code

2012-05-20 © Joomlapolis.com 2004-2014 28

6900 lines of specifcations (core CBSubs CB plugin) only

– 41 controllers– 50 views– 56 models– 397 “queries”– 650 settings

`ONE’ MORE THING… AHA?WOW DOES MUCH MORE: includes:

– Database-agonistic query-compiler and schema– Formal dataset query specifcations– Inheritence– Permissions & View Access Levels handling

Handles 100% of the Admin interface– Toolbar, Menus– Online Help– All feld Types– Tables, Graphs, Widgets, …

And is event-driven and event-triggering

2012-05-20 © Joomlapolis.com 2004-2014 29

Database scheme description

● Handles:● Creation● Checks● Upgrades● Downgrades

automaticaly

<table name="#__comprofiler_tabs" class="" strict="false" drop="never"> <columns> <column name="tabid" type="sql:int(11)" auto_increment="23" /> <column name="title" type="sql:varchar(50)" /> <column name="description" type="sql:text" null="true" /> <column name="ordering" type="sql:int(11)" /> <column name="ordering_register" type="sql:int(11)" default="10" /> <column name="width" type="sql:varchar(10)" default=".5" /> <column name="enabled" type="sql:tinyint(1)" default="1" /> <column name="pluginclass" type="sql:varchar(255)" null="true" /> <column name="pluginid" type="sql:int(11)" null="true" /> <column name="fields" type="sql:tinyint(1)" default="1" /> <column name="params" type="sql:mediumtext" null="true" /> <column name="sys" type="sql:tinyint(4)" /> <column name="displaytype" type="sql:varchar(255)" /> <column name="position" type="sql:varchar(255)" /> <column name="viewaccesslevel" type="sql:int(10)" unsigned="true" initialvalue="1" initialvaluetype="int" /> <column name="useraccessgroupid" type="sql:int(9)" default="-2" mandatory="false" /> <column name="cssclass" type="sql:varchar(255)" null="true" /> </columns> <indexes> <index name="PRIMARY" type="primary"> <column name="tabid" /> </index> <index name="enabled_position_ordering"> <column name="enabled" /> <column name="position" /> <column name="ordering" /> </index> <index name="orderreg_enabled_pos_order"> <column name="enabled" /> <column name="ordering_register" /> <column name="position" /> <column name="ordering" /> </index> </indexes></table>

© Joomlapolis.com 2004-2014

Database queries compiler

Handles automatically:

• SQL queries • Mix-ins • Searches • Filters

<filter name="tabid" value="tabid" valuetype="sql:string" type="data"> <data name="tab" type="sql:multiplerows" table="#__comprofiler_tabs" key="id"> <rows> <field name="tabid" as="value" /> <field name="title" as="text" /> </rows> <orderby> <field name="position" ordering="ASC" /> <field name="ordering" ordering="ASC" /> </orderby> <where> <column name="fields" operator="=" value="1" type="sql:field" valuetype="const:int" /> </where> </data></filter>

XML Query can be used in TableBrowser:Example: CB 2.0 Tab Manager, with “Tools”

JDAY-FR-2014 © Joomlapolis.com 2004-2014 32

3/4/2011 © Joomlapolis.com 2004-2014 33

Mor

e

XML TableBrowser example: CBSubs™ plans

Presentation:– Hierarchies with sub-plans– Yes-no radios/clickable toggles– Links– Css classes– Text formatting– Hidden columns– Permissions– Dynamic attributes

33

Extending and Inheriting

Two-way inherience: Extend and inherit:

• Explicit (<extend>) • Implicit (<inherit>)

<extend toxpathnode="/cbxml/types" fromxpathnode="/*/types/type" file="xml/models/model.blogs" />

<extend toxpathnode="../following-sibling::*[1]/params/fieldset" fromxpathnode="/cbxml/views/view[@name="radiosettings"]/*" file="xml/admin.lib.processors" mode="prepend">

<inherit name="editgatewayinherit" from="/*/views/view[@name='editgenericgateway']">

Extension example 1: CB Privacy on CB Field

3/4/2011 © Joomlapolis.com 2004-2014 36

Mor

e

Extension example 2: CBSubs™ integrations

CBSubs™ plan edit is automatically extended with available integrations. By pure XML AHAWOW magic:

36

Extension example 3: CB 2.0 plugins can extend Joomla menus


Permissions and View Access Levels

Handles:● Conf guring

● Enforcing<actions ui="admin" permission="core.manage" permissionasset="com_comprofiler"> <action name="users" request="task" action="showusers" permission="core.manage" permissionasset="com_users"> <showview view="showusers" mode="show" /> </action></actions>

<tablebrowser name="fieldsbrowser" table="#__comprofiler_fields" class="CB\Database\Table\FieldsTable" permissionasset="com_comprofiler.fields">

<param name="permissions_fields" type="permissions" assetname="com_comprofiler.fields" label="" display="2lines"> <permission name="core.manage" label="Manage" description="Allows viewing of Fields Management." /> <permission name="core.delete" label="Delete" description="Allows deleting of fields." /> <permission name="core.create" label="Create" description="Allows creating new fields." /> <permission name="core.edit" label="Edit" description="Allows editing existing fields and editing the state of existing fields." /> <permission name="core.edit.state" label="Edit State" description="Allows editing the state of existing fields." /></param>

Toolbars and online Help

<toolbarmenu>:

<toolbarmenu> <menu name="apply" action="apply" task="act=apply" label="Save" /> <menu name="save" action="save" task="act=save" label="Save & Close" /> <menu name="cancel" action="linkAction" task="cancel" link="cb_action:fields" label="Cancel" /> <menu name="help" action="help" ref="https://www.joomlapolis.com/help/cb/2/permissions" label="Help" /></toolbarmenu>

Example XML toolbars: CB 2.0 Field edit


Menu buttons

<menugroup>:

<menugroup name="mailersettingsmenu" type="buttonMenu" label="Mailer menu" description=""> <menu name="mailers" action="cb_action:show_mailers" label="Mailers" description="Configure Mailers"> <fields> <field name="mailers_count" label="active mailers" type="int"> <data name="mailers_count" type="sql:count" table="#__cbsubs_mailer_automessages" key="published" value="1" valuetype="const:int" /> </field> </fields> </menu> <menu name="mailerautomation" action="cb_action:mailer_sendmail" label="Automation Settings" description="Configure Automation for Mailers"> </menu> <menu name="mailersentmessages" action="cb_action:show_mailersentmessages" label="Sent Messages" description="Show mailer sent messages"> <fields> <field name="mailer_used_count" label="messages sent" type="int"> <data name="mailer_used_count" type="sql:count" table="#__cbsubs_mailer_sentmails" key="state" value="A" valuetype="const:string" /> </field> </fields> </menu> <showview name="integrationsglobals" mode="edit" type="plugins" groups="user/plug_cbpaidsubscriptions/plugin,user[cbsubs.]" action="" path="payintegrationmailersettingsmenu" /></menugroup>

30.05.14 © Joomlapolis.com 2004-2014 42

Example menus: CBSubs™ Panel

42

DEMO

http://localhost/~brunner/1.5/administrator/index2.php?option=com_comprofiler&task=editPlugin&cid=500

Bar graphs and graphs

<f eld type=”bargraph”> and <plot>:<field name="graph_paid_amounts" type="bargraph" key="paid_mc_gross_total" width="10%" label="Completed" description="Relative amount to other days displayed." align="left"> <data name="0" type="const:int" /></field>

<field name="id" type="plot" rowspan="all" class="cbregAdminTotalGraph" width="50%" label="Graph" description="This graph shows the table content in a graphical way"> <plot name="front_plot"> <series> <serie label="New"> <data missing="0"> <y name="newregs" type="int" /> <x name="time_paid_date" type="date" /> </data> <lines show="true" /> <points show="true" /> </serie> <serie label="Upgrades"> <data missing="0"> <y name="upgrades" type="int" /> <x name="time_paid_date" type="date" /> </data> <lines show="true" /> <points show="true" /> </serie> </series> <format> <yaxis min="0" tickDecimals="0" /> <xaxis mode="time" timeformat="%d %b %y" /> <grid markings="cbPlotWeekends" /> <legend noColumns="4" position="nw" container="th.cbregAdminTotalGraph" /> </format> </plot></field>

30.05.14 © Joomlapolis.com 2004-2014 44

Example Bargraphs and graphs: CBSubs™ panel

Shows detailed weekly activities:● In numbers, bar-graph and graphs,● Transactions, Payments, Registrations, Upgrades, renewals

44

Events generation

<trigger>:

<trigger group="user/plug_cbpaidsubscriptions/plugin" event="onxmlBeforeCbSubsDisplayOrSaveInvoice" results="extend" toxpathnode="../fieldset" mode="replaceorappend" keyattribute="name" />

30.05.14 © Joomlapolis.com 2004-2014 46

Community Builder 2.0 uses AHAWOW

● Community Builder handles site members:

100 % Libre GPL !● Registration process,user profles, lists, photos, videos, ...

30.05.14 © Joomlapolis.com 2004-2014 47

CBSubs™ GPL 3.0 uses AHAWOW

47

DEMO

http://www.afuj.fr/creeruncompte.html

30.05.14 © Joomlapolis.com 2004-2014 48

CB 2.0 Software Architecture

Joomla!

CBLib 2.0 = CB Framework (Joomla library)

CB 2.0 (Joomla component, modules)

CB Plugin 2.0 (Joomla plugin)

30.05.14 © Joomlapolis.com 2004-2014 49

CB 2.0 Software Architecture

Joomla!

CBLib\Cms

CBLib\CBLib

CBLib\CB

CB 2.0 (component, modules)

CBLibComposer

CB Plugin

Libraries

\AhaWow

30.05.14 © Joomlapolis.com 2004-2014 50

CBLib 2.0 Software principles

● CBLib 2.0 has a modern software architecture:● Leightweight (“Small is beautifull”, completely modular● Uses the new PHP 5.3 functionalities, in particular:

● Namespacing● Interfaces● Closures and function variables

● Uses extensively:● Depedency Injection Containers● Class Interfaces● Inversion of Control● Aspect Oriented Programming● Entity-Relation-Processor● Model-View-Controller

30.05.14 © Joomlapolis.com 2004-2014 51

CBLib 2.0 namespaced packages

● CBLib 2.0 includes mostly independent packages:● AhaWow™ : Specifcation-based execution● Dependency Injection (DI) Containers● Observers+Mappers● Languages (incl. Pluralizations + Multi-keys)● Auto-loaders for PHP, XML and languages● Permissions and Access handler● Database Schema checker and upgrader● Database Query Compiler● Image Library● LESS + Bootstrap 3● Legacy Compatibility Layer

30.05.14 © Joomlapolis.com 2004-2014 53

Conclusion

Using a formal approach it is possible to specify software instead of implementing it.

This has many advantages:

● Beautiful

● Functional

● Consistent

● Durable

● Verifable

● Auditable

And AHA?WOW! ™ is now here after 5 years of research!

Rings a bell ?

30.05.14 © Joomlapolis.com 2004-2014 54


● Good Architecture brings:● Beauty● Functionality● Consistency● Durability● Happyness !

● Lack of architecture results in:● The reverse (unless you are very lucky !)

● Good architects are hard to f nd and hard to keep.

Remember that slide ?

30.05.14 © Joomlapolis – 2004 - 2014 55

Thanks !

CBLib\AhaWow and Community Builder 2.0:An exciting software architecture!

JAB14 Honey I shrunk the code

Technology