F3X12 FLOW3 Project Lifecycle

Project Lifecycle

TheProject LifecycleProject Setup, Deplyoment and Maintenance of FLOW3 projects.

About me

Christopher HlubekCTO networkteam GmbH

FLOW3 & TYPO3 Phoenix Core Team

SoftwareDevelopmentLifecycle

Requirements

Architecture

Design

ImplementationTest

Deploy

Support

Architecture Test Deploy Support

SoftwareDevelopmentLifecycle

Code is not everything

Non-Functional

Project Risks

Framework updates and changes

Bugs in application code

Bugs in framework code

Bugs in 3rd party code

Erroneous deployment ...

Architecture

Compositionof a FLOW3 application

Distribution

Configuration

Build Application Packages

Framework Packages

Community Packages

Data

Web

Project Setup

Create a distribution

git clone --recursive -o typo3 \ git://git.typo3.org/FLOW3/Distributions/Base.git MyDistribution

git remote add origin ssh://myrepo.local/path/to/distribution.git

Versioning with Git

Your Git Server

Distribution

Application Packages

TYPO3 Git Server

Community PackagesTYPO3 / Community Git Server

Framework Packages

simple

Emergency Hotfix - or -

Core Hacks

Your Git Server

Distribution

Application Packages

TYPO3 Git Server

Community PackagesTYPO3 / Community Git Server

Framework Packages

more flexibleVersioning with Git

Package Repository

Git Vendor Branching

[TASK] Lorem ipsum ...[BUGFIX] Some bugfix ...[FEATURE] Some feature ...

[TASK] Latest ...vendor/masterorigin/masterSubmodule

Pointer

git checkout master

git commit ...

git push origin master

Create a hotfix

Package Repository

Git Vendor Branching

[TASK] Lorem ipsum ...[BUGFIX] Some bugfix ...[FEATURE] Some feature ...

[TASK] Latest ...vendor/master

SubmodulePointer [HOTFIX] Custom patchorigin/master

git fetch vendor

git merge vendor/master master

git push origin master

Update the Package

Package Repository

Git Vendor Branching

[TASK] Lorem ipsum ...[BUGFIX] Some bugfix ...[FEATURE] Some feature ...

[TASK] Latest ...

[HOTFIX] Custom patch

origin/masterSubmodule

Pointer Merge remote-tracking ...

[TASK] New change ...vendor/master

Test

Unit Tests vs. Functional Tests

FLOW3 Framework

Application Package

Application Class / Unit

Database

Functional Test

Unit Test}

Unit Tests

Test complex logic in isolation

Find the source of a problem

Should test a restricted set of classes

Functional Tests

Test code with the framework and database

Check that a feature works

Isolation is difficult

Use mocks or fixtures for data or external services

Where to test? High risk of failure

High complexity

Hard to test manually

More TestsLess Tests

Test strategies

101

Don‘t test everythingTest the important stuff

Test every feature with one functional test

(at least)

Reproduce bugswith tests

(they won‘t come back)

Deployment

Deployment Deployment

Deployment Deployment Deployment

Environments

Live Staging

Integration Load-Testing

DevelopmentDevelopment

Manual deployment# ssh [email protected] $ cd /var/www/projectuser123 $ git pull --rebaseuser123 $ git submodule inituser123 $ git submodule updateuser123 $ rm -rf Data/Temporary/Productionuser123 $ ./flow3 doctrine:migrateuser123 $ ./flow3 cache:warmup

# ssh [email protected] $ cd /home/myproject/public_htmluser123 $ git pull --rebaseuser123 $ git submodule inituser123 $ git submodule updateuser123 $ rm -rf Data/Temporary/Productionuser123 $ ./flow3 doctrine:migrateuser123 $ ./flow3 cache:warmup

# ssh [email protected] $ cd /home/myproject/public_htmluser123 $ git pull --rebaseuser123 $ git submodule inituser123 $ git submodule updateuser123 $ rm -rf Data/Temporary/Productionuser123 $ ./flow3 doctrine:migrateuser123 $ ./flow3 cache:warmup

Lot of workErroneous

Automated deployment

A custom script?

Rollback on errors?

Distributed deployment?

Meet Surf

Deployment with Surf

SSH

CI Server /Dev Host

Surf

Node

Node

Git Repositories Git

The Surf Model

ServerNode

Code / ArtifactsApplication

Node

Deployment EnvironmentsDeployment

Wor

kflo

w

Deployment process

initialize

update

migrate

finalize

test

switch

cleanup

Create release structure

Git update

Migration (e.g. Doctrine)

Finalize release

Test release (smoke test)

Publish release to live state

Cleanup old releases

Stages

INFO Deploying vagrantINFO Using workflow "Simple workflow"DEBUG ====== Stage initialize ======DEBUG **** Node vagrant ****DEBUG * Application FLOW3 *DEBUG Executing stage "initialize" on "vagrant" for application FLOW3DEBUG Execute task "typo3.surf:createdirectories" on "vagrant" for application "FLOW3"DEBUG $vagrant: "test -d /home/planetflow3/deploy"DEBUG $vagrant: "mkdir -p /home/planetflow3/deploy/releases;mkdir -p /home/planetflow3/deploy/shared;mkdir -p /home/planetflow3/deploy/releases/20120331105449;cd /home/planetflow3/deploy/releases;ln -snf ./20120331105449 next"DEBUG Execute task "typo3.surf:flow3:createdirectories" on "vagrant" for application "FLOW3"DEBUG $vagrant: "mkdir -p /home/planetflow3/deploy/shared/Data/Logs;mkdir -p /home/planetflow3/deploy/shared/Data/Persistent;mkdir -p /home/planetflow3/deploy/shared/Configuration"DEBUG ====== Stage update ======DEBUG **** Node vagrant ****DEBUG * Application FLOW3 *DEBUG Executing stage "update" on "vagrant" for application FLOW3DEBUG Execute task "typo3.surf:gitcheckout" on "vagrant" for application "FLOW3"DEBUG $vagrant: "git ls-remote git://github.com/chlu/Planetflow3-Distribution.git refs/heads/master | awk '{print $1 }'"INFO > 7662553d5e666261a3a5bc2d6122096e7e21e40aDEBUG $vagrant: "if [ -d /home/planetflow3/deploy/cache/localgitclone ]; then cd /home/planetflow3/deploy/cache/localgitclone && git fetch -q origin && git reset -q --hard 7662553d5e666261a3a5bc2d6122096e7e21e40a && git submodule -q init && for mod in `git submodule status | awk '{ print $2 }'`; do git config -f .git/config submodule.${mod}.url `git config -f .gitmodules --get submodule.${mod}.url` && echo synced $mod; done && git submodule -q sync && git submodule -q update && git clean -q -d -x -ff; else git clone -q git://github.com/chlu/Planetflow3-Distribution.git /home/planetflow3/deploy/cache/localgitclone && cd /home/planetflow3/deploy/cache/localgitclone && git checkout -q -b deploy 7662553d5e666261a3a5bc2d6122096e7e21e40a && git submodule -q init && git submodule -q sync && git submodule -q update; fi"INFO > synced Build/CommonINFO > synced Packages/Application/LibtextcatINFO > synced Packages/Application/Planetflow3INFO > synced Packages/Framework/Doctrine.CommonINFO > synced Packages/Framework/Doctrine.DBALINFO > synced Packages/Framework/Doctrine.ORMINFO > synced Packages/Framework/Symfony.Component.YamlINFO > synced Packages/Framework/TYPO3.FLOW3INFO > synced Packages/Framework/TYPO3.FluidINFO > synced Packages/Framework/TYPO3.PartyDEBUG $vagrant: "cp -RPp /home/planetflow3/deploy/cache/localgitclone/. /home/planetflow3/deploy/releases/20120331105449 && (echo 7662553d5e666261a3a5bc2d6122096e7e21e40a > /home/planetflow3/deploy/releases/20120331105449REVISION)"DEBUG Task "typo3.surf:flow3:symlinkdata" after "typo3.surf:gitcheckout" for application FLOW3DEBUG Execute task "typo3.surf:flow3:symlinkdata" on "vagrant" for application "FLOW3"DEBUG $vagrant: "mkdir -p /home/planetflow3/deploy/releases/20120331105449/Data;cd /home/planetflow3/deploy/releases/20120331105449;ln -sf ../../../shared/Data/Logs ./Data/Logs;ln -sf ../../../shared/Data/Persistent ./Data/Persistent"DEBUG Task "typo3.surf:flow3:symlinkconfiguration" after "typo3.surf:gitcheckout" for application FLOW3DEBUG Execute task "typo3.surf:flow3:symlinkconfiguration" on "vagrant" for application "FLOW3"DEBUG $vagrant: "cd /home/planetflow3/deploy/releases/20120331105449/Configuration;rm -f Production/*;rmdir Production;mkdir -p ../../../shared/Configuration/Production;ln -snf ../../../shared/Configuration/Production Production"INFO > rmdir: failed to remove `Production': No such file or directoryDEBUG ====== Stage migrate ======DEBUG **** Node vagrant ****DEBUG * Application FLOW3 *DEBUG Executing stage "migrate" on "vagrant" for application FLOW3DEBUG Execute task "typo3.surf:flow3:migrate" on "vagrant" for application "FLOW3"DEBUG $vagrant: "cd /home/planetflow3/deploy/releases/20120331105449 && FLOW3_CONTEXT=Production ./flow3 typo3.flow3:doctrine:migrate"

Deployment structure

< Document Root

Deployment Features

Clustered deployment

Configuration overrides

Smoke tests

Custom tasks

Flexible workflows

Questions?

?