Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm
Post on 08-May-2015
2267 Views
Preview:
DESCRIPTION
Transcript
Configuration Managementwith Subversion and RPM
www.immobilienscout24.de
Berlin | 19.05.2011 | Schlomo SchapiroSystemarchitekt, Open Source Evangelist
License: http://creativecommons.org/licenses/by-nc-nd/3.0/
The Linux Way of Doing Things
Slide 2 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Agenda
www.immobilienscout24.de: Behind the scenes
The Linux Way of Doing Things
RPM Deployment
Configuration Management in Subversion
YADT – an Augmented Deployment Tool
Slide 3 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
www.immobilienscout24.de
1.7 billion page impressions per monthhttp://www.immobilienscout24.de/de/anbieten/kennzahlen.jsp
Linux and standard open source toolchain
RHEL
LAMP server
Tomcat, Spring, Hibernate …
MySQL and Oracle
2 data centres with async WAN, >600 VM, SAN, NAS …
24/7 operations
Agile development, now going toward „Continuous Delivery“
Automate, Automate, Automate
Slide 4 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Slide 5 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
What problem do we solve?
Web
Container
Java VM
Linux OS
Physical/Virtual HW
Virtualization Layer
PHP/Perl
Apache
PHP AppStandalone
MySQL
Syslog SNMP
Postfix
Standalone
Manage entire platform above (virtual) hardware layer
Slide 6 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
The UNIX Way of Thinking
Tomcat
Apache Java VM
AppWeb Tomcat
Web
PHP/Perl
Web
MySQL Syslog Net-SNMP
PostfixSSHStandalone
Linux OS
physical/virtual HW
On the Linux level
If we look very closely
Everything is just a file!
And think UNIX
Programs
Libraries
Software configuration
Operational configuration
Environmental configuration
Even Startup dependencies
And also what to run and when
Reboot is the final test!
Slide 7 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Linux tools we like to use
Software Management:
RPM Packages
Versioning & Transactions
Validation and Rollback
Deploy & run actions: post-install, pre-remove, pre-install …
YUM Repositories
Proven to work with millions of systems
Release Upgrades, Repo Mirrors, Downgrades, Provisioning …
Bash & Python scripts:
/etc/init.d scripts for service management
YADT for multi-server system management
Automate, Automate, Automate :-)
Slide 8 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Configuration = Software
Same tools usable for software and configuration
One tool to rule them all
Software & Configuration
Dependencies between software and configuration
RPM file conflicts enforce separation of concerns→
Doing this with a 100% coverage is probably a new idea!
Slide 9 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Configuration Management in SVN
ConfigSVN
post-commitYUM repos
Slide 10 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Big Picture – Static Structure
ConfigSVN
YUMRepository
IS24 softwareAutomated RPM
creation
config├── host│ └── tstweb02│├── loc│ └── tst│ ├── VARIABLES│ │ ├── RPM_REQUIRES│ │ ├── RPM_PROVIDES│ │ ├── DB_HOST│ │ └── DB_USER│ └── etc│ └── is24│ ├── web.properties│ └── db.properties├── typ│ └── web│├── loctyp│ └── tstweb│ └── etc│ └── is24│ └── web.properties└── all ├── VARIABLES │ └── SYSLOG_HOST └── etc └── is24 └── system.properties
is24-config-tstweb01-1.0-$rev.rpm:/etc/is24/system.properties/etc/is24/db.properties/etc/is24/web.properties
[root@tstweb01 ~]# yum update
svn ci
db.host=@@@DB_HOST@@@db.user=@@@DB_USER@@@db.port=3306
Post-commit hook createsis24-config-$hostname RPM● svn export● patch VARIABLES● fill in metadata
svn co
loghost=log.domain.com
Slide 11 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Static Structure – Pros and Cons
Variables follow samestructure as config data
Only one tool (SVN) needed
GUI talks only to SVN
Versioning and changetracking for variables
Variables and config files change together (atomic) easy rollback→
Is this simple overlaying structure sufficient?
Simplify the world so that it is good enough!
ConfigSVN
YUMRepository
IS24 softwareAutomated RPM
creation
config├── host│ └── tstweb02│├── loc│ └── tst│ ├── VARIABLES│ │ ├── RPM_REQUIRES│ │ ├── RPM_PROVIDES│ │ ├── DB_HOST│ │ └── DB_USER│ └── etc│ └── is24│ ├── web.properties│ └── db.properties├── typ│ └── web│├── loctyp│ └── tstweb│ └── etc│ └── is24│ └── web.properties└── all ├── VARIABLES │ └── SYSLOG_HOST └── etc └── is24 └── system.properties
is24-config-tstweb01-1.0-$rev.rpm:/etc/is24/system.properties/etc/is24/db.properties/etc/is24/web.properties
[root@tstweb01 ~]# yum update
svn ci
db.host=@@@DB_HOST@@@db.user=@@@DB_USER@@@db.port=3306
Post-commit hook createsis24-config-$hostname RPM● svn export● patch VARIABLES● fill in metadata
svn co
loghost=log.domain.com
KISS!
Slide 12 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Big Picture – Group Assignment
ConfigSVN
YUMRepository
IS24 softwareAutomated RPM
creationPost-commit
hookcreates
is24-config-$hostnameRPM
svn ci
db.host=@@@DB_HOST@@@db.user=@@@DB_USER@@@db.port=3306
config├── all│ └── etc│ └── is24│ └── system.properties├── groups│ ├── tst│ │ └── etc│ │ └── is24│ │ ├── web.properties│ │ └── db.properties│ ├── apache│ │ └── etc│ │ └── is24│ │ └── httpd.conf│ └── webapp│ └── etc│ └── is24│ └── web.properties└── host ├── testserver-263 └── tstweb02
meta├── hosts│ ├── ttestserver-263│ │ ├── SPECIAL_KEY=value│ │ └── memberOf=apache│ └── tstweb02│ ├── memberOf=webapp│ └── memberOf=tst└── groups ├── webapp │ ├── member=tstweb02 │ ├── memberOf=tomcat │ └── RPM_REQUIRES=is24-webapp-env, is24-jcr └── tst ├── member=tstweb02 ├── DB_USER=tuv-db-user └── DB_HOST=dbserver-20
svn ci
Slide 13 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Group Assignment – Pros and Cons
Variables follow samestructure as config data
Only one tool (SVN) needed
GUI talks only to SVN
Versioning and changetracking for variables
Variables and config files change together (atomic) easy rollback→
Free assignment of hosts to groups, nested groups
Groups define function and pull in configuration and software
Final result is combination of all groups (in nesting order)
Not any more so much KEEP IT SIMPLE
ConfigSVN
YUMRepository
IS24 softwareAutomated RPM
creationPost-commit
hookcreates
is24-config-$hostnameRPM
svn ci
db.host=@@@DB_HOST@@@db.user=@@@DB_USER@@@db.port=3306
config├── all│ └── etc│ └── is24│ └── system.properties├── groups│ ├── tuv│ │ └── etc│ │ └── is24│ │ ├── web.properties│ │ └── db.properties│ ├── apache│ │ └── etc│ │ └── is24│ │ └── httpd.conf│ └── webapp│ └── etc│ └── is24│ └── web.properties└── host ├── testserver-263 └── tstweb02
meta├── hosts│ ├── srv001│ │ ├── SPECIAL_KEY=value│ │ └── memberOf=apache│ └── testserver-263│ ├── memberOf=webapp│ └── memberOf=tuv└── groups ├── webapp │ ├── member=testserver-263 │ ├── memberOf=tomcat │ └── RPM_REQUIRES=is24-webapp-env, is24-jcr └── tuv ├── member=testserver-263 ├── DB_USER=tuv-db-user └── DB_HOST=dbserver-20
svn ci
Slide 14 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
System Personalisation: RPM Dependencies
is24-tomcat is24-base-env is24-appl-env
is24-tomcat-data
is24-jcr-base is24-sitemapassembler
is24-jtools is24-sitemapassembler-config
is24-contentrules
jdk
is24-mod_jkhttpd
Requiresis24-config-tstweb02
is24-jcr
Slide 15 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
YADT – an Augmented Deployment Tool
Meta Control Layer
Open Source – GNU General Public License
100% Python / Bash
Local file management through RPM
Global service managent through YADT
Model datacenter dependencies:
RPM packages
Servers and services
Between services on different servers
Stateless design – current status is the state, define target states
All actions restartable by following the dependency model
SVN RPM post-commit hook: Maintained as part of YADT→
Slide 16 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Immobilien Scout GmbH
http://bit.ly/is24greatplace
Operate Germanys Leading Real Estate Listing Portal: www.immobilienscout24.de
Linux, XEN, Apache, Tomcat, Java, LAMP, Oracle, MySQL, Nagios, DRBD, RPM
Open Source & DevOps Company
Always seeking to hire new skill:
Admins
System Developers
System Engineers
Architects
Java & PHP Developers
Mobile (iOS & Android) Developers
Slide 17 | Configuration Management with SVN and RPM | Schlomo Schapiro
http://code.google.com/p/yadt/
Kontakt:Immobilien Scout GmbHAndreasstraße 1010243 Berlin
Fon: +49 30 243 01-1229 Email: schlomo.schapiro@immobilienscout24.deURL: www.immobilienscout24.de
Thank you very much!Please contact me for further questions and discussions.
top related