How do we drive tech changes

How do we drive tech changes?

Jaewoo Ahn (https://www.linkedin.com/in/lancersahn)Platform Architecture TeamSK Planet

https://www.linkedin.com/in/lancersahn

Disclaimers for Non-Koreans

• This slide is translated version. Originally it was written in Korean. (http://www.slideshare.net/saltynut/how-do-we-drive-tech-changes )

• Development environments and cultures in Korea might be quite different from yours. In Korea,– Almost projects(I bet over 80%) still use Java as backend and most popular combination is Java

1.6/Tomcat 6/Spring 3.x/MyBatis/SVN. It’s hard to find(I bet under 20%) somewhere use Java 1.7+, Spring 4.x, Any ORM(JPA, Hibernate, …), and other languages.

– There are many projects developed by SI companies (outsourcing), not their own developers.

– Scrum/Agile is generally NOT welcomed, due to Korean culture. People is familiar to just follow directions from their boss and (almost) nobody want to pay more money on their request changes.

– Well, feel comfortable? Then you might enjoy this.

– p.s. Somebody can blame me these are not TRUE. Believe or not, it’s up to you.

• I’m not English-native person, and therefore you can find many poor writings even I did my best. You are welcome to suggest/correct it.

• Got it? Keep go ahead!

http://www.slideshare.net/saltynut/how-do-we-drive-tech-changes

Changes?

‘People hate change…

and that’s because people hate change…

I want to be sure that you get my point.

People really hate change.

They really, really do.’

- Tom DeMarco/Timothy Lister, Peopleware

- (Orginally) Steve McMenamin, The Atlantic Systems Guild, 1996

Nothing different on Software

• Should we use it? I’m tired to learn new one.

• Why can’t we do it with current technologies?

• Did it proved? Is anybody use it in our company?

• If we got some troubles, will you be liable for that?

Belling the cat

And it should be considered that nothing is more difficult to handle, more doubtful of success, nor more dangerous to manage, than to put oneself at the head of introducing new orders. For the introducer has all those who benefit from the old orders as enemies, and he has lukewarm defenders in all those who might benefit from the new orders.

- Niccolo Machiavelli, Il Principe, 1513

Still we need changes

• New technology draws “fun”.

• Technical changes might change your viewpoint and scopes what you can.

• Old developers become outdated with old-fashioned technologies, but young developers excited with new technologies.

• You can easily find early-adopters in OSS communities trying to prove it instead of you.(forget your company & country, be global!)

Agony when you develop new one

• Just use same/safe technology/methodology before

• Try for new/challenging ones

• Which one is more “funny” for you project members?

• Yes, I know some people doesn’t have any time to consider “fun”. Sorry.

So our team decided

• We think ‘change’ cause more ‘fun’, so we are going to change something.

• If we say it seems to be ‘correct’/‘good’ way ‘in principle’/’ideally’, just do it.

Phase 1

Firstly, let’s create

a ‘New’ Service

with ‘New’ technologies

using ‘New’ methodologies

Phase 1

Firstly, let’s create

a ‘New’ Service

with ‘New’ technologies

using ‘New’ methodologies

Don’t need to consider existing/legacy ones

free to use any ones what devs want

Make people working better

Backend

FrontendWeb ApplicationWeb Application

Phase 1 : Architecture

Presentation

Controller

Business

Data Access

Database

Existing systems’ Architecture

Frontend

Backend

Database

UI

REST API Service

New system’s Architecture


• Explicitly divide frontend and backend, enabling separated development and deployment

• REST API should be used between Frontend and Backend, and should be stateless architecture.

Phase 1 : Frontend

Web Application

Presentation

Controller

Business

Data Access

JSP

Sitemesh

JQuery

Existing systems’ Frontend

Frontend

HTML

CSS

Angular.js

Bootstrap

Less

Grunt

Bower

Karma

New system’s Frontend

Phase 1 : Frontend

Web Application

Presentation

Controller

Business

Data Access

JSP

Sitemesh

JQuery

Existing systems’ Frontend

Frontend

HTML

CSS

Angular.js

Bootstrap

Less

Grunt

Bower

Karma

New system’s FrontendFrontend

MVC Framework

FrontendTask Runner

(Build, Test, Run)

FrontendPackage Manager

FrontendTest Runner

CSSFramework

CSSHelper

Phase 1 : Backend

Java 1.6

Tomcat 6

Spring 3.x + XML Conf.

Spring MVC

MyBatis

Maven

Existing systems’ Backend

a.k.a. De Facto Standard(?) Java set in Korea

New system’s Backend

Java 1.7

Tomcat 7

Spring 4.x + Java Conf.

Spring MVC

Spring Data JPA + Hibernate

Maven

Phase 1 : Backend

• Goodbye MyBatis & Hello ORM– Model-Driven Design

– DDL is auto-generated (not for Production)

– Stop wasting time to draw ERD

– MySQL is for running, H2 is for Unit Test

• ORM performance concerns?– If we need, use Second-Level Cache

– If not enough, use Native Query

– If not enough, tune DB

Phase 1 : Methodology/Process

• Project Charter– Overview, Scope, Milestones

– Communication Points(Wiki, JIRA, Agile Board, Git)

– Principles, FAQ

• Begin to use Scrum– Empowered with JIRA Agile

– Sprint Planning/Execution/Review

• Git– At the beginning, we didn’t force any branch policy

– Let people feel friendly with Git


• Project Principles

– During a sprint period, all members should concentrate on sprint backlog.

– All sprint backlog should be done during a sprint, and demonstrated + ready to deploy.

– Dog-fooding

– Please suggests voluntarily something needed, something missed, problems, and better

ideas. Help each others even it’s not assigned to you.

– Don’t hide your development process and results. Be open-minded for other’s comment.

– Be proud of yourself – we are creating best practice on our team and company.

Backend DeveloperFrontend Developer

Phase 1 – Methodology/ProcessAnalyze User Story

Design Contract/APIs

Implement Frontend Implement Backend

Mock/Unit Test Unit Test

Consume Actual APIs Load Test

User Story is done

Phase 1 Results

• Released beta version of “New” Service successfully

• Learned and proved “New” technologies

• Adopted new methodology/process

• Positive Feedbacks from stakeholders

• Most of all, project members feel‘fulfilling’ and ‘fun’

All right, go for next!

Phase 2

Let’s refactor & build

‘Existing’ services + @

with ‘Newer’ technologies

using ‘Newer’ methodologies

by ‘More’ People.

Phase 2

Let’s refactor & build

‘Existing’ services + @

with ‘Newer’ technologies

using ‘Newer’ methodologies

by ‘More’ People.

How do we migrate & transit?

There’s a lot of new things!

Experiments for better improvements

What about more than 2 pizzas team?


• As Is : Monolithic Architecture

Web Application for “User”

Web Application for “Admin”

API Service for “App”

“Central” Database

A.UI A.Biz

B.UI B.Biz

A.UI A.Biz

C.UI C.Biz

D.UI D.Biz

D.Biz

A.Biz

A B

C D


• To Be : Micro-Service Architecture(MSA)

AUI

A Service

BUI

B Service

CUI

C Service

DUI

D Service

A DB

B DB

CDB

DDB

ContentRouter

APIGateway

ContentRouter

ServiceRegistry

EventBroker

ConfigService

DeployService

User Endpoint

Admin Endpoint


• Why we choose MSA– Divide not only Frontend and Backend but also feature boundaries

– Force to maintain contract based on API

– Easy to build/deploy, Easy to scale, Isolate fail-effects

– Micro-Service is easier to understand and develop

– We can apply polyglot programming and multi-framework on each Micro-services


• Why we choose MSA– Divide not only Frontend and Backend but also feature boundaries

Synchronize with company’s engineer career paths (Frontend/Backend)

– Force to maintain contract based on APINothing can be done without APIs

– Easy to build/deploy, Easy to scale, Isolate fail-effectsYes, it’s isolated by each Micro-Services

– Micro-Service is easier to understand and developReduced code size make it easier to understand by anyone

– We can apply polyglot programming and multi-framework on each Micro-servicesExperiment them as you want(If fails, do it another way)Maybe it makes easy to hire your developers


• Of course, there are many drawbacks

– How do we fine-grain into Micro-Services?

– How do we manage contract (API)?

– A feature could be done by only one developers, but now we need to co-work with Frontend developer and other micro-service developers

– How do we manage session states?

– How do we handle dependencies and transaction among services?

– Code efforts might be duplicated

– Headaches for deploy/operations


• Of course, there are many drawbacks– How do we fine-grain into Micro-Services?







How do we solve these?









How do we solve these?

Ready for pay?

Sorry, I’ll discuss at next time(In fact, we’re still doing)

and you can google for answers








– Headaches for deploy/operations ?

Most of all,We enjoy our efforts to overcome these drawbacks, and we believe that is worth to pay for MSA’s benefit.

Remind: If we say it seems to be ‘correct’/‘good’ way ‘in principle’/’ideally’, just do it.

Phase 2 : Frontend

AUI

BUI

CUI

DUI

ContentRouter

ContentRouter

사용자용 Endpoint

관리자용 Endpoint

Single Origin/Endpoint for Browsers

Basis for enabling UIVersioning and A/B Testing

Log/Analytics Develop independently, paired with Micro-Service Backend

In production service, it’s composited into main pages with other UI

E2E(End-to-End) TestUsing Selenium for E2E test on user’s viewpoint

Phase 2 : BackendJava 1.7

Tomcat 7

Spring 4.x + Java Conf.

Java 1.8

Embedded Tomcat

Spring Boot

Node.js

Express

MySQL

Redis

Others…

Groovy

Go

ASP.NET 5

Vert.x

Others…

Polyglot,Multi-

Framework

NextCandidates

Polyglot,NoSql

Another NoSql

or Cloud

PlanetSpace

OurFile Storage

Cloud


• Enhance Scrum– Apply estimation for planning

– Manage Release note with JIRA Versions

– Synchronize with JIRA Issue StatusOpen -> Progress -> Resolve(Code Commit/Link) -> Close(master merge & deployed)

• Git Branch Policy– Master Branch : deploy for Production/Stage

– Dev Branch : shared one for sprint development

– Feature Branch : Created by JIRA Issue Id

– Code Reviews and Test when merge Feature to Dev, and Dev to Master


• Pair Programming– Pair by Frontend, Backend, Test

• 2 Scrums– To maintain 2-pizzas

• Rules & Conventions– Make implicit knowledge among team members into explicit

documentation

– Shared among Scrums

• Code Workshop during Sprint Review– Share each other’s know-how

Phase 2 is still progressing

We cannot make it

We’re sure we can do it,since we enjoy the changes

Let’s change.

End of Slides

How do we drive tech changes

Software

new technologies

new service

new methodologiesdont

new methodologiesphase

head ofintroducing new

technical changes

people doesnt

tech changes