S6 the guardian of line today automation test

The Guardian of LINE TODAY Automation Test

Kent Han

• LINE TODAY

• Challenges

• CI and Automation Test

CI Pipeline

Unit Test

Functional Test

Service Monitoring

• Wrap Up

AGENDA

LINE TODAY

FastTrack RegularTrack

2016.03 - Fast Track as Proof Of Concept 2016.06 - Regular Track as long-term product

GLN - Global LINE News

GLN - Global LINE News

TW 18m

TH 36m

IR 3m

ID 32m

JP 58m

TW TH ID IR

(RTL)

GLN-GlobalLINENewsGLN - Global LINE News

Fast Track - 02/26 (TW, TH, ID, US, MM) Regular Track - 06/16 (TW, TH, ID) 10/04 (IR)

3/2 3/5 3/8 3/113/143/173/203/233/263/29 4/1 4/4 4/7 4/104/134/164/194/224/254/28 5/1 5/4 5/7 5/105/135/165/195/225/255/285/31 6/3 6/6 6/9 6/126/156/186/216/246/276/30 7/3 7/6 7/9 7/127/157/187/217/247/277/30 8/2 8/5 8/8 8/11 8/148/178/208/238/268/29 9/1 9/4 9/7 9/10

DUBDPV

Fast Track

Regular Track

ChallengesSwiftly adjust to survive, or die

Feature requests are diverse

Agile is the Only Solution

How Fast is Your Agile

CI + Automation Test

Jenkins 2.0

CI Pipeline by Jenkinsews

• Jenkins IFTTT LINE Notify

• https://ifttt.com/line

CI Notification

Testing Pyramid

https://watirmelon.blog/tag/testing-pyramid/

Unit test vs. Integration test • Speed • Stable • Easy to troubleshoot

Testing Pyramid

PullRequestmustbecheckedbyunittest

Unit Test

Static Code Analysis• Checkstyle • FindBugs • PMD

• Duplicate Code

Testing Tools

• Python

• Robot Framework

• Selenium

• Appium

• Jenkins

• Gatling

Functional Test

Why Did We Adopt Robot Framework?

• Human-readable test cases for Maintainability

• Python and Java support for Versatility

• --rerunfailed for flaky tests

*** Test Cases ***Add A Comment To An Article [Tags] comment RAT New An Article In Category Add Comment To The Article 1 Times Comment Count Of The Article Is 1

Reply A Comment To A Comment [Tags] reply RAT New An Article In Category Add Comment To The Article 1 Times Comment Count Of The Article Is 1 Reply To The Comment With Text Comment Comment Should Be Included In Article

Vote Up To A Comment [Tags] vote RAT New An Article In Category Add Comment To The Article 1 Times Comment Count Of The Article Is 1 Vote The Comment With UP

Robot Framework Report

Content Working Flow

Feeding Test

• Average 5,500 articles imported a day

• Should ensure the input stream

• Fake articles for testing

Thumbnail

Image

Video

Youtube

<?xml version='1.0' encoding='UTF-8'?><articles> <UUID>c11ae751-b180-11e6-96c0-985aeb8c0520</UUID> <time>1461117682767</time> <article> <ID>c11b1973-b180-11e6-8d03-985aeb8c0520</ID> <nativeCountry>TW</nativeCountry> <language>zh</language> <startYmdtUnix>1479907723508</startYmdtUnix> <endYmdtUnix>1571117682767</endYmdtUnix> <title>20161123212843_all_medias</title> <category> </category> <publishTimeUnix>1479907723508</publishTimeUnix> <contents> <image> <description>Mountain is beautiful!</description> <url>yama_1.png</url> </image> <video> <description>Kobe Top 1</description> <thumbnail>thumbnail_1.jpg</thumbnail> <url>kobe_1.mp4</url> </video> <video> <description>Youtube</description> <url>https://www.youtube.com/embed/c7rCyll5AeY</url> </video> <text> <content> Google GoogleAlphabet

Larry Page Alphabet Google Sundar Pichi

Google Larry PageGoogle ......

Alphabet </content> </text> </contents> </article></articles>

CMS Test

• OA Digest

• Timeline Digest

• Category publication

• When new features are added, basic functions should not fail

Front-end Test

• Selenium covers the most

• Appium for the mobile-only feature

• API test as a tool to generate testing data

• Internal tools

iMON - application error

NSight - system error

• Health check script

Web response time

RabbitMQ management api

ElasticSearch cluster/node/stats/status api

Service Monitoring

0

100

200

300

400

v2.1 v2.2 v2.3 v3.0 v3.1 v3.2 v3.3

major bugs total bugs

Bug Stats

Wrap UpQuality is everyone’s KPI

Find a pain point in retrospective then improve in next sprint

THANK YOU