Heavy lifting and easy carrying. - s3.amazonaws.com · Heavy lifting and easy carrying. or how to using docker engine for Oracle products. SESSION ID GLEB OTOCHKIN

Heavy lifting and easy carrying.or how to using docker engine for Oracle products.

SESSION IDGLEB OTOCHKIN

© The Pythian Group Inc., 2018

Gleb Otochkin

Started to work with data in 1992At Pythian since 2008

Area of expertise: ● Data Integration● Oracle RAC● Oracle engineered systems● Virtualization● Performance tuning● Big Data

[email protected]@sky_vst

Principal Consultant

© The Pythian Group Inc., 2018 2

mailto:[email protected]


Systems currently managed by Pythian

EXPERIENCED

Pythian experts in 35 countries

GLOBAL

Millennia of experience gathered and shared

over 19 years

EXPERTS

11,800 2 400

3


What we were thinking first when we heard “Docker” before?.

4


Or sometimes ...

5


Who (or what) is the docker?

6


Now it is for most of us

7

AGENDA

© The Pythian Group Inc., 2018 8

• Docker introduction.• Available docker resources.• Creating our own docker from scratch.• Customizing existing resources.• Docker tips and tricks.• QA


What is the Docker. Some basics about Docker:

• Based on cgroups and kernel namespaces.

• Using a Union capable FS (overlay)

• Docker is a high level API to manage layers on Union FS and processes..

• It can be run in collaboration with other services and tools.

• Linux and Windows base images.

• Layers with copy on write container on top of it..

9


Processes

Docker engine.Basic look to a Docker engine.

10

Infrastructure

OS

Docker Engine

Base image

Processes

RDBMS I

Processes

RDBMS II

Base image

App


Docker Storage.Basic look to a Docker storage.

11


Docker Storage.Basic look to a Docker storage.

12

Base Image (RO)

Add packages (RO)

Copy scripts (RO)

Run scripts (RO)

Container I (RW) Container I (RW)


Business cases

13


Business cases.Short summary

• Where to use:•Development.

•Tests.

•Microservices.

• Reasons:• Isolation of environment.

• Save storage.

• Make changes without affecting a base.

• Create new containers in seconds.

• Commit changes to a new container.

14

Resources.

Where to find and what is useful..


Docker home. Good source for verified base images

• Home for Docker: https://www.docker.com

• Docker store: https://store.docker.com/publishers/oracle

• Docker documentation: https://docs.docker.com/

• Docker blog: https://blog.docker.com

• Docker official repository:• Perl: https://hub.docker.com/_/perl/

• Ruby: https://hub.docker.com/_/ruby/

• Go: https://hub.docker.com/_/golang

16

https://blog.docker.com

https://hub.docker.com/_/perl/

https://hub.docker.com/_/ruby/

https://hub.docker.com/_/golang


Github. Oracle and other

• Oracle has github for dockerimages : https://github.com/oracle/docker-images

• Oracle site for containers: https://developer.oracle.com/containers

• Docker topic on github: https://github.com/topics/docker

• Labs on the Docker’s github: https://github.com/docker/labs

17

Creating our own docker.

Starting from a small step.


Installing docker. Installing using yum on linux.

• Switch enabled=1 in the section [ol7_addons] if you use Oracle Linux 7.

• Run: [root@dockerhost01 ~]# yum install docker-engine

• Start the service: [root@dockerhost01 ~]# systemctl start docker.service

• Add yourself to the “docker” group: [root@dockerhost01 ~]# group gpasswd -a otochkin docker

• Check the service: [root@dockerhost01 ~]# docker run hello-world[d

19


Dockerfile instructions. Basic structure and commands.

• Build images using “$ docker build “ command.

• The “$ docker build --help” will provide list of options.

• To name your image - option “-t”

• Here is command to build an image with name “myimage” and tag “01”[d

20

[otochkin@dockerhost01 layer01]$ docker build -t myimage:01 .Sending build context to Docker daemon 2.048kBStep 1/1 : FROM oraclelinux:7-slim ---> 9870bebfb1d5Successfully built 9870bebfb1d5Successfully tagged myimage:01



• Docker look for “Dockerfile” with instructions and a context (PATH,URL).

• By default Docker look for file “Dockerfile” in the directory.

• The file instructions:• Directives(optional) are going first: directive=value• Comments are starting from: “#”• Instructions are starting from: “FROM “

21


Dockerfile instructions.

22

Instruction Description Instruction Description

FROM sets the Base Image for subsequent instructions.

RUN executes any commands in a new layer

ADD copies new files, directories or remote file URLs

USER Yes

COPY copies new files or directories VOLUME mounts external volume to container

ENV sets the environment variable WORKDIR sets the working directory for other instructions

EXPOSE specified network ports at runtime

ONBUILD Yes

LABEL adds metadata to an image CMD provide defaults for an executing container


Dockerfile instructions. .dockerignore fileTo to exclude files and directories from context for the Docker daemon.

23

Instruction Description Comments

# comment Used for comments - ignored

* Exclude files and directories *.tmp all files in root directory with ext “tmp”

! Exception from the excluded set !notex.txt - will not be excluded

** Any number of directories **/*.bak - any files with .bak from all directories

● The file is read and all conditions applied from up to down.



• A simplest Dockerfile.

24

[otochkin@dockerhost01 layer01]$ cat Dockerfile # HOW TO BUILD THE IMAGE# -----------------------# Run: # $ docker build -t oracle/linux7 . ##FROM oraclelinux:7-slim



• Let’s make it bit more complex.

25

[otochkin@dockerhost01 layer01]$ cat Dockerfile.01# HOW TO BUILD THE IMAGE# -----------------------# Run:# $ docker build -t oracle/linux7:new -f Dockerfile01##FROM oraclelinux:7-slimRUN yum -y update



• Run it:

26

[otochkin@dockerhost01 layer01]$ docker build -f Dockerfile.01 -t oracle/linux7:new .Sending build context to Docker daemon 3.072kBStep 1/2 : FROM oraclelinux:7-slim ---> 9870bebfb1d5Step 2/2 : RUN yum -y update ---> Running in ebf9d704d704Loaded plugins: ovl...



• Check size for components:

27

[otochkin@dockerhost01 layer01]$ docker history 9d78cba36022IMAGE CREATED CREATED BY SIZE COMMENT9d78cba36022 23 seconds ago /bin/sh -c yum -y update 1.02GB9870bebfb1d5 2 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B<missing> 2 months ago /bin/sh -c #(nop) ADD file:9061281688fde55... 118MB<missing> 3 months ago /bin/sh -c #(nop) MAINTAINER Oracle Linux... 0B



• Adding one more “RUN”.

28

[otochkin@dockerhost01 layer01]$ cat Dockerfile.01# HOW TO BUILD THE IMAGE# -----------------------# Run:# $ docker build -t oracle/linux7:new -f Dockerfile01##FROM oraclelinux:7-slimRUN yum -y updateRUN yum -y install oracle-database-server-12cR2-preinstall



• Check layers:

29

[otochkin@dockerhost01 layer01]$ docker history 1d633491b262IMAGE CREATED CREATED BY SIZE COMMENT1d633491b262 52 seconds ago /bin/sh -c yum -y install oracle-database-... 163MB446a380bfb1b About a minute ago /bin/sh -c yum -y update 1.02GB9870bebfb1d5 2 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B<missing> 2 months ago /bin/sh -c #(nop) ADD file:9061281688fde55... 118MB<missing> 3 months ago /bin/sh -c #(nop) MAINTAINER Oracle Linux... 0B



• Running all commands in one “RUN” block.

30

[otochkin@dockerhost01 layer01]$ cat Dockerfile.03# HOW TO BUILD THE IMAGE# -----------------------# Run:# $ docker build -t oracle/linux7:new -f Dockerfile01##FROM oraclelinux:7-slimRUN yum -y update && \ yum -y install oracle-database-server-12cR2-preinstall



• Check layers.

• We’ve just removed one intermediate layer.

31

[otochkin@dockerhost01 layer01]$ docker history 398cb5e11c45IMAGE CREATED CREATED BY SIZE COMMENT398cb5e11c45 39 seconds ago /bin/sh -c yum -y update && yum -y ins... 1.17GB9870bebfb1d5 2 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B<missing> 2 months ago /bin/sh -c #(nop) ADD file:9061281688fde55... 118MB<missing> 3 months ago /bin/sh -c #(nop) MAINTAINER Oracle Linux... 0B



• Why is it so important? Let’s repeat with a bigger file.

32

[otochkin@dockerhost01 layer02]$ cat Dockerfile.05# HOW TO BUILD THE IMAGE# -----------------------# Run:# $ docker build -t oracle/linux7:file -f Dockerfile.05##FROM oraclelinux:7-slimRUN groupadd -g 54321 oinstall && groupadd -g 54322 dba && useradd -u 54321 -g 54321 -G oinstall,dba oracle && \ mkdir -p /u01/app/oracle/ords && chown -R oracle:oinstall /u01/app/oracleRUN dd if=/dev/zero of=/u01/app/oracle/ords/file bs=1024k count=1024RUN chown -R oracle:oinstall /u01/app/oracle/ords



• We have 2 layers about 1 Gb each.

33

[otochkin@dockerhost01 layer02]$ docker history f5341bb821cdIMAGE CREATED CREATED BY SIZE COMMENTf5341bb821cd 22 seconds ago /bin/sh -c chown -R oracle:oinstall /u01/a... 1.07GBce0c92dbd846 40 seconds ago /bin/sh -c dd if=/dev/zero of=/u01/app/ora... 1.07GB9b3e75a95ab5 56 seconds ago /bin/sh -c groupadd -g 54321 oinstall && g... 3.28kB9870bebfb1d5 2 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B<missing> 2 months ago /bin/sh -c #(nop) ADD file:9061281688fde55... 118MB<missing> 3 months ago /bin/sh -c #(nop) MAINTAINER Oracle Linux... 0B



• What if we use only one “RUN” block?

34

[otochkin@dockerhost01 layer02]$ cat Dockerfile.06# HOW TO BUILD THE IMAGE# -----------------------# Run:# $ docker build -t oracle/linux7:file01 -f Dockerfile.06 .##FROM oraclelinux:7-slimRUN groupadd -g 54321 oinstall && groupadd -g 54322 dba && useradd -u 54321 -g 54321 -G oinstall,dba oracle && \ mkdir -p /u01/app/oracle/ords && chown -R oracle:oinstall /u01/app/oracleRUN dd if=/dev/zero of=/u01/app/oracle/ords/file bs=1024k count=1024 && \ chown -R oracle:oinstall /u01/app/oracle/ords



• Only one layer.

35

[otochkin@dockerhost01 layer02]$ docker history ebf8c760c692IMAGE CREATED CREATED BY SIZE COMMENTebf8c760c692 24 seconds ago /bin/sh -c dd if=/dev/zero of=/u01/app/ora... 1.07GB9b3e75a95ab5 8 minutes ago /bin/sh -c groupadd -g 54321 oinstall && g... 3.28kB9870bebfb1d5 2 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B<missing> 2 months ago /bin/sh -c #(nop) ADD file:9061281688fde55... 118MB<missing> 3 months ago /bin/sh -c #(nop) MAINTAINER Oracle Linux... 0B



• Compare two images.

36

[otochkin@dockerhost01 layer02]$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEoracle/linux7 file01 ebf8c760c692 16 seconds ago 1.19GBoracle/linux7 file f5341bb821cd 7 minutes ago 2.27GB

Customizing reference configurations.Rebuilding using pre-built blocks.


Using reference configurations. Oracle supplied configurations.

• Oracle maintains a Github repository with docker scenarios.

• They can be used for:• As an example.

• As a base image.

• As a component to build environment.

38



• Oracle database docker image as a base image for your custom scenario.

• For custom ODI image:

39

# Pull base image of Oracle database (used as repository for ODI)# From the Oracle Registry# -------------------------FROM oracle/database:12.2.0.1-ee

# Maintainer



• One of the reference images can be part of complex architecture.

40

Processes

Oracle reference image

Processes

RDBMS I

Processes

RDBMS II

Custom image

App I

Processes

Custom image

App II

Docker tips.

Tips and tricks


Tips and tricks. Some advices and best practices.

• Check the latest documentation for Docker (most of best practices are there).

• Make your image as much close to final container state as possible.

• Create less layers.

• Use COPY instead of ADD.

• Use “wget” and “curl” instead of ADD <web src>.

• Use multi-stage builds.

• Put intensive IO out of docker FS.

• Order/sort multi-line commands for better maintenance.

42


Dockerfile tips. Make your image as much close to final container state as possible..

• Make your container as ephemeral as possible.

43

Image

Container

Image

Container


Dockerfile tips. Make your image as much close to final container state as possible..

• Make your container as ephemeral as possible.

44

Base Image

ADD

COPY

RUN

Container I (RW) Container II (RW)

RUN

Base Image

COPY

Container I (RW)

RUN

Container II (RW)


Dockerfile tips. Use COPY when it is sufficient.

• “ADD” vs “COPY”.• “ADD” can use URL as a source to add files to image.

• If the source a know archive (tar.gz) it will be unpacked.

• “ADD” invalidates cache including “RUN” cache.

• “COPY” does not invalidate existing build cache.

• Using COPY is more preferable if you don’t need special features of ADD

45


Dockerfile tips. RUN with wget or curl vs ADD.

• “ADD” ADD http://sourcesite.com/app.tar.xz /usr/src/appsrc/ tar -xJf /usr/src/appsrc/app.tar.xz -C /usr/src/appsrc

• “RUN” RUN curl -SL http://sourcesite.com/big.tar.xz \ | tar -xJC /usr/src/appsrc \ && make -C /usr/src/appsrc all

46


Dockerfile tips. Multi-stage builds.

FROM myapp:04COPY ……FROM oradb:12cCOPY …RUN …

47

oradb:12c image

myapp:04 image

Final image

Dockerfile

Build


External volume

Docker FS

Dockerfile tips. Move IO out of the Docker FS

• Docker FS is COW (copy on write). Use the “VOLUME” instruction.

48

Infrastructure

OS

Docker Engine

Base image RDBMSBase image

App Container DB Container


Dockerfile tips. Sort commands in a RUN block.

• Sorting commands in “RUN” RUN mkdir /u01/app/build/logs && \ Abc01.sh && \ Bdc02.sh && \ … Zfc.sh

49


And finally we are here!

50


QA

51


THANK YOUEmail: [email protected]://pythian.com/blog@sky_vst

52

mailto:[email protected]

https://pythian.com/blog

Heavy lifting and easy carrying. - s3.amazonaws.com · Heavy lifting and easy carrying. or how to using docker engine for Oracle products. SESSION ID GLEB OTOCHKIN

Documents