This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Docker Configuration
Release 6.7.0
Details are provided to set up Docker for:
l Genic DataManager
l GenicWorkFlow
l Genic DataLoader
l GDM - GCBRT Bridge
Pre-requisites
These tools need to be installed before configuring docker.
l Docker
l Docker Compose
Note: For installation details, refer to the Docker website: https://docs.docker.com/
Docker Compose FileTheDocker Compose tool is used to define and runmulti-container Docker applications, using a YAML file toconfigure the application's services. You need to update the YAML file by specifying the environment variablesfor the docker images to be downloaded.
Important: When runningmulti-hosts, with GDMandGWF containers on two different hosts, separateinstances of the Docker configuration file must be used for each of the two servers to set up theconfiguration.
Details for these environment settings are specified in the YAML file:
You need to create and register a Docker ID to access the Docker hub for downloading the container images.Your Docker ID becomes your user name to access the Docker hub.
TO REGISTER FOR A DOCKER ID
1. Launch the following Docker hub web page: https://hub.docker.com/.
2. ClickSign up for Docker Hub.3. Enter the following:
l Docker ID: Usernamewhich will be your Docker IDl Password: Passwordl Email: Valid email address
Once the user ID is created, contactEnverus Trading & RiskCustomer Support / PS / to add your ID tothe external users list. This will allow you to have read-only access to the docker images.
Configuring DockerFollow these steps on the local machine where docker is installed and run.
Note: Details of the various environment parameters required for configuring docker are provided in theYAML file.
Important: The badge indicates the new addition for the current release.
TO CONFIGURE DOCKER
1. ContactEnverus Trading & RiskCustomer Support / PS / to get a sample docker project.
2. Create a docker folder under /opt/ as /opt/docker.3. Select and copy the sample docker YAML file from the project, e.g docker-compose.yml into
/opt/docker folder.4. Edit the docker-compose.yml file for the required docker configuration.5. Copy theWorkflow_Custom.jar to ./gdm-gwf-config/customResources.
./gdm-gwf-config/customResources is the location where the custom jars are placedmanually, onthe selected local drive.
Note: TheWorkflow_Custom.jar is just an example custom jar name. You can rename the jar file name,as preferred.
6. To download the image:
a. Login to the docker hub, by running the following command:
docker login
b. Enter the registered user name and password.
7. Run the following command:
docker-compose up -d
This commandwill download the image from the docker hub and start the container, bymapping thedetails specified in the docker-compose.yml file.
Note: Highlighted parameters in the YAML file are optional and should be edited by removing the # touncomment the line, as required.For boolean parameters, true / false valuesmust be entered in lower case.
The server setting and data setting values are exposed in the docker compose.yml file.a. With the SERVER_SETTING / DATA_SETTINGvalue set to true and the values defined in the
docker compose file, the database is now updated automatically with the values, upon server start-up.
Note: The Server_Setting values are in JSON array format.
b. Set the value as false if you want to restart the server without making any changes to the SERVER_SETTINGor DATA_ACCESS_SETTING.
9. For GDL:
Configuring the distribution datasources andmapping of datasources.transformUriis done using the docker-compose.yml file. You can provide inputs for these files in the docker-compose file.
Note: For more details onGDL, see theGDLReferenceGuide.
11. ForWorkflow:a. Once the image is downloaded, amessage is displayed on the server start up confirming the transfer
of custom libraries into ./gdm-gwf-config/customResources.
b. Once connected to the server, launch theGDMClient, go to theWorkflow node >WorkflowDeployment and deploy theWorkflow_custom.jar that was copied to ./gdm-gwf-config/customResources.
Note: You can copymultiple custom jar files into the ./gdm-gwf-config/customResourcesfolder.
Dynamic allocation of port numbers Enter the required port numbers for dynamicallocation.
For Windows:
1. Set the port number in theEnvironmental Variables > SystemVariables of your system.e.g. GDM_OUT_PORT=1111 , GDM_OUT_DEBUG_PORT=2222
2. Add these parameters to the docker-compose file before thedocker-compose up section.
3. Start the server, which will now startusing the exposed port numbers.
Note: You can run the command docker-compose config to check and confirmthe exposed port numbers.
For Linux:
1. Stop the server.
2. Run the following command in theconsole, to set the port number as anenvironment variable:export [env_var_name]=[env_value]e.g. (export GDM_PORT=83) , (exportDEBUG_PORT=8787)
3. Add these parameters in the docker-compose file.
4. Start the server, which will now startusing the exposed port numbers.
Note: You can run the command docker-compose config to check and confirmthe exposed port numbers.
DATA_CACHING= To enable data caching set this variable to true;otherwise set to false.
Database details for Options database
When a separate database is used for loading Options data, the following options database connection detailsneed to be included in the docker-compose file.
OPTION_HOST= Enter the Options database host machine IP:
[HOST_MACHINE_IP]
OPTION_SERVER_NAME= Enter the Options database server name:
[SERVER_NAME]
OPTION_PORT= Enter Options database port:
[DATABASE_PORT]
OPTION_NAME= Enter the Options database name:
[DATABASE_NAME]
OPTION_USERNAME= Enter the Options database username:
[USERNAME]
OPTION_PASSWORD= Enter the Options database password:
DISABLE_HIBERNATE_FORMULAS= To enable reading of formula data, set this to true.
Note: If you set this to false, formula data isnot read.
When set to true, a new property GCB_PD_FORMULA is created automatically, which sendsall formula data to the GCB server.
Note: Once the formula data is sent to the GCBserver, the property value is rolled back tofalse, which will enable the server to send onlythe updated values.
Important: You cannot the delete the GCB_PD_FORMULA property from the dockercompose file.
JAVA_OPTS= Specify the required RAM for the JAVA OPTSenvironment variable:
Note: The port detail included in the parameter allowsyou to start the gdm-cas server using thepreferred port number.
As an example, port 8080 is specified here. Youcan choose to expose any preferred portnumber. The port number specified heremustbe the same as provided underPorts section.
e.g. -XX:MinRAMPercentage=60.0 -XX:MaxRAMPercentage=80.0Djava.net.preferIPv4Stack=true -Djboss.http.port=8080
Where:
-XX:MinRAMPercentage is theminimum JVMheap size, set as a percentage of the total memory.
-XX:MaxRAMPercentage is themaximum Javaheap size, set as a percentage of the total memory.
Important: The values for XX:MinRAMPercentage and -XX:MaxRAMPercentagemust be provided as adecimal value.
It is recommended that you do not enter thevalue for -XX:MaxRAMPercentage 100%.
Use the command $docker stats to viewthememory usage.
WEBSOCKET_SERVER URL= Enter the application server IP address and port.
l To enable distributed caching whenGDM is run inmulti-host environment.
GDM_HOST andGWF_HOST are the host names of themachine where gdm-cas and gdm-gwfservices (containers) are running, respectively.
DISTRIBUTED_CACHE_EXT_ADDRESS= To enable distributed caching for GDM onmulti hostenvironment, enter the external address:
e.g. GDM_HOST
This is themachine name or the IP address of themachine where GDM server is running.
DISTRIBUTED_CACHE_HOSTS= If distributed cache setting is set to true, enter thedistributed cache host address:
e.g. [GDM_HOST[7800], [GWF_HOST[7801]
hostname= Enter the IP address of themachine where gdm-cascontainer is run.
e.g. [MACHINE_IP]
SERVER_SETTING
SERVER_SETTING= TRUE
Important: For server settings, the combination of Domain and Key details are unique. If a value is notdefined for a server setting key, then the result will be a null pointer exception. Hence, ensure a value isentered in this field.
STARTUP_DIR genicWorkflow/startUp/ The workflow serverreads this directorywhen theGDM server isstarted. Any<filename>,deployment.properties stored in thedirectory is read. Whendeployments areupdated, they areautomatically re-deployed.
JSP_PATH genicWorkflow/GWF/ Location of JSP formsused in workflow taskthat are saved anddisplayed in the GDMClient.
DATA_ACCESS_SETTING
Important: If a value is not defined for a data access setting key, then the result will be a null pointerexception. Hence, ensure a value is entered in this field.
By default, the GDM Docker server is started in INFO mode.
Important: If you are updating the docker compose file and its version is prior to 6.7.0, ensure that thelatest customization.cli file is replaced in gdm-config/cli folder.
l To enable distributed caching whenGWF is run inmulti-host environment.
GWF_HOST andGDM_HOST are the host names of themachineswhere the gdm-cas and gdm-gwfservices (containers) are running, respectively.
DISTRIBUTED_CACHE_BIND_ADDRESS=
If distributed cache setting is set to true, enter thedistributed cache bind address:
e.g. gdm-gwf
DISTRIBUTED_CACHE_EXT_ADDRESS= To enable distributed caching for GDM onmulti-hostenvironment, enter the address:
e.g. GWF_HOST
This is themachine name or the IP address of themachine where GWF server is running.
DISTRIBUTED_CACHE_HOSTS= If distributed cache setting (DISTRIBUTED_CACHE) is set to true, enter the distributed cachehost address:e.g. [GDM_HOST[7800], [GWF_HOST[7801]
command:
By default, the GWF docker server is started in INFO mode.
Important: If you are updating the docker compose file and its version is prior to 6.7.0, ensure that thelatest customization.cli file is replaced in gdm-config/cli folder.
The optionDS variables need to be updated when a separate database is used for loading Options data. Beloware the optionDS connection details to be included in the docker-compose file.
sql.optionDS.driverClassName= TheOptions DS driver class name.
[oracle.jdbc.driver.OracleDriver]
sql.optionDS.url= In the SQL options DS URL, enter themachine IPand the database name:
gdm.server.password= Enter the GDM server password:
[PASSWORD]
websocket url= Enter the GDM application server IP address andport:
ws://[IP address:port]
env_name= Enter the environment variable name:e. g: SUPERFAST_LOADER.
The environment namewill bemapped in the gdl-standalone-server.xml and gdl.properties files tomonitor the data loaded from GDL, using the GWSweb application.
log_type= Enter the log type as: DEBUG.
This will generate a detailed log report of the serveraction.
Note: The default log_type is INFO.
webmonitoring_url= Enter the GDM application server url:
http://server_url:port
JAVA_OPTS= Specify the required RAM for the JAVA OPTS environmentvariable:
e.g. -XX:MinRAMPercentage=60.0 -XX:MaxRAMPercentage=80.0Djava.net.preferIPv4Stack=true
When a file is processed, the following folders are created:
l failedDelivery: When a file fails to load, a failedDelivery folder is created. This folder will include a .csvfile containing the file name, client name, datasource, failed delivery date, Ftp path and error message.
l delivery: When a file is processed successfully, a folder called delivery is created. This folder will includea .csv file containing the filename, start time, end time and the actual time taken to load the file.
l .error: When the file processing has failed, a .error folder is created where all the files that have failed tobe processed are stored.
l .done: When a file is processed successfully, a .done folder is created where all the files that aresuccessfully processed are stored.
archiveLoadedFile= To archive files that are successfully processed intothe .done folder, set this variable to true.
Note: When set to false, loaded files aredeleted and they will not be available in the.done folder.
By default, this option is set to true.
uploadFailedFileToFtp= To upload the failed files to FTP, set this variable totrue. Once the files are uploaded to FTP, the filesfrom the local directory will be deleted.
Important: Once failed files are uploaded to thefailed folder in FTP, these files will not beprocessed. Manual action is required toprocess the files.
When set to false, failed files will not be uploaded toFTP and will remain in the local .error directory.
By default, this option is set to false.
Important: An exceptionmessage is displayedif user with insufficient permission try to accessthe failed folder present in the system. Usershould be provided with enough permission tocreate it.
distribution.datasources= Enter the datasources values; comma separated:
e.g. distribution.datasources=ARGUS_DLNGD,GDL,APX_EL_UK,EEX_ELEC_AUC_INT_FR,EEX_GS_SPOT
datasources.transformUri= Enter the transform values; comma separated.
e.g. datasources.transformURI=ICE_GASOIL=transform://ICE_GASOIL_CUSTOMIZE_PROPERTIES_XSL
Specific to the Linux environment, in addition todefining the datasource transformUri, youcan definemultiple inputs by extending the uris fromdatasources.transformUri1 todatasources.transformUrinth.
Note: If you are updating the docker composefile and its version is prior to 6.6.0, ensure thatthe latest gcb-bridge.xml file is replaced ingcbrt-bridge-conf folder.