Building a Docker container with SDL Web 8

Hello all, first of all best wishes everyone for the new year. Let it be a great year with lot of technological innovation. I already saw the first cool one of the year, the oculus Rift will finally be available for pre-order from 6th of January 😀

Recently I wrote a post about docker-compose, some of the questions that have reached me is actually how do you create such a Docker container for the SDL Web 8 micro-services. As it is not that difficult, I thought it would be nice to just write a small post on how to create a docker container using the SDL Web 8 Discovery Service as an example.

Docker Build & Running

It is good to realise that docker containers have roughly two main stages, creating (building) of the container and running it. The creation and building is generally done by an automated build process and hopefully provided by a vendor. These days a lot of containers are published publically on Docker Hub: https://hub.docker.com/

However in case of SDL Web 8 we are dealing with a proprietary licensed product which makes this slightly more challenged. So in this post I assume that you own a copy of the Software and have a License for the software itself.

SDL Web 8 Discovery Service

For those not knowing what is the SDL Web 8 Discovery Service, it is a small OData based REST Webservice that describes the available resources in a SDL Web Content Delivery environment. The Webservice is based on Java Spring-boot which in turn uses embedded Tomcat to run the Service. The service has a simple dependency on a Relational Database where the discovery information is stored and it exposes the webservice on port 8082.

Let’s get Started

So let’s get started, what is needed to get this working? Well the first thing is that we need to prepare the directory structure to put inside the Docker container. A docker container in essence is a virtual machine with an isolated directory structure. We will copy our Service artefacts into this docker container so that we can use this to start a Java process.

Directory layout

You start out by creating a directory structure that we can use to put the discovery-service in a docker container. See the following directory structure in the screenshot, it is important to following this precisely.
dirstructure

What to put where:
\discovery-docker\config\cd_ambient_conf.xml
This is copied from your SDL Web 8 copy: ‘SDL Web 8 Core\Content Delivery\roles\discovery\standalone\config’

\discovery-docker\config\cd_storage_conf.xml
For this file use the provided sample config in this blog below in the configuration files section.

\discovery-docker\config\logback.xml
For this file use the provided sample config in this blog below in the configuration files section.

\discovery-docker\config\cd_licenses.xml
Your favorite products license file 🙂

\discovery-docker\bin\start.sh
This is a Docker specific start bash script, see below for the contents. Do not use the one provided with Web 8

\discovery-docker\lib
Copied from Web 8 software ‘SDL Web 8 Core\Content Delivery\roles\discovery\standalone\lib’

\discovery-docker\services
Copied from Web 8 software ‘SDL Web 8 Core\Content Delivery\roles\discovery\standalone\services’)

\Dockerfile
Last but not least we need a Dockerfile in the root, this is a text file, the filename is literally ‘Dockerfile’

For the lib and services folder these need to be a direct copy of the artefacts of the SDL Web 8 cd-layout. For the config files use the above storage configuration, and use the sample versions of the cd_ambient_conf.xml and make sure your cd_Licenses.xml file is present.

Configuration files

Any discovery-service requires a storage configuration to point the service to where the database is. At this moment this file is not parameterizable. Meaning that on the moment you build the docker container you already need to know where the database host is located. This is something that should be prevent in the product itself, but for the moment we will just accept it. So you end up with following storage config for the moment:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration Version="8.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:noNamespaceSchemaLocation="schemas/cd_storage_conf.xsd">
    <Global>
        <ObjectCache Enabled="false"/>
        <Storages>
            <StorageBindings>
                <Bundle src="odata_dao_bundle.xml" />
            </StorageBindings>
            <Storage Type="persistence" Id="defaultdb" dialect="MSSQL" Class="com.tridion.storage.persistence.JPADAOFactory">
                <Pool Type="jdbc" Size="10" MonitorInterval="60" IdleTimeout="120" CheckoutTimeout="120" />
                <DataSource Class="com.microsoft.sqlserver.jdbc.SQLServerDataSource">
                    <Property Name="serverName" Value="SERVER_NAME" />
                    <Property Name="portNumber" Value="1433" />
                    <Property Name="databaseName" Value="DATABASE_NAME" />
                    <Property Name="user" Value="USER_NAME" />
                    <Property Name="password" Value="PASSWORD" />
                </DataSource>
            </Storage>
        </Storages>
    </Global>

    <ItemTypes defaultStorageId="defaultdb" cached="false"/>
</Configuration>

Logging

Because logging happens differently in Docker, it is simplest for now to simply log the output to the console. In a further stage it would be recommended to switch this to a logstash based setup.

You can use this sample to create a console logger for the discovery-service:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <property name="log.pattern" value="%date %-5level %logger{0} - %message%n"/>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${log.pattern}</pattern>
        </encoder>
    </appender>

    <logger name="com.sdl" level="info"/>
    <logger name="com.tridion" level="info"/>

    <root level="OFF">
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

Ambient configuration

By default the ambient configuration used has the service secured with OAuth, for this test I disable that by setting the following properties to false (found in the ambient sample file):

<Security OAuthEnabled="false">
<Rules Enabled="false"/>

Putting it all together

Shell script used to start the discovery service, please store this with path: ‘discovery-docker\bin\start.sh’.

Please make sure to use this version instead of the version provided by the SDL Web 8 Discovery-Service version.
For those Windows Peeps out there, please make sure you store this file with Unix file endings, else you will get errors 🙂

#!/bin/sh

# Java options and system properties to pass to the JVM when starting the service. For example:
# JVM_OPTIONS="-Xrs -Xms256m -Xmx512m -Dmy.system.property=/var/share"
JVM_OPTIONS="-Xrs -Xms256m -Xmx512m"
SERVER_PORT=--server.port=8082

BASEDIR=$(dirname $0)
CLASS_PATH=.:config:bin:lib/*
CLASS_NAME="com.sdl.delivery.service.ServiceContainer"
PID_FILE="sdl-service-container.pid"

cd $BASEDIR/..

ARGUMENTS=()
for ARG in $@
do
    if [[ $ARG == --server\.port=* ]]
    then
        SERVER_PORT=$ARG
    else
        ARGUMENTS+=($ARG)
    fi
done
ARGUMENTS+=($SERVER_PORT)

for SERVICE_DIR in `find services -type d`
do
    CLASS_PATH=$SERVICE_DIR:$SERVICE_DIR/*:$CLASS_PATH
done

echo "Starting service."

java -cp $CLASS_PATH $JVM_OPTIONS $CLASS_NAME ${ARGUMENTS[@]}

The docker file looks as following, it actually simply copies over the discovery-docker folder into the container:

FROM java:8
COPY discovery-docker /
RUN chmod +x /bin/start.sh

CMD bash -C '/bin/start.sh'

EXPOSE 8082
MAINTAINER Renze de Vries

Building it

Now we have put the right directory structure together, the next step is actually building the docker container so you can run it. In order to do this, make sure you logged into a docker-machine aware shell (Docker quickstart terminal).

Then simply in the top directory that contains the ‘Dockerfile’ and the ‘discovery-docker’ directory run the following command:
docker build -t discovery-service .

After this command has run you should be able to see that the build image was correctly build like following: docker images

This should give an output like this:

REPOSITORY                    TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
discovery-service             latest              aee569d5be25        9 seconds ago       697.7 MB

Running it

Now we have created this beautiful docker container, but we obviously want to run it. So let’s run the following command to start the container: docker run -p 8082:8082 discovery-service

This will start the process in the same window, you should see output like this for the discovery-service we dockerised:

Starting service.
2016-01-06 08:58:09,819 INFO  ServiceContainer - Starting CD service container with parameters [--server.port=8082]
2016-01-06 08:58:10,697 INFO  ServiceContainer - Starting ServiceContainer v8.1.0-1228 on ee33e87949c3 with PID 11 (/lib/service-container-core-8.1.0-1228.jar started by root in /)
<<cut to much text>>
2016-01-06 08:58:26,214 INFO  ServiceContainer - Started ServiceContainer in 16.155 seconds (JVM running for 16.98)
2016-01-06 08:58:26,217 INFO  ServiceContainer - CD service container started

Understandable you probably do not want to have a window open for every container, so you can simply run the container in detach mode by adding the ‘-d’ startup flag, the command then becomes:
docker run -d -p 8082:8082 discovery-service

Show running containers with docker ps yields something like this:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
ee33e87949c3        discovery-service   "/bin/sh -c 'bash -C "   4 minutes ago       Up 4 minutes        0.0.0.0:8082->8082/tcp   focused_thompson

If you want to stop your running container that can be done with this command: docker stop ee33e87949c3

Keep in mind that stopping a container does not remove it, so it can be re-used next time. If you want to permanently get rid of your container you can use the rm command: docker rm ee33e87949c3

Conclusion

So there you have it we dockerised one of the webservices provided as part of the SDL Web 8 product. As you see from above it is relatively simple to build a docker container. Most of the steps I have described are actually product specific and something you would do for any product installation.

For most products that contain proprietary software you would go through a similar process, so I think most of these steps can be translated to any software product out there. The steps you can skip are the product specific configuration steps.

I hope this will lead to many companies building docker containers for their proprietary products. For the moment it should at least make the lives of developers, testers and anyone else that wants to try your product a whole lot simpler.

Please do not hesitate to ask question on this topic in the comments or via the mail renze at renarj.nl

Devoxx 2015 Summary :)

I went to the Devoxx 2015 conference in Belgium this year and thought I would give a quick summary from the Devoxx conference with some take-aways that I took myself while it is still fresh. 

  • Micro-service are hot
    • Lot of presentations based on microservices with Spring-boot 
    • Docker, Docker-compose and Swarm are really becoming big lots of talks on getting started as a developer or in the cloud
    • Slightly related, cluster management is getting quite some tooling (Kubernetes, Swarm, etc.)
  • The next big thing seems to really be about Machine Learning and Data streams
    • Instead of offline analytics, we analyse Streams online and offline and combine data (Apache Spark Stream, Akka Streams, Java Streams)
    • Standard has been developed around this called Reactive Streams initiative
      • Will most likely be adopted as part of JDK 9
  • JDK 9 is all about modularisation
    • No longer will you be required to drag the entire JDK with you, only the parts you depend on
    • Introducing a linker to package a mobile version of your app with minimal JRE dependencies
    • No support for multiple versions -> this needs to be resolved by build tools
    • Expected slow adoption by Oracle due to breaking changes (internal classes removed, that where being used, etc.)
  • Chaos engineering
  • Fun topics
    • Developing with robotics with Aldebaran (Yeah the NAO 😀 which i also have, see my other posts )
    • Inspirational talk about silver bullet syndrome: https://www.youtube.com/watch?v=3wyd6J3yjcs (hint, there is no silver bullet 🙂
    • Capturing the air around you for $7.
      • Really loved this session, using a computer and a small radio receiver he ‘hijacked’ the wireless audio stream from a session next door 🙂

All the sessions where recorded, and from this year they have put them on youtube:

https://www.youtube.com/channel/UCCBVCTuk6uJrN3iFV_3vurg/videos

Robotics and Home Automation the next step

In one of my first posts I talked about Robotics, in particular a Raspberry PI project with Java to create a spider robot. That project was great fun, and in the end it tasted like I should really finally take the next step.

A few years ago I was at a conference in San Francisco the JavaOne, there was a company there called Aldebaran to demo their little Robot. Instantly I knew that Robot was so great I wanted to experiment with one. However it was very hard to get one, and not so affordable.

Although the affordable part has not changed, I am happy to say that I did manage to acquire one almost 3 years after meeting him the first time. I am happy to introduce my new buddy that we call “Peppy”:

IMG_1188

Aldebaran NAO Robot

The above robot is a NAO robot from a company called Aldebaran. He is one of the more advanced Humanoid research platform robots you can get these days at a relatively affordable level. He has 25 degrees of freedom, 2 cameras, 4 microphones and several other sensors all over his body. The robot also has Wifi and is powered by an Intel Atom processor running a Linux OS based on Gentoo distribution.

What is great about the robot is the great SDK that comes with it. Out of the box there is a graphical studio called Choregraphe which has a blocks based editor allowing you to develop behaviours for your robot. This is very easy and I could already quickly assemble this little dialog (In Dutch):

The robot also comes with a Python and Java SDK. The cool part about this SDK is that it does not matter if it runs remotely or on the robot itself. I will write a bit more about the SDK in future articles.

Goal

I have had the robot already for a few weeks, and often I get asked what am I planning to do with it. Well I want to fuse the world of home automation and robotics with each other. Would it not be great if you open the door and a robot can greet you? And you can tell your robot your preferences and he can automatically arrange things for you.

Is this any different than a Siri, perhaps not so much, but the robot can make it more personal and physical which is something Siri simply cannot do. In the coming months I will explore and research this topic more, and hopefully share a bit more about this, stay tuned..

About me and my Blog

Just posting my first blog entry, and thought better just start with the About section and tell you all a bit about me 🙂

I am Renze de Vries a technology enthusiast, or gadget freak as some people would perhaps say :). I am very keen on exploring the new and awesome things that happen out there.

I started out as a Java developer in my first job, and evolved quickly into a leadership role. These days I am a technology manager at SDL plc and at the same time have my own Company called Oberasoftware

I spend most of the time building new and innovative product concepts with my teams. Some interesting products I have developed is my own NoSQL database called JasDB that I built @ Oberasoftware.

More recently I have been heavily investing in the internet of things and robotics @ Oberasoftware. My goal is to fully automate my home including a Robot that greets you at the door and helps and assists you.

Next to this I act as a innovator in SDL where I am introducing concepts like Open Sourcing, and evangelising new technologies in the company. One of the fruits of this is that under my leadership SDL has released their very first open source framework the OData library.

I hope on this blog I can expose a bit of the cool things that come across my path and hopefully share a bit of the experiences or sometimes challenges that i face in my day to day jobs.