Setting up an SAP Hybris Cluster with Jenkins

Minimum requirements

1 Jenkins server
1 Apache server (working as load balancer) with IP 192.168.0.1
2 Hybris servers (one working as importer and application server and the other one working only as application server) with IPs 192.168.0.2 and 192.168.0.3

Steps needed

Setting firewall unlocks between Jenkins and Apache/Hybris servers.

Create a "jenkins" user on all servers.

Set up SSH access for "jenkins" user on all nodes.
Create configuration files and push them into any code repository of your choice  (SVN, Mercurial, Git, etc.). Note: this example is using Mercurial.

Create Apache "httpd.conf" file

To use the HTTP Apache server as load balancer we need a configuration file like the described below:

Listen 80
Header add Set-Cookie: "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED 

<Proxy balancer://mycluster>
   BalancerMember http://192.168.0.2:9001 route=node1
   BalancerMember http://192.168.0.3:9001 route=node2
</Proxy> 

ProxyPass / balancer://mycluster/ stickysession=ROUTEID
ProxyPassReverse / balancer://mycluster/ 

<Location /balancer-manager>
   SetHandler balancer-manager
   Order Deny,Allow
   Deny from all
   Allow from all
</Location>

This load balancer server will use sticky sessions, because Hybris says that it improves the response time.

Save that content into a file called "httpd.conf" and push it into a code repository, this file will be used inside Jenkins job later (all configuration files should be compressed into config.tar.gz file).

Setting up SSH access in Jenkins

First of all, you need to load every server information into "SSH remote hosts" section, to do this you need to be Jenkins admin and go to

Jenkins -> Manage Jenkins -> Configure System

Create a new View tab

To organize the related cluster jobs, we will create a new view using the + button shown in the image below

We will call it "My-Cluster" and choose the "List View" option.

After that point you can add a description and click on OK button.

Create the "Load-Balancer" job

Click now on "New Item" to create your first Jenkins job and call it "Load-Balancer". After that choose the "Freestyle project" option and click on OK button.

After that choose the "Execute shell" option

Add in the Command area the text below and save it

sshpass -p "jenkins-password" scp -o StrictHostKeyChecking=no config.tar.gz jenkins@192.168.0.1:/home/jenkins/

Where "config.tar.gz" has the httpd.conf file to be used later.

Install SSH Jenkins plugin

Go to 

Jenkins -> Manage Jenkins -> Manage Plugins

Click on "Available" tab and filter by "SSH", after that choose the "SSH plugin" and click on "Download now and install after restart"

Execute a shell script on remote host

Edit the "Load-Balancer" job and add a new build step

After that, choose the SSH site and add the text below inside Command section

#!/bin/bash
echo "Enable Apache modules"
sudo a2enmod rewrite
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod proxy_ajp
sudo a2enmod proxy_balancer
sudo a2enmod proxy_connect
sudo a2enmod headers

echo "Stopping Apache"
sudo /etc/init.d/apache2 stop

echo "Extract config files"
sudo tar -xzf config.tar.gz

echo "Copy files into Apache available sites"
sudo cp -r config/apache-conf/node1/* /etc/apache2/sites-available

echo "Copy static content"
sudo cp -r config/apache-conf/www/* /var/www

echo "Remove old files"
rm -rf config
rm config.tar.gz

echo "Enable Apache sites"
sudo a2ensite my-shop.com
sudo a2ensite hybris

echo "Starting Apache"
sudo /etc/init.d/apache2 start

Save it and run this Jenkins job. If no errors were found, check your Apache server content.

Create the Hybris build job

Now we'll create a new Jenkins job, who will be responsible for building the Hybris platform. We will call it "Hybris-build" job.

To complete this step we need to install the "Mercurial" plugin (because I use this kind of code repository).

After the installation you can add the Mercurial repository in your "Hybris-build" job like the image below

Add also a shell build step with the information below

# get local.properties for jenkins build
mv config/properties/JENKINS_local.properties config/local.properties 

# link hybris platform zips to avoid copying them to speed up build
mkdir -p temp
ln -s -f /opt/ftp/hybrisplatform/* temp/ 

# copy hybris platform
cd config/env_build
printf '1\n' | ant -lib lib/jsch-0.1.50.jar 

# setup project structure
mkdir -p hybris
cp -R bin hybris/
cp -R config hybris/ 

# clean up workspace
rm -R bin
rm -R config
if [ -d "temp" ]; then
   rm -R temp
fi

After that we need an "Invoke Ant"  step, look at the image to have an idea

Add another shell build step with:

# replace jenkins path before Hybris zip the whole files
find . -name '*.java' -or -name '*.conf' -or -name '*.properties' -or -name '*.xml' | xargs perl -pi -e 's/opt\/jenkins\/jobs\/Hybris-build\/workspace/opt/g'
sed -i -e 's|/opt/jenkins/jobs/.*/workspace/hybris|/opt/hybris|g' $WORKSPACE/hybris/bin/platform/tomcat-6/conf/server.xml

Add another "Invoke Ant" and "Execute shell" steps like the image below:

As the last step, you need to archive the results to be used from the deploy job, to do this you need to add a "post-build action", checkout the image below

And fill it in with:

Create the deploy job

The last step in this process is to create the job responsible to stop the Tomcat server, deploy and start Tomcat again.

First, create a new freestyle Jenkins job called "Hybris-deploy".

Inside the Hybris-deploy job, you need to create a boolean parameter to update the database if it's needed.

To fulfil this step, we need to install first the "Copy Artifact" plugin using the Plugin Manager and restart Jenkins.

After Jenkins restart, we can add the copy artifact step

And then give in the build job name

Now we will copy the artifact content into the Hybris servers, for that reason we need a new "Execute shell" step with the content below

echo "Copy all files into Hybris node 1"
sshpass -p "jenkins-password" scp -o StrictHostKeyChecking=no hybrisServer-AllExtensions.zip hybrisServer-Platform.zip jenkins@192.168.0.2:/opt
sshpass -p "jenkins-password" scp -o StrictHostKeyChecking=no config.tar.gz jenkins@192.168.0.2:/opt/hybris/ 

echo "Copy all files into Hybris node 2"
sshpass -p "jenkins-password" scp -o StrictHostKeyChecking=no hybrisServer-AllExtensions.zip hybrisServer-Platform.zip jenkins@192.168.0.3:/opt
sshpass -p "jenkins-password" scp -o StrictHostKeyChecking=no config.tar.gz jenkins@192.168.0.3:/opt/hybris/

And now a new "Execute shell script on remote host using SSH" with the next script:

#!/bin/bash

echo "Stop tomcat in NODE_01"

sudo /etc/init.d/hybristomcat stop

### go to /opt ###

cd /opt

### extract new code ###

sudo unzip -o hybrisServer-Platform.zip

sudo unzip -o hybrisServer-AllExtensions.zip

sudo unzip -o hybrisServer-Config.zip

### provide proper local.properties ###

sudo cp -r hybris/config/P_NODE_01_local.properties hybris/config/local.properties

### remove archives after extraction ###

rm hybrisServer-Platform.zip

rm hybrisServer-AllExtensions.zip

rm hybrisServer-Config.zip

### set execution permissions ###

sudo chmod -R 755 /opt/hybris

### navigate to platform and invoke ant to 'activate' new config ###

cd hybris/bin/platform

. setantenv.sh

ant

# hybris uses wrong wrapper -> replace

sudo cp -r /opt/hybris/bin/platform/tomcat/bin/wrapper-linux-x86-64 /opt/hybris/bin/platform/tomcat/bin/wrapper-linux-x86-32

### copy tomcat running script into linux init daemon ###

sudo cp /opt/hybris/config/tomcat/hybristomcat /etc/init.d/

sudo chmod 755 /etc/init.d/hybristomcat

### trigger DB update if needed ###

echo "DBUpdate: " ${DBUpdate}

if [ "${DBUpdate}" == "true" ] ;

then

    echo "Running DB update with updatesystem..."

    ant updatesystem -Dtenant=master

fi

### start hybris server after code-exchange ###

echo "Starting hybris server in node 1"

sudo service hybristomcat start

 

### wait for hybris server to startup ###

cd /opt/hybris/log/tomcat/

tail -f `ls -tr | grep console | tail -n 1` | while read LOGLINE

do

    [[ "${LOGLINE}" == *"INFORMATION: Server startup in"* ]] && pkill - P $$ tail

done

And a similar one for NODE_02

#!/bin/bash

echo "Stop tomcat in NODE_02"

sudo /etc/init.d/hybristomcat stop

### go to /opt ###

cd /opt

### extract new code ###

sudo unzip -o hybrisServer-Platform.zip

sudo unzip -o hybrisServer-AllExtensions.zip

sudo unzip -o hybrisServer-Config.zip

### provide proper local.properties ###

sudo cp -r hybris/config/P_NODE_02_local.properties hybris/config/local.properties

### remove archives after extraction ###

rm hybrisServer-Platform.zip

rm hybrisServer-AllExtensions.zip

rm hybrisServer-Config.zip

### set execution permissions ###

sudo chmod -R 755 /opt/hybris

### navigate to platform and invoke ant to 'activate' new config ###

cd hybris/bin/platform

. setantenv.sh

ant

# hybris uses wrong wrapper -> replace

sudo cp -r /opt/hybris/bin/platform/tomcat/bin/wrapper-linux-x86-64 /opt/hybris/bin/platform/tomcat/bin/wrapper-linux-x86-32

### copy tomcat running script into linux init daemon ###

sudo cp /opt/hybris/config/tomcat/hybristomcat /etc/init.d/

sudo chmod 755 /etc/init.d/hybristomcat

### start hybris server after code-exchange ###

echo "Starting hybris server in node 2"

sudo service hybristomcat start

Well, I'm very sure that you can improve the script and add a JUnit test job in the middle ;-)
Have fun and let me know if you got it running!
Kind regards,
Celso Kurrle