Page tree
Skip to end of metadata
Go to start of metadata

Note :

Deploy Cytomine on multiple servers is not so easy, you need to have some knowledge in Linux and Docker.

On this page, I will split Cytomine in only 4 machines and give the complete code for only 2 examples :

  • IMS (full code below)
  • IIP1 (full code below)
  • IIP2
  • The last services (Core, postgis, etc.)

On each, we will install Cytomine-bootstrap.

If you understood the following explainations, you will be able to deploy on more machines.

 

Global rules :

As you deploy on multiple servers, some componants (Core, IMs, etc.) are on the other servers. So, all images are not needed and it would be a good thing to not compile them. You will saving up HD memory and time.

To identify which images are needed, follow the rule "If I want the service X on this machine, I check into the Dockerfile its ancestor and if X is not the ancestor of another Dockerfile. Then, I check into start_deploy.sh if another service is linked to it."

We will also need the nginx image.

 

Then, update the main scripts to delete the references on the deleted images : 

  • create_docker_images.sh 
  • reporting.sh 
  • start_deploy.sh 
  • clean_docker_keep_data.sh

You will also need to update the nginx/nginx.conf file and nginx/deploy.sh

Example : IMS

In the Dockerfile, we see "FROM cytomine/tomcat7" So we need tomcat7 and its ancestors.

Recursively, we have the following hierarchy :

IMS - tomcat7 - java7 - base

because IMS doesn't have any child.

We keep nginx.

We look at the start_deploy file too to see if another service is needed.

bioformat is linked with IMS, we will keep bioformat on the same machine than ims.

You can now delete the folders other than IMS, tomcat7, java7, nginx, bioformat and base.

We can delete useless scripts : update_core_war.sh

Update the other to delete the references on the deleted images : 

  • create_docker_images.sh 
  • reporting.sh 
  • start_deploy.sh 
  • clean_docker_keep_data.sh 

For example, the new create_docker_images.sh is now

#get all the config values.
. ./configuration.sh
cd base && docker build -t="cytomine/base" .
cd ../java7 && docker build -t="cytomine/java7" .
cd ../tomcat7 && docker build -t="cytomine/tomcat7" .
cd ../bioformat && docker build -t="cytomine/bioformat" .
cd ../ims && docker build -t="cytomine/ims" .
cd ../nginx && docker build -t="cytomine/nginx" .
cd ..
echo DONE

 

clean_docker_keep_data.sh

docker stop bioformat
docker rm -v bioformat
docker stop ims
docker rm -v ims
docker stop nginx
docker rm -v nginx

 

reporting

rm -r ./reporting.tgz
docker cp ims:/usr/share/tomcat7/.grails/imageserverconfig.properties ./reporting/ims
docker cp ims:/var/lib/tomcat7/logs/catalina.out ./reporting/ims
mv ./reporting/ims/imageserverconfig.properties ./reporting/configurationIMS.properties
tail -n 200 ./reporting/ims/catalina.out > ./reporting/catalinaIMS.out
rm -r ./reporting/ims
cp ./configuration.sh ./reporting/configuration.sh
cp ./start_deploy.sh ./reporting/start_deploy.sh
tar -zcvf reporting.tgz reporting
rm -r ./reporting

 

start_deploy

Don't forget to delete the link (ex : nginx link to core)

The nginx will only be the proxy to ims so you can delete the useless variables.

Be careful, the IMS_PUB_KEY is in the database. So you need to launch Core first, select the key values into the database and set the goo value instead of $(cat /proc/sys/kernel/random/uuid).

 

#get all the config values.
. ./configuration.sh
nb_docker=$(echo "$(sudo docker ps)" | wc -l)
nb_docker=$((nb_docker-1)) # remove the header line

if [ $BIOFORMAT_ENABLED = true ]
then
	docker run -p 22 -d --name bioformat -v $IMS_STORAGE_PATH:$IMS_STORAGE_PATH \
	-e BIOFORMAT_PORT=$BIOFORMAT_PORT \
	-e BIOFORMAT_JAR_URL=$BIOFORMAT_JAR_URL \
	cytomine/bioformat
	nb_docker=$((nb_docker+1))
fi
IMS_PUB_KEY=your_value
IMS_PRIV_KEY=your_value
# create IMS docker
docker run -p 22 -v $IMS_STORAGE_PATH:$IMS_STORAGE_PATH -m 8g -d --name ims \
-v /tmp/uploaded/ \
-e IIP_OFF_URL=$IIP_OFF_URL \
-e IIP_VENT_URL=$IIP_VENT_URL \
-e IIP_CYTO_URL=$IIP_CYTO_URL \
-e IIP_JP2_URL=$IIP_JP2_URL \
-e IMS_URLS=$IMS_URLS \
-e IMS_STORAGE_PATH=$IMS_STORAGE_PATH \
-e IMS_BUFFER_PATH=$IMS_BUFFER_PATH \
-e WAR_URL=$IMS_WAR_URL \
-e DOC_URL=$IMS_DOC_URL \
-e IS_LOCAL=$IS_LOCAL \
-e HAS_GLUSTER=$HAS_GLUSTER \
-e CORE_URL=$CORE_URL \
-e IMS_PUB_KEY=$IMS_PUB_KEY \
-e IMS_PRIV_KEY=$IMS_PRIV_KEY \
-e BIOFORMAT_ENABLED=$BIOFORMAT_ENABLED \
-e BIOFORMAT_LOCATION=$BIOFORMAT_ALIAS \
-e BIOFORMAT_PORT=$BIOFORMAT_PORT \
cytomine/ims
nb_docker=$((nb_docker+1))
# add a dynamic link to bioformat
if [ $BIOFORMAT_ENABLED = true ]
then
	BIOFORMAT_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' bioformat)
	docker exec ims /bin/bash -c "echo $BIOFORMAT_IP       $BIOFORMAT_ALIAS >>  /etc/hosts"
fi

IMS_ALIAS=ims
# create nginx docker
	docker run -m 1g -d -p 22 -p 80:80 --link ims:$IMS_ALIAS \
	--volumes-from ims  \
	--name nginx \
	-e IMS_URLS="$IMS_URLS" \
	-e IMS_ALIAS=$IMS_ALIAS \
	-e UPLOAD_URL=$UPLOAD_URL \
	cytomine/nginx
nb_docker=$((nb_docker+1))
# checking
running_containers=$(sudo docker ps)
nb_started_docker=$(echo "$running_containers" | wc -l)
nb_started_docker=$((nb_started_docker-1)) # remove the header line
#echo "number of started docker = $nb_started_docker"
#echo "number of asked docker = $nb_docker"
if [ $nb_started_docker -eq $nb_docker ]
then
        touch ./.cookies
else
	if ! echo "$running_containers" | grep -q -w nginx; then echo "nginx container is not running !"; fi
	if ! echo "$running_containers" | grep -q -w ims; then echo "ims container is not running !"; fi
	if [ $BIOFORMAT_ENABLED = true ]
	then
		if ! echo "$running_containers" | grep -q -w bioformat; then echo "bioformat container is not running !"; fi
	fi
        echo "Please check into your docker logs."
        #echo "A problem occurs. Please check into your docker logs."
fi

echo "End of the installation."

 

Now we can see the used variables and delete the unused into the configuration file

CORE_URL=localhost-core
IMS_URLS="[localhost-ims,localhost-ims2]"
UPLOAD_URL=localhost-upload
IIP_OFF_URL=localhost-iip-base
IIP_VENT_URL=localhost-iip-ventana
IIP_CYTO_URL=localhost-iip-cyto
IIP_JP2_URL=localhost-iip-jp2000
HAS_GLUSTER=false
GLUSTER_SERVER=
VOLUME=aurora
IS_LOCAL=true
IMS_STORAGE_PATH=/data
IMS_BUFFER_PATH=/data/_buffer
BIOFORMAT_ENABLED="true"
# You don't to change the datas below this line instead of advanced customization
# ---------------------------
IMS_WAR_URL="https://github.com/cytomine/Cytomine-IMS/releases/download/v1.0/IMS.war"
IMS_DOC_URL="https://github.com/cytomine/Cytomine-IMS/releases/download/v1.0/restapidoc.json"
BIOFORMAT_JAR_URL="https://github.com/cytomine/Cytomine-tools/releases/download/v1.0/BioFormatStandAlone.tar.gz"
BIOFORMAT_ALIAS="bioformat"
BIOFORMAT_PORT="4321"

 

nginx/nginx.conf.sample

Delete the reference to the uninstalled containers.

#user  nobody;
worker_processes  1;
daemon off;
error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    server_names_hash_bucket_size  128;
    log_format  main  '$host - $remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  120;
    proxy_connect_timeout 75;
IMS_URLS_CONFIG

    server {
	        client_max_body_size 0;
	        listen       80;
	        server_name  UPLOAD_URL;
	        #charset koi8-r;
	        #access_log  logs/host.access.log  main;
			# Upload form should be submitted to this location
			location /upload {
				proxy_read_timeout 600;
				# Pass altered request body to this location
				upload_pass   @test;
				# Store files to this directory
				# The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
				upload_store /tmp/uploaded;
				# Allow uploaded files to be read only by user
				upload_store_access user:rw group:rw all:rw;
				# Set specified fields in request body
				upload_set_form_field $upload_field_name.name "$upload_file_name";
				upload_set_form_field $upload_field_name.content_type "$upload_content_type";
				upload_set_form_field $upload_field_name.path "$upload_tmp_path";
				# Inform backend about hash and size of a file
				upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5";
				upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";
				upload_pass_form_field "^submit$|^description$";
				upload_pass_args on;
				add_header 'Access-Control-Allow-Credentials' 'false';
				add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
				add_header 'Access-Control-Allow-Headers' 'Content-Type, Content-Range, Content-Disposition, Content-Description, date, dateFull, authorization,content-type-full';
				add_header 'Access-Control-Max-Age' '1728000';
				add_header 'Access-Control-Allow-Origin' '*';
			}
			# Pass altered request body to a backend
			location @test {
				add_header 'Access-Control-Allow-Origin' '*';
				proxy_pass   http://IMS_ALIAS:8080;
				proxy_read_timeout 600;
			}
			error_page   500 502 503 504  /50x.html;
			location = /50x.html {
				root   html;
			}
	}
}

 

nginx/deploy.sh

Idem. Delete unused variables 

echo "Beginning of the deployment"
/etc/init.d/ssh start
echo "SSH started"
sed -i "s/IMS_ALIAS/$IMS_ALIAS/g" /tmp/nginx.conf.sample
sed -i "s/UPLOAD_URL/$UPLOAD_URL/g" /tmp/nginx.conf.sample

IMS_URLS_CONFIG=""
### transform the ims urls for the config file ###
arr=$(echo $IMS_URLS | tr "," "\n")
arr=$(echo $arr | tr "[" "\n")
arr=$(echo $arr | tr "]" "\n")
for x in $arr
do
	sed -i "s/IMS_URLS_CONFIG/   server { \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                client_max_body_size 0; \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                listen       80; \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                server_name  $x; \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                location \/ { \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                        add_header Access-Control-Allow-Origin *; \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/			proxy_set_header Host \$host; \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                        proxy_pass http:\/\/$IMS_ALIAS:8080; \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/                } \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
	sed -i "s/IMS_URLS_CONFIG/    } \\`echo -e '\n\r'` \\`echo -e '\n\r'` IMS_URLS_CONFIG/g" /tmp/nginx.conf.sample
done
sed -i "s/IMS_URLS_CONFIG//g" /tmp/nginx.conf.sample
### END transform the ims urls for the config file ###
mv /tmp/nginx.conf.sample /usr/local/nginx/conf/nginx.conf
mkdir -p /tmp/uploaded && chmod 777 /tmp/uploaded
echo "Launch of nginx"
/usr/local/nginx/sbin/nginx
echo "End of the deployment"
tail -F /usr/local/nginx/logs/access.log

 

You can now run the create_docker_images & start_deploy scripts.

 

Example 2 : IIP

Here, we will put 2 of the 4 IIP on one machine (iipOff & iipJ2).

git clone of Cytomine-bootstrap as explained here

We can see in start_deploy than iip use memcached and via the Dockerfile, we can see the hierarchy :

base - iipbase - iipX

and nginx are still mandatory

delete all the folder who are not needed.

We can delete useless scripts : update_core_war.sh, update_ims_war.sh

Update the other to delete the references on the deleted images : 

  • create_docker_images.sh 
  • reporting.sh 
  • start_deploy.sh 
  • clean_docker_keep_data.sh  

 

Follow the same way than previously to update create_docker_images.sh and reporting.sh.

In start_deploy.sh

Don't forget to delete the link (ex : nginx link to core)

The nginx will only be the proxy to ims so you can delete the useless variables.

Be carefull,I delete memcached 3 & 4. Si I updated the "link" into the run command for the 2 iip

 

#get all the config values.
. ./configuration.sh
nb_docker=$(echo "$(sudo docker ps)" | wc -l)
nb_docker=$((nb_docker-1)) # remove the header line
# create memcached docker
docker run -d -e MEMCACHED_PASS="mypass" --name memcached1 cytomine/memcached
nb_docker=$((nb_docker+1))
docker run -d -e MEMCACHED_PASS="mypass" --name memcached2 cytomine/memcached
nb_docker=$((nb_docker+1))

# create IIP dockers
docker run -p 22 --privileged -d --name iipOff -v $IMS_STORAGE_PATH:$IMS_STORAGE_PATH \
--link memcached1:memcached \
-e IIP_ALIAS="iip_officiel" \
-e GLUSTER_SERVER=$GLUSTER_SERVER \
-e VOLUME=$VOLUME \
-e IMS_STORAGE_PATH=$IMS_STORAGE_PATH \
-e HAS_GLUSTER=$HAS_GLUSTER \
cytomine/iipofficiel
nb_docker=$((nb_docker+1))
docker run -p 22 --privileged -d --name iipJ2 -v $IMS_STORAGE_PATH:$IMS_STORAGE_PATH \
--link memcached2:memcached \
-e IIP_ALIAS="iip_jpeg2000" \
-e GLUSTER_SERVER=$GLUSTER_SERVER \
-e VOLUME=$VOLUME \
-e IMS_STORAGE_PATH=$IMS_STORAGE_PATH \
-e HAS_GLUSTER=$HAS_GLUSTER \
cytomine/iipjpeg2000
nb_docker=$((nb_docker+1))

# create nginx docker
	docker run -m 1g -d -p 22 -p 80:80  \
	--link iipOff:iip_officiel  \
	--link iipJ2:iip_jpeg2000 \
	--name nginx \
	-e IIP_OFF_URL=$IIP_OFF_URL \
	-e IIP_JP2_URL=$IIP_JP2_URL \
	cytomine/nginx
nb_docker=$((nb_docker+1))
# checking
running_containers=$(sudo docker ps)
nb_started_docker=$(echo "$running_containers" | wc -l)
nb_started_docker=$((nb_started_docker-1)) # remove the header line
#echo "number of started docker = $nb_started_docker"
#echo "number of asked docker = $nb_docker"
if [ $nb_started_docker -eq $nb_docker ]
then
        touch ./.cookies
else
	if ! echo "$running_containers" | grep -q -w nginx; then echo "nginx container is not running !"; fi
	if ! echo "$running_containers" | grep -q -w memcached1; then echo "memcached1 container is not running !"; fi
	if ! echo "$running_containers" | grep -q -w memcached2; then echo "memcached2 container is not running !"; fi
	if ! echo "$running_containers" | grep -q -w iipOff; then echo "iipOff container is not running !"; fi
	if ! echo "$running_containers" | grep -q -w iipJ2; then echo "iipJ2 container is not running !"; fi
        echo "Please check into your docker logs."
        #echo "A problem occurs. Please check into your docker logs."
fi

echo "End of the installation."

 

Now we can see the used variables and delete the unused into the configuration file

IIP_OFF_URL=localhost-iip-base
IIP_JP2_URL=localhost-iip-jp2000
HAS_GLUSTER=false
GLUSTER_SERVER=
VOLUME=aurora
IS_LOCAL=true
IMS_STORAGE_PATH=/data

# You don't to change the datas below this line instead of advanced customization
# ---------------------------
MEMCACHED_PASS="mypass"

 

nginx

 

nginx/nginx.conf.sample

Delete the reference to the uninstalled containers.

#user  nobody;
worker_processes  1;
daemon off;
error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    server_names_hash_bucket_size  128;
    log_format  main  '$host - $remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  120;
    proxy_connect_timeout 75;
    #gzip  on;
        server {
                listen       80;
                server_name  IIP_OFF_URL;
                location / {
                        proxy_set_header X-Forwarded-Host $host;
			proxy_set_header Host $host;
                        proxy_set_header X-Forwarded-Server $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_pass http://iip_officiel:80;
                }
        }
        server {
                listen       80;
                server_name  IIP_JP2_URL;
                location / {
                        proxy_set_header X-Forwarded-Host $host;
			proxy_set_header Host $host;
                        proxy_set_header X-Forwarded-Server $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_pass http://iip_jpeg2000:80;
                }
        }
}

 

nginx/deploy.sh

Idem. Delete unused variables 

echo "Beginning of the deployment"
/etc/init.d/ssh start
echo "SSH started"
sed -i "s/IIP_OFF_URL/$IIP_OFF_URL/g" /tmp/nginx.conf.sample
sed -i "s/IIP_JP2_URL/$IIP_JP2_URL/g" /tmp/nginx.conf.sample

mv /tmp/nginx.conf.sample /usr/local/nginx/conf/nginx.conf
mkdir -p /tmp/uploaded && chmod 777 /tmp/uploaded
echo "Launch of nginx"
/usr/local/nginx/sbin/nginx
echo "End of the deployment"
tail -F /usr/local/nginx/logs/access.log

 

You can now run the create_docker_images & start_deploy scripts.

  • No labels