Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 

Code Block
execute_command = "python algo/my_algorithm/script.py --cytomine_host $host --cytomine_public_key $publicKey --cytomine_private_key $privateKey " + "--cytomine_base_path /api/ " + "--cytomine_working_path /software_router/algo/detect_sample/ " + "--cytomine_id_software $cytomine_id_software " + "--cytomine_id_project $cytomine_id_project " + "--cytomine_id_image $cytomine_id_image " + "--cytomine_predict_term $cytomine_predict_term " + "--cytomine_max_image_size $cytomine_max_image_size " + "--cytomine_erode_iterations $cytomine_erode_iterations " + "--cytomine_dilate_iterations $cytomine_dilate_iterations " + "--cytomine_athreshold_blocksize $cytomine_athreshold_blocksize " + "--cytomine_athreshold_constant $cytomine_athreshold_constant " + "--verbose true "

 

In this command, the following arguments are mandatory (in addition to your own software parameters):
                            "--cytomine_host $host "
                            "--cytomine_public_key $publicKey "
                            "--cytomine_private_key $privateKey "
                            "--cytomine_base_path /api/ "
                            "--cytomine_id_software $cytomine_id_software "
                            "--cytomine_id_project $cytomine_id_project "

 

For logging you also have to add after the add_software call these two lines: 
Code Block
conn.add_software_parameter("cytomine_id_software", software.id, "Number",0, True, 400, True)
conn.add_software_parameter("cytomine_id_project", software.id, "Number", 0, True, 500, True)

 

If some of your software parameters are related to some Cytomine "objects", you can specify a service to display user-friendly information, e.g. the following command will then allow to have a drop-down list to select on which image to apply your algorithm, and you will see in the Cytomine-WebUI drop-down list images and thumbnails (rather than imageids) in the "Run Job" execution dialog box (see screenshot below):
Code Block
conn.add_software_parameter("cytomine_id_image", software.id, "Domain", "", True, "", False, "/api/project/$currentProject$/imageinstance.json", "instanceFilename", "instanceFilename")

 

 

Here is a full add_software.py example that will add the "my_algorithm" algorithm (it requires the script.py file to be in the docker container as explained above. In this example the script.py is based on the same code than https://github.com/cytomine/Cytomine-python-datamining/blob/master/cytomine-applications/detect_sample/detect_sample.py and it can be tested on the DEMO-SEGMENTATION-TISSUE toy data project). You have to adapt variables to your cytomine_host, keys, location of your script file in the docker container, location of your cytomine_working_path in the docker container:

 

Code Block
import cytomine
import sys
#Connect to cytomine, edit connection values
cytomine_host=""  # Cytomine core URL
cytomine_public_key=""  # Your public key
cytomine_private_key=" # Your private key
id_project=XXX  #Integer value corresponding to Cytomine project identifier where to add the software

#Connection to Cytomine Core
conn = cytomine.Cytomine(cytomine_host, cytomine_public_key, cytomine_private_key, base_path = '/api/', working_path = '/tmp/', verbose= True)

execute_command = "python algo/my_algorithm/script.py --cytomine_host $host --cytomine_public_key $publicKey --cytomine_private_key $privateKey " + "--cytomine_base_path /api/ " + "--cytomine_working_path /software_router/algo/my_algorithm/ " + "--cytomine_id_software $cyto
mine_id_software " + "--cytomine_id_project $cytomine_id_project " + "--cytomine_id_image $cytomine_id_image " + "--cytomine_predict_term $cytomine_predict_term " + "--cytomine_max_image_size $cytomine_max_image_size " + "--cytomine_erode_iterations $cytomine_erode_iterations " + "--
cytomine_dilate_iterations $cytomine_dilate_iterations " + "--cytomine_athreshold_blocksize $cytomine_athreshold_blocksize " + "--cytomine_athreshold_constant $cytomine_athreshold_constant " + "--verbose true "
#define software parameter template
software = conn.add_software("My_algorithm", "createRabbitJobWithArgsService","ValidateAnnotation", execute_command)
conn.add_software_parameter("cytomine_max_image_size", software.id, "Number", 2048, True, 10, False)
conn.add_software_parameter("cytomine_erode_iterations", software.id, "Number", 3, True, 30, False)
conn.add_software_parameter("cytomine_dilate_iterations", software.id, "Number", 3, True, 40, False)
conn.add_software_parameter("cytomine_athreshold_constant", software.id, "Number", 5, True, 50, False)
conn.add_software_parameter("cytomine_athreshold_blocksize", software.id, "Number", 951, True, 60, False)

#for logging (set by server)
conn.add_software_parameter("cytomine_id_software", software.id, "Number",0, True, 400, True)
conn.add_software_parameter("cytomine_id_project", software.id, "Number", 0, True, 500, True)
# for user-friendly drop-down menu
conn.add_software_parameter("cytomine_id_image", software.id, "Domain", "", True, "", False, "/api/project/$currentProject$/imageinstance.json", "instanceFilename", "instanceFilename")
conn.add_software_parameter("cytomine_predict_term", software.id, "Domain", "", True, "", False, "/api/project/$currentProject$/term.json", "name", "name")
#add software to a given project
addSoftwareProject = conn.add_software_project(id_project,software.id)
 

 

Execute the add_software.py script to add the software to your Cytomine Core:

Code Block
python add_software.py

 

You will then be able to execute your software throught the Cytomine-WebUI. In practice, Cytomine-Core and RabbitMQ will append the parameter values encoded by the end-user in Cytomine-WebUI dialog box through the command-line arguments that executes the script contained in the software_router container.

Please notice in the second screenshot the user-friendly drop-down image list for Images and the list of ontology term names):

 

 

 

2. Logs and debugging

It is recommended that in your software you include some communications with Cytomine-Core to give information to the end-user about the status of its running job (these status messages will be displayed in the Jobs tab in the Cytomine-WebUI). It can be done by using the update_job_status, e.g. in Python (see other examples in our Cytomine-Datamining applications):

 

Code Block
progress_msg = "Analyzing image %s (%d / %d )..." %(id_image,i,len(images))
job = conn.update_job_status(job, status = job.RUNNING, progress = progress, status_comment = progress_msg)

These status messages will be displayed in the Cytomine-WebUI Job tab:

 

However, for debugging, these status messages might not be enough. To see the full execution logs, you can enter into the software_router docker container and see the outputs of your job.

 

Code Block
sudo docker-enter software_router
cd /software_router/algo/   ;; code
cd /software_router/algo/logs/ 
ls -altr
 
root@6041e74c788b:/software_router/algo/logs# ls -altr
total 620
-rw-r--r--  1 root root 349657 Feb 10 12:15 queueSoftwareTissueDetect-10-2-2017_12-15-40-423.log
-rw-r--r--  1 root root     98 Feb 10 12:29 queueSoftwareMy_algorithm-10-2-2017_12-29-09-101.log
drwxr-xr-x 18 root root   4096 Feb 10 12:30 ..
drwxr-xr-x  2 root root   4096 Feb 10 12:30 .
-rw-r--r--  1 root root 266401 Feb 10 12:30 queueSoftwareMy_algorithm-10-2-2017_12-30-44-096.log