# Write a Cytomine App

# Requirements

To write Cytomine apps, you need to know how to interact with Cytomine programmatically. If you are not yet familiar with Cytomine HTTP API, have a look at Interact with Cytomine guide.

The Cytomine API client libraries provides many helpers to create new Cytomine apps. A majority of Cytomine apps are written in Python, with the help of the Cytomine API client for Python library. This guide focuses on how to create a Cytomine app in Python. See Installation and Usage instructions if you are not yet ready to use the API client for Python.

# Setup

In this guide, we will create a Cytomine app called MyFirstCytomineApp.

First, create a new directory to gather the files to be created for our app.

mkdir MyFirstCytomineApp
cd MyFirstCytomineApp

1
2

# Create the JSON descriptor

In the app directory, create a new JSON file descriptor.json.

touch descriptor.json

In this file written in JSON (opens new window), we will write all meta information related to your app. We start by providing basic information such as the name for our new app and a textual description:

{
    "name": "MyFirstCytomineApp",
    "description": "This Cytomine App is a first test.",
    "schema-version": "cytomine-0.1"
}

1
2
3
4
5

This declarative file follows a pre-defined schema. It means that JSON keys such as name, description and schema-version are defined in a specification and accept only some values and/or types as values. The JSON descriptor reference summarizes the expected formalism for the cytomine-0.1 version of the schema.

To be added to Cytomine, the app descriptor must strictly conform to the JSON descriptor reference.

The next step is to declare the parameters we want for our app, so that we can interact with the app. We will add an alpha parameter to our app:

{
    "name": "MyFirstCytomineApp",
    "description": "This Cytomine App is a first test.",
    "schema-version": "cytomine-0.1",
    "inputs": [
        {
            "id": "alpha",
            "name": "Alpha",
            "description": "This is the alpha parameter",
            "type": "Number",
            "optional": false,
            "default-value": 0.5
        }
    ]
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

There are new keys to declare a parameter:

id: A short, unique, informative identifier containing only alphanumeric characters and underscores. It is the variable name for this parameter in the source code.
name: A human-readable Cytomine app parameter name.
type: The Cytomine app parameter type. See type in the JSON descriptor reference for supported types.
optional: Whether the parameter should always be given (false) or not (true).
default-value: The default value for the parameter.

The alpha parameter is not related with Cytomine but with your source code.

# Authentication parameters

To make the connection with a Cytomine server, you have to provide additional parameters, not directly related to the app you want to create. Just like you provide host, public_key and private_key to authenticate in Cytomine scripts, Cytomine apps have to declare Cytomine authentication parameters:

cytomine_host
cytomine_public_key
cytomine_private_key
cytomine_id_project
cytomine_id_software

Mandatory Cytomine authentication parameters

The 5 Cytomine authentication parameters are mandatory and must be declared as parameters in every Cytomine app.

Complete the descriptor as follows:

{
    "name": "MyFirstCytomineApp",
    "description": "This Cytomine App is a first test.",
    "schema-version": "cytomine-0.1",
    "inputs": [
        {
            "id": "alpha",
            "name": "Alpha",
            "description": "This is the alpha parameter",
            "type": "Number",
            "optional": false,
            "default-value": 0.5
        },
        {
            "id": "cytomine_host",
            "name": "Cytomine host",
            "set-by-server": true,
            "optional": false,
            "type": "String"
        },
        {
            "id": "cytomine_public_key",
            "name": "Cytomine public key",
            "set-by-server": true,
            "optional": false,
            "type": "String"
        },
        {
            "id": "cytomine_private_key",
            "name": "Cytomine private key",
            "set-by-server": true,
            "optional": false,
            "type": "String"
        },
        {
            "id": "cytomine_id_project",
            "name": "Cytomine project ID",
            "set-by-server": true,
            "optional": false,
            "type": "Number"
        },
        {
            "id": "cytomine_id_software",
            "name": "Cytomine software ID",
            "set-by-server": true,
            "optional": false,
            "type": "Number"
        }
    ]
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

These 5 Cytomine authentication parameters have set-by-server set to true because they are automatically filled by the Cytomine server (when the app is released).

JSON descriptor template for new apps

{
	"name": "My-Software-Name",
	"description": "An optional description of the software",
	"inputs": [
		{
			"id": "cytomine_host",
			"name": "Cytomine host",
			"set-by-server": true,
			"optional": false,
			"type": "String"
		},
		{
			"id": "cytomine_public_key",
			"name": "Cytomine public key",
			"set-by-server": true,
			"optional": false,
			"type": "String"
		},
		{
			"id": "cytomine_private_key",
			"name": "Cytomine private key",
			"set-by-server": true,
			"optional": false,
			"type": "String"
		},
		{
			"id": "cytomine_id_project",
			"name": "Cytomine project ID",
			"set-by-server": true,
			"optional": false,
			"type": "Number"
		},
		{
			"id": "cytomine_id_software",
			"name": "Cytomine software ID",
			"set-by-server": true,
			"optional": false,
			"type": "Number"
		}
	],
	"schema-version": "cytomine-0.1"
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

# Publish the JSON descriptor

Once the descriptor is ready and all app parameters are declared, it has to be published to Cytomine.

From MyFirstCytomineApp folder, open an interactive Python shell and enter:

host = "https://mycytomine.com"
public_key = "AAA"
private_key = "ZZZ"
from cytomine import Cytomine
from cytomine.utilities.descriptor_reader import read_descriptor
with Cytomine(host, public_key, private_key) as c:
	read_descriptor("descriptor.json")

1
2
3
4
5
6
7

where https://mycytomine.com has been replaced by your actual Cytomine instance and, AAA and ZZZ by your API keys. See Interact with Cytomine to learn how to retrieve your API keys if needed.

Your app is now declared in Cytomine!

In the graphical interface, click Algorithms in the Cytomine bar. In the Algorithms page, click Show filters and switch off Only last releases and Only runnable. You should see MyFirstCytomineApp in the list.

If you enabled Developer mode in your Account, the app ID is displayed in the expanded details.

# Enable the app in projects

Cytomine apps can only be run from a project (thus not at a Cytomine platform level). To test your Cytomine app, you need to enable the app in at least one of your projects.

See Enable a Cytomine app in a project in the User Guide.

# App entrypoint

When a Cytomine app is run, the first steps consist in:

retrieving the values associated to the app parameters for this run
authenticating as a run of the app to the Cytomine server

The API client for Python library provides the handy interface CytomineJob that wraps these steps:

from cytomine import CytomineJob
def main(argv):
     with CytomineJob.from_cli(argv) as cj:
        # Implements your software here.
        # Will print the parameters with their values
        print(cj.parameters)
if __name__ == "__main__":
    import sys
    main(sys.argv[1:])

1
2
3
4
5
6
7
8
9
10
11
12

Save this in app.py in MyFirstCytomineApp folder.

Run this script with

python app.py --cytomine_host https://mycytomine.com --cytomine_public_key AAA --cytomine_private_key ZZZ --cytomine_id_project 42 --cytomine_id_software 123 --alpha 0.5

where you have substituted https//mycytomine.com with your actual Cytomine server, AAA and ZZZ with your API keys, 42 with the project ID where you have enabled the app and 123 with the software ID you retrieved at previous step.

As a result, the parameter values are printed on screen. You have just launched the first execution of your Cytomine app!

# What happened?

Behind the scenes, the Python script:

authenticated you to the Cytomine server
parsed the command-line using parameter names and types from the JSON descriptor
asked the Cytomine server to execute the following code as an execution of the Cytomine app given in parameter (here: 123) on your behalf
saved the parsed parameters to the Cytomine server, in agreement with the JSON descriptor
made these parameters available in cj.parameters as a Python dict
run itself in the given project (here: project 42)

TIP

A Cytomine app always runs on the human launcher behalf. It means that the app is only able to manipulate data the human launcher is able to manipulate, in agreement with its access rights in the project and its role at platform level.

# Inspect Cytomine app executions

This first execution has been recorded in the Cytomine server. In the graphical interface, open the project where you launched the app. Open the Analysis tab from the project sidebar. The execution should be referenced as a successful analysis. If you expand the line relative to this execution, you can retrieve the parameter values you used from the command line you executed from your computer.

You have something similar to

# From scripts to apps

As we noticed in this guide, the main differences between Cytomine scripts and Cytomine apps are

Cytomine apps need to be defined in a formal way with the JSON descriptor
Cytomine app executions are referenced on the Cytomine server, for every execution
Cytomine app executions manipulate Cytomine data as non-human user acting on your behalf

Put differently, annotations created in a Cytomine app are thus Analysis annotations (created by an app execution acting on your behalf) while those created in a Cytomine script are User annotations (created by you, like in the graphical interface)

To adapt a Cytomine script source code to a Cytomine app, you have to replace

with Cytomine.connect(host, public_key, privated_key) as c:

with CytomineJob.from_cli(argv) as cj:

where argv is the input command-line (without program name) that is parsed according to the JSON descriptor.

# Manage dependencies

When your app is growing, you will probably rely on some existing dependencies. In Python apps, you will probably need some libraries like Numpy (opens new window) or Scikit-Learn (opens new window) at some point. Sometimes you need some libraries only working on specific operating system versions, or make direct call to binary executables.

Managing all these dependencies (and dependencies of dependencies...) is challenging. Running your Cytomine app with different versions of one of the app dependencies could potentially lead to different results or even break the Cytomine app if the updated dependency introduces breaking changes.

To prevent these situations, we recommend defining the environment in which your Cytomine app has to be executed. It can be achieved with container engines such as Docker (opens new window).

Do I need this ?

It is not strictly mandatory to define an execution environment to run your Cytomine app. For example, the first app execution explained in this guide do not define such an environment.

However, it is a recommended practice enabling result reproducibility and let you share your Cytomine app.

It is a mandatory step to release a Cytomine app and make it executable from Cytomine graphical interface.

Docker lets you run a container image which behaves like a compiled static file with executable code that can be run as an isolated process. When executed, this container runs your code (here, the Cytomine app) in an execution environment you defined, which is isolated from the host machine. With Docker, the image recipe is written in a Dockerfile (opens new window).

To continue and execute Docker containers from your machine, you need to Get Docker (opens new window) installed on your computed.

If you don't have experience with Docker, read first Get started guide by Docker (opens new window) Part 1, 2 and 3. In this tutorial, you will have to create a -very simple- Dockerfile in order to run your algorithm inside a container.

# Create the Dockerfile

The Dockerfile is a recipe which define your execution environment. For details, see the Dockerfile reference (opens new window).

# Base image

The Dockerfile starts with a FROM instruction which initializes a new build stage and sets the base image for subsequent instructions.

Cytomine ULiège R&D provides base images for Python and Java with the appropriate Cytomine API client library pre-installed:

cytomineuliege/software-python3-base (opens new window): A Python 3 environment execution with Cytomine API client library for Python (based on official Python Docker image (opens new window))
cytomineuliege/software-java8-base (opens new window): A Java 8 environment execution with Cytomine API client library for Java (based on official OpenJDK Docker image (opens new window))

Another option is to start from a clean operating system (such as Alpine (opens new window) or Ubuntu (opens new window)) and install all dependencies yourself (such as Python and the Cytomine API client library for Python).

# Install dependencies

Add your Cytomine app dependencies with the RUN instruction, which works like bash commands. A Python dependency can be installed with pip for example:

RUN pip install my_package1 my_package2

# Copy files

The app files have to be copied into the Dockerfile, with the ADD instruction.

RUN mkdir -p /MyFirstCytomineApp
ADD app.py /MyFirstCytomineApp/app.py

1
2

# App entrypoint

The ENTRYPOINT instruction indicates which command is executed as container entrypoint.

ENTRYPOINT ["python", "/MyFirstCytomineApp/app.py"]

# Example

Taking back our example Cytomine app MyFirstCytomine, we decide to rely on Cytomine API client for Python v2.1.0, and we add numpy as a Python dependency:

FROM cytomineuliege/software-python3-base:v2.1.0
RUN pip install numpy
RUN mkdir -p /MyFirstCytomineApp
ADD app.py /MyFirstCytomineApp/app.py
ENTRYPOINT ["python", "/MyFirstCytomineApp/app.py"]

1
2
3
4
5

In MyFirstCytomineApp directory, save this in a file called Dockerfile (without extension).

# Build and run

To build the Docker container image, from your MyFirstCytomineApp directory, run:

docker build -t myfirstcytomineapp .

An image myfirstcytomineapp has been built. Docker requires image names in lowercase. Your source code and its dependencies have been packed into a reproducible execution environment.

To run the Cytomine app container from your machine, run:

docker run myfirstcytomineapp --cytomine_host https://mycytomine.com --cytomine_public_key AAA --cytomine_private_key ZZZ --cytomine_id_project 42 --cytomine_id_software 123 --alpha 0.5

As a result, the parameter values are printed on screen. You have just launched the first execution of your Cytomine app running from its isolated environment! As previously, the run is saved to Cytomine.

# What to do next?

We learned how to create complete programs manipulating Cytomine data, run them from your computer and save results to Cytomine. Every execution is reported to Cytomine, with the parameters applied to the program and the results of each program execution can then be compared from the graphical interface.

We can go one step further. At this stage, you cannot share your Cytomine app with other collaborators nor let them use your app in their projects. Good news: your Cytomine app can be released to be plugged to Cytomine server and directly launched from the graphical interface on dedicated hardware. See how to release a Cytomine app.

← Concepts Release a Cytomine App →