The main domains used during upload are:
AbstractImage: An image on Cytomine database. An image could be add to many projects,
Project: A workspace where you can view and use images,
ImageInstance: An instance of AbstractImage in a project (link between Project and AbstractImage),
Storage: Virtual directory where you can store a file,
UploadedFile: A file being uploaded. Later, it will give rise to an AbstractImage,
Property: key-value that you can add to a Cytomine domain (AbstractImage, ImageInstance, Project,...)
For more information regarding the package Image/Storage, see Package Image - Storage page.
1 uploaded file = 1
UploadedFile object except in these situations :
- 1 Archive file. We decompress and reupload the content. The original file is tagged as the parent of the last
- 1 file which need an heavy conversion. We reupload the result of conversion. The convertible file is tagged as the parent of the converted
After these processes, an
AbstractImage is created from the "last"
UploadedFile. As conversion has been done, this
AbstractImage reference a supported format.
The full upload architecture on Cytomine contains:
CORE: Cytomine main server (e.g. beta.cytomine.be, localhost-core)
IMS: Cytomine image management server (e.g. upload.cytomine.be, localhost-ims)
CLIENT: Cytomine client (Java, Python,...or the JS web client)
STORAGE/IMAGE SERVER: Server storing and delivering images
VIEWER JS: Cytomine js viewer
Complete upload procedure
1. Client send an HTTP message to IMS (POST IMS/upload) with image(s) and some metadata.
The main metadata are:
|cytomine||The cytomine (Core) url where to add the image. An IMS may be used for multiple Core||Mandatory|
|idStorage||The storage where to copy the image||Mandatory|
|idProject||If not null, link the image with this project (create a new ImageInstance from the AbstractImage)||Optional, default null|
|sync||Set the upload synchronized||Optional, default false|
|keys||List of properties keys (join with “,”). |
The key(i) will be link with values(i) and this key-value entry will be insert as a property for the image (AbstractImage).
values and keys must have the same number of items!
|Optional, default null|
|values||List of properties values (join with “,”). See keys definition||Optional, default nul|
2. IMS checks parameters and authentify the user. The communication between Core and IMS is made trough the Java client. (
3. IMS (UploadService) creates an
UploadedFile instance on Cytomine-Core with the upload and copy it to its storage. The file path follows the convention
4. IMS (UploadService) tries to detect the format. A supported format is either
- natively supported (Pyramidal TIFF, JPEG2000, OpenSlide supported formats)
- supported after a light conversion (JPEG, PNG, BMP, DICOM, PlanarTIFF, HuronTIFF, ...). See step 6 for light conversion.
- supported after a heavy conversion (all formats convertible to TIFF by BioFormat). See step 5 for heavy conversion.
If sync parameter is set to true, the conversion and deployment is done sequentially and not in background. In the case where
sync=false, the next steps will be processed in background.
5. For the files which need a heavy conversion, they are converted using BioFormat. The result of conversion is reuploaed and the convertible file is tagged as the parent of the converted
UploadedFile, as previously explained. Depending on the format, the process may take some time.
6. For the files which need a light conversion, the format is convertible by VIPS and it creates a new pyramidal TIFF image, which is stored in the same directory as the original image, but this time with a random name (and the .tif extension).
7. IMS asks Core to create an AbstractImage image based on the
UploadedFile and the novel corresponding pyramidal TIFF image. If a project was specified, Core will create and
ImageInstance to make the link between this project and the freshly created
8. IMS responds to the client. The reponse mainly contains: (200=OK),
|status||HTTP response status. 200 = OK|
|uploadedFile||The uploaded file (JSON)|
|images||A list of AbstractImage (generally with only 1 item). |
This property is only available if sync parameter = true!
(if async, response is send before the AbstractImage creation)
Upload possible errors
Errors may happens at some step:
|#||Step summary||If an error happens here...|
|Step 1||Client send message to IMS||The client should gives you an error (network error,...)|
|Step 2-3||IMS retrieve parameters and image||The IMS should send you an error. It could be an auth error (bad keys) or parameters errors (keys.size!=values.size, bad project...)|
|If you are in async mode (by default, see sync parameter), IMS will response to the client after step 4. The reponse contains a property “uploadFile”. You can frequently get the uploaded file with a client to check the new status|
|Step 3->8||IMS process and create image on Core||Only if sync=true, you will get an error|
Uploaded file status
Each UploadedFile has an associated status.
|0||UPLOADED||Default status when a new UploadedFile is added to Core (at step 3-4)|
|1||CONVERTED||A file with heavy conversion has been converted (at step 5)|
|2||DEPLOYED||The file has been possibly converted by VIPS and a corresponding AbstractImage is created (from end of step 6)|
|3||ERROR_FORMAT||The format is not detected and the upload is aborted (at step 4)|
|4||ERROR_CONVERT||There is an error during heavy conversion (at step 5)|
|5||UNCROMPRESSED||For archive format, but not used ?|
|6||TO_DEPLOY||The format has been detected, waiting for deployment (between end of step 4 and step 6)|
For more information regarding image/storage domain, see Package Image - Storage.
- For more information regarding the IMS, see Part 7: IMS.
- For more information regarding the image server (IIP server), see Part 8: Image server.
- For more information regarding the image formats supported (natively, after light conversion or after heavy conversion), see Guide: Image format.
Upload files with a client.
Upload with Java
If you use async mode, you may want to retrieve the upload status
Upload with Python
See example (outdated?) here: https://github.com/cytomine/Cytomine-python-client/blob/master/client/examples/upload_image.py