Workflow Service Manual

The UNICORE Workflow service provides advanced workflow processing capabilities using UNICORE resources. The Workflow service provides graphs of activities including high-level control constructs (for-each, while, if-then-else, etc), and submits and manages the execution of single UNICORE jobs.

The Workflow service offers a REST API for workflow submission and management and uses an easy-to-understand workflow description syntax in JSON format.

Thanks to a flexible internal workflow model and execution engine, the Workflow service can be in principle extended with custom workflow parsers and custom activities.

The Workflow service supports the full range of authentication options provided by UNICORE and uses JWT tokens for delegated authentication when submitting jobs to the execution sites.

For more information about UNICORE visit https://www.unicore.eu.

Installing and setting up the UNICORE Workflow engine

This chapter covers basic installation of the Workflow engine and the integration of the workflow services into an existing UNICORE system.

As a general note, the Workflow engine is based on a UNICORE/X instance. General UNICORE configuration concepts (such as user authentication, gateway integration, shared registry, attribute sources) fully apply, and you should refer to the UNICORE/X manual for such details.

Prerequisites

Java 11 or later

An existing UNICORE installation with Gateway, Shared Registry and one ore more UNICORE/X execution systems.

A server certificate (for production use)

Updating from previous versions

If you update from 7.x, please note that it is a major update, and we suggest installing from scratch based on the template config files. The required changes are very similar to the UNICORE/X 7.x to 8.x update.

Installation

The workflow system is available either as part of the UNICORE Server bundle (tar.gz or zip), or as separate Linux packages (deb or rpm) on the UNICORE project website at sourceforge.

The basic installation procedure is completely analogous to the installation of a UNICORE/X server.

If you downloaded the UNICORE server bundle, please untar the tar.gz, review the configure.properties file and edit the parameters to integrate the workflow services into your existing UNICORE environment. Then call ./configure.py to apply your settings to the configuration files. Finally use ./install.py to install the workflow server files to the selected installation directory.
If using the Linux packages, simply install using the package manager of your system and review the config files.

Setup

After installation, there are some manual steps needed to integrate the new server into your UNICORE installation.

Gateway: edit gateway/conf/connections.properties and add the connection data for the Workflow server. For example,
```
WORKFLOW = https://localhost:7700
```
XUUDB: if you chose to use an XUUDB for the Workflow server, you might have to add entries to the XUUDB to allow users access to the workflow engine. Optionally, you can edit the GCID used by the workflow/servorch servers, so that existing entries in the XUUDB will match.
Registry: if the registry is setup to use access control (which is the default), you need to allow the Workflow server to register in the Registry. The exact procedure depends on how you configured your Registry, please refer to the section Access control in the Registry manual. If you’re using default certificates and the XUUDB, the required entries can be added as follows:
```
$ cd xuudb
$ bin/admin.sh add REGISTRY <workflow>/conf/workflow.pem nobody server
```

Verifying the installation

Using the UNICORE commandline client, you can check whether the new server is available and accessible:

$ ucc system-info -l

should include output such as

Checking for <Workflow submission> endpoint ...
... OK, found 1 endpoint(s)
 * https://localhost:8080/WORKFLOW/rest/workflows
  * server v8.0.0 CN=Demo Workflow,O=UNICORE,C=EU
  * authenticated as: 'CN=Demo User, O=UNICORE, C=EU' role='user'

The “authenticated as:” line should list you as user.

Some more info about the server can be obtained via

$ ucc rest get https://localhost:8080/WORKFLOW/rest/workflows

Running a test job

Using UCC again, you can submit workflows

$ ucc workflow-submit /path/to/samples/date1.json

and get the ID of your new workflow back, e.g.

https://localhost:8080/WORKFLOW/rest/workflows/86686f72-b732-42e8-b14d-a8bd514e7edf

API documentation

Since version 8.0, the Workflow engine exclusively uses a RESTful API for all operations including job submission.

You can find an API reference and usage examples in Workflow description.

Configuration of the Workflow server

This chapter covers configuration options for the Workflow server. Since the Workflow server is running in the same underlying environment (UNICORE Services Environment, USE), a lot of the basic configuration options are documented in the UNICORE/X manual.

Note

The configuration files in the distribution are commented, and contain example settings for all the options listed here.

Depending on how you installed the server, the files are located on:

/etc/unicore/workflow (Linux package)
<basedir>/workflow/conf (standalone installer)

Workflow processing

Some details of the workflow engine’s behaviour can be configured. All these settings are made in uas.config.

Limits

To avoid too many tasks submitted (possibly erroneously) from a workflow, various limits can be set:

workflow.maxActivitiesPerGroup limits the total number of tasks submitted for a single group (i.e. (sub-)workflow). By default, this limit is 1000, ie. a maximum number of 1000 jobs can be created by a single group. Note, that it is not possible to limit the total number of jobs for any workflow, it can only be applied to individual parts of the workflow (such as loops).
workflow.forEachMaxConcurrentActivities limits the maximum number of tasks in a for-each group that can be active at the same time (default: 20).

Resubmission

The workflow engine will (in some cases) resubmit failed tasks to the service orchestrator. To completely switch off the resubmission,

workflow.resubmitDisable=true

To change the maximum number of resubmissions from the default 3,

workflow.resubmitLimit=3

Cleanup behaviour

This controls the behaviour when a workflow is removed (automatically or by the user). By default, the workflow engine will remove all child jobs, but will keep the storage where the files are. This can be controlled using two properties:

workflow.cleanupStorage remove storage when workflow is destroyed (default: false)
workflow.cleanupJobs remove jobs when workflow is destroyed (default: true)

XNJS settings

The workflow engine uses the XNJS library for processing workflows. Some settings for modifying the behaviour are available, and are usually found in the workflow server’s container.properties file.

An important characteristic is the number of threads used by the workflow engine for processing. Note, this does not control the number of concurrent activities, etc., since all XNJS processing is asynchronous. The default number (4) is usually fine.

This properties is set via

XNJS.numberofworkers=4

Property reference

A complete reference of the properties for configuring the Workflow server is given in the following table:

Property name	Type	Default value / mandatory	Description
workflow.additionalSettings	filesystem path		Optional file containing additional settings (e.g. XNJS.* settings) only used for the workflow service.
workflow.cleanupJobs	[true, false]	true	Whether to remove child jobs when the workflow is destroyed.
workflow.cleanupStorage	[true, false]	false	Whether to cleanup the workflow storage when the workflow is destroyed.
workflow.fastPollingInterval	integer >= 1	20	Interval in seconds for (slow) polling of job states.
workflow.forEachConcurrentActivities	integer >= 1	100	Default maximum number of concurrent for-each iterations (user can increase this).
workflow.forEachMaxConcurrentActivities	integer >= 1	200	Hard limit on the number of concurrent for-each iterations.
workflow.internalMode	[true, false]	false	Internal mode: Workflow service only uses services deployed in the same UNICORE/X instance.
workflow.maxActivitiesPerGroup	integer >= 1	1000	Maximum number of workflow activities per activity group.
workflow.pollingInterval	integer >= 1	600	Interval in seconds for (slow) polling of job states.
workflow.resubmitDisable	[true, false]	false	Whether to disable automatic re-submission of failed jobs.
workflow.resubmitLimit	integer >= 1	3	Maximum number of re-submissions of failed jobs.
workflow.xnjsConfiguration	string	n/a	(deprecated)

Updating an existing UNICORE Workflow service

This chapter covers the steps required to update an existing workflow installation (version 8.x).

Stop the server

Stop the workflow server.

Backup

You should make a backup of your existing data and, if necessary, your config files.

Update jar files

The Java libraries have to be replaced with the new versions.

Update config files

Compare the config files from the new version to your existing one. Check the changelog for new features that might require updates to config files.

Restart the servers

Restart the workflow server, and check the logs for any suspicious error messages!