CrystalFlow is a hypothetical workflow with low-medium complexity that adequately illustrates the benefits and characteristics of scientific computational workflows.
📝 Note: The code, executables, and test data for the crystal workflow is available on ORNL's public GitLab.
About the Crystal Workflow
The workflow is a crystal shaped graph as shown in the figure below.
In the above workflow, each of the boxes represent process and arrows represent the dependency between connected processes. For example, process P1 produces a data file that is consumed by processes P2 and P3.
Each of the 6 processes of this workflow are implemented in C and bash. Either may be used in the workflows shown below. The initial input file is pre-prepared. The code and data for these workflows are available on open HPC condo at /software/T/B/D. A Makefile will build the C executables found in the directory named c. The following sections show how this workflow may be orchestrated using each of the three workflow management tools.
Nextflow
The following code snippet shows how the above workflow would be expressed in Nextflow.
Assuming the above workflow is saved in a file named crystal.nf, it could be run as follows:
$modulepurge$moduleloadPE-gnu$moduleloadjava/1.8.0_131$moduleloadnextflow$$nextflowruncrystal.nfNEXTFLOW~version0.27.6Launching`crystal.nf` [thirsty_allen] -revision:e3b42d107d[warm up] executor >local[db/d513da] Submitted process > p1[89/e16494] Submitted process > p2[c3/9d4ddd] Submitted process > p3[0d/5406b9] Submitted process > p4[cf/4b94bb] Submitted process > p5[c2/3bae00] Submitted process > p6
Makeflow
The following code snippet shows how the crystal workflow would be implemented using Makeflow.
The following code snippet shows the Swift implementation of the crystal workflow. Note that the Swift implementation invokes the C version of executables but it can equally invoke the bash version.
