A Viash component can be translated into one or more engines: Native, Docker; and one or more runners: Executable, Nextflow. Each of these engines and runners result in a different artifact:
Native engine: a single executable is generated which runs natively on the host system. This assumes all dependencies are already installed by the user and is therefore not reproducible. Requirements: Bash.
Docker engine: a single executable is generated but it runs inside a Docker container instead. The Docker engine specification can help you install custom dependencies and will take care of auto-mounting input/output files. Requirements: Bash, Docker.
Executable runner: a bash script is generated and can be run directly. The executable runner supports both the native and docker engine.
Nextflow runner: a Nextflow module which can be used as a standalone workflow or as a module in a larger workflow. Requirements: Nextflow and a containerization engine (e.g. Docker, Singularity, Podman).
Below we will create our first Viash component using any of the languages natively supported by Viash.
Create a script
When creating a new Viash component, you can write a new script or use a pre-existing script. Below is a script that simply copies an input file to an output destination.
The par variable(s) appear to be hard coded, but they’re not! When running this script with Viash, Viash will strip away the section between VIASH START and VIASH END, and replace it with parameter values at runtime. The values included in this script are thus entirely for development and debugging purposes. More information on how this works will be given in Variables and meta-variables.
Create a config
A Viash config file is a YAML file that describes the functionality of a component as well as the engine(s) and runner(s) it targets.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_bash
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_bash:latest'
[warning] Could not pull from 'example_bash:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_bash:latest' with Dockerfile
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/bash/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/bash/foo.txt'.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_csharp
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_csharp:latest'
[warning] Could not pull from 'example_csharp:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_csharp:latest' with Dockerfile
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/csharp/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/csharp/foo.txt'.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_js
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_js:latest'
[warning] Could not pull from 'example_js:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_js:latest' with Dockerfile
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/js/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/js/foo.txt'
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_python
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_python:latest'
[warning] Could not pull from 'example_python:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_python:latest' with Dockerfile
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/python/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/python/foo.txt'.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_r
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_r:latest'
[warning] Could not pull from 'example_r:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_r:latest' with Dockerfile
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/r/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/r/foo.txt'.
[1] TRUE
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_scala
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_scala:latest'
[warning] Could not pull from 'example_scala:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_scala:latest' with Dockerfile
warning: 1 deprecation; re-run with -deprecation for details
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/scala/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/scala/foo.txt'.
Note
The double dash (--) between the viash command and the arguments is used to signify the end of the arguments passed to Viash and the start of those passed to the script. If you forgot to add these, you’ll get an error similar to this:
viash run config.vsh.yaml \--input foo.txt \--output bar.txt
[scallop] Error: Unknown option 'input'
Build an executable
We will now turn the Viash component into an executable.
Use the viash build command to generate an executable.
To display its documentation, run the executable with just the --help argument:
target/example_bash--help
example_bash
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
To display its documentation, run the executable with just the --help argument:
target/example_csharp--help
example_csharp
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
To display its documentation, run the executable with just the --help argument:
target/example_js--help
example_js
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
To display its documentation, run the executable with just the --help argument:
target/example_python--help
example_python
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
To display its documentation, run the executable with just the --help argument:
target/example_r--help
example_r
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
To display its documentation, run the executable with just the --help argument:
target/example_scala--help
example_scala
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
Viash built in Computational Requirements:
---cpus=INT
Number of CPUs to use
---memory=STRING
Amount of memory to use. Examples: 4GB, 3MiB.
Viash built in Docker:
---setup=STRATEGY
Setup the docker container. Options are: alwaysbuild, alwayscachedbuild, ifneedbebuild, ifneedbecachedbuild, alwayspull, alwayspullelsebuild, alwayspullelsecachedbuild, ifneedbepull, ifneedbepullelsebuild, ifneedbepullelsecachedbuild, push, pushifnotpresent, donothing.
Default: ifneedbepullelsecachedbuild
---dockerfile
Print the dockerfile to stdout.
---docker_run_args=ARG
Provide runtime arguments to Docker. See the documentation on `docker run` for more information.
---docker_image_id
Print the docker image id to stdout.
---debug
Enter the docker container for debugging purposes.
Viash built in Engines:
---engine=ENGINE_ID
Specify the engine to use. Options are: docker, native.
Default: docker
This executable takes a file as input and will create an output file.
Running the executable
Running an executable is the same as any other executable on your system.
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/bash/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/bash/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 411 Jan 16 10:39 config.vsh.yaml
-rw-r--r-- 1 runner docker 411 Jan 16 10:40 foo.txt
-rw-r--r-- 1 runner docker 411 Jan 16 10:41 output.txt
-rwxr-xr-x 1 runner docker 181 Jan 16 10:39 script.sh
drwxr-xr-x 2 runner docker 4096 Jan 16 10:41 target
You can run the executable by providing a value for --input and --output:
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/csharp/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/csharp/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 450 Jan 16 10:39 config.vsh.yaml
-rw-r--r-- 1 runner docker 450 Jan 16 10:39 foo.txt
-rw-r--r-- 1 runner docker 450 Jan 16 10:39 output.txt
-rw-r--r-- 1 runner docker 237 Jan 16 10:39 script.csx
drwxr-xr-x 2 runner docker 4096 Jan 16 10:41 target
You can run the executable by providing a value for --input and --output:
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/js/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/js/output.txt'
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 428 Jan 16 10:39 config.vsh.yaml
-rw-r--r-- 1 runner docker 428 Jan 16 10:40 foo.txt
-rw-r--r-- 1 runner docker 428 Jan 16 10:41 output.txt
-rwxr-xr-x 1 runner docker 282 Jan 16 10:39 script.js
drwxr-xr-x 2 runner docker 4096 Jan 16 10:41 target
You can run the executable by providing a value for --input and --output:
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/python/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/python/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 423 Jan 16 10:39 config.vsh.yaml
-rw-r--r-- 1 runner docker 423 Jan 16 10:40 foo.txt
-rw-r--r-- 1 runner docker 423 Jan 16 10:41 output.txt
-rwxr-xr-x 1 runner docker 216 Jan 16 10:39 script.py
drwxr-xr-x 2 runner docker 4096 Jan 16 10:41 target
You can run the executable by providing a value for --input and --output:
warning: 1 deprecation; re-run with -deprecation for details
Copying '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/scala/config.vsh.yaml' to '/viash_automount/tmp/RtmpkivNMw/create_new_component58fb722e0655/scala/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 462 Jan 16 10:39 config.vsh.yaml
-rw-r--r-- 1 runner docker 462 Jan 16 10:41 foo.txt
-rw-r--r-- 1 runner docker 462 Jan 16 10:41 output.txt
-rw-r--r-- 1 runner docker 435 Jan 16 10:39 script.scala
drwxr-xr-x 2 runner docker 4096 Jan 16 10:41 target