Expression Reference

Literals

Literal values can be used to insert data directly into an expression:

• Strings: "How are you?", 'How are you?'
• Numbers: 40, 30.123
• Arrays: [1, 2, 3]
• Objects: { one: 1, two: 2 }
• Booleans: true, false

Math

Expressions allow you to operate on values. The following operators are available:

• Addition: +
• Subtraction: -
• Division: /
• Division and integer truncation: //
• Division remainder: %
• Multiplication: *
• Power: **

You can use them like this:

2 + 3
=> 5

10 / 5
=> 2

3 * 2
=> 6

Tests and Comparisons

Comparison operators may be used to return a Boolean based on the comparison result of two values:

• Equal (value only): ==
• Strict Equal (value and type): ===
• Not Equal (value only): !=
• Strict Not Equal (value and type): !==
• Greater Than: >
• Greater Than Or Equal: >=
• Less Than: <
• Less Than Or Equal: <=

Expressions also support an “if-else” syntax similar to Python. It can be used to choose from two values:

'Yes' if (variables.Foo == true) else 'No'

This is also useful for selecting default values when a variable is optional and may not be defined:

variables.Foo if variables.Foo else 'Foo'

Built-in Environment Variables

To access the built-in environment variables that are passed to step modules that run as subprocesses, use the following expression:

env.<var name>

The following values are available:

• AGENT_VERSION
• WORKSPACE_PATH
• RUN_PATH
• PROJECT_ID
• RUN_ID
• RUN_NUMBER
• PIPELINE_ID
• PIPELINE_REVISION
• PIPELINE_REVISION_ID
• PIPELINE_NAME
• JOB_ID (only available if the run was a job)

Note that operating system environment variables are not propagated to the values available in the env.* expressions. However, these values are merged into the environment for subprocesses executed by pipelines, such as shell scripts and CLI-drive step modules.

Pipeline Helper Functions

The following top-level helper functions are available when running a pipeline. They’re used in pipeline step properties to help connect steps together.

find_parent_step_by_type(step, type)

Searches a step’s parent dependency chain for the first step with the given type. If multiple parent steps are found at the same distance, an arbitrary parent will be chosen.

Parameters
step	The step where the search starts. This should be a full step object, which can be retrieved using steps.<step id>, or simply provide step to search from the current step if the expression is evaluating in a step property.
type	The type of step to return.

Returns

A step object, or null if there is no parent step with the given type. See the step module reference for examples.

find_step_by_id(id)

Finds a step in the pipeline by id. Note that you can also access a step by steps.<id>

Parameters
id	The id of the step to return.

Returns

A step object, or null if there is no step with the given id. See the step module reference for examples.

find_steps_by_tag(tag)

Finds all steps in the pipeline with the given tag.

Parameters
tag	The tag on the steps to find.

Returns

An array of step objects. See the step module reference for examples.

find_steps_by_type(type)

Finds all steps in the pipeline with the given type.

Parameters
type	The step type, such as “file”.

Returns

An array of step objects. See the step module reference for examples.

Other Helper Functions

The following top-level helper functions are available in any expression.

credential(id)

Loads a credential’s data. This helper is not needed for step modules with credential support, however it is often useful for manipulating credential data directly in a pipeline step.

Parameters
id	The ID of the credential to retrieve.

Returns

A credential data object. See the credential reference for more information about credential types.

Examples

To retrieve the password from a Username/Password credential with ID myCred:

credential("myCred").password

random_string(generator, length)

Generates a random string using one of several built-in generator functions.

Parameters
generator	The type of string to generate. Allowed values: windowsPassword, alphaNumeric, alphaNumericLower, alphaNumericUpper, alpha, alphaLower, alphaUpper, numeric.
length	The number of characters in the generated string. Defaults to 8.

Returns

A randomly generated string.

Examples

To create a random password guaranteed to pass Windows validation (this also works for most Linux systems):

random_string('windowsPassword')
=> "kQi3n49S"

To create a sequence of 10 number characters:

random_string('numeric', 10)
=> "9234576823"

read_file(path, encoding)

Reads a local file on the runner. Only supports text files.

Parameters
path	Local path to an existing file.
encoding	Text encoding of the file. Defaults to “utf8”.

Returns

The content of the file as a string.

Examples

To read a file called “example.txt” from the current run directory:

read_file("example.txt")

To read and parse a JSON file:

read_file("package.json") | parse_json
(read_file("package.json") | parse_json).version

Filters

Filters are similar to helper functions, except they are “piped” an input value using the | character. For example,

[1, 2, 3] | join(',')
=> "1,2,3"

This allows chaining of filters to perform complex operations on data:

findStepsByTag('vm') | slice(2) | json_query('[*].properties.name') | join(',')
=> "vm1,vm2"

All filters from Nunjucks are available, as well as some additional filters that are listed below.

Filters used commonly in Sophos Factory are documented in more detail here. For a complete list of built-in filters, see the Nunjucks built-in filters.

append(val)

Appends val to the filtered value which must be an array and returns the updated array.

Parameters
val	The value to append to the array. Can be any valid JSON type.

Returns

The updated value of the array.

Examples

Example 1.

[] | append('foo')

=> [ 'foo' ]

Example 2.

[] | append('foo') | append('bar') | append('baz')

=> [ 'foo', 'bar', 'baz' ]

base64decode(str)

Parameters
str	The base64-encoded string to decode.

Returns

The UTF-8 decoded result.

Examples

'dXNlcm5hbWU6cGFzc3dvcmQ=' | base64decode

=> username:password

base64encode(str)

Parameters
str	The UTF-8 string to encode.

Returns The base64-encoded result.

Examples

'username:password' | base64encode

=> dXNlcm5hbWU6cGFzc3dvcmQ=

capitalize()

Make the first letter uppercase, and the rest lower case.

Returns

The capitalized string.

date_format(formatter, timezone)

Creates a formatted string representing the current date/time. This helper utilizes the date-fns library to perform the formatting, and also supports zzz timezone tokens.

Parameters
formatter	A formatter string. See here for full formatter documentation.
timezone	An IANA timezone identifier, such as “America/Los_Angeles”.

Returns

The current date/time as a formatted string.

Examples

To produce a calendar date in the format [month]/[day]/[year]:

date_format('MM/dd/yyyy')
=> "11/27/2021"

To produce a date string in ISO 8601 format with a specific timezone:

date_format('yyyy-MM-dd\'T\'HH:mm:ssXXX', 'Pacific/Honolulu')
=> "2021-04-15T11:53:41-10:00"

default(value)

Provides a default value when the filtered value is undefined.

Parameters
value	The default value to use.

Returns

The default value if the filtered value is undefined, otherwise the filtered value.

first()

Returns the first value of a list.

Returns

The first item.

hmac(type, data, secret)

Computes an HMAC signature for a given string and secret string.

Parameters
type	The hashing algorithm to use. Possible values are md5, sha1, sha256, sha224, sha512, sha384, sha3, and ripemd160.
data	The secret data string.
secret	The secret string.

Returns

An HMAC signature string.

join(delimiter)

Return a string which is the concatenation of the strings in a sequence.

Parameters
delimiter	The character to insert between items. Optional.

Returns

The joined string.

Examples

To convert an array of FQDN segments into a single FQDN string:

['app1', 'mydomain', 'com'] | join('.')
=> "app1.mydomain.com"

json_query(query)

The json_query filter lets you query a complex data structure. It can be used to select sub-values from an object or list of objects, like those returned by the find*() helper functions.

json_query utilizes JMESPath as the query language. Visit the JMESPath Examples page to learn more about JMESPath queries and to try out queries using their embedded tester.

Parameters
query	The query string.

Returns

The query result.

Examples

To retrieve a list of names from all steps tagged with the rdsh tag:

findStepsByTag('rdsh') | json_query('[*].properties.name')
=> ["Virtual Machine 1", "Virtual Machine 2"]

last()

Returns the last value of a list.

Returns

The last item.

merge(obj)

Merges two objects together. obj will be merged into the input object and will override keys with the same name. Performs a shallow merge.

Parameters
obj	The object to merge into the input object

Returns

A new object which is the result of merging the two input objects.

merge_deep(obj)

Merges two objects together. obj will be merged into the input object and will override keys with the same name. Performs a recursive merge. Note that array values will be replaced, rather than appended.

Parameters
obj	The object to merge into the input object

Returns

A new object which is the result of merging the two input objects.

parse_json()

Parses a JSON string. JSON data structures can be represented natively in Sophos Factory’s engine.

Returns

The result of parsing the JSON.

Examples

To parse a JSON string:

'{"foo": "bar"}' | parse_json
=> {"foo": "bar"}

To read and parse a JSON file:

read_file("package.json") | parse_json
(read_file("package.json") | parse_json).version

parse_xml()

Parses an XML string, such as that returned by an HTTP response, into a dictionary data structure.

The dictionary contains keys matching the tag names in the XML. Attributes are placed in a sub-dictionary under the special $ key. Inner node text is placed under the special _ key. Child nodes are placed recursively as arrays in their respective tag keys, each having the same structure.

parse_xml utilizes xml2js to perform the parsing.

Returns

An object containing the parsed XML.

Examples

{| vars.my_xml_string | parse_xml |}

Given an XML string:

<foo>
    <bar name="1">Bar 1</bar>
    <bar name="2">Bar 2</bar>
</foo>

parse_xml will produce:

{
    "foo": {
        "bar": [
        {
            "_": "Bar 1",
            "$": {
            "name": "1"
            }
        },
        {
            "_": "Bar 2",
            "$": {
            "name": "2"
            }
        }
        ]
    }
}

dump_xml()

Converts an object to an XML string.

dump_xml utilizes xml2js to perform the string conversion. See also parse_xml above for more information about special object keys.

Returns

An XML string.

parse_yaml()

Parses a YAML string. At minimum, JSON-compatible YAML features are supported.

Returns

An object containing the parsed YAML.

Examples

To read and parse a YAML file:

read_file("myDeployment.yml") | parse_yaml
(read_file("myDeployment.yml") | parse_yaml).kind

quote()

Escapes and wraps a string so that it can safely be provided as a single command line argument in a shell.

Returns

The escaped string, which can be provided as a shell command argument.

random()

Selects a random item from a list.

Returns

The randomly chosen item.

replace(search, value)

Replaces every occurence of search with value.

Parameters
search	The string to be replaced.
value	The replacement string.

Returns

The new string with values replaced.

Examples

Replace hyphens with underscores in a string:

'foo-bar-baz' | replace('-', '_')

=> "foo_bar_baz"

set_var(var[, val])

Sets a pipeline variable and returns the filtered value.

The set_var() filter requires one or two parameters in addition to the filtered value. The value assigned to the variable is different in each case. However, in both cases, the filtered value is reflected onto the filter pipeline.

Parameters
var	A string containing the name of the variable to be set. (Required) If the value of var is ‘foo’, the variable vars.foo will be set. If vars.foo does not
exist, it will be created.
val	The value to be assigned to the variable in place of the filtered value. (Optional)

Note: |

Returns

The filtered value

Examples

'ham' | set_var('spam')

=> ham
=> the value of `vars.spam` is set to 'ham'

'ham' | set_var('spam', 'yam')

=> ham
=> the value of `vars.spam` is set to 'yam'

split(delimiter)

Divides a string into an ordered list of substrings, puts these substrings into an array, and returns the array.

Parameters
delimiter	The character sequence to split on.

Returns

An array containing the substrings.

ternary(valueA, valueB)

This filter can be used instead of the if-else pattern. It is equivalent to the ternary operator in most programming languages.

Returns valueA if the filtered value is truthy, otherwise returns valueB.

Parameters
valueA	Any value.
valueB	Any value.

Returns

valueA or valueB depending on the filtered value.

Examples

The following expression evaluates to “Yes” if the value of variables.Foo is true:

(variables.Foo == true) | ternary('Yes', 'No')
=> "Yes"

unique()

Returns the unique values of the filtered value which must be an array.

Returns

The unique members of the array.

Examples

Example 1.

[] | unique

=> []

Example 2.

['a', 'b', 'a'] | unique

=> [ 'a', 'b' ]

Example 3.

'abacadae' | split('') | unique | join('')

=> 'abcde'

unset_var(var)

Unsets a pipeline variable and returns the filtered value.

Parameters
var	A string containing the name of the variable to be unset. (Required) If the value of var is ‘foo’, the variable vars.foo will be unset.

Note: |

Returns

The filtered value

Examples

'ham' | unset_var('spam')

=> ham
=> the value of `vars.spam` is undefined

set_var('spam')

=> undefined
=> the value of `vars.spam` is set to undefined

Test Expressions

Test expressions perform a comparison or check, and evaluate to true or false. They use the “is” syntax:

<expression to test> is <test function>

e.g., the following expression evaluates to true if the step with id step1 has the content property defined:

steps.step1.properties.content is defined

The following test functions are available:

deepequalto(value)

Returns true if the input is deeply equal to the value, and false otherwise. This test recursively compares the properties of objects and arrays by value.

defined()

Returns true if the input is defined, and false otherwise.

divisibleby(value)

Returns true if the input is divisible by the value, and false otherwise.

empty()

Returns true if the input is null, undefined, or an empty string, object, or array.

equalto(value)

Returns true if the input is strictly equal to the value, and false otherwise. Aliased as sameas(value) and eq(value).

even()

Returns true if the input is an even number, and false otherwise.

falsy()

Returns true if the input is falsy, which includes false, 0, null, undefined, and empty strings, and false otherwise.

greaterthan(value)

Returns true if the input is greater than the value, and false otherwise. Aliased as gt().

iterable()

Returns true if the input is iterable, which includes arrays and strings, and false otherwise.

lessthan(value)

Returns true if the input is less than the value, and false otherwise. Aliased as lt().

lower()

Returns true if the input is an all-lowercase string , and false otherwise.

ne(value)

Returns true if the input is not strictly equal to the value, and false otherwise.

nil()

Returns true if the input is null or undefined.

null()

Returns true if the input is null, and false otherwise.

number()

Returns true if the input is a number, and false otherwise.

odd()

Returns true if the input is an odd number, and false otherwise.

string()

Returns true if the input is a string, and false otherwise.

truthy()

Returns true if the input is truthy, which includes true, 1, and non-empty strings, and false otherwise.

undefined()

Returns true if the input is undefined, and false otherwise.

upper()

Returns true if the input is an all-uppercase string, and false otherwise.