Blog
End-to-End tests with Venom

End-to-End tests with Venom

December 17, 2021

A tech-inspired green snake

What is Venom?

Venom is an Open-Source tool for writing automated end-to-end and integration tests. It is published by OVH, a French cloud company. It is built on top of a popular Go framework called Cobra (hence the name). It aims to be easy and simple to use. It doesn’t require you to write your testCases in Python/Java/JavaScript like many other e2e testing tools. You declare your test cases using a simple .yaml file that resembles CI job declarations. It is written in go and can be easily extendable.

Writing testCases

Writing tests in Venom is simple. It uses the concept of executors, which are the underlying code that gets executed for a given testCase. In the example below, the executor is specified with the type keyword.

name: Title of TestSuite
testcases:
- name: example of a GET http testcase
 steps:
 - type: http # This test is using the http executor.
   method: GET
   url: https://some-url.com/products
   assertions:
   - result.statuscode ShouldEqual 200
   - result.timeseconds ShouldBeLessThan 1
   - result.body ShouldContainSubstring Owner

Variables, arguments & helpers

Variables can be defined on the testsuite level. To use the same example as above:

name: Title of TestSuite
vars:
 baseUrl: 'https://some-url.com'
testcases:
- name: example of a GET http testcase
 steps:
 - type: http # This test is using the http executor.
   method: GET
   url: '{{.baseUrl}}/products' # using the testsuit-level environment variable
   assertions:
   - result.statuscode ShouldEqual 200
   - result.timeseconds ShouldBeLessThan 1
   - result.body ShouldContainSubstring Owner

They can also be specified in the command line, using the --var flag, like such:

venom run --var="baseUrl=https://some-url.com"

Or by defining them in a file, and using the flag --var-from-file:

venom run --var-from-file="stagingEnv.yaml"

You can also use normal environment variables, as long as you prefix them with VENOM_VAR_. As a fallback strategy, if venom can’t find a given environment variable, it will try to find one in the environment variables. For example:

VENOM_VAR_baseUrl = https://some-url.com

Venom also comes with several environment variable helpers, such as upper, lower, camelcase, toJSON, and so on. A full list can be found in the project’s documentation. To use it, you need to pipe the value to the helper. For example:

{{.myvar | upper}} # will transform the value of `myvar` to uppercase
{{.myvar | camelcase }} # here something like 'new value' becomes 'newValue'

Venom also contains a few builtin variables where you can get things like the name of the file ({{.venom.testsuite.filename}})

It also allows you to extract the output of one test into a variable, so you can use in the subsequent testcases.

Test executors

The http executor in the example above, as the name says, makes HTTP requests.

Venom comes with support for many types of executors out of the box (and I would expect the list to grow over time):

Executor description
amqp Publish/subscribe to AMQP 1.0 compatible broker (QPID, ActiveMQ, etc)
dbfixtures Load fixtures into MySQL and PostgreSQL databases. Uses testfixtures
exec This is the default executor, executes a script
grpc Executes a GRPC Request
http Executes a HTTP Request
imap Used to test if mail is received from your application
kafka Used to use read/write on a Kafka topic
mqtt Used to read and write MQTT topics
rabbitmq Used to publish/subscribe on a RabbitMQ
readfile Executor that can read a file. This can be useful when testing files generated by your software
redis Execute commands into Redis
smtp Used for sending SMTP emails
sql Execute SQL queries into databases (MySQL, PostgresQL, and Oracle)
ssh Execute a script on a remote server via SSH
web This can be used for browser testing, it navigates to a web page and you can assert certain behaviors

Under the hood, executors implement an Executor interface:

// Executor execute a testStep.
type Executor interface {
  // Run run a Test Step
  Run(ctx context.Content, TestStep) (interface{}, error)
}

A generic example of a custom executor looks like this:

// Name of executor
const Name = "myexecutor"

// New returns a new Executor
func New() venom.Executor {
    return &Executor{}
}

// Executor struct
type Executor struct {
    Command string `json:"command,omitempty" yaml:"command,omitempty"`
}

// Result represents a step result
type Result struct {
    Code        int    `json:"code,omitempty" yaml:"code,omitempty"`
    Command     string `json:"command,omitempty" yaml:"command,omitempty"`
    Systemout   string   `json:"systemout,omitempty" yaml:"systemout,omitempty"` // put in testcase.Systemout by venom if present
    Systemerr   string   `json:"systemerr,omitempty" yaml:"systemerr,omitempty"` // put in testcase.Systemerr by venom if present
}


// GetDefaultAssertions return default assertions for this executor
// Optional
func (Executor) GetDefaultAssertions() *venom.StepAssertions {
    return &venom.StepAssertions{Assertions: []venom.Assertion{"result.code ShouldEqual 0"}}
}

// Run execute TestStep
func (Executor)    Run(ctx context.Context, step venom.TestStep) (interface{}, error) {
    // transform step to Executor Instance
    var e Executor
    if err := mapstructure.Decode(step, &e); err != nil {
        return nil, err
    }

    systemout := "foo"
    ouputCode := 0

    // prepare result
    r := Result{
        Code:    ouputCode, // return Output Code
        Command: e.Command, // return Command executed
        Systemout: systemout, // return Output string
    }

    return r, nil
}

But the beauty of Venom is that you don’t need to know any of this unless you are writing your own executor. If you find yourself repeating the same testCase logic, you can also create user-defined executors:

executor: hello
input:
  myarg: {}
steps:
- script: echo "{\"hello\":\"{{.input.myarg}}\"}"
  assertions:
  - result.code ShouldEqual 0
output:
  display:
    hello: "{{.result.systemoutjson.hello}}"
  all: "{{.result.systemoutjson}}"

Now, by defining a re-usable hello executor (that is using the default exec executor behind the scenes), you can re-use it in other testCases:

name: testsuite with the `hello` executor
testcases:
- name: example test
  steps:
  - type: hello # this is the custom executor that we've declared above.
    myarg: World
    assertions:
    - result.display.hello ShouldContainSubstring World
    - result.alljson.hello ShouldContainSubstring World

Configuring with .venomrc

Finally, instead of providing arguments from the command line, you can create a .venomrc file in your project and set the configuration there. Venom will read those values before running.

variables:
  - foo=bar
variables_files:
  - my_var_file.yaml
stop_on_failure: true
format: xml
output_dir: output
lib_dir: lib
verbosity: 3
An introduction to Protocol Buffers 3How I organize (most of) my Go microservices