Testing

Both unit and integration tests are written using pytest. To run the unit tests:

pytest tests/unit

Integration tests require running services and evaluate multiple components at the same time.

pytest tests/integration

Functional tests ensure that the system functions correctly end-to-end, with resources and load as close to production as possible.

pytest tests/functional

To test that multiple producers and multiple consumers are able to share workloads, a sample Procfile is provided spawning four processes like below:

pub1: wms --port 8080 --spec-file ./conf/sites-default.yaml scrape
pub2: wms --port 8081 --spec-file ./conf/sites-test.yaml    scrape
sub1: wms --port 8083 persist
sub2: wms --port 8084 persist

It can be executed with honcho start. Ideally this would be run with pytest during functional testing.

Quality Assurance

When the Terraform cluster has been deployed, one can simulate a real-world scenario with two producers and two consumers (the env should be named staging):

WMS_ENV=prod honcho start

Running integration tests on Cloud services tend to produce flaky results because of network latency in particular.

Coverage

Test coverage can be computing from the top-level directory, producing an HTML report

make cov