Rule Testing

A test is a small collection of several key pieces of information - the input data, expected output data and the strategy of execution. Such a test can be executed and the system then compares the results of the current rule to the expected results of the tests and give you either just a success or a comparison of differences between the real output and the expected.

Test suites are just organizational units for tests. You can think of them as folder in Decisionrules, they mostly allow for better understanding of the test , separate test into logical blocks and make it easier to batch manage tests if necessary. They also make selecting a group of tests to run much easier.

When you run a set of test a test run is created, when the execution finished you will find all the results of your tests in this run. You can execute only one test, multiple or all of them, in either case the test run will be where you find your results.

There are several menus for tests each of them with a specific purpose. In any rule ( except Integration flows and Lookup tables ) you will find the Tests tab right next to designer - this is a place to manage and execute tests for your rule. In the left panel of the rule you will find a list of your executed test runs and you can visualize the results right in the rule for debugging. Lastly in the Space - tests menu you can manage and run tests for the entire space at once.

For creating tests you have several options, the easiest one is available right in the test bench where you can save your execution as test without having to paste everything manually, you can create the test from scratch in the tests panel or you can use the API.

Regular users of Decisionrules are used to the Test bench, which allows them to fill in data and visualize the execution of the rule right in the UI. For any such execution you are provided an option to “Save as Test” and after selecting a test suite in which the test is created and a name you will have created your first test without having to paste in everything by hand. This way you can also be sure that the test fits the current rule perfectly.

When the UI is not enough for the creation of the test ( maybe you have too large of a number of test to input them manually ) you can take advantage of our test API. This API will allow you to Create, update, delete and export tests as you would expect and is a part of the Management API sharing the same API keys.

For running tests you have a few options depending on what you want to achieve. If you want to tests a particular rule, for example after making some changes, you can go into the Tests panel and either select the tests you want to run and click “Run selected”, or just click “Run all”.

If you want to run tests across multiple different rules, you can do this via the Space menu - Tests. This menu works the same way as the rule level panel does, but allows you to select tests across all the rules in the space or just run all available tests in the space.

Any test execution creates a Test Run, these can be either seen in the space level menu ( here you will see all runs across the space ) or in the left panel in the rule designer which will have all the Test Runs which contained any test from the particular rule.

When you open up the test run and select a particular test that failed you will see a comparison of the expected results ( what was defined in the test ) and the actual output which the current rule gave.

When looking at the test results ( whether in the space menu or in the rule ) you have the option to inspect. If you select this , you are taken into the rule and the data of the test in loaded into the test bench. At the same time you will see the visualization of the executed logic as if you ran the rule with Debug option.

What is recommended is that after the tests are ran and some fail, the user should visualize the results to be able to determine why the logic doesn’t work the way it is expected. After they find the mistake, they can fix it and rerun the test just using the test bench ( this does not create the test run, but the test bench will contain a badge notifying the user about whether the output conforms to the preexisting test ). After making changes it is recommended to rerun all tests again to make sure that some fixes did not accidentally affect other tests.

The test suites are mostly for organization, they also make it easier to select a group of tests with just one click. The number of the test suites that are created is up to the user, so whatever makes your life easier is the best.

Running selected tests is mainly recommended when either fixing errors or designing the rules, it is a useful tool to quickly see that the changes you are making actually fix the errors you are trying to fix.

After you are content with the rules, it is recommended to Run all tests to make sure that everyting works as expected. Running all test is also useful when preparing for migrations, you can run all the tests in the space to make sure that the rules are production ready.

It can happen, that your rule return in its output some dynamic field that will always cause the test to fail ( imagine returning the current date, such a test would only work for 24 hours). Because of this each test has the option to select Ignored outputs, where you can select output fields that will not be counted towards the failure of the test and will be omitted from the output comparison.