| tdc - Adding test cases for tdc |
| |
| Author: Lucas Bates - lucasb@mojatatu.com |
| |
| ADDING TEST CASES |
| ----------------- |
| |
| User-defined tests should be added by defining a separate JSON file. This |
| will help prevent conflicts when updating the repository. Refer to |
| template.json for the required JSON format for test cases. |
| |
| Include the 'id' field, but do not assign a value. Running tdc with the -i |
| option will generate a unique ID for that test case. |
| |
| tdc will recursively search the 'tc' subdirectory for .json files. Any |
| test case files you create in these directories will automatically be included. |
| If you wish to store your custom test cases elsewhere, be sure to run tdc |
| with the -f argument and the path to your file. |
| |
| Be aware of required escape characters in the JSON data - particularly when |
| defining the match pattern. Refer to the tctests.json file for examples when |
| in doubt. |
| |
| |
| TEST CASE STRUCTURE |
| ------------------- |
| |
| Each test case has required data: |
| |
| id: A unique alphanumeric value to identify a particular test case |
| name: Descriptive name that explains the command under test |
| category: A list of single-word descriptions covering what the command |
| under test is testing. Example: filter, actions, u32, gact, etc. |
| setup: The list of commands required to ensure the command under test |
| succeeds. For example: if testing a filter, the command to create |
| the qdisc would appear here. |
| cmdUnderTest: The tc command being tested itself. |
| expExitCode: The code returned by the command under test upon its termination. |
| tdc will compare this value against the actual returned value. |
| verifyCmd: The tc command to be run to verify successful execution. |
| For example: if the command under test creates a gact action, |
| verifyCmd should be "$TC actions show action gact" |
| matchPattern: A regular expression to be applied against the output of the |
| verifyCmd to prove the command under test succeeded. This pattern |
| should be as specific as possible so that a false positive is not |
| matched. |
| matchCount: How many times the regex in matchPattern should match. A value |
| of 0 is acceptable. |
| teardown: The list of commands to clean up after the test is completed. |
| The environment should be returned to the same state as when |
| this test was started: qdiscs deleted, actions flushed, etc. |
| |
| |
| SETUP/TEARDOWN ERRORS |
| --------------------- |
| |
| If an error is detected during the setup/teardown process, execution of the |
| tests will immediately stop with an error message and the namespace in which |
| the tests are run will be destroyed. This is to prevent inaccurate results |
| in the test cases. |
| |
| Repeated failures of the setup/teardown may indicate a problem with the test |
| case, or possibly even a bug in one of the commands that are not being tested. |
| |
| It's possible to include acceptable exit codes with the setup/teardown command |
| so that it doesn't halt the script for an error that doesn't matter. Turn the |
| individual command into a list, with the command being first, followed by all |
| acceptable exit codes for the command. |
| |
