Lua Test

Overview

The lua_test binary program allows users to execute one or more Lua-defined test scripts against a running N2SVCD RestServerApp application (supported by a LogicApp application). This program uses the StartRun REST API endpoint to begin executing each test run, and then uses the QueryRun REST API endpoint to periodically check each test run until it completes.

This program may be used to either:

1. A single test run, which executes a single instance.
2. A single test run, which executes multiple instances (specify --total greater than one).
3. Multiple test runs, each of which run in sequence (specify more than one .lua file).
4. Multiple test runs, running in parallel (specify more than one .lua file, and specify --parallel greater than one).

The default installation location for this binary is /usr/share/n2svcd/bin/lua_test.

Running lua_test --help will show the program usage.

usage: lua_test <lua-file> [... <lua-file>]
        --host <rest-svc-host>          Default = 'localhost'
        --port <rest-svc-port>          Default = 9010
        --quiet                         (suppress progress noise)
        --minimal                       (suppress even more progress noise)
        --timestamp                     (show timestamp on test pass/fail)
        --space                         (output a newline between tests)
        --failure-only                  (when running multiple test scripts only print failed tests in summary)
        --summary-totals                (when running multiple test scripts print totals in summary)
        --parallel <run-count>          The maximum number of test runs to start in parallel
        --google-test-output <out-file> Output a Google test formatted XML file that can be used by invoking applications to display run statistics.
        --retry-amount <amount>         Amount of times to try re-running a test. If a test fails it will be scheduled for a retry at the end
                                        of the test run. If successful it will be marked as such in the exit code and test output.
                                        Any failures will be reported as a non destructive failure. Defaults to 0 unless --google-test-output is defined then it is 3.

        --trace_level <0|1|2|3>         (request n2svcd to return debug traces for the first/only call in the run)
        --trace <0|1|2|3>               (request n2svcd to return debug traces for the first/only call in the run)
        --debug                         (--trace_level = 1)
        --dump                          (--trace_level = 2)
        --spam                          (--trace_level = 3)

        --total <total>                 (requires --cps and/or --backlog)
        --ramp <ramp-seconds>           (requires --cps and/or --backlog)
        --cps <cps>
        --backlog <backlog>

        --initial-args <JSON>           A single JSON object structure to be passed to the Lua script as the "..." vararg.
        --initial-args-file <filename>  Filename containing JSON object structure to be passed to the Lua script as the "..." vararg.

        --global-vars <JSON>            A single JSON object structure of global variable keys to set in the Lua script context.
        --global-vars-file <filename>   Filename containing JSON object structure of global variable keys to set in the Lua script context.

        --library-directory <root-dir>  Root directory containing library '.lua' files to upload, with require location derived from path/file.
                                        E.g. within this directory path/to/utils.lua is reachable as: local utils = require "path.to.utils"

Usage

lua_test can be used to run a single Lua test definition, in which case a single test summary will be output.

./lua_test ../test/regression/set2-INAP/tester_app/gb001_a_party_hangup.lua

[1/1][START ../test/regression/set2-INAP/tester_app/gb001_a_party_hangup.lua]
[STARTED tr_172115_7098_1 ...]
[COMPLETED 1(1) of 1 instances in 0.090 seconds.  Aborted = 0, Failed = 0, Cancelled = 0, Rate = 11.152/sec]
>>> FIRST CALL CHECKS
... [06] [PASS] Received INAP opcode FurnishChargingInformation [34] as expected.
... [06] [PASS] String 'FCI.[0].tag_hex' is present and has expected value '001c'.
... [06] [PASS] String 'FCI.[0].value_hex' is present and has expected value '0100ffffffffffffffffffffffffffffffffffff'.
... [06] [PASS] String 'FCI.[1].tag_hex' is present and has expected value '002c'.
... [06] [PASS] String 'FCI.[1].value_hex' is present and has expected value '0200000000000000001000000000000000000000'.
... [06] [PASS] String 'FCI.[2].tag_hex' is present and has expected value '004c'.
... [06] [PASS] String 'FCI.[2].value_hex' is present and has expected value '0000000000000017228700000049300000005042'.
... [06] [PASS] String 'FCI.[3].tag_hex' is present and has expected value '999c'.
... [06] [PASS] String 'FCI.[3].value_hex' is present and has expected value '0000000000000000000000000000000000000000'.
... [09] [PASS] Received INAP opcode ApplyCharging [35] as expected.
... [09] [PASS] Integer 'R1.maxConversationTime' is present and has expected value 120.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.warningTimeBeforeExpiry' is present and has expected value 110.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.messageID.elementaryMessageID' is present and has expected value 500.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.numberOfRepetitions' is present and has expected value 1.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.duration' is present and has expected value 0.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.interval' is present and has expected value 0.
... [12] [PASS] Received INAP opcode RequestReportBCSMEvent [23] as expected.
... [12] [PASS] Integer 'BCSM(5,2)monitorMode' is present and has expected value 0.
... [12] [PASS] Integer 'BCSM(6,2)monitorMode' is present and has expected value 0.
... [12] [PASS] Element 'dPSpecificCriteria' exists as expected (and is defined).
... [12] [PASS] Integer 'BCSM(6,2)dPSpecificCriteria.applicationTimer' is present and has expected value 300.
... [12] [PASS] Integer 'BCSM(10,1)monitorMode' is present and has expected value 1.
... [12] [PASS] Integer 'BCSM(4)monitorMode' is present and has expected value 0.
... [12] [PASS] Integer 'BCSM(9,2)monitorMode' is present and has expected value 0.
... [12] [PASS] Integer 'BCSM(9,1)monitorMode' is present and has expected value 1.
... [15] [PASS] Received INAP opcode Connect [20] as expected.
... [15] [PASS] String 'destinationRoutingAddress_digits' is present and has expected value '049340221'.
... [15] [PASS] Integer 'destinationRoutingAddress_noa' is present and has expected value 3.
... [16] [PASS] This is a static pass string.
... [17] [PASS] Correctly received DRA digits 049340221.
>>> First Instance all 30 tests returned PASS.

Multiple Runs

Each Lua filename specified on the command line is a single Test Run.

Each Test Run is initiated by sending a StartRun REST request to the RestServerApp.

By default, each Test Run is run in sequence. A new Test Run is started only after the previous Test Run has been reported as completed by a CheckRun REST request. However, the --parallel command-line option will instruct lua_test to run more than one Test Run in parallel.

When executing more than one Test Run, an additional print-out at the end of all test runs will be displayed, giving a summary of the test run results.

./lua_test ../test/regression/set2-INAP/tester_app/*.lua
...

SUMMARY OF 18 RUNS.
  IDX 1st-PASS/FAIL : #START #FINISH #ABORT #FAILED #CANCEL : Lua Filename
    1       30    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb001_a_party_hangup.lua
    2       31    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb002_b_party_hangup.lua
    3       33    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb003_timer_expires.lua
    4       40    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb004_post_call_interaction.lua
    5       28    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb100_simple_charged.lua
    6        5    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap001_idp_ext_rc_logic.lua
    7       16    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap002_ctr_pa_pacui.lua
    8       12    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap003_sinap_fci.lua
    9       11    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap004_etc_pa_pacui.lua
   10       18    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap004_etc_pa_pacui_double.lua
   11        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap005_etc_pa_pacui_error.lua
   12        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap006_etc_pa_pacui_abort.lua
   13       14    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap007_connect_bcsm_cir.lua
   14        6    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap100_simple_connect.lua
   15        4    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap101_vars_connect.lua
   16        4    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap102_idp_abort.lua
   17        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap200_intester_camel.lua
   18        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap400_intester_camel4.lua

Common Options

The following common options may be specified when executing lua_test.

Load Testing Options

These options specify that each single Test Run will execute the exact same test script multiple times.

Initial Variables

The lua_test may pass initial variables into the script execution.

These are passed as the args (first parameter) into the Lua chunk code, and may be accessed via the ... variable.

For example:

local args = ...

Global Variables

The lua_test may set global variables which will be visible to the executing Lua script.