Exam BDD Framework >

Extension for Concordion BDD framework

Overview

Exam is oriented on declarative end-to-end gray-box application testing in a way a manual tester would do it: send request, verify response/database/message queue etc.

manual
Figure 1. Manual testing

Hence, the Exam functionality falls apart in different libraries (plugins) that are tailored for specific kinds of checks: database interactions, message queue interactions, http interactions, file system interactions etc. and may be used separately.

exam
Figure 2. Exam testing

Each library consist of a plugin class that should be configured and attached to ExamExtension and set of commands that can be used for instrumenting .adoc specification files.

Exam could be enabled by implementing io.github.adven27.concordion.extensions.exam.core.AbstractSpecs class:

class Specs : AbstractSpecs() {

    override fun init() = ExamExtension(
        WsPlugin(
            host = "localhost",
            port = 8080
        ),
        DbPlugin(
            driver = "org.postgresql.Driver",
            url = "jdbc:postgresql://localhost:5432/postgres",
            user = "postgres",
            password = "postgres"
        )
    )

    /*  Methods in order of execution: */

    override fun beforeSetUp() {
        // Optional: Run some actions BEFORE Exam set up
    }
    override fun beforeSutStart() {
        // Optional: Run some actions AFTER Exam set up and BEFORE start of a System Under Test (SUT)
    }
    override fun startSut() {
        // Start SUT before specs suite if needed
    }
    override fun stopSut() {
        // Stop SUT after specs suite if needed
    }
    override fun afterSutStop() {
        // Optional: Run some actions AFTER stop of a SUT and BEFORE Exam tear down
    }
    override fun afterTearDown() {
        // Optional: Run some actions AFTER Exam tear down
    }
}

Core

Core commands enable abstract interactions and verifying capabilities.

Commands

e-eq

Verifies an equality of instrumented block and instrumentation expression result.

Attributes
verifier

Optional. Default: text.

Content type verifier. Built-in values: text, json, xml.
await

Optional. Default: 4s,0s,1s.

Format: <at_most>,<poll_delay>,<poll_interval>

Where:

at_most

How long to wait before failing the verification
poll_delay

How long to wait before starting verification
poll_interval

How long to wait between verification attempts
Waiting is disabled by default. Check will fail immediately in case of mismatch unless the await attribute is set.
Usage
Given `greeting()` method returns `Hello World!`
expected result is: [{eq}greeting()]_Hello World!_

Given greeting() method returns Hello World! expected result is: Hello World!

JSON verifying
Given `someJson()` method returns `{"result": 1}`
expected result is: [{verifier}json {eq}someJson()]_{"result": "{any-num}" }_. +
Short version: [{eq-json}someJson()]_{"result": "{any-num}"}_. +
On listing:
[source,json,{eq-json}someJson()]
--
{"result": "{{number}}"}
--

Given someJson() method returns {"result": 1} expected result is: {"result": "${json-unit.any-number}" }.
Short version: {"result": "${json-unit.any-number}"}.
On listing:

{"result": "${json-unit.any-number}"}
XML verifying
Given `someXml()` method returns `<result>1</result>`
expected result is: [{verifier}xml {eq}someXml()]_<result>{any-num}</result>_. +
Short version: [{eq-xml}someXml()]_<result>{any-num}</result>_. +
On listing:
[source,xml,{eq-xml}someXml()]
--
<result>{{number}}</result>
--

Given someXml() method returns <result>1</result> expected result is: <result>${xml-unit.any-number}</result>.
Short version: <result>${xml-unit.any-number}</result>.
On listing:

<result>${xml-unit.any-number}</result>

e-set

Sets a specification variable which could be used in other commands expressions.

Usage
The greeting for user [{set}firstName]_Bob_ will be: [{eq}greetingFor(#firstName)]_Hello Bob!_

The greeting for user Bob will be: Hello Bob!

Variable can be initialized with text, object and file content.

Initialization with string from template:
When:: Set `someText` = [{set}someText]`some text`
and set `template` = [{set}template]`someText: {{someText}}, current timestamp: {{iso (at)}}`
Then:: `template` equals [{eq}#template]`someText: {{someText}}, current timestamp: {{iso (at)}}`
When

Set someText = some text and set template = someText: some text, current timestamp: 2024-08-24T16:42:28.883

Then

template equals someText: some text, current timestamp: 2024-08-24T16:42:28.883

Initialization with object:
When:: Set `mapObject` = [{set}mapObject]`{{map a='1' b='2'}}`
Then::
`mapObject.a` equals [{eq}#mapObject.a]`1` +
`mapObject.b` equals [{eq}#mapObject.b]`2`
When

Set mapObject = {a=1, b=2}

Then

mapObject.a equals 1
mapObject.b equals 2

Initialization with string from file template:
When:: Set `fileTemplate` = [{set}fileTemplate]`{{file '/data/core/file.txt' v='123'}}`
Then:: `fileTemplate` equals [{eq}#fileTemplate]`File template: v = 123, someText = some text`
Given file /data/core/file.txt
File template: v = {{v}}, someText = {{someText}}
When

Set fileTemplate = File template: v = 123, someText = some text

Then

fileTemplate equals File template: v = 123, someText = some text

Initialization with listing:
.Given `someJson`:
[source,json,{set}someJson]
----
{ "a" : 1 }
----
Then `someJson` equals [{eq-json}#someJson]`{ "a" : 1 }`
Given someJson:
{ "a" : 1 }

Then someJson equals { "a" : 1 }

Hidden set
When:: Set `hidden` = [{set}hidden hide]#1#
Then:: `hidden` equals [{eq}#hidden]`1`
When

Set hidden = 1

Then

hidden equals 1

e-execute

Wrapper of concordion execute command

Usage
The full name [{exec}#result = split(#TEXT)]_Jane Smith_
will be broken into first name [{eq}#result.first]_Jane_
and last name [{eq}#result.second]_Smith_

The full name Jane Smith will be broken into first name Jane and last name Smith

In case, when an input parameter occurs after an output we want to check, we can solve this problem by putting an execute command on the paragraph.

Unusual sentence structures
[{exec-on-paragraph}#greeting = greetingFor(#firstName)]
The greeting "[{eq}#greeting]#Hello Bob!#"
should be given to user [{set}#firstName]#Bob# when he logs in.

The greeting "Hello Bob!" should be given to user Bob when he logs in.
Execute on table
Given:: Variable `someVar` = [{set}someVar]`123`
Then::
+
.Lowercase test
[%header, cols="1,1,2a,2a,3,3", {exec}"#r = lowercase(#param)"]
|===
|[{set}param]#Param#
|[{eq}#r.text]#Text#
|[{eq-json}#r.json]#JSON#
|[{eq-xml}#r.xml]#XML#
|[{echo}#r.json]#Echo JSON#
|[{echo}#r.xml]#Echo XML#

|[.json]`ABC {{someVar}}`
|abc {{someVar}}
|[.json]`{"result": "abc {{someVar}}"}`
|[.xml]`<result>abc {{someVar}}</result>`
|[pre language-json details]`{empty}`
|[pre language-xml]`{empty}`

|aBc {{someVar}}
|abc {{someVar}}
|[source,json]
----
{"result": "abc {{someVar}}"}
----
|[source,xml]
----
<result>abc {{someVar}}</result>
----
|[pre language-json]`{empty}`
|[pre language-xml details]`{empty}`
|===
Given

Variable someVar = 123

Then
Lowercase test
Param Text JSON XML Echo JSON Echo XML

ABC 123

abc 123

{"result": "abc 123"}

<result>abc 123</result>

{ "result": "abc 123" }

<result>abc 123</result>

aBc 123

abc 123

 
{"result": "abc 123"}
 
<result>abc 123</result>

{ "result": "abc 123" }

<result>abc 123</result>
Checking emptiness:
.Empty string test
[{exec}#result = getEmptyString()]
,===
[{eq}#result]#Text#, [{eq-json}#result]#JSON#, [{eq-xml}#result]#XML#

,,
,===
Empty string test
Text JSON XML
     

e-verify-rows

Wrapper of concordion verify-rows command

Usage
.Set up users
[{exec}setUpUser(#TEXT)]
1. john.lennon
2. ringo.starr

.Add more users
[{exec}setUpUser(#TEXT)]
- george.harrison
- paul.mccartney

Searching for [{set}searchString]*arr* will return:

.Search result
[{verify-rows-best-match}#username : search(#searchString)]
,===
[{eq}#username]#Matching Usernames#

george.harrison
ringo.starr
,===
Set up users

  1. john.lennon

  2. ringo.starr

Add more users

  • george.harrison

  • paul.mccartney

Searching for arr will return:

Search result
Matching Usernames

george.harrison

ringo.starr

e-run

Usage
link:Nested.adoc[Some nested spec, {run}]

Some nested spec

e-echo

Usage
Echo _firstName_: [{echo}#firstName]_{empty}_

Echo firstName: Bob

example command

Asciidoc named examples interpreted as Concordion examples.

Sidebar with id = before transformed to “before” examples command.

Usage
[{before}]
.Before each example
****
Split the full name [{exec}#result = split(#TEXT)]_Jane Smith_
****

.Dummy
====
Result contains first name [{eq}#result.first]_Jane_
and last name [{eq}#result.second]_Smith_
====

[{ExpectedToFail}]
.Dummy
====
When:: Wrong expectations
Then:: Result contains first name [{eq}#result.first]_Wrong_ and last name [{eq}#result.second]_Wrong_
====

Before each example

Split the full name Jane Smith

Result contains first name Jane and last name Smith

Log File
Example 2. DummyExpectedToFail
When

Wrong expectations

Then

Result contains first name 

Text mismatch
Expected: "Wrong"
     but: was "Jane"
WrongJane
and last name 
Text mismatch
Expected: "Wrong"
     but: was "Smith"
WrongSmith

Handlebar support

The Handlebars templates may be used for templating values in Exam commands.

Exam provides following built-in helpers groups:

Date helpers

{{at '-2d' '-1m'}} will produce: object of class java.util.Date = "Thu Aug 22 16:41:28 MSK 2024" {{now '-2d' '-1m'}} will produce: object of class java.util.Date = "Thu Aug 22 16:41:28 MSK 2024" {{today '-1y' '-2M' '-3d'}} will produce: object of class java.util.Date = "Wed Jun 21 00:00:00 MSK 2023" {{parse '01.02.2000 10:20' 'dd.MM.yyyy HH:mm'}} will produce: object of class java.util.Date = "Tue Feb 01 10:20:00 MSK 2000" {{shift date '-2d' '1m'}} will produce: object of class java.util.Date = "Sat Jan 01 10:21:00 MSK 2000" (given context has variables: {date=Mon Jan 03 10:20:00 MSK 2000}) {{weeksAgo 2}} will produce: object of class java.util.Date = "Sat Aug 10 00:00:00 MSK 2024" {{daysAgo 2}} will produce: object of class java.util.Date = "Thu Aug 22 00:00:00 MSK 2024" {{format date "yyyy-MM-dd'T'HH:mm"}} will produce: "2000-01-02T10:20" (given context has variables: {date=Sun Jan 02 10:20:00 MSK 2000})

Data matcher helpers

{{notNull}} will produce: "${json-unit.not-null}" (given context has variables: {placeholder_type=json}) {{string}} will produce: "${json-unit.any-string}" (given context has variables: {placeholder_type=json}) {{number}} will produce: "${json-unit.any-number}" (given context has variables: {placeholder_type=json}) {{bool}} will produce: "${json-unit.any-boolean}" (given context has variables: {placeholder_type=json}) {{ignore}} will produce: "${json-unit.ignore}" (given context has variables: {placeholder_type=json}) {{uuid}} will produce: "${json-unit.matches:uuid}" (given context has variables: {placeholder_type=json}) {{regex '\d+'}} will produce: "${json-unit.regex}\d+" (given context has variables: {placeholder_type=json}) {{after (today)}} will produce: "${json-unit.matches:after}2024-08-24T00:00" (given context has variables: {placeholder_type=json}) {{before (today)}} will produce: "${json-unit.matches:before}2024-08-24T00:00" (given context has variables: {placeholder_type=json}) {{formattedAndWithin 'yyyy-MM-dd' '5s' (today)}} will produce: "${json-unit.matches:formattedAndWithin}yyyy-MM-dd|$|5s|$|2024-08-24T00:00" (given context has variables: {placeholder_type=json}) {{within '5s' (today)}} will produce: "${json-unit.matches:within}5s|$|2024-08-24T00:00" (given context has variables: {placeholder_type=json}) {{formattedAs "yyyy-MM-dd'T'hh:mm:ss"}} will produce: "${json-unit.matches:formattedAs}yyyy-MM-dd'T'hh:mm:ss" (given context has variables: {placeholder_type=json}) {{iso date}} will produce: "2000-01-02T10:20:11.123" (given context has variables: {date=Sun Jan 02 10:20:11 MSK 2000}) {{isoDate date}} will produce: "2000-01-02" (given context has variables: {date=Sun Jan 02 10:20:11 MSK 2000}) {{matches 'name' 'params'}} will produce: "${json-unit.matches:name}params" (given context has variables: {placeholder_type=json})

Misc helpers

{{vars v1='a' v2=2}} will produce: "" {{set 1 "someVar"}} will produce: object of class java.lang.Integer = "1" {{fnn var1 var2 "default value"}} will produce: "default value" {{map key='value'}} will produce: object of class java.util.Collections$SingletonMap = "{key=value}" {{ls '1' '2'}} will produce: object of class java.util.Arrays$ArrayList = "[1, 2]" {{json (map f1='1' f2=(ls '1' '2'))}} will produce: "{"f1":"1","f2":["1","2"]}" {{prettyJson '{"a": 1}'}} will produce: "{ "a": 1 }" {{NULL}} will produce: null {{exist obj}} will produce: object of class java.lang.Boolean = "true" (given context has variables: {obj=any}) {{eval '#var'}} will produce: object of class java.lang.Integer = "2" (given context has variables: {var=2}) {{resolve 'today is {{today}}' var='val'}} will produce: "today is Sat Aug 24 00:00:00 MSK 2024" {{file '/hb/some-file.txt' var1='val1' var2='val2'}} will produce: "today is Sat Aug 24 00:00:00 MSK 2024" {{jsonPath '{"root": {"nested": 1}}' '$.root.nested'}} will produce: object of class java.lang.Integer = "1" {{math '+' 1 2}} will produce: object of class java.lang.Double = "3.0" {{prop 'system.property' 'optional default'}} will produce: "optional default" {{env 'env.property' 'optional default'}} will produce: "optional default"

Data types matchers:
Given::
+
.`dataJson`:
[source,json,{set}dataJson]
{
  "string": "some string",
  "number": 123,
  "bool": true,
  "ignore": "anything 123",
  "regex": "123"
}

Then::
+
.`dataJson` matches:
[source,json,{eq-json}#dataJson]
{
  "string": "{{string}}",
  "number": "{{number}}",
  "bool": "{{bool}}",
  "ignore": "{{ignore}}",
  "regex": "{{regex '\\d+'}}"
}
Given
dataJson:
{
  "string": "some string",
  "number": 123,
  "bool": true,
  "ignore": "anything 123",
  "regex": "123"
}
Then
dataJson matches:
{
  "string": "${json-unit.any-string}",
  "number": "${json-unit.any-number}",
  "bool": "${json-unit.any-boolean}",
  "ignore": "${json-unit.ignore}",
  "regex": "${json-unit.regex}\\d+"
}
Check with matcher and set actual to variable:
.Check and set
[{exec}#r = lowercase(#param)]
|===
|[{set}param]#Param# |[{eq}#r.text]#Text#

|ABC |{any-str}>>str1
|DEF |{any}>>str2
|===
Actual value is matched and set to variable: str1 = [{eq}#str1]`abc`, str2 = [{eq}#str2]`def`
Check and set
Param Text

ABC

${text-unit.any-string}

DEF

${text-unit.ignore}

Actual value is matched and set to variable: str1 = abc, str2 = def

Date/time matchers:
Given::
+
.`dtJson`:
[source,json,{set}dtJson]
--
{
  "customFormat": "{{format (at) "yyyy/MM/dd'T'HH:mm.ss"}}",
  "isoDate": "{{isoDate (at)}}",
  "iso": "{{iso (at)}}",

  "customFormatAndWithinNow": "{{format (at) "yyyy/MM/dd'T'HH:mm.ss"}}",
  "isoDateAndWithinNow": "{{isoDate (at)}}",
  "isoAndWithinNow": "{{iso (at)}}",

  "customFormatAndWithinSpecifiedDate": "{{format (at) "yyyy/MM/dd'T'HH:mm.ss"}}",
  "isoDateAndWithinSpecifiedDate": "{{isoDate (at)}}",
  "isoAndWithinSpecifiedDate": "{{iso (at)}}",

  "afterSpecifiedDate": "{{iso (at)}}",
  "beforeSpecifiedDate": "{{iso (at)}}"
}
--
Then::
+
.`dtJson` matches:
[source,json,{eq}#dtJson,{verifier}json]
--
{
"customFormat": "{{formattedAs 'yyyy/MM/dd\'T\'HH:mm.ss'}}",
"isoDate": "{{isoDate}}",
"iso": "{{iso}}",

"customFormatAndWithinNow": "{{formattedAndWithin 'yyyy/MM/dd\'T\'HH:mm.ss' '25s'}}",
"isoDateAndWithinNow": "{{isoDate '1d'}}",
"isoAndWithinNow": "{{iso '25s'}}",

"customFormatAndWithinSpecifiedDate": "{{formattedAndWithin 'yyyy/MM/dd\'T\'HH:mm.ss' '25s' (now)}}",
"isoDateAndWithinSpecifiedDate": "{{isoDate '1d' (now)}}",
"isoAndWithinSpecifiedDate": "{{iso '25s' (now)}}",

"afterSpecifiedDate": "{{after (at '-1h')}}",
"beforeSpecifiedDate": "{{before (at '+1h')}}"
}
--
Given
dtJson:
{
  "customFormat": "2024/08/24T16:42.28",
  "isoDate": "2024-08-24",
  "iso": "2024-08-24T16:42:28.883",

  "customFormatAndWithinNow": "2024/08/24T16:42.28",
  "isoDateAndWithinNow": "2024-08-24",
  "isoAndWithinNow": "2024-08-24T16:42:28.883",

  "customFormatAndWithinSpecifiedDate": "2024/08/24T16:42.28",
  "isoDateAndWithinSpecifiedDate": "2024-08-24",
  "isoAndWithinSpecifiedDate": "2024-08-24T16:42:28.883",

  "afterSpecifiedDate": "2024-08-24T16:42:28.883",
  "beforeSpecifiedDate": "2024-08-24T16:42:28.883"
}
Then
dtJson matches:
{
"customFormat": "${json-unit.matches:formattedAs}yyyy/MM/dd'T'HH:mm.ss",
"isoDate": "${json-unit.matches:formattedAs}yyyy-MM-dd",
"iso": "${json-unit.matches:formattedAs}ISO_LOCAL",

"customFormatAndWithinNow": "${json-unit.matches:formattedAndWithin}yyyy/MM/dd'T'HH:mm.ss|$|25s|$|2024-08-24T16:42:41.308",
"isoDateAndWithinNow": "${json-unit.matches:formattedAndWithin}yyyy-MM-dd|$|1d|$|2024-08-24T16:42:41.308",
"isoAndWithinNow": "${json-unit.matches:formattedAndWithin}ISO_LOCAL|$|25s|$|2024-08-24T16:42:41.308",

"customFormatAndWithinSpecifiedDate": "${json-unit.matches:formattedAndWithin}yyyy/MM/dd'T'HH:mm.ss|$|25s|$|2024-08-24T16:42:41.308",
"isoDateAndWithinSpecifiedDate": "${json-unit.matches:formattedAndWithin}yyyy-MM-dd|$|1d|$|2024-08-24T16:42:41.308",
"isoAndWithinSpecifiedDate": "${json-unit.matches:formattedAndWithin}ISO_LOCAL|$|25s|$|2024-08-24T16:42:41.308",

"afterSpecifiedDate": "${json-unit.matches:after}2024-08-24T15:42:28.883",
"beforeSpecifiedDate": "${json-unit.matches:before}2024-08-24T17:42:28.883"
}

Web Service plugin

WsPlugin enables to document REST/SOAP API.

ws
Dependency
testImplementation "io.github.adven27:exam-ws:2024.0.10"
Configuration
class Specs : AbstractSpecs() {
    override fun init() = ExamExtension(
        WsPlugin(
            host = "localhost",
            port = 8080
        )
    )

e-http

Check HTTP interaction

Attributes
verifier

Optional. Default: text.

Content type verifier. Built-in values: text, json, xml.
await

Optional. Default: 4s,0s,1s.

Format: <at_most>,<poll_delay>,<poll_interval>

Where:

at_most

How long to wait before failing the verification
poll_delay

How long to wait before starting verification
poll_interval

How long to wait between verification attempts
Waiting is disabled by default. Check will fail immediately in case of mismatch unless the await attribute is set.
Usage
.HTTP API check
[{http}]
--
.Given request:
[source,httprequest]
----
GET /mirror/request
Content-Type: application/json
Authorization: token
Accept-Language: ru
Cookie: cookie1=c1; cookie2=c2
----

.Expected response:
[source,httprequest]
----
200
Content-Type: application/json

{
  "GET": "/mirror/request",
  "headers": {
    "host": "{{ignore}}",
    "content-type": "application/json",
    "authorization": "token",
    "accept-language": "ru",
    "cookie": "cookie1=c1; cookie2=c2"
  },
  "cookies": { "cookie1": "c1", "cookie2": "c2" }
}
----
--
HTTP API check
Given request:
GET /mirror/request HTTP/1.1
Host: localhost:8888
Content-Type: application/json
Authorization: token
Accept-Language: ru
Cookie: cookie1=c1; cookie2=c2
Expected response:
SOAP
.SOAP in collapsible blocks
[{http}]
--
.SOAP request
[{collapse}]
=====
[source,httprequest]
----
POST /mirror/soap
Content-Type: application/soap+xml

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Body>
        <ns2:getItemRequest xmlns:ns2="http://ws.io">
            <date>{{iso (at)}}</date>
        </ns2:getItemRequest>
    </soap:Body>
</soap:Envelope>
----
=====

.SOAP response
[{collapse}]
=====
[source,httprequest]
----
200

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Body>
        <ns2:getItemRequest xmlns:ns2="http://ws.io">
            <date>{{iso (at)}}</date>
        </ns2:getItemRequest>
    </soap:Body>
</soap:Envelope>
----
=====
--
SOAP in collapsible blocks
SOAP request 
POST /mirror/soap HTTP/1.1
Host: localhost:8888
Content-Type: application/soap+xml
Content-Length: 259

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <ns2:getItemRequest xmlns:ns2="http://ws.io">
      <date>2024-08-24T16:42:28.883</date>
    </ns2:getItemRequest>
  </soap:Body>
</soap:Envelope>
SOAP response
Data-driven check
.Data-driven check
[{http}]
--
.Request
[source,httprequest]
----
{{file req}}
----
.Response
[source,httprequest]
----
{{file resp}}
----

[{http-where}]
,===
,req, resp

GET, /data/ws/get.http, /data/ws/get-resp.http
POST, /data/ws/post.http, /data/ws/post-resp.http
PUT, /data/ws/put.http, /data/ws/put-resp.http
DELETE, /data/ws/delete.http, /data/ws/delete-resp.http
,===
--
Data-driven check
Put interactions to variable
.Put interactions to variable
[source,httprequest,{http}interactions]
----
GET /mirror/request?p=1
Content-Type: application/json
----

.Check interactions
[%header,cols="4,2,0",{verify-rows}"#i : #interactions"]
|===
|[{eq}#i.req.raw]#Request#
|[{jsonIgnoreExtraFields} {eq}#i.resp.body]#Response#
|[{eq}#i.resp.statusCode]#Status#

|[pre language-http]`GET /mirror/request?p=1 HTTP/1.1
Host: localhost:8888
Content-Type: application/json
{blank}
{blank}`
|[pre language-json]`{"GET":"/mirror/request?p=1"}`
|200
|===

Доступ к полям через jsonPath:

`{{jsonPath interactions.[0].resp.body '$.GET'}}` = [e-set=r]`{{jsonPath interactions.[0].resp.body '$.GET'}}`
Put interactions to variable
GET /mirror/request?p=1 HTTP/1.1
Host: localhost:8888
Content-Type: application/json
Check interactions
Request Response Status

GET /mirror/request?p=1 HTTP/1.1 Host: localhost:8888 Content-Type: application/json

{"GET":"/mirror/request?p=1"}

200

Доступ к полям через jsonPath:

{{jsonPath interactions.[0].resp.body '$.GET'}} = /mirror/request?p=1

Custom content type verifier
import io.github.adven27.concordion.extensions.exam.core.AbstractSpecs
import io.github.adven27.concordion.extensions.exam.core.ExamExtension
import io.github.adven27.concordion.extensions.exam.core.JsonVerifier
import net.javacrumbs.jsonunit.core.Option.IGNORING_EXTRA_FIELDS

class Specs : AbstractSpecs() {

    override fun init() = ExamExtension(
        //...
    ).withVerifiers(
        "jsonIgnoreExtraFields" to JsonVerifier { it.withOptions(IGNORING_EXTRA_FIELDS) }
    )
}
Use custom content verifier
.Custom content type verifier
[{http}]
--
[source,httprequest]
----
POST /mirror/request
Content-Type: application/json

{}
----

.Check with registered `jsonIgnoreExtraFields` verifier
[source,httprequest,{jsonIgnoreExtraFields}]
----
200
Content-Type: application/json

{
  "POST": "/mirror/request",
  "body": {},
  "headers": {
    "host": "{{ignore}}",
    "content-type": "application/json",
    "content-length": "{{ignore}}"
  }
}
----
--
Custom content type verifier
POST /mirror/request HTTP/1.1
Host: localhost:8888
Content-Type: application/json
Content-Length: 2

{}
Check with registered jsonIgnoreExtraFields verifier

Details

SQL Database plugin

DbPlugin is a wrapper around DbUnit library that enables setting up and verification of database.

db
Figure 3. Exam integration with database
Dependency
testImplementation "io.github.adven27:exam-db:2024.0.10"
Configuration
class Specs : AbstractSpecs() {
    override fun init() = ExamExtension(
        DbPlugin(
            driver = "org.postgresql.Driver",
            url = "jdbc:postgresql://localhost:5432/postgres",
            user = "postgres",
            password = "postgres"
        )
    )
    //...
}
Commands

e-db-set

Applies datasets to database.

Attributes
operation

Optional. Default: clean_insert.

DbUnit operation
ds

Optional. Default: default.

Datasource to apply dataset. By default, only 1 datasource with name default exists, more can be configured in DbPlugin.

Applying tables

Creates DbUnit dataset for specified table(s) and applies it to database.

Execute single table
.customer
[{db-set}]
|===
|name |balance |is_active |comment |created_at

|Alex |10.01 |true |{nil} |{1d-}
|===
customer
namebalancecommentis_activecreated_at
Alex10.01(null)true2024-08-23T16:42:28.883
Execute single table from CSV file
.product
[%header, {db-set}]
,===
//include::../data/db/product.csv[]
id, name, price, rating, disabled, created_at, meta_json
1, Model A, 100.50,       10, false,{{at '-1d'}},"{
  ""id"": ""a"",
  ""code"": 1
}"
2, Model B,    200, {{NULL}}, true,{{at}},"{
  ""id"": ""b"",
  ""code"": 2
}"
,===
product
idnamepriceratingdisabledcreated_atmeta_json
1Model A100.5010false2024-08-23T16:42:28.883
{
  "id": "a",
  "code": 1
}
2Model B200(null)true2024-08-24T16:42:28.883
{
  "id": "b",
  "code": 2
}
Execute several tables transactionally
.Dataset
[{db-set}]
--
.cart
,===
id, name

1, Cart 1
,===

.item
,===
cart_id, name, price

1, item 1, 10.01
,===
--
cart
idname
1Cart 1
item
namepricecart_id
item 110.011

Applying DbUnit datasets

Applies specified DbUnit datasets to database.

Execute single DbUnit dataset
.Dataset
[,xml,{db-set}]
----
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
  <PERSON ID="1" NAME="Adam" AGE="10" BIRTHDAY="{{today}}" BIRTH_TIME="11:30:00"/>
  <PERSON_FIELDS ID="11..19" NAME="a1" VAL="{{a1}}" PERSON_ID="1"/>
  <PERSON_FIELDS ID="11..19" NAME="a2" VAL="{{a2}}" PERSON_ID="1"/>
</dataset>
----

1 2

Dataset
PERSON
idnameagebirthdaybirth_time
1Adam102024-08-24T00:00:0011:30:00
PERSON_FIELDS
idnamevalperson_id
11a111
12a221
Execute several DbUnit datasets:
.Dataset
[{db-set}]
--
.carl.xml
[,xml]
----
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
  <PERSON ID="2" NAME="Carl" AGE="30" BIRTHDAY="{{parse '2003-03-03'}}" BIRTH_TIME="11:30:00"/>
  <PERSON_FIELDS ID="41..40" NAME="c1" VAL="vc" PERSON_ID="2"/>
</dataset>
----

.bob.json
[,json]
----
{
  "PERSON": [
    {"id": 10, "name": "Bob", "age": 20, "BIRTHDAY": "2000-10-10 00:00:00", "BIRTH_TIME": "03:40:00"}
  ],
  "PERSON_FIELDS": [
    {"id": "21..30", "name": "b1", "val": "1", "person_id": 10},
    {"id": "21..30", "name": "b2", "val": "2", "person_id": 10}
  ]
}
----
--
PERSON
idnameagebirthdaybirth_time
2Carl302003-03-03T00:00:0011:30:00
10Bob202000-10-10 00:00:0003:40:00
PERSON_FIELDS
idnamevalperson_id
41c1vc2
21b1110
22b2210

e-db-check

Verifies database state.

Attributes
where

Optional. Default: not set.

WHERE clause in database-specific SQL format to filter actual dataset
orderBy

Optional. Default: columns in order of declaration.

List of columns to sort records before verifying
ds

Optional. Default: default.

Datasource to apply dataset. By default, only 1 datasource with name default exists, more can be configured in DbPlugin.
await

Optional. Default: 4s,0s,1s.

Format: <at_most>,<poll_delay>,<poll_interval>

Where:

at_most

How long to wait before failing the verification
poll_delay

How long to wait before starting verification
poll_interval

How long to wait between verification attempts
Waiting is disabled by default. Check will fail immediately in case of mismatch unless the await attribute is set.

Verifying tables

Creates DbUnit dataset for table and verifies it against a database.

Check single table
.customer
[{db-check}]
|===
|name |balance |is_active |comment |created_at

|Alex |10.01 |true |{{NULL}} |{{at '-1d'}}
|===
customer
namebalanceis_activecommentcreated_at
Alex10.01true(null)2024-08-23T16:42:28.883
Check single table from CSV file
.product
[%header,{db-check}]
,===
//include::../data/db/product.csv[]
id, name, price, rating, disabled, created_at, meta_json
1, Model A, 100.50,       10, false,{{at '-1d'}},"{
  ""id"": ""a"",
  ""code"": 1
}"
2, Model B,    200, {{NULL}}, true,{{at}},"{
  ""id"": ""b"",
  ""code"": 2
}"
,===
product
idnamepriceratingdisabledcreated_atmeta_json
1Model A100.5010false2024-08-23T16:42:28.883
{
  "id": "a",
  "code": 1
}
2Model B200.00(null)true2024-08-24T16:42:28.883
{
  "id": "b",
  "code": 2
}
Check with matchers
.product
[{db-check}]
|===
|name |price |rating |disabled |created_at |meta_json

|{{string}}
|{{number}}>>priceA
|{{notNull}}
|{{bool}}>>disabledA
|{{within '15s' (at '-1d')}}
|{"id": "{{string}}", "code": "{{number}}"}

|{{regex '.*'}}>>nameB
|{{ignore}}
|{{ignore}}>>ratingB
|true
|{{within '1m'}}>>createdB
|{"id": "{{regex '\\w'}}", "code": "{{ignore}}"}
|===

nameB = [{eq}#nameB]`Model B`
priceA = [{eq}#priceA]`100.50`
ratingB = [{eq}#ratingB]`(null)`
disabledA = [{eq}#disabledA]`false`
createdB = [{eq}#createdB]`{{format (at) 'yyyy-MM-dd HH:mm:ss.SSS'}}`
product
namepriceratingdisabledcreated_atmeta_json
Model A100.5010false2024-08-23T16:42:28.883
{
  "id": "a",
  "code": 1
}
Model B200.00(null)true2024-08-24T16:42:28.883
{
  "id": "b",
  "code": 2
}

nameB = Model B priceA = 100.50 ratingB = (null) disabledA = false createdB = 2024-08-24 16:42:28.883

Check ordered
.product
[{db-check}, {db-orderBy}'rating, id']
,===
name, rating

{{string}}, {{ignore}}
{{string}}, 10
,===
product
namerating
Model B(null)
Model A10
Check emptiness
.product
[{db-check}]
|===
|===

Given empty table:

product
EMPTY

Then:

product
EMPTY

Verifying DbUnit datasets

Verifies DbUnit datasets against a database.

Check single dataset
.Dataset
[,xml,{db-check}]
----
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
  <cart name="Cart 1"/>
  <item name="item 1" price="10.01" cart_id="1"/>
</dataset>
----

1 2

Dataset
   cart
name
Cart 1
   item
namepricecart_id
item 110.011
Check several datasets:
.Dataset
[{db-check}]
--
.carl.xml
[,xml]
----
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
  <PERSON ID="2" NAME="Carl" AGE="30" BIRTHDAY="{{parse '2003-03-03'}}" BIRTH_TIME="11:30:00"/>
  <PERSON_FIELDS ID="41..40" NAME="c1" VAL="vc" PERSON_ID="2"/>
</dataset>
----

.bob.json
[,json]
----
{
  "PERSON": [
    {"id": 10, "name": "Bob", "age": 20, "BIRTHDAY": "2000-10-10 00:00:00", "BIRTH_TIME": "03:40:00"}
  ],
  "PERSON_FIELDS": [
    {"id": "21..30", "name": "b1", "val": "1", "person_id": 10},
    {"id": "21..30", "name": "b2", "val": "2", "person_id": 10}
  ]
}
----
--
Dataset
PERSON
IDNAMEAGEBIRTHDAYBIRTH_TIME
2Carl302003-03-03T00:00:0011:30:00
10Bob202000-10-10T00:00:0003:40:00
PERSON_FIELDS
IDNAMEVALPERSON_ID
21b1110
22b2210
41c1vc2

e-db-show

Prints content of specified database table and optionally generates DbUnit dataset.

Attributes
where

Optional. Default: not set.

WHERE clause in database-specific SQL format to filter actual dataset
ds

Optional. Default: default.

Datasource to apply dataset. By default, only 1 datasource with name default exists, more can be configured in DbPlugin.
saveTo

Optional. Default: not set.

Path in 'srs/test/resources/' to store generated dataset
Usage
.Show all and generate dataset
[{db-show}person, {db-show-save-to}/data/db/person.xml]
,===
,===

.Show specific columns
[{db-show}person]
,===
id, name

,===

.Show specific rows
[{db-show}person, {db-where}"name='Bob'"]
,===
id, name

,===
Show all and generate dataset
idnameagebirthdaybirth_time
2Carl302003-03-03T00:00:0011:30:00
10Bob202000-10-10T00:00:0003:40:00
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
  <person id="2" name="Carl" age="30" birthday="2003-03-03 00:00:00.0" birth_time="11:30:00"/>
  <person id="10" name="Bob" age="20" birthday="2000-10-10 00:00:00.0" birth_time="03:40:00"/>
  <person_fields id="21" name="b1" val="1" person_id="10"/>
  <person_fields id="22" name="b2" val="2" person_id="10"/>
  <person_fields id="41" name="c1" val="vc" person_id="2"/>
</dataset>
Show specific columns
idname
2Carl
10Bob
Show specific rows
idname
10Bob

e-db-clean

Cleans specified tables in a database.

Attributes
ds

Optional. Default: default.

Datasource to apply dataset. By default, only 1 datasource with name default exists, more can be configured in DbPlugin.
Usage
Clean tables: [{db-clean}]#person, person_fields#
Tables will be cleaned up in the order opposite to declaration, hence the "referenced" tables should go first and the "referencing" - last.

Clean tables: person, person_fields

Details

Message Queue plugin

MqPlugin enables to set up and verify the state of some message queue. A message queue is represented as an implementation of the io.github.adven27.concordion.extensions.exam.mq.MqTester interface and that implementation is responsible for interacting with that specific queue.

mq

There are several out-of-the-box implementations that can be used directly or as an example for custom ones:

Dependency
testImplementation "io.github.adven27:exam-mq:2024.0.10"
Configuration
open class Specs : AbstractSpecs() {
    override fun init() = ExamExtension(
        MqPlugin(
            "someRabbitQueue" to RabbitTester(
                port = 5432,
                sendConfig = SendConfig("someRoutingKey"),
                receiveConfig = ReceiveConfig("someQueueName")
            ),
            "dummyQueue" to object : MqTester {
                private val queue = ArrayDeque<Message>()

                /* open connection*/
                override fun start() = Unit

                /* close connection*/
                override fun stop() = Unit

                override fun send(message: Message) {
                    queue += message
                }

                override fun receive(): List<Message> = queue.map { queue.poll() }
                override fun purge() = queue.clear()
                override fun accumulateOnRetries(): Boolean = true
            }
        )
    )
MqPlugin accepts a map of a <queue alias name> to a MqTester implementation. Queue alias is a queue logical name: e-mq-set=out
Commands

e-mq-set

Sends messages to specified queue.

Usage
:queue2: myAnotherQueue

.Sending message to {queue2}
[source,json,e-mq-set={queue2}]
----
{ "a": 1, "d": "dd" }
----
Sending message to myAnotherQueue
{
  "a": 1,
  "d": "dd"
}
Sending message with headers and params which could be used by MqTester:
.Sending message with headers and params to {queue2}
[e-mq-set={queue2}]
--
.Params
[{mq-params}]
,===
key, some-kafka-message-key
partition, 1
,===

.Headers
[{mq-headers}]
,===
h1, 1
h2, 2
,===

[source,json]
----
{"timestamp": "{{iso (at '-1h')}}"}
----
--
Sending message with headers and params to myAnotherQueue
Params

key

some-kafka-message-key

partition

1
Headers

h1

1

h2

2

 
{
  "timestamp": "2024-08-24T15:42:28.883"
}
Sending several messages:
.Sending several messages to myQueue
[{mq-set}myQueue]
|===
| Message without headers:
[source,json]
----
{{file '/specs/../data/mq/msg.json' msg=(map v1='a' v2='b')}}
----

| Message with headers and params which could be used by `MqTester`

.Params
[{mq-params}]
,===
key, some-kafka-message-key
partition, 1
,===

.Headers
[{mq-headers}]
,===
h1, {{isoDate (at '-1d')}}
h2, 2
,===

.Payload
[source,json]
----
{{file '/specs/../data/mq/msg.json' msg=(map v1='c' v2='d')}}
----
|===
Sending several messages to myQueue

Message without headers:

{
  "date": "2024-08-24",
  "var1": "a",
  "var2": "b",
  "var3": "3"
}

Message with headers and params which could be used by MqTester

Params

key

some-kafka-message-key

partition

1
Headers

h1

2024-08-23

h2

2
Payload
{
  "date": "2024-08-24",
  "var1": "c",
  "var2": "d",
  "var3": "3"
}

e-mq-check

Check messages in queue.

Attributes
contains

Optional. Default: EXACT.

Messages list verifying mode. Supported values: EXACT, EXACT_IN_ANY_ORDER.
verifier

Optional. Default: text.

Content type verifier. Built-in values: text, json, xml.
await

Optional. Default: 4s,0s,1s.

Format: <at_most>,<poll_delay>,<poll_interval>

Where:

at_most

How long to wait before failing the verification
poll_delay

How long to wait before starting verification
poll_interval

How long to wait between verification attempts
Waiting is disabled by default. Check will fail immediately in case of mismatch unless the await attribute is set.
Usage
.Check messages in myQueue
[{mq-check}myQueue]
|===
| Expected message ignore headers:

[source,json]
----
{{file '/specs/../data/mq/msg.json' msg=(map v1='a' v2='b')}}
----

| Expected message containing headers and params:

[{mq-params}]
,===
key, some-kafka-message-key
,===

[{mq-headers}]
,===
h1, {{isoDate (at '-1d')}}
,===

[source,json]
----
{{file '/specs/../data/mq/msg.json' msg=(map v1='c' v2='d')}}
----
|===
Check messages in myQueue
{ "date": "2024-08-24", "var1": "a", "var2": "b", "var3": "3" }
Params
key
some-kafka-message-key
Headers
h1
2024-08-23
{ "date": "2024-08-24", "var1": "c", "var2": "d", "var3": "3" }
Check emptiness
.Check myQueue is empty
[{mq-check}myQueue]
|===
|===
Check myQueue is empty
EMPTY
Custom content type verifier
import io.github.adven27.concordion.extensions.exam.core.AbstractSpecs
import io.github.adven27.concordion.extensions.exam.core.ExamExtension
import io.github.adven27.concordion.extensions.exam.core.JsonVerifier
import net.javacrumbs.jsonunit.core.Option.IGNORING_EXTRA_FIELDS

class Specs : AbstractSpecs() {

    override fun init() = ExamExtension(
        //...
    ).withVerifiers(
        "jsonIgnoreExtraFields" to JsonVerifier { it.withOptions(IGNORING_EXTRA_FIELDS) }
    )
}
Use custom content verifier
.Check with registered `jsonIgnoreExtraFields` verifier
[{mq-check}myQueue]
|===
| Expected ISO-formatted `date` and ignore extra fields:

[source,json,{jsonIgnoreExtraFields}]
----
{"date":  "{{isoDate}}"}
----
|===
Given
myQueue has message
{
  "date": "2024-08-24",
  "var1": "1",
  "var2": "2",
  "var3": "3"
}
Then

Check with registered jsonIgnoreExtraFields verifier
{ "date": "${test-unit.matches:formattedAs}yyyy-MM-dd" }

e-mq-clean

Cleans specified queues.

Usage
Queues are empty: [{mq-clean}]_myQueue, myAnotherQueue_

Queues are empty: myQueue, myAnotherQueue

Details

Results generated by Concordion
in 12318 ms on 24-Aug-2024 at 16:42:53 MSK
Log File