Skip to content

Conversation

@lorenzboguhn
Copy link

Hi @yugui,
I hope you are well.
We have used your test module and i found it very unhandy to interpret failure messages for tests.
For example, suppose we have a Kubernetes manifest in our code in which there is an image url.
The error messages are hard to read if we have a function to override the url in specific cases with other values and the image url itself is an object or very log string.
It is really annoying to scroll an awful lot to compare actual and expected in such cases.
Moreover, it is really hard to see if there are just subtle changes in the url path.

Main Changes:

  • Output is also an object in case of failure
  • Failure messages are interpreted as code and not formatted in a string
  • files are able to evaluate (functions are hidden)

If you need any further infos/examples or other changes send me a message.

Example:

Changes the output of jsonnetunit/test/std_matchers_failure_test.jsonnet from

unknown:Location { line: 1, column: 1 }-Location { line: 1, column: 1 } | unknown error: RUNTIME ERROR: Failed 11/11 test cases:
testEq: Expected 1 to be 2
testGe: Expected 1 to be greater than or equal to 2
testGt: Expected 1 to be greater than 2
testGtEq: Expected 1 to be greater than 1
testLe: Expected 2 to be less than or equal to 1
testLt: Expected 2 to be less than 1
testLtEq: Expected 2 to be less than 2
testNe: Expected 1 not to be 1
testThatFunction: Expected 1 to satisfy the function
testThatObject: Expected 1 to satisfy {"actual": 1, "result": false}
testThatObjectDesc: Expected 1 to satisfy the condition that the value is 2
	jsonnetunit/test.libsonnet:44:7-20	object <anonymous>
	Field "verify"	
	During manifestation	

to:

{
   "verify": {
      "failures": [
         {
            "testEq": {
               "Expected": 1,
               "to be": 2
            }
         },
         {
            "testGe": {
               "Expected": 1,
               "to be greater than or equal to": 2
            }
         },
         {
            "testGt": {
               "Expected": 1,
               "to be greater than": 2
            }
         },
         {
            "testGtEq": {
               "Expected": 1,
               "to be greater than": 1
            }
         },
         {
            "testLe": {
               "Expected": 2,
               "to be less than or equal to": 1
            }
         },
         {
            "testLt": {
               "Expected": 2,
               "to be less than": 1
            }
         },
         {
            "testLtEq": {
               "Expected": 2,
               "to be less than": 2
            }
         },
         {
            "testNe": {
               "expected": 1,
               "not to be": 1
            }
         },
         {
            "testThatFunction": {
               "Expected": 1,
               "to satisfy": "the function"
            }
         },
         {
            "testThatObject": {
               "Expected": 1,
               "to satisfy": {
                  "actual": 1,
                  "result": false
               }
            }
         },
         {
            "testThatObjectDesc": {
               "Expected": 1,
               "to satisfy": "the condition that the value is 2"
            }
         }
      ],
      "result": "Failed 11/11 test cases"
   }
}

Best Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants