Sketch: new testing API

[itamarst]

The current testing API:

1. Duplicate parts of eliot.parse.
2. Changes Logger global state, which means it's hard to do things like additionally logging all messages via py.test plugin.

New design:

1. Rely on eliot.parse for validation of messages (will likely port some of the utility methods from eliot.testing.)
2. Messages are captured via a destination, so it's possible to add other destinations as well.
3. Downside: tracebacks on errors may be less nice, not sure yet.

Any thoughts? @exarkun @thedrow @jonathanj

[thedrow]

Just neatpicking.
This looks good otherwise.

[thedrow]

Can't you just use __qualname__?
It's available since Python 3.3.

[exarkun]

Python 2.7 is still supported.

[itamarst]

Not anymore! But when that code was written, it was supported.

[thedrow]

If using __qualname__ is possible you can simply use operator.attrgetter("__qualname__") instead of class_fpqn.

[exarkun]

Any thoughts? @exarkun @thedrow @jonathanj

Can you write some developer-facing prose/howto docs for this feature? I will read and comment on those. Skimming the current diff, it's not obvious to me how this is meant to be used. I'd rather comment on UX of the API than implementation details. :)

[itamarst]

I'll write some docs when I have time, yeah.

But quick version is:

class MyTests(unittest.TestCase):
    def test_mything(self):
        logs = logs_for_pyunit(self)  # start capturing logs
        do_my_thing()  # run normal test code

        # Ok now you can make assertions about logs:
        # 1. I expect a ZeroDivisionError to be logged.
        assert len(logs.remove_expected_tracebacks(ZeroDivisionError)) == 1
        # 2. I expect a my_action action:
        assert logs.messages[0]["action_type"] == "my_action"
        # Or something like
        [action] = logs.parse()
        # Now I have a eliot.parse.WrittenAction to play with

[itamarst]

If you have a custom JSON encoding class:

def test_mytest(self):
    # Check that encodes using custom JSON encoder (and then decodes with default json.loads):
    logs = logs_for_pyunit(self, encode=CustomEncoder().encode)
    # ...