Repository navigation
Add live end-to-end test against the real neuPrint brain - #2
Merged
Merged
Conversation
Fetches the mini brain from the server with an empty cache, builds the network and checks it settles to finite motor output. Skipped when no token is set, so offline runs are unchanged. Adds make test-live and passes the token through CI when the repo has one.
Lets you evaluate the untrained connectome as a baseline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
tests/test_live_brain.py: fetches the DNge104 mini brain from neuPrint with an empty cache, builds the network, settles it and checks the motor output is finite. Skipped whenNEUPRINT_APPLICATION_CREDENTIALSis unset, so offline runs are unchanged.make test-live.NEUPRINT_TOKENrepo secret to the tests when one exists; otherwise the live test skips.ZeroDivisionErrorin the trainers when run with--episodes 0, which makes it possible to evaluate the untrained connectome as a baseline.Checked by hand against the real data
flynet train --fullwith hebbian, gradient and rl, 10 episodes each: all run to completion.make lint docstyle typecheckclean, 82 tests passing.Worth knowing
The untrained connectome already reaches 100% success on the foraging task (~125 steps), so the current success rates don't show that training helps. A harder task would be needed for that.