-
Notifications
You must be signed in to change notification settings - Fork 1
Large test
antonnazaruk edited this page Jun 10, 2020
·
7 revisions
Large tests are running over component(s) deployed to environment by the same Continuous Deployment pipeline that deploys to production.
Large Tests can be reflected by following test suites:
- End-To-End
- Functional
- Load/Stress/Performance (NFR gates)
- Security
- Smoke/Sanity
- any other ones which are running over deployed components
The question a large test attempts to answer is, “Does the product operate the way a user would expect (from functional and non-functional requirements perspective) and produce the desired results?”.
Large Tests tend to be much slower than medium tests, they rely on a full PROD-like deployment up and running alongside real (not stubbed/mocked) infrastructure services.
Possible phases/places where Large Tests can be running :
- Pull Requests checks (in case if they are relatively fast and overall PR time doesn't go beyond ~15mins)
- Functional Test Suite in Continuous Deployment pipeline (after-deployment step). If their run takes too long -- it's expected to have separate Smoke/Sanity test suite extracted for that purpose and Functional ones running by separate pipeline.
- NFRs gate in Continuous Deployment pipeline. It's expected that desired/existed application benchmarks have been already collected by performance tests and put as a NFR's thresholds/gates. Metrics collected during NFR gate tests are to be trended in time.