The Integrated Evaluation Framework (IEvaluate) provides a flexible, centralized platform for validating and monitoring artificial intelligence applications. Specifically, it provides a framework through which automated regression testing can be done against established baselines as well as support for receipt of live feedback of deployed pipelines for continuous improvement.
IEvaluate can be extended to support any variety of AI pipelines through the plugin system, although it is specifically written with the intention of supporting clinical applications, and in particular clinical information extraction and phenotyping tasks.
IEvaluate-Parent (You are here) - The parent project that contains all components of the framework. Clone this to build the whole stack
IEvaluate-API - The API provides a relatively static point of access for extensions to the IEvaluate Framework, most notably through the use of Plugins to introduce new integrations and application types. It is recommended that all extension plugins interact as much as possible with classes from this API as behaviour consistency is guaranteed
IEvaluate-Core - The Integrated Evaluation Framework's core functionality and several default implementations of the API
IEvaluate-REST - Exposes IEvaluate's capabilities as a RESTful service
IEvaluate-Web - Interacts with the RESTful service to provide a front-end to IEvaluate's capabilities
IEvaluate-ES-Plugin - An IEvaluate Plugin that integrates with an underlying Elasticsearch index for storage and search capabilities
IEvaluate-ML-Plugin - An IEvaluate Plugin that leverages Elasticsearch's innate ML capabilities to accelerate the debugging process, as well as utilizing anomaly detection to monitor application errors in a live environment
IEvaluate-ClinicalAI-Plugin - An example implementation of an IEvaluate plugin for typical Clinical AI Tasks