DARE: Data Auditing for Reliability Evaluation

Data Auditing for Reliability Evaluation (DARE) is a python library that assesses the reliability of individual machine learning (ML) model predictions and attempts to address certain data trustworthiness concerns in data-driven models. DARE is suitable for ML-integrated control systems that require and implement real-time/fast sensor measurements with data-driven ML model predictions. It evaluates prediction reliability by analyzing the proximity and locality of the training data used to derive a prediction outcome. The core purpose of DARE is to provide evidence that a ML model is validated for a specific test input. DARE estimates reliability by using a kernel distance function on the model's training data. It then determines the relevance of new test samples to the training dataset. DARE also includes a method to visualize the training data as a 2D temperature slices. Ultimately, DARE helps users determine whether a prediction should be accepted or rejected based on the verification and validation knowledge within the training dataset. It is beneficial for groups that rely on data-driven ML-based architectures and can be implemented for real-time systems that utilize ML-models to any degree. When using DARE, users will receive a score for every prediction. This score indicates a test sample's relevance to the training set and implies a prediction's reliability. It is important to note that DARE is still under development (alpha-phase) and ongoing efforts focus on improving performance and accuracy of predictions as well as investigating applications for other advanced modeling methods. DARE can theoretically be applied to any ML model, however, has only been tested on feedforward and long-short-term memory models.

This software is under copyright. To purchase a license, please use the 'Contact Us' form on this page. We will respond as promptly as possible.