Knowledge-Based Training of Learning Architectures Under Input Sensitivity Constraints for Improved Explainability

Abstract

The traditional machine learning (ML) training problem is unconstrained and lacks an explicit formulation of the underlying driving phenomena. Such a formulation, based solely on experimental data, does not ensure the delivery of qualitative knowledge among variables due to many theoretical issues in the optimization task. This study further tightens Artificial Neural Networks (ANNs) training by including input sensitivities as additional constraints and applies to regression and classification tasks based on literature data. In theory, such sensitivity represents the change direction of the target variable per change in measurements from indicators. The resulting nonlinear optimization problem is solved th rough a rigorous solver and includes the sensitivity expressions through algorithmic differentiation. Compared to traditional methods, with an acceptable decrease in the prediction capability, the proposed model delivers more intuitive, explainable, and experimentally verifiable predictions under input variable variations, under robustness to overfitting, while serving robust identification tasks. A classification case study includes a patient-oriented clinical decision support system development based on the impact of cancer-indicating variables. A competitive test prediction accuracy is obtained compared to commonly used algorithms despite 10 % decrease in the training. The regression case is built upon the energy load estimation to account for prominent considerations to obtain desired sensitivity patterns and proposed methodology delivers significant accuracy drop compared to some formulations to address knowledge patterns. The approach delivers a compatible pattern with practitioner expertise and is compared to widely used machine learning algorithms, whose performances are evaluated through common statistics in addition to multi-variable response graphs.