Automated Deep Learning Model Development Based on Weight Sensitivity and Model Selection Statistics

Abstract

Current sustainable production and consumption processes call for technological integration with the realm of computational modeling especially in the form of sophisticated data-driven architectures. Advanced mathematical formulations are essential for deep learning approach to account for revealing patterns under nonlinear and complex interactions to enable better prediction capabilities for subsequent optimization and control tasks. Bayesian Information Criterion and Akaike Information Criterion are introduced as additional constraints to a mixed-integer training problem which employs a parameter sensitivity related objective function, unlike traditional methods which minimize the training error under fixed architecture. The resulting comprehensive optimization formulation is flexible as a simultaneous approach is introduced through algorithmic differentiation to benefit from advanced solvers to handle computational challenges and theoretical issues. Proposed formulation delivers 40% reduction, in architecture with high accuracy. The performance of the approach is compared to fully connected traditional methods on two different case studies from large scale chemical plants.