Automated machine learning and neural architecture optimization

Abstract

Automated machine learning (AutoML) and neural architecture optimization (NAO) represent pivotal components in the landscape of machine learning and artificial intelligence. This paper extensively explores these domains, aiming to delineate their significance, methodologies, cutting-edge techniques, challenges, and emerging trends. AutoML streamlines and democratizes machine learning by automating intricate procedures, such as algorithm selection and hyperparameter tuning. Conversely, NAO automates the design of neural network architectures, a critical aspect for optimizing deep learning model performance. Both domains have made substantial advancements, significantly impacting research, industry practices, and societal applications. Through a series of experiments, classifier accuracy, NAO model selection based on hidden unit count, and learning curve analysis were investigated. The results underscored the efficacy of machine learning models, the substantial impact of architectural choices on test accuracy, and the significance of selecting an optimal number of training epochs for model convergence. These findings offer valuable insights into the potential and limitations of AutoML and NAO, emphasizing the transformative potential of automation and optimization within the machine learning field. Additionally, this study highlights the imperative for further research to explore synergies between AutoML and NAO, aiming to bridge the gap between model selection, architecture design, and hyperparameter tuning. Such endeavors hold promise in opening new frontiers in automated machine learning methodologies.