Date of Award


Document Type


Degree Name

Master of Science in Operations Research


Department of Operational Sciences

First Advisor

Daniel W. Steeneck, PhD


In the world of machine learning, neural networks have become a powerful pattern recognition technique that gives a user the ability to interpret high-dimensional data whereas conventional methods, such as logistic regression, would fail. There exists many different types of neural networks, each containing its own set of hyper-parameters that are dependent on the type of analysis required, but the focus of this paper will be on the hyper-parameters of convolutional neural networks. Convolutional neural networks are commonly used for classifications of visual imagery. For example, if you were to build a network for the purpose of predicting a specific animal, it would hopefully output, with high fidelity, the correct classification of a new animal introduced to the model. Traditionally, hyper-parameters were rarely optimized because it required a lot of computational power and time. If hyper-parameters were adjusted, analysts would manually change a few hyper-parameters, re-run the model, and hopefully get a better classification accuracy. However, because of the advancements in technology, hyper-parameter tuning can now be done through complex and powerful optimization algorithms to improve the model. This paper implements and compares three different optimization techniques: random search, Bayesian Optimization with Gaussian Process, and tree of parzen estimator approach. The best performing technique is then improved through the Kiefer-Wolfowitz approximation.

AFIT Designator


DTIC Accession Number