Date of Award

3-21-2019

Document Type

Thesis

Degree Name

Master of Science in Operations Research

Department

Department of Operational Sciences

First Advisor

Daniel W. Steeneck, PhD

Abstract

In the world of machine learning, neural networks have become a powerful pattern recognition technique that gives a user the ability to interpret high-dimensional data whereas conventional methods, such as logistic regression, would fail. There exists many different types of neural networks, each containing its own set of hyper-parameters that are dependent on the type of analysis required, but the focus of this paper will be on the hyper-parameters of convolutional neural networks. Convolutional neural networks are commonly used for classifications of visual imagery. For example, if you were to build a network for the purpose of predicting a specific animal, it would hopefully output, with high fidelity, the correct classification of a new animal introduced to the model. Traditionally, hyper-parameters were rarely optimized because it required a lot of computational power and time. If hyper-parameters were adjusted, analysts would manually change a few hyper-parameters, re-run the model, and hopefully get a better classification accuracy. However, because of the advancements in technology, hyper-parameter tuning can now be done through complex and powerful optimization algorithms to improve the model. This paper implements and compares three different optimization techniques: random search, Bayesian Optimization with Gaussian Process, and tree of parzen estimator approach. The best performing technique is then improved through the Kiefer-Wolfowitz approximation.

AFIT Designator

AFIT-ENS-MS-19-M-105

DTIC Accession Number

AD1077380

Download

Share

COinS