Optimization of deep learning hyperparameters to predict amphibious aircraft aerodynamic coefficients using grid search cross validation

Hyperparameter optimization is an important stage in modeling using machine learning, especially using deep learning algorithm, in which there are many combinations of parameters that will be used to produce a high-accuracy model. There are various methods for hyperparameter optimization of a model architecture, ranging from simple methods, namely trial error, to the most advanced and complex method, such as genetic algorithms. This paper will explain the hyperparameter optimization using the Grid Search Cross Validation (GSVC) method which is relatively simple but quite efficient in calculation time and produces an acceptable model accuracy. The hyperparameter to be optimized are optimizer and activation function. The options for optimizer are Adam, Adamax, Nadam, Stochastic Gradient Descent (SGD), and Root Mean Squared Propagation (RMSProp). The options for activation functions are TanH, Sigmoid, Elu and Relu. The options for optimizer and activation function are combined and selected using GSCV to provide the best performance at a given network architecture. The GSCV method will be used to optimize a model for aerodynamic coefficient of an amphibious aircraft (N219A) based on test data in wind tunnels, i.e. namely CL (coefficient of lift), C_D (coefficient of drag) and C_M25 (coefficient of bending moment). The best combination of hyperparameters for the aerodynamic coefficient model is (Adam & Sigmoid) for C_L, (Adam & Elu) for CD and (Adamax & Elu) for C_M25. The results of validation using testing data produce the following RMSE (Root Mean Square Error) and R² (R-Squared) i.e. (0.0137 & 0.9949) for CL; (0.0113 & 0.9969) for C_D and (0.02096 & 0.9984) for C_M25 show that the error of the prediction in C_L, C_D and C_M25 is acceptable and the relationship between independent and dependent variables are quite strong.