Recently, deep learning has become a pervasive tool in prediction problems for structured and/or unstructured big data in various areas including science and engineering. In particular, deep neural network models (i.e. a basic core model of deep learning) can be viewed as an extension of statistical models by going through the incorporation of hidden layers. In this paper, we study the relationship between both models in terms of model structures and model learning. For this purpose, we also compare the predictive performances of both models, with two practical examples.