پیش بینی بارش با استفاده از شبکه عصبی عمیق (خودرمزگذار پشته‌ای تنک با نویززدا مبتنی بر نرون سخت)

نوع مقاله : مقاله پژوهشی

نویسندگان

1 پژوهشکده اقلیم شناسی-گروه بلایای جوی اقلیمی-مشهد-ایران

2 مدیر گروه مهندسی برق دانشگاه آزاد اسلامی مشهد

3 استادیار، پژوهشگاه هواشناسی علوم جو- پژوهشکده اقلیم شناسی و تغییر اقلیم

چکیده

در جهان امروز، توسعه سریع و پایدار، هدف اصلی تمامی کشورها می‌باشد. اصلیترین محدودیت پیش روی توسعه پایدار، محدودیتهای اقلیمی، از جمله بارش ناکافی همراه با پراکندگی نامناسب مکانی- زمانی است و بیشترین همبستگی را با حوادث ناگوار طبیعی دارد. یکی از روشهای مورد استفاده برای پیش‌بینی در حوزه‌های مختلف شبکه‌های عصبی مصنوعی می‌باشد که این شبکه‌ها به خاطر استفاده از معماری سطحی و کم‌عمق با ویژگیهای دستکاری شده ممکن است نتوانند دقت لازم را ارائه دهند. شبکه عصبی عمیق مشکلاتی مثل بیش‌برازش را برطرف می‌‌کند و همچنین هرچقدر عمق شبکه‌ها بیشتر باشند سطوح انتزاع بیشتری را یاد می‌گیرند. هدف اصلی این تحقیق، بالا بردن دقت پیش‌بینی بارش ساعتی منطقه خراسانرضوی با استفاده از یکی از روشهای شبکه عصبی‌عمیق است. در این تحقیق ما یک معماری شبکه عصبی عمیق با روش خودرمزگذار پشتهای نویززدا مبتنی بر نرون سخت بصورت تنک (RSDSAE) را برای پیشبینی بارش کوتاه مدت ارائه میدهیم. به منظور بهبود دقت، شبکههای عصبی سخت (RNNs) به عدم قطعیت بارش کمک میکنند و برای بهبود سرعت و صحت از ترکیب الگوریتم تنک (Spars) با مدل فوق مورد استفاده قرار گرفته است و همچنین از دادههای بارش پیشبینی شده خروجی مدل WRF استفاده کردیم و آزمایش‌ها بر روی دادههای باران و توسط دو معیار RMSE، MAE و با پنج خودرمزگذار به ترتیب 0.7912 و 0.7662 محاسبه گردیده است که توانسته عملکرد بهتری نسبت به مدل (RSDAE) از خود نشان دهد.



کلمات کلیدی: شبکه عصبی‌عمیق، پیشبینی بارش، خودرمزگذار تنک پشتهای نویززدا

کلیدواژه‌ها


عنوان مقاله [English]

Precipitation Forecasting with deep neural network (Rough Neuron Based stacked denoising sparse autoencoder )

نویسندگان [English]

  • sharareh malboosi 1
  • Syyed Javad Seyyed Mahdavi 2
  • Morteza Pakdaman 3
1 Climatology Research Institute-Climatic Disasters Group-Mashhad-Iran
2 Director of Electrical Engineering, Islamic Azad University of Mashhad
3 Assistant Professor, Climatological Research Institute (CRI), Mashhad, Iran
چکیده [English]

Abstract:

In today's world, rapid and sustainable development is the ultimate goal of all countries. The main constraint to sustainable development is the climatic conditions of countries, including rainfall, which is most correlated with natural disasters. One of the methods used for prediction is deep neural networks. The main purpose of this study is to increase the accuracy of hourly rainfall forecasting in Khorasan Razavi region using one of the deep neural network methods. In this study, we present a deep neural network architecture using the Ron Neuron-Based Random Neural Decomposition Stochastic (RSDSAE) method for short-term precipitation prediction. In order to improve the accuracy, rough neural networks (RNNs) contribute to the rainfall uncertainty and to improve the speed and accuracy, the combination of the sparse algorithm with the above model has been used and We also used the predicted output data output of the WRF model, and tests are calculated on rain data and by two criteria of RMSE, MAE and with five self-esteem, respectively, 0.7912 and 0.7662, and has been able to better performance than the model (RSDAE) of itself Show.

Keywords: Deep Neural Network, Precipitation Prediction, stacked denoising sparse autoencoder















1. Introduction:

Rainfall is a climatic factor that affects many human activities such as agriculture, construction, electricity generation, etc. Therefore, having a proper method for predicting rainfall can take preventive and mitigating measures for natural disasters. Over the past few years, deep neural networks (DNNs) have been used as a successful mechanism for solving complex problems in areas such as machine vision, image recognition, etc. (Hern´andez, et al., 2016). Deep neural network is a set of multi-layered architectures that have been trained using unsupervised algorithms and have challenges in training deep neural networks such as slow network training, overprocessing, and so on. Most existing neural network models have three drawbacks: (Khodayar, et al., 2017)

1- Most of the architecture is shallow.

2. Some methods require handmade engineering features that are tedious.

3- Most methods do not have direct knowledge about rainfall uncertainty.

In this study, to solve problems one and two, we combined the proposed method of stacked denoising sparse autoencoder (SDSAE) and to solve problem three, we combined the proposed method with Neuron Rough.



2. Materials and methods:

2-1. Data and area of study:

In this study, the study area is Khorasan Razavi province and the output of networked data is the WRF model, which is a mid-scale regional forecasting model. We considered eight days of regional precipitation hourly data as target variable data for training and testing algorithms. We prepared the data used from the output of two regional forecast model implementations (mid-scale atmospheric forecast model) on 26 and 28 October 2018 for each implementation with 4 days forecast.



2-2. Steps to perform the proposed method:

The proposed method has five main steps:

1) Receive WRF output prediction data,

2) Apply noise to input data.

3) Validation of deep network parameters.

4) Unsupervised learning with RSDAE to build deep multilayer network.

5) Learning by monitoring (fine tuning) by BP algorithm with SGD method





2-3. Evaluation criteria

To evaluate the proposed method SRDSAE and RSDAE, we use two criteria, mean squared error (RMSE), absolute mean error (MAE).



3. Research results:



Predicting rainfall is one of the most important and fundamental challenges for researchers. Properly predicting rainfall improves the lives of the general public and even the proper planning of governments for the correct use of rainfall. Various algorithms and methods have been proposed for forecasting issues in recent years. Deep learning methods are one of the most popular methods for predictive problems. In this research, we have used the stacked denoising autoencoder method to predict rainfall and combine this method with neuron-rough and sparse algorithm to increase the accuracy of the forecast. In this research, we applied the proposed method on the output data of the WRF mid-scale forecast model in Khorasan Razavi province.

The proposed RSDSAE(Rough Neuron Based stacked denoising sparse autoencoder) method and RSDAE (Rough Neuron Based stacked denoising autoencoder) method have been compared and analyzed for better comparison in precipitation forecasting based on two evaluation parameters RMSE and MAE. The proposed method has been able to reduce the forecast error by reducing the RSDAE network and subsequently reduce the RMSE and MAE criteria, or in other words, improve the rainfall forecast.



References:

1- Hern´Andez, E. et al., 2016. Rainfall Prediction: A Deep Learning Approach. in International Publishing Switzerland , Springer, pp. HAIS, LNAI 9648, pp. 151–162.

2- Khodayar, M., Kaynak, O. & Khodayar, M. E., 2017. Rough Deep Neural Architecture for Short-term Wind Speed Forecasting. IEEE, pp. 1551-3203 (c).

3- Kumar, V., Nandi, G. & Kala, R., 2014. Static Hand Gesture Recognition using Stacked Denoising Sparse Autoencoders. Robotics and Artificial Intelligence Laboratory.IEEE, pp. 978-1-4799-5173-4).

کلیدواژه‌ها [English]

  • Deep Neural Network
  • Precipitation Prediction
  • stacked denoising sparse autoencoder
  • Ahmadi, A. , Daraiee, M. , Salamzadeh, A & Jafary, M. 1392. Artificial intelligence and business opportunities: identifying the functionsof artificial intelligence in creating a competitive advantage for technology businesses(a study of the computer games industry).Publication in entrepreneurship development, 6th period,2nd issue, pp.7-26.
  • Russakovsky , O. et al., 2015. ImageNet Large Scale Visual Recognition Challenge.. International Journal of Computer Vision, pp. 115(3):211-52.
  • Voulodimos, N. D. A. D. a. E. P., 2018. Deep Learning for Computer Vision: A Brief Review. Hindawi, Computational Intelligence and Neuroscience, p. 7068349.
  • Alain, G. & Bengio , Y., 2014. What regularized auto-encoders learn from the data-generating distribution.. The Journal of Machine Learning Research, pp. 15(1):3563-93.
  • Barcelona & Memisevic , R., 2011. Gradient-based learning of higher-order image features.. IEEE international conference on computer vision (ICCV), p. 1591–1598.
  • Bengio , Y., Courville , A. & Vincent , P., IEEE Transactions on 2013. Representation learning: A review and new perspectives.. Pattern Analysis and Machine Intelligence, pp. 35(8):1798-828.
  • Bishop, C. M., 2007. Pattern Recognition and Machine Learning. New York: (Information Science and Statistics), Springer.
  • Chandra B, S. R., 2014. Adaptive noise schedule for denoising autoencoder. Springer International Publishing,, pp. 535-542.
  • Deng, J., Zhang, Z., Marchi, E. & Schuller, B., 2013. Sparse autoencoder based feature transfer learning for speech emotion recognition.. IEEE, New York, pp. 511-516.
  • Ganin, Y. et al., 2016. Domain-adversarial training of neural networks. Mach Learn Res, p. 17:1–35.
  • Guo, Y. et al., 2016. Deep learning for visual understanding, A review. Neurocomputing 187, p. 27–48.
  • Hern´andez, E. et al., 2016. Rainfall Prediction: A Deep Learning Approach. in International Publishing Switzerland , Springer, pp. HAIS, LNAI 9648, pp. 151–162.
  • Hinton, G. E., Osindero, S. & Teh, Y.-W., 2006. A fast learning algorithm for deep belief nets. Neural Computation, vol. 18, no. 7, p. pp. 1527–1554.
  • Hinton, G. E. & Salakhutdinov, R. R., 2006. Reducing the dimensionality of data with neural networks. Science 313 (5786), p. 504–507.
  • Ibrahim, M., Ibrahim, F. & Al-Jumaily, A. A., 2018. Auto-Encoder based Deep Learning for Surface Electromyography Signal Processing. Advances in Science, Technology and Engineering Systems Journal, Volume Vol. 3, No. 1, pp. 94-102.
  • Ioffe, S. & Szegedy, C., 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. International Conference on Machine Learning, pp. 37: 448-56.
  • Jiang, X., Zhang, Y., Zhang, W. & Xiao, X., 2013. A novel sparse auto-encoder for deep unsupervised learning. in: Proceedings of the ICACI.
  • Jordan, M. I. & Mitchell, T. M., 2015. Machine learning: trends, perspectives, and prospects. Science, p. 349:255–60.
  • Khodayar, M., Kaynak, O. & Khodayar, M. E., 2017. Rough Deep Neural Architecture for Short-term Wind Speed Forecasting. IEEE, pp. 1551-3203 (c).
  • Kumar, V., Nandi, G. & Kala, R., 2014. Static Hand Gesture Recognition using Stacked Denoising Sparse Autoencoders. Robotics and Artificial Intelligence Laboratory.IEEE, pp. 978-1-4799-5173-4.
  • LeCun, Y., Bengio, Y. & Hinton, G., 2015. Deep learning. Nature, p. 521:436–44.
  • Lee MK, H. D., 2012. Dimensionality reduction of radio map with nonlinear autoencoder. Electron Lett, 48(11), pp. 655-657.
  • Lemme A, R. R. S. J., 2012. Online learning and generalization of parts-based image representations.. Neural Netw, Volume 33, pp. 194-203.
  • Lin, S.-Y., Chiang, C.-C., Hung, Z.-S. & Zou, Y.-H., 2017. A Dynamic Data-Driven Fine-Tuning Approach for Stacked Auto-Encoder Neural Network. s.l., The Fourteenth IEEE International Conference on e-Business Engineering, Department of Information Management.
  • Liu J, C. G. L. Z. L. Y. L. H. L. X., 2013. Predicting protein structural classes with autoencoder neural networks. IEEE, Piscataway, pp. 1894-1899.
  • Liu, R., Yang, B., Zio, E. & Chen, X., 2018. Artificial intelligence for fault diagnosis of rotating machinery: A review.. Mechanical Systems and Signal Processing 108, p. 33–47.
  • Meng L, 2016. Research of stacked denoising sparse autoencoder. Springer.
  • Meng, L., Ding, S., Zhang, N. & Zhang, J., 2016. Research of stacked denoising sparse autoencoder. Springer,The Natural Computing Applications Forum.
  • Mesnil, G. et al., 2012. Unsupervised and Transfer Learning Challenge: a Deep Learning Approach. ICML Unsupervised and Transfer Learning.. pp. 27:97-110.
  • Miotto, R. et al., 2017. Deep learning for healthcare: review, opportunities and challenges. Published by Oxford University Press , Oxford , Briefings in Bioinformatics, p. 1–11.
  • M, K., 2017. Rough Deep Neural Architecture for Short-term Wind Speed Forecasting,. IEEE Okyay Kaynak, pp. 1551-3203.
  • Murphy, K. P., 2012. Machine learning: a probabilistic perspective. MIT press.
  • Olshausen , B. & Field , D., 1997. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision research.. pp. 37(23):3311-25.
  • OLSHAUSEN, B. A. & FIELD, D. . J., 1997. Sparse Coding with an Overcomplete Basis Set: Strategy Employed by V1?. Pergamon, Volume vol. 37, no. 23, pp. pp. 3311-3325.
  • Lingras, 1996. “Rough Neural Networks,”. IPMU, pp. 1445-1450.
  • Quoc, V. L., 2013. Building high-level features using large scale unsupervised learning. in: Proceedings of the ICASSP.
  • Raina , R., Battle , A., Lee , H. & Packer , B., 2007. Self-taught learning: transfer learning from unlabeled data.. s.l., s.n.
  • Rifai , S. et al., 2011. Contractive auto-encoders: Explicit invariance during feature extraction. s.l., s.n.
  • Sagha, H., Cummins, N. & Schuller, B., 2017. Stacked denoising autoencoders for sentiment analysis: a review. WIREs Data Mining Knowl Discov, p. e1212.
  • Shen, C., 2017. A trans-disciplinary review of deep learning research for water resources scientists. Civil and Environmental Engineering,Pennsylvania State University, University Park, p. PA16802.
  • Srivastava, N. et al., 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, pp. 15: 1929-58.
  • Tan CC, E. C., 2011. Using autoencoders for mammogram compression.. J Med Syst, 35(1), p. 4958.
  • Vidhya, K. & Geetha, T., 2017. Rough set theory for document clustering:A review. Journal of Intelligent & Fuzzy Systems 32, p. pp 2165–2185.
  • Vincent, . P., Larochelle, H., Bengio, Y. & Manzago, P.-A., 2008. Extracting and composing robust features with denoising autoencoder. New York, USA l, ACM Press, p. pp. 1096–1103.
  • Wu , R. et al., 2015. Deep Image: Scaling up Image Recognition.. CoRR., p. abs 1501.02876.
  • Bengio, P. L. D. P. a. H. L., 2007. “Greedy layerwise layerwise training of deep networks,”. s.l., s.n.
  • Yang, . J., Yu , K., Gong , Y. & Huang, . T., 2009. Linear spatial pyramid matching using sparse coding for image classification.. s.l., s.n.
  • Zhang, J., Shan, S., Kan, M. & Chen, X., 2014. Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment. in: Proceedings of the ECCV.
  • Zhang, T., Liu, D. & Yue, D., 2017. Rough Neuron based RBF Neural Networks for Short-Term Load Forecasting. s.l., s.n.
  • Zhou, Y., Arpit, D., Nwogu, I. & Govindaraju, V., 2014. Is joint training better for deep autoencoders?. pp. 1405,1380.
  • Zuo,. H. et al., 2015. Transfer learning in hierarchical feature spaces. Taipei, Taiwan, ACM, p. 183–188.