NEYRON TARMOQLARIDA FAOLLASHTIRISH FUNKSIYALARINING TAQQOSIY TAHLILI: SIGMOID, TANH VA RELU FUNKSIYALARINING NAZARIY VA AMALIY JIHATLARI
Keywords:
faollashtirish funksiyalari, neyron tarmoqlar, sigmoid funksiya, giperbolik tangens, to'g'rilangan chiziqli birlik, chuqur o'rganish, gradient tushish, yo'qolib borayotgan gradient muammosiAbstract
Ushbu maqola neyron tarmoqlardagi faollashtirish funksiyalarining keng qamrovli tadqiqotiga bag'ishlangan bo'lib, sigmoid funksiya, giperbolik tangens va to'g'rilangan chiziqli birlikka alohida e'tibor qaratilgan. Ish har bir faollashtirish funksiyasining matematik asoslarini ochib beradi, ularning hisoblash xususiyatlarini tahlil qiladi va zamonaviy chuqur o'rganish arxitekturalarida amaliy qo'llanilishini o'rganadi. Yo'qolib borayotgan gradient muammosi, hisoblash samaradorligi va turli faollashtirish funksiyalarining ma'lumotlardagi nochiziqli bog'liqliklarni modellashtirish qobiliyatiga alohida e'tibor beriladi. Tadqiqot faollashtirish funksiyasini tanlash yaqinlashish tezligi, model aniqligi va neyron tarmoqning umumiy ishlashiga sezilarli ta'sir ko'rsatishini ko'rsatadi. Tahlil natijalari har bir funksiyaning turli mashinali o'rganish vazifalarida, jumladan, tasvirlarni tasniflash, tabiiy tilni qayta ishlash va naqshlarni tanishda afzalliklari va cheklovlarini ko'rsatadi. Ish sun'iy intellekt sohasidagi tadqiqotchilar va amaliyotchilar uchun hal qilinayotgan muammoning o'ziga xosligi va neyron tarmoq arxitekturasiga qarab faollashtirish funksiyalarini optimal tanlash bo'yicha amaliy tavsiyalar beradi.
References
1. Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press, Cambridge, Massachusetts, 2016. 775 p.
2. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521(7553): 436-444.
3. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems. 2012; 25: 1097-1105.
4. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. 2011; 15: 315-323.
5. He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. Proceedings of the IEEE International Conference on Computer Vision. 2015; 1026-1034.
6. Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. Proceedings of the Twenty-seventh International Conference on Machine Learning. 2010; 807-814.
7. Maas AL, Hannun AY, Ng AY. Rectifier nonlinearities improve neural network acoustic models. Proceedings of the International Conference on Machine Learning. 2013; 30(1): 3.
8. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016; 770-778.
9. Clevert DA, Unterthiner T, Hochreiter S. Fast and accurate deep network learning by exponential linear units. International Conference on Learning Representations. 2016.
10. Klambauer G, Unterthiner T, Mayr A, Hochreiter S. Self-normalizing neural networks. Advances in Neural Information Processing Systems. 2017; 30: 971-980.
11. Ramachandran P, Zoph B, Le QV. Searching for activation functions. arXiv preprint arXiv:1710.05941. 2017.
