Automatic Ship Detection and Classification using Machine Learning from Remote Sensing Images on Apache Spark
Öz
Ship detection and classification is very important for port and coastal security. Due to maritime safety and traffic control, high-resolution images of ships should be obtained. High resolution color remote sensing ship images taken from short distances provide advantages in ship detection applications. But the analysis of these high-dimensional images is complicated and requires long time. Dividing the image data into smaller blocks and representing them with a vector with distinctive and independent features facilitates the analysis process. For this reason, a block division method is applied first, dividing the image data into small pixel blocks. These obtained image blocks are also represented by the hybrid feature vectors. These feature vectors are created by adding the sub-features extracted from the color and texture properties of the images one after another. Using the obtained hybrid vectors, the images are classified using machine learning methods on Apache Spark. Classification studies were realized using Naive Bayes, Decision Trees and Random Forest methods in the MLlib. The analysis of the images was realized much faster with the clustering architecture created on Apache Spark platform. According to the obtained classification results, 99.62% classification success was achieved by using Random Forest method. In addition, an average of 3.4 times acceleration was achieved by running each method on 1 master + 4 workers clustering architecture on Spark. The analysis results obtained are presented in detail in the experimental studies section.
Anahtar Kelimeler
Apache Spark Classification Clustering Machine Learning Remote Sensing Ship Detection
Destekleyen Kurum
Scientific Research Projects Unit of Karabuk University
Proje Numarası
FYL-2019-2044
Teşekkür
This work was supported by the Scientific Research Projects Unit of Karabuk University under project number FYL-2019-2044.
Kaynakça
- Bi, F., Hou, J., Chen, L., Yang, Z., Wang, Y., 2019. “Ship Detection for Optical Remote Sensing Images Based on Visual Attention Enhanced Network”. Sensors, 8, 4634-4646, 2015.
- Cavallaro G, Riedel M, Richerzhagen M, Benediktsson JA, Plaza A. “On Understanding Big Data Impacts in Remotely Sensed Image Classification Using Support Vector Machine Methods”. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8, 4634-4646, 2015.
- Chen, Z., Chen D., Zhang Y., Cheng X., Zhang M., 2020. “Deep learning for autonomous ship-oriented small ship detection”. Safety Science, 130.
- Cortes C, Vapnik Vladimir. “Support-Vector Networks”, Machine Learning, 20, 273-297 (1995).
- Ergül M, Alatan AA. “Geospatial Object Recognition From High Resolution Satellite Imagery”. 2013 21st Signal Processing and Communications Applications Conference (SIU), Haspolat, Turkey, 24-26 April 2013.
- Gonzalez RC, Woods RE, Eddins SL. Digital Image Processing using Matlab. New Jersey, Prentice Hall, 2003.
- Han J, Kamber M, Pei J. Data Mining: Concepts and Techniques. Waltham, MA, USA: Elsevier, Second Edition, 2006
- Kanjir, U., Greidanus, H., Ostir, K., 2018. “Vessel detection and classification from spaceborne optical images: A literature survey”. Remote Sensing of Environment, 207 (1-26).
- Kavzaoğlu T, Çölkesen İ. “Karar Ağaçları ile Uydu Görüntülerinin Sınıflandırılması: Kocaeli Örneği”, Electronic Journal of Map Technologies, 2, 2010.
- Kaya Ç, Yıldız O. “Makine Öğrenmesi Teknikleriyle Saldırı Tespiti: Karşılaştırmalı Analiz”. Marmara Fen Bilimleri Dergisi
- Li, H. Chen, L., Li, F., Huang M., 2019. “Ship detection and tracking method for satellite video based on multiscale saliency and surrounding contrast analysis”. Applied Remote Sensing 13 (2).
- Li Y, Zhang H, Guo Q, Li X. “Machine Learning Methods for Ship Detection in Satellite Images”.
- Liu, Y., Cuı, H.Y., Kuang, Z., Lı, G.Q., 2017. ITM Web of Conferences, 12.
- Man W, Ji Y, Zhang Z. “Image Classification Based on Improved Random Forest Algorithm”, 2018 the 3rd IEEE International Conference on Cloud Computing and Big Data Analysis, 20-22 April 2018, Chengdu, China.
- Meng, X., Bradley, J., Yavuz, B., Sparks, E., Venkataraman, S., Liu, L., et al. (2015). MLlib: Machine Learning in Apache Spark.
- Morillas JRA, Garsia IC, Zölzer U. “Ship Detection Based on SVM Using Color and Texture Features”. 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3-5 September 2015.
- Nie T, He B, Bi G, Zhang Y, Wang W. “A Method of Ship Detection under Complex Background”. International Journal of Geo-Information, 2017.
- Oğul İÜ, Özcan C, Hakdağlı Ö. “Fast Text Classification with Naive Bayes Method on Apache Spark”. 2017 25th Signal Processing and Communications Applications Conference (SIU), Antalya, Turkey.
- Oğul İÜ, Özcan C, Hakdağlı Ö. “Text Classification with Spark Support Vector Machine”. 1. Ulusal Bulut Bilişim ve Büyük Veri Sempozyumu, Antalya, 2017.
- Özcan C, Ersoy O, Oğul İÜ. “Classification of SAR Image Patches with Apache Spark Using GLCM Texture Features”. International Conference on Advanced Technologies, 3rd World Conference on Big Data, İzmir, 28 - 30 Nisan 2018.
- Özcan C, Ersoy O, Oğul İÜ. “Fast texture classification of denoised SAR image patches using GLCM on Spark”, Turkish Journal of Electrical Engineering & Computer Sciences,28, 2020.
- Temizkan E, Bilge HŞ. “Airport Detection by Combining Geometric and Texture Features on RASAT Satellite Images”. 2017 25th Signal Processing and Communications Applications Conference (SIU), Antalya, Turkey, 15-18 May 2017.
- Wang N, Chen F, Yu B, Qin Y. “Segmentation of large-scale remotely sensed images on a Spark platform: A strategy for handling massive image tiles with the MapReduce model”. ISPRS Journal of Photogrammetry and Remote Sensing, 162, 137-147, 2020.
- Wang. Z, Yang T., Zhang H. 2020. “Land contained sea area ship detection using spaceborne image”. Pattern Recognition Letters, 130 (125-131).
- Yang, X., Sun, H., Fun, K., Yang, J., Sun, X., Yan, M., Guo, Z., 2018. “Automatic Ship Detection of Remote Sensing Images from Google Earth in Complex Scenes Based on Multi-Scale Rotation Dense Feature Pyramid Networks”. Remote Sens, 132, 10.
- Yuan, Q., Shen, H., Li, T., Li, Z., Li, S., Jiang, Y., Xu, H., Tan, W., Yang Q., Wang, J., Gao, J., Zhang, L., 2020. “Deep learning in environmental remote sensing: Achievements and challenges”. Remote Sensing of Environment, 241.
- Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Medjahed I. “Spark: Cluster Computing with Working Sets”.
Apache Spark Makine Öğrenimi Kullanılarak Uzaktan Algılama Görüntülerinden Otomatik Gemi Tespiti ve Sınıflandırma
Öz
Gemi tespiti ve sınıflandırması, liman ve kıyı güvenliği açısından çok önemlidir. Deniz güvenliği ve trafik kontrolü nedeniyle, gemilerin yüksek çözünürlüklü görüntülerinin elde edilmesi gerekmektedir. Kısa mesafeden çekilmiş yüksek çözünürlüklü renkli uzaktan algılama gemi görüntüleri, gemi tespiti uygulamalarında avantaj sağlamaktadır. Fakat yüksek boyutlu bu görüntülerin analiz edilmesi süreci karmaşık ve uzun süreler gerektirmektedir. Görüntü verilerinin daha küçük parçalara bölünmesi ve bu parçalardan elde edilen ayırt edici ve bağımsız özelliklere sahip bir vektörle temsil edilmesi analiz işlemini kolaylaştırmaktadır. Bu nedenle, öncelikle görüntü verilerini küçük piksel bloklarına bölen bir blok bölümü yöntemi uygulanır. Elde edilen bu görüntü bloklarının da hibrit bir öznitelik vektörleri ile temsil edilmesi gerçekleştirilir. Bu öznitelik vektörleri, görüntülerin renk ve doku özelliklerinden çıkarılan alt özelliklerin birbiri ardına eklenmesi ile oluşturulur. Elde edilen hibrit vektörler Apache Spark'daki makine öğrenmesi yöntemleri ile kullanılarak görüntülerin sınıflandırılması sağlanmıştır. MLlib kütüphanesinde bulunan Naif Bayes, Karar Ağaçları ve Rastgele Orman yöntemleri kullanılarak sınıflandırma çalışmaları gerçekleştirilmiştir. Görüntülerin Apache Spark ortamında analiz edilmesi oluşturulan kümeleme mimarisi ile çok daha hızlı bir şekilde gerçekleştirilmiştir. Ayrıca her bir yöntemin Spark 1 master + 4 worker kümeleme mimarisi üzerinde çalıştırılması sonucu ortalama 3.4 kata yakın hızlanma sağlanmıştır.
Anahtar Kelimeler
Apache Spark Sınıflandırma Kümeleme Makine Öğrenmesi Uzaktan Algılama Gemi Tespiti
Proje Numarası
FYL-2019-2044
Kaynakça
- Bi, F., Hou, J., Chen, L., Yang, Z., Wang, Y., 2019. “Ship Detection for Optical Remote Sensing Images Based on Visual Attention Enhanced Network”. Sensors, 8, 4634-4646, 2015.
- Cavallaro G, Riedel M, Richerzhagen M, Benediktsson JA, Plaza A. “On Understanding Big Data Impacts in Remotely Sensed Image Classification Using Support Vector Machine Methods”. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8, 4634-4646, 2015.
- Chen, Z., Chen D., Zhang Y., Cheng X., Zhang M., 2020. “Deep learning for autonomous ship-oriented small ship detection”. Safety Science, 130.
- Cortes C, Vapnik Vladimir. “Support-Vector Networks”, Machine Learning, 20, 273-297 (1995).
- Ergül M, Alatan AA. “Geospatial Object Recognition From High Resolution Satellite Imagery”. 2013 21st Signal Processing and Communications Applications Conference (SIU), Haspolat, Turkey, 24-26 April 2013.
- Gonzalez RC, Woods RE, Eddins SL. Digital Image Processing using Matlab. New Jersey, Prentice Hall, 2003.
- Han J, Kamber M, Pei J. Data Mining: Concepts and Techniques. Waltham, MA, USA: Elsevier, Second Edition, 2006
- Kanjir, U., Greidanus, H., Ostir, K., 2018. “Vessel detection and classification from spaceborne optical images: A literature survey”. Remote Sensing of Environment, 207 (1-26).
- Kavzaoğlu T, Çölkesen İ. “Karar Ağaçları ile Uydu Görüntülerinin Sınıflandırılması: Kocaeli Örneği”, Electronic Journal of Map Technologies, 2, 2010.
- Kaya Ç, Yıldız O. “Makine Öğrenmesi Teknikleriyle Saldırı Tespiti: Karşılaştırmalı Analiz”. Marmara Fen Bilimleri Dergisi
- Li, H. Chen, L., Li, F., Huang M., 2019. “Ship detection and tracking method for satellite video based on multiscale saliency and surrounding contrast analysis”. Applied Remote Sensing 13 (2).
- Li Y, Zhang H, Guo Q, Li X. “Machine Learning Methods for Ship Detection in Satellite Images”.
- Liu, Y., Cuı, H.Y., Kuang, Z., Lı, G.Q., 2017. ITM Web of Conferences, 12.
- Man W, Ji Y, Zhang Z. “Image Classification Based on Improved Random Forest Algorithm”, 2018 the 3rd IEEE International Conference on Cloud Computing and Big Data Analysis, 20-22 April 2018, Chengdu, China.
- Meng, X., Bradley, J., Yavuz, B., Sparks, E., Venkataraman, S., Liu, L., et al. (2015). MLlib: Machine Learning in Apache Spark.
- Morillas JRA, Garsia IC, Zölzer U. “Ship Detection Based on SVM Using Color and Texture Features”. 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3-5 September 2015.
- Nie T, He B, Bi G, Zhang Y, Wang W. “A Method of Ship Detection under Complex Background”. International Journal of Geo-Information, 2017.
- Oğul İÜ, Özcan C, Hakdağlı Ö. “Fast Text Classification with Naive Bayes Method on Apache Spark”. 2017 25th Signal Processing and Communications Applications Conference (SIU), Antalya, Turkey.
- Oğul İÜ, Özcan C, Hakdağlı Ö. “Text Classification with Spark Support Vector Machine”. 1. Ulusal Bulut Bilişim ve Büyük Veri Sempozyumu, Antalya, 2017.
- Özcan C, Ersoy O, Oğul İÜ. “Classification of SAR Image Patches with Apache Spark Using GLCM Texture Features”. International Conference on Advanced Technologies, 3rd World Conference on Big Data, İzmir, 28 - 30 Nisan 2018.
- Özcan C, Ersoy O, Oğul İÜ. “Fast texture classification of denoised SAR image patches using GLCM on Spark”, Turkish Journal of Electrical Engineering & Computer Sciences,28, 2020.
- Temizkan E, Bilge HŞ. “Airport Detection by Combining Geometric and Texture Features on RASAT Satellite Images”. 2017 25th Signal Processing and Communications Applications Conference (SIU), Antalya, Turkey, 15-18 May 2017.
- Wang N, Chen F, Yu B, Qin Y. “Segmentation of large-scale remotely sensed images on a Spark platform: A strategy for handling massive image tiles with the MapReduce model”. ISPRS Journal of Photogrammetry and Remote Sensing, 162, 137-147, 2020.
- Wang. Z, Yang T., Zhang H. 2020. “Land contained sea area ship detection using spaceborne image”. Pattern Recognition Letters, 130 (125-131).
- Yang, X., Sun, H., Fun, K., Yang, J., Sun, X., Yan, M., Guo, Z., 2018. “Automatic Ship Detection of Remote Sensing Images from Google Earth in Complex Scenes Based on Multi-Scale Rotation Dense Feature Pyramid Networks”. Remote Sens, 132, 10.
- Yuan, Q., Shen, H., Li, T., Li, Z., Li, S., Jiang, Y., Xu, H., Tan, W., Yang Q., Wang, J., Gao, J., Zhang, L., 2020. “Deep learning in environmental remote sensing: Achievements and challenges”. Remote Sensing of Environment, 241.
- Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Medjahed I. “Spark: Cluster Computing with Working Sets”.
Ayrıntılar
| Birincil Dil | İngilizce |
|---|---|
| Konular | Yapay Zeka |
| Bölüm | Araştırma Makalesi |
| Yazarlar | |
| Proje Numarası | FYL-2019-2044 |
| Yayımlanma Tarihi | 23 Eylül 2021 |
| Gönderilme Tarihi | 21 Temmuz 2020 |
| Yayımlandığı Sayı | Yıl 2021 Cilt: 4 Sayı: 2 |
Kaynak Göster
Cited By
Survey on Deep Learning-Based Marine Object Detection
Journal of Advanced Transportation
https://doi.org/10.1155/2021/5808206
Deep Learning based Ship Variants Classification Using Different Scale Images
Journal of Intelligent Systems: Theory and Applications
https://doi.org/10.38016/jista.1118740
Makine Öğrenme Teknikleri Kullanılarak Kükürt Giderme İşleminde Kullanılan Malzeme Miktarının Tahmini
Journal of Intelligent Systems: Theory and Applications
https://doi.org/10.38016/jista.993853
Zeki Sistemler Teori ve Uygulamaları Dergisi