IoT for the Maritime Industry: Challenges and Emerging Applications

—The Internet of things (IoT) ecosystem provides a platform for the connectivity of interrelated smart devices to automate manual processes and reduce labor costs. IoT has brought signiﬁcant beneﬁts to all industries, including maritime, as various objects (e.g., ports, ships, agents, etc.) are connected to gather and share information within the maritime ecosystem. The innovative technological aspects of IoT are promoting the effective collaboration between the research community and the maritime industry, for enhancing the performance of maritime transportation systems. Therefore, this study discusses recent advances delivered by the IoT and other emerging technologies, like machine learning (ML) and computer vision (CV), for smart maritime transportation systems (SMTSs). In particular, this paper presents two speciﬁc use cases of SMTSs, namely, predictive maintenance and container damage/seal inspection. Moreover, the key beneﬁts of integrating IoT with ML and CV are highlighted for the above-mentioned use cases. Finally, a discussion is presented to highlight key opportunities along with foreseeable future challenges in adopting these new technologies by the maritime industry.


I. INTRODUCTION
M ARITIME transportation is fundamental to the global supply chain.Conventional transportation systems are not inter-or intra-connected.However, with the advancements of information and communication technologies, the concept of connecting everything into smart maritime transportation systems (SMTSs) has emerged [1].SMTSs use computing, telecommunication, radio-location, internet of things (IoT), and/or automation technologies to improve the performance, management, and safety of transportation systems [2].In addition, SMTSs communicate information to maritime stakeholders about congestion levels, alternative modes of transportation, or alternate routes.Smart containers are also an essential component of SMTSs, as they enable fine-grained tracking of goods across continents [3].A terminal is considered "digital" or "smart" if all of its objects are fully interconnected via different communication technologies, e.g., IoT, cloud, etc., to exchange information.A combination of smart sensors and actuators, wireless devices, data centers, and other IoT-based port services form the main infrastructure of smart ports, This research was funded by the European Union's Horizon Europe program for Research and Innovation through the aerOS project under Grant No. 101069732.
enabling port authorities to provide more reliable information and various new services to their customers.
A real-time smart port at Xiamen Ocean Gate has been implemented [4] as the first automated container terminal in China that follows the standards of a global automated terminal handling system.Furthermore, it is considered as the first terminal in the world to deploy 5G mobile networks and cloud technologies in its operations [5].The authors of [6] also proposed several solutions for digitalization in global supply chains, connecting cities and ports via maritime informatics, and sustainable shipping.Furthermore, they also argue that the future of shipping relies on exploiting collaboration via digital data sharing.European ports have also taken initiatives to become smart ports [7].For example, the Port of Rotterdam is using IoT-based sensors to enable advanced intelligence and create digital twins, the Port of Hamburg is using 5G-based networks to monitor the critical infrastructure by enabling virtual reality, while the Port of Seville is using mobile network technology to monitor goods and traffic in the port in real-time.
Moreover, the sustainability of short sea shipping (SSS) is central to a clean, safe, and efficient European Union (EU) transport system, researched by related Sea Traffic Management (STM) EU projects (such as STMV and STEAM).The study presented in [8] reports on key challenges for advancing reliability, quality, and safety, and removing unnecessary costs and delays at SSS hubs, with a particular focus on Eastern Mediterranean.Specifically, it considers the effect of port-2port (P2P) communication on port efficiency by investigating the factors influencing the various waiting times at the Port of Limassol, Cyprus, both qualitatively and quantitatively.Finally, measures are proposed for improving agent performance based on the principles of Port Collaborative Decision Making, including P2P communication, data sharing, and transparency among all stakeholders involved in a port call process, and open dissemination of agent-specific key performance indicators (KPIs).
Cargo handling at the Port of Limassol is currently performed by several quay cranes and straddle carriers.These cranes are heavy machines that have multiple subsystems running internally that are managed by different programmable logic controllers (PLCs).The PLCs take input directly from the cranes, execute specific programmed logic, and produce output that allows terminal personnel to operate the crane.PLCs are therefore the most accurate representation of a port crane's status [9].However, Big Data, artificial intelligence (AI)/ machine learning (ML), and IoT technologies, currently rely on remote servers and/or cloud technologies.So there is a gap between where the data is most accurate, and where the analysis and predictions are performed.This mismatch impacts real-time observability at the upper levels of the terminal system due to higher latency between the central server and the source from which the data originates.These constraints limit the efficiency of the terminal, as it cannot prevent possible unforeseen problems such as outages.Furthermore, there is a need to combine latest technologies with the IoT paradigm in order to enhance the performance of maritime ports.For instance, combination of IoT and ML/deep learning (DL) can enhance the lifetime of maritime equipment and reduce costs through accurate predictive maintenance (PdM).In another scenario, IoT with computer vision (CV) can help reduce human intervention in risky environments, e.g., security seal inspection and damage detection for containers during loading/unloading.
Therefore, this study focuses on how IoT can benefit the maritime industry, especially when combined with other emerging technologies.First, we highlight the research challenges and future directions for the adoption of IoT by the maritime industry.Then, we discuss two specific use cases of SMTSs, namely, PdM and container damage/seal inspection, where IoT is combined with ML/AI and computer vision technologies, respectively.

II. CHALLENGES FOR IOT ADOPTION
In the shipping industry, a huge amount of data is generated from various sources and formats.The various sources include traffic data, weather data, port call data, cargo data, water level data, and maritime equipment data.In this section, the main open issues related to the deployment of IoT in the maritime industry are presented below.Real-time data collection and transfer: Data collection and transmission are considered two of the main issues in the use of IoT, ML, and CV technologies in the maritime industry.This is because the data collected by current technologies may be incorrect, unreliable, or incomplete in certain locations or at certain times due to the frequent movement of ships at sea.In addition, a ship is usually equipped with multiple sensors that generate a huge amount of data, and transmitting the data to data centers for further processing is inefficient, creating a major challenge and uncertainty.For example, each sensor requires a particular bandwidth to transmit the data to the database.Hence, new sensor technologies are needed to improve data quality, and high-tech IoT-based communication systems to speed up the data transmission speed [10].Also, automating data acquisition instead of manual input will improve the quality of data.Security and privacy concerns: Although modern maritime transportation systems benefit greatly from IoT and other communication technologies, security and reliability risks have increased significantly [11].The involvement of various maritime stakeholders in planning and managing maritime traffic flows further exacerbates these challenges.Therefore, there is an urgent need to develop an IoT-based collaborative processing system that unifies the modular structure and integrates multiple modules involved in maritime transportation systems.In addition, a common and controlled access mechanism that cannot be manipulated or tampered with by unauthorized parties is also an essential requirement for SMTSs.Moreover, intercommunication and the integration of heterogeneous technologies into IoT-enabled SMTSs offer opportunities not only for the industries but also for cyber criminals.Cyber threat intelligence is an effective security strategy that uses AI models to understand cyber-attacks and can effectively protect IoT-enabled data [12].In addition, the research community must take advantage of the latest technologies (e.g., ML, CV, blockchain, etc.) by combining them with IoT-assisted systems to find secure and reliable solutions for SMTSs at sea.Big Data: In the maritime industry, huge amounts of data are generated from various sources every day.For example, Marine Traffic, an automatic identification system (AIS) vessel tracking website, reports collecting 520 million AIS messages daily involving 180 thousand distinct vessels from 3000 active AIS stations worldwide [13].Port authorities and various port stakeholders (e.g., cargo terminals, tug operators) also collect data on the arrival, berthing, loading, unloading, relocation, anchoring, and departure of ships from ports [8], [14].Various sensors are also deployed at sea to record data on various oceanographic, environmental, and meteorological parameters of interest, with data volumes reaching up to 5 GB per day [15].Therefore, developing and designing a reliable and suitable storage architecture to meet the requirements of Big Data in the maritime industry has become a major challenge.Another challenge is the timeliness of processing maritime big data.In the maritime industry, fast and accurate decisions are needed to avoid hazards that relies on fast data processing.Hence, big data processing systems should be able to handle diverse and huge amounts of data, which are constantly increasing.There are several general big data frameworks for processing real-time data, such as Apache Kafka, Elasticsearch, OpenSearch, MongoDB, etc. So, developing an efficient big data framework that can process large maritime data sets is still an open challenge.
Emerging technologies: To ensure higher efficiency of SMTSs, new and emerging technologies must be adopted.In this regard, frugal AI, CV, ML, and IoT, especially when integrated with each other, can play a crucial role in reducing costs and increasing efficiency in smart ports.By using frugal AI models, the system can be lighter and less computationally intensive, which can be beneficial in situations where resources are limited.To prevent cyberattacks on SMTSs, satellite IoT and high-altitude platform solutions can be deployed.With increased GPS jamming and spoofing attacks on maritime systems, satellite IoT can serve as a complementary long-range Fig. 1: A view of the container terminal at the Port of Limassol, Cyprus solution and thus be beneficial in many ways [16].

III. IOT FOR PREDICTIVE MAINTENANCE
Container handling equipment (such as ship-to-shore (STS) cranes, gantry cranes, straddle carriers, or reach stackers) require an optimal maintenance process due to the heavy loads, long operating times, and diverse weather conditions, motivating the need for a precise PdM system.Figure 1 shows the container terminal with several cranes and other equipment at the Port of Limassol, Cyprus.Preventive maintenance has traditionally been performed using supervisory control and data acquisition (SCADA) systems set up with human-coded thresholds, warning rules, and configurations to determine when a machine's condition requires repair or even replacement.However, this semi-manual approach does not take into account the more complex dynamic behavior of the machines, nor the contextual data that relates to the operational process as a whole.For this reason, and thanks to recent advances in AI and IoT, the implementation of ML-based solutions is seen as the next functional step that can lead to significant cost savings, higher predictability, and better availability.PdM results in up-to-the-minute knowledge of health status, which allows neither waiting for equipment to shut down (i.e., reactive maintenance) nor performing maintenance when it is not required (i.e., preventive maintenance).In PdM, the IoT can play an important role in predicting faults/failures at an early stage.For example, a number of appropriate sensors can be deployed in machinery/engines to minimize the risk of negligent failures of key components and increase their efficiency [17].More comprehensive monitoring provides realtime information such as cargo temperature, gas emissions, and other important data that can help optimize operations, reduce maintenance costs, and increase the safety of the entire ecosystem [18].
Based on the current literature, a significant amount of work has already been carried out regarding ML and IoT approaches to solve PdM problems in various industrial environments.This, along with previous research on how the effectiveness of quay container cranes is affected by reduced speed and breakdowns [19] lies the basis for developing PdM approaches (proposed by combining IoT with ML models) applied to port machinery (such as STS cranes and straddle carriers).In addition, most ML projects require sufficient historical data to help understand past failures.This includes general characteristics such as mechanical properties, average usage, and operating conditions.However, even when sufficient data is available, the selection, training, and inference process of the most appropriate ML model for the computational capabilities of the pilot project's infrastructure elements is paramount.Although cloud computing can support predictive analytics solutions, running these models on a remote server introduces potential latency issues that can lead to delayed response times, depending on the quality of connectivity to send data from the device to the cloud over the internet.Edge technology can be considered as a way to optimize the speed and performance of predictive analytics by running ML models locally.Frugal AI models are seen as the optimum approach to (i) predict the remaining useful life of assets with regression models and (ii) predict failures within a given time window with classification models for STS cranes and straddle carriers.In the envisioned architecture, far-edge devices could be used for collecting data from sensors and the machine's PLC, and performing basic pre-processing of the data.Edge processing will be performed for local ML training and inferencing for identifying the need for PdM.Finally, cloud resources could be used for more intense ML training and testing.

IV. IOT FOR CONTAINER SEAL INSPECTION
Containers play an important role in the maritime industry worldwide.Containerization has improved the way cargo is transported around the world by ensuring the safety of cargo in transit.To ensure the security of containers, they are sealed with security seals that prevent an unauthorized entry (a pictorial example of security seals is presented in Fig. 2).During transit, customs officers need to inspect security seals when containers pass through the container terminal gate.The current mechanism for checking the seals relies on visual inspection by humans, which can be time-consuming, laborintensive, and potentially dangerous.Within the digitalization journey, CV is a rapidly growing field of AI that has many potential advantages in various industries [20], particularly in manufacturing and industrial settings, where visual inspections are critical to ensure product quality and safety.One significant advantage of CV is its ability to automate visual inspection processes.With the help of CV algorithms, it is possible to automate quality control, defect detection, and inspection processes, resulting in increased accuracy, speed, and costeffectiveness.
One of the most important sources of data for the terminal is the camera system.Thanks to the explosion of customizable internet protocol television (IPTV) cameras, the captured video streams can be used in combination with CV to improve various aspects of the port.In particular, it is important to track the continuous flow of incoming and outgoing containers.Therefore, the development of CV and IoT-based systems will allow the terminal to automatically identify containers with damage and verify the presence of proper container seals without requiring human intervention.These two features will be offered to container terminals' customers, providing added value from the quality checkpoint of view.In addition, frugal AI models can provide the automated and low-latency analyses of those video streams on the edge, avoiding human mistakes and at the same time reducing safety risks, thus achieving a secure, and trustable environment for workers.
Based on current literature, several studies exist that are employing IoT and CV for visual inspection processes in different industrial settings.However, we found only a few studies that deal with the maritime industry using CV, IoT, and edge computing based solutions.In the maritime industry, edge technology is considered the way of optimizing the speed and performance of CV analytics by performing ML locally, since despite cloud computing can support CV solutions, performing the inference process on a remote server may lead to potential latency issues, which may cause an unsafe environment for terminal workers.Therefore, an edge-based architecture of container damage/seal inspection via CV and ML technologies would be the most suitable solution.In the envisioned architecture, far-edge devices (i.e., IPTV customizable cameras) could be used for collecting video streams and performing basic pre-processing.Edge processing will be performed for local ML training and inference for identifying the presence of container damages or the absence of safety seals.Finally, cloud resources could be used for more intense ML training and testing.

V. CONCLUSION
This paper reviews the current status of IoT in SMTSs.The main research gaps identified in this study are: 1) the need to deploy IoT in the maritime industry, 2) the integration of IoT with other emerging technologies such as ML, edge computing, and CV, 3) data security and privacy, and 4) big data collection and storage, while the edge technology-based solution is proposed to optimize the speed and performance of predictive analytics by running ML and CV models locally.Furthermore, these challenges can also be addressed by com-bining emerging technologies, i.e.IoT, ML, edge computing, and CV, in the maritime industry.This paper explores two such use cases of SMTSs, while combining IoT with other emerging technologies; i.e., ML/DL for PdM, and CV for container damage/seal inspection.
Fig. 2: A container with the security seal shown in a green square; the possible alternative positions for the security seals are shown in black squares.