“While autonomous technology is available, the biggest challenge is building trust in automated systems that can function safely in many different maritime situations without human intervention,” explains DNV’s SAFEMATE Project Manager Grunde Løvoll.

Decision support system for safe navigation

“Automated bridge systems are increasingly taking navigational functions out of the hands of the crew and therefore we need to find ways of ensuring these systems make the correct decisions.”

SAFEMATE is therefore focused on evolving a decision support system for safe navigation that can detect obstacles and threats in the marine environment, interpret this information and communicate a routing solution to the on-board operator. The system is designed to give enhanced situational awareness for object detection and collision avoidance, based on images and data gathered from cameras, sensors, radar, and the vessel’s AIS, with a human in the loop.

A prototype system developed by project partner Kongsberg Maritime is now being tested in a pilot project on the operational ferry Bastø VI serving the route between Moss and Horten in Norway. This ferry service is run by Torghatten unit Bastø Fosen, which is also participating in the three-year research programme, along with the Norwegian University of Science and Technology (NTNU).

Evaluating route visualization on user interface

Løvoll explains that the system has three main functional elements – to detect objects around the ship, to provide collision avoidance advice for the navigation system operator, and to give a visualization of the marine environment with the suggested route on the human–machine interface, or HMI.

“Rather than starting with a fully automated ship, we are narrowing the scope by focusing on a relatively small aspect of vessel autonomy to verify the reliability of automated navigation with this system,” he says.

“This means we can develop trust in the product without taking big risks and give the operator confidence that automated navigation works within the boundaries of safety.”

Test bed for situational awareness technology

The prototype system installed on the Bastø VI has yielded useful results to inform research during the first year of the project. This has enabled the system operator to evaluate navigational advice given on a fixed repetitive course across a relatively crowded stretch of the Oslofjord traversed by commercial vessels, ferries and leisure boats.

The SAFEMATE project is an important proving ground for newly developed technology and processes that will ultimately be able to support safe automated navigation. Torghatten is taking a proactive role to facilitate adoption of this technology as part of its efforts to achieve autonomous shipping,” says Torghatten’s Chief Technology Officer Jan-Egil Wagnild.

“So far, the project has provided valuable insights into the challenges of and possible solutions to the introduction of smarter navigational decision-supporting systems. This is an important step towards autonomous functionality for the maritime industry.”

Supporting the human navigator on board

The SAFEMATE project enables Kongsberg Maritime to further develop its situational awareness solution by testing it in real-life operations. The supplier is also working together with DNV on developing the necessary assurance regime for the navigational decision support to be integrated into its automated bridge systems thereby enhancing safety and efficiency.

“Basically, we are introducing an electronic navigator with automated functionality to support the human navigator on board by increasing knowledge and awareness of the marine environment, as well as providing routing advice,” explains Kongsberg Maritime’s Product Director for Bridge Systems, Børre Flaglien.

An innovative element is the use of cameras positioned around the ship to stream video the external surroundings, which augments radar, sensor and AIS data being constantly fed into the Kongsberg Bridge System to give the navigator a more accurate picture of obstacles and potential hazards. The system software then assimilates data input from different sources to generate navigational advice using high-powered processing capacity.

Proving data reliability on a large scale

Flaglien says that hundreds of gigabytes of data generated daily from the pilot project are presently being logged and timestamped for extensive analysis in a laboratory to determine the reliability of this data by comparing it with actual actions taken by the navigator. “There is a lot of number crunching going on behind the scenes for deep-learning algorithms in relation to target acquisition. This is part of the fine-tuning of the system to ensure that automated advice is accurate,” he says.

While the system is still at an evolutionary stage, Flaglien says: “This is the most promising initiative in a long time for enhancing navigational safety in shipping and such decision support technology could ultimately be applicable for other automated systems on board.”

Assurance and verification framework for automated system testing

DNV’s role in the joint industry project is to provide an assurance and verification framework for system testing and development, as well as to further develop class rules and standards for such automated systems that are still in gestation, according to Løvoll.

He says collaboration with third-party vendors like Kongsberg Maritime is necessary to determine performance criteria for such standards. DNV is also using the SAFEMATE project to mature its draft guideline for object detection and tracking systems.

As part of this assurance role, DNV is testing the decision support system in simulations of a wide range of real-life operational scenarios by using bridge and other simulators at NTNU under a subproject led by Are Jørgensen, Senior Principal Engineer, Digital Ship Systems at DNV.

This enables the operator to adjust factors such as vessel speed and position to see how the system responds and is thus a more cost-effective method of testing than conducting numerous sea trials, Jørgensen explains.

Understanding the system’s decisions is essential

The testing and verification process involves establishing a framework of processes and tools that can assess the safety performance of the system during the design stage and while in operation.

Naturally, the project is a work in progress with several technical and regulatory challenges still to be overcome such as integrating object detection data from different sources, quality assurance of system advice and reinterpretation of the IMO’s COLREG rules to prevent collisions at sea so these are aligned with computerized systems.

“One of the challenges is explicability; the human navigator needs to be able to see and understand why the system chose the selected route so that they can make their own evaluation. Another is the quality of the routing advice; it needs to be reliable on a consistent basis as otherwise it is suboptimal,” Jørgensen says.

Putting AI through its paces

A primary concern is therefore the reliability of routing advice for the operator generated by artificial intelligence (AI) based on machine learning, which demands a new approach to safety assessment and verification.

Flaglien explains that the use of machine learning is limited to object detection and object classification to build a situational awareness map around the vessel. Route advice – such as a deviation manoeuvre – is then computed using repetitive, deterministic methods in line with COLREG requirements.

A further issue is generating a coherent model for situational awareness by incorporating data from different sources as the fusion of sensor and radar data could give misleading object detection, for example by interpreting one vessel as two, he explains. This could obviously cause problems for the system in determining the optimal trajectory in a situation where there are multiple vessels ahead.

Adhering to the rules of the road at sea

Jørgensen also points out that the COLREG rules, which were developed for human navigators and last updated in 1972, poses a significant challenge for automated collision avoidance algorithms. The reason being that many of the rules in COLREG are of a qualitative nature and need to be interpreted before they can be applied in practice. Algorithms, on the other hand, are in need of clear parameters and boundaries to be developed and verified. Still, autonomous systems need to adhere to COLREG, as any other ship, and this is an important research challenge within SAFEMATE and other projects concerning automated navigation.

SAFEMATE, which is partially funded by the Research Council of Norway, has a distance to go in tackling these myriad challenges but can make significant headway in facilitating unmanned vessel operations, according to Løvoll. “We believe this can be an important stepping stone to autonomous shipping,” he says.

Agencies

Marex Media