Concept

Outdoor UAV 350 RTK provides overhead images and videos that give a better understanding of the disaster area and of how the buildings have collapsed. This information is quickly processed  and delivered at the coordination team, thus having  at their disposal a 3D updated map of the area as well as images and videos to then determine which specific buildings to task the first responders to carry out search and rescue activities (trying to estimate the location of survivors and to extract them). The outdoor drone also allows thermal scanning of the area and quick measurement of distances of objects of interest (e.g., persons) to support a variety of search and rescue tasks. All above information are available in real time at the C3I/IMS Platform (HQ and Mobile).

The UAV DJI Mini 3 Pro is used for quick (25’ min flight time) live video streaming and images taking of a small area (1 worksite or 2-3 neighboring worksites).
The purpose is to give first responders a tool that is rapidly (15 secs) deployable and will support their instantaneous enhancement of situational awareness. It takes the images and videos that are live streamed to Command, Control, Coordination and Intelligence Platform and Incident Commander App (C3I/IMS).

Auxilliary Communication systems provides the satellite links to ensure communication redundancy.

Quadrupedal (i.e., legged) robots (aka ANYmal) carry two small indoor drones (aka OWLs), a crawling robot (aka SNAKE), a robotic arm to perform object manipulation tasks (e.g., door opening, valve turning), a set of cameras (visual and thermal) to have a visual and to model/map the uncharted surrounding whilst autonomous exploration is in progress.

The quadrupedal robots are operated by first responders using an Augmented Reality (AR) headset but can also autonomously perform interior exploration.

The AR services, make use of Hololens 2 AR glasses to provide

• AR-based intuitive control of heterogeneous robotic devices ranging from teleoperation to high-level instructions in case of (semi)autonomous devices.
• An AR–based User-interface for first responders and field operators that will offer an information enhanced real-world view.
• AR-based visualization of alerts and checklists at the view of the operator.
• AR based videoconferencing between head quarters and field personnel.

Once a small opening is identified in which high probability for survivors exists the SNAKE robot finds its way into the opening to explore hard-to-reach places.

SNAKE envisions to expand the search area in confined spaces that are difficult for ANYMALs, OWLs and existing manipulators (e.g. entering a small hole). Monitoring is done by two or more cameras: a front-facing rear camera at the base to observe the head insertion, and a wide-angle camera focused on the face. The device sucks in gas from the SNAKE head and monitors concentrations of oxygen, carbon dioxide, combustible gas, and toxic gas. Remote control is performed by an operator based on camera images.

A swarm of UAVs is used to rapidly and autonomously explore deeper into the collapsed structure. It is carried by the ANYMAL robot to the collapsed sites and is autonomously deployed performing quick exploration and real time mapping of the uncharted interior.

Incident Commander (Mobile) App exchanges information with the Headquarters

All information generated by the entirety of filed devices (map and 3D modelling of interior, detection of voids and access points and objects of interest such as valves, doors, pipes, vitals and position of responders, gas sensing, etc.) is seamlessly transmitted to the first response team outside and visualized in the Incident Commander App, to increase their situational awareness. The position of the deployed devices as well as of the personnel is also depicted onto the map.

The entirety of information generated by a single team and the information exchanged among teams and between the HQ is integrated to the Command, Control, Coordination and Intelligence Platform (C3I/IMS). The head quarters are able to see information from all teams and from external sources over a map-based environment and each team can see the information generated by the team members and by their equipment. In this way a Common Operational Picture is created allowing team leaders and HQ personnel to take decisions about what further resources are required, where to put them, and how to reduce the risks to the rescuers operating in the hazard zone. Furthermore, any piece of information can also be exchanged among teams whilst chat and voice based communication, access to availability of resources and commands are always enabled.

It will also include AI based optimizations in terms of logistics managements and actionable intelligence.

As the rescuers/team members make their way into the collapsed structure their position is continuously monitored by the wearables. Such information is transmitted to the team leader (and possibly to team members) knowing at all times position of each team member location.

As the rescuers/team members make their way into the collapsed structure their vitals and environmental conditions are continuously monitored by Wearin’ fully modular wearable solution. The solution will detect health and environmental threats (like heat stress, man-down, respiratory and heart problems, high gas values…) and generate an alarm in real-time to provide the command centre and team leaders with critical information via the most appropriate media to take the appropriate actions.

Novel Human-Machine teaming framework boosts collaboration among first responders teams and with their response tool.

Rapidly deployable Wi-Fi mesh access points ensure rich and ubiquitous connectivity at the worksite area and inside the buildings of interest. These Wi-Fi access points creates a mesh of network that can be accessed by devices and tools in the SYNERGISE tool kit to ensure seamless communications. Some of these access points are also carried by the robots, whilst entering the building/interior of a collapsed structures and dropped appropriately inside the area to extend the coverage.