USEFUL: A car equipped with AI and optical sensors that will collect vast amounts of data for autonomous driving

Autonomous driving is the future of urban mobility, especially in an increasingly ageing society. In the medium term, journeys will be made in vehicles capable of making real-time decisions without human intervention in all types of situations, and the first trials have already begun in cities like San Francisco, Phoenix, and Singapore. Autonomous driving will help optimise energy consumption and traffic management, reduce accidents, and improve mobility for groups such as the elderly or disabled.

Achieving this transformation requires large volumes of precise data from detectors with various operational modes, enabling autonomous systems to interpret their surroundings and act safely, even in complex environments. This necessitates training advanced artificial intelligence (AI) algorithms, and for these algorithms to be reliable in all circumstances, they need a broad and diverse set of data to generate realistic and generalised information that helps the AI learn and make robust decisions.

In this context, the Spanish State Research Agency, under the Ministry of Science, Innovation and Universities, through the Ecological and Digital Transition program, has provided funding of €273,000 to the UPC's CD6 group to launch the USEFUL project. In this project, CD6 has equipped a low-emission vehicle with an integrated and computerised optical sensor system, which includes a high-resolution solid-state LIDAR (a type of laser sensor with a photodetector that measures distances) provided by UPC spin-off Beamagine; a high-precision geolocation and odometry system without the need for GPS (sensor data measurement in motion) provided by Anello Photonics; and a large-scale data storage system provided by QNAP.

The system used in the vehicle over six months includes a processing unit approved for autonomous driving systems that not only manages the acquisition of data from all sensors but also runs artificial intelligence neural networks for specific real-time applications, such as obstacle detection or 3D mapping of the environment.

With USEFUL, millions of images from multiple sensors, some of which are disruptive in the field of autonomous vehicles, will be gathered and converted into usable data to design reliable autonomous driving systems.

The data will be anonymised (not identifying pedestrians or vehicles) and incorporated into a public dataset that will allow other researchers to develop related algorithms. The goal is for the dataset to be as diverse as possible, capturing rural, urban, motorway, and other scenarios; including all types of vehicles, especially the most vulnerable such as bicycles or scooters; and collecting data in real weather conditions and realistic environments where sensor performance may vary. In fact, this data will be used to develop advanced applications for object detection on roads, environment mapping, movement prediction of road users, and digital twin creation, among other things, with greater reliability.

Additionally, USEFUL will allow testing different AI algorithm training methodologies to compare the most effective way of achieving secure perception systems. The vehicle will also enable future testing and validation of new types of autonomous driving sensors, new sensor distributions, or the development of detailed mapping applications for all types of environments.

The project is funded by the Ecological and Digital Transition program of the State Research Agency and is supported by Beamagine, Anello Photonics, QNAP, the School of Industrial, Aerospace and Audiovisual Engineering of Terrassa (ESEIAAT), and the Faculty of Optics and Optometry of Terrassa (FOOT) of the UPC. USEFUL began in January 2023 and will conclude in December 2024.