Federated deep learning for privacy-preserving sensor fusion in autonomous vehicles

The upsurge of autonomous vehicles (AVs) is mainly supported by colossal multimodal sensor data acquired from various sources, including cameras, LiDAR, and radar systems. Nonetheless, the pooling of such data processed across different vehicles and organizations raises significant privacy, security, and compliance issues concerning international data protection laws. Our contribution in this research is the federated deep learning (FDL) framework that is capable of performing privacy-preserving sensor fusion without the need for raw data sharing across multiple cloud platforms. The proposed system is built upon the CNN-LSTM hybrid architectures for the extraction of multimodal features and also employs Federated Averaging (FedAvg) for the distributed model aggregation. The experiments are carried out on three open-source datasets, KITTI, nuScenes, and Waymo Open Dataset, that represent real-world driving scenarios with different types of sensors. The results reveal that federated deep learning is a suitable technique for the establishment of learning pipelines in AVs that are privacy-compliant across fleets and provide a robust basis for the development of future intelligent transportation systems.