Umeå University's logo

Cooperative perception has emerged as a key paradigm for enhancing environmental understanding in multi-agent systems by fusing sensory information from multiple agents to achieve more comprehensive and accurate perception than single-agent approaches.Despite its demonstrated benefits, existing cooperative perception methods face critical limitations in practical deployments, primarily due to model heterogeneity, latency, and limited communication bandwidth.

This Ph.D. thesis addresses the gap between the theoretical promise of cooperative perception and its practical deployment by systematically investigating how to design cooperative perception systems that are robust, efficient, and scalable under realistic constraints. The main objective of this research is to develop unified frameworks that enable effective multi-agent perception.

To this end, the thesis proposes a series of novel methods targeting these challenges.First, as a foundational study, InputMix is proposed to balance the contributions of heterogeneous sensors in joint training scenarios. Second, an intermediate model-agnostic cooperative perception framework is introduced to enable modular training and seamless collaboration among agents with heterogeneous models. Third, the Latency-Robust Cooperative Perception (LRCP) framework is developed to mitigate the adverse effects of temporal misalignment among agents. Fourth, a lightweight, codebook-free feature compression framework is designed to reduce communication overhead while preserving perceptual performance. Finally, these components are integrated into a unified framework.

Extensive experiments on public benchmark datasets demonstrate that the proposed methods achieve perception performance comparable to the ideal scenario under latency constraints, while enabling effective collaboration among heterogeneous agents and substantially reducing communication bandwidth.

The main contributions of this thesis lie in establishing practical cooperative perception frameworks that collectively address multiple fundamental challenges in multi-agent perception. The findings of this research have broader implications for large-scale autonomous systems, including connected autonomous vehicles and distributed robotic platforms, where reliable cooperative perception under communication and system heterogeneity constraints is essential.