Vulnerability of privacy-preserving visual localization against diffusion-based attacks

The rapid deployment of visual localization (VL) in aug-mented/virtual reality (AR/VR) and autonomous systems makes privacy-preserving localization a critical societal necessity. Existing VL systems rely on cloud-based 3D scene representation storage and transmit client side features to a remote server, exposing users to potential reconstruction attacks from intercepted privacy preserving representations. The aim of this paper is therefore to investigate what sensitive information can an adversary actually recover, by comparing multiple privacy-preserving solutions within the literature. We define privacy as the inability to recover personally identifiable information from these representations, acknowledging that general scene details do not inherently represent a privacy breach. To model adversarial behavior and to thoroughly measure a method’s  degree of privacy preservation, we introduce a new privacy attack that trains a conditional diffusion model to reconstruct images from privacy-preserving representations. This reconstruction quality serves as a direct proxy for the amount of sensitive information each representation contains. Leveraging this unified attack protocol, we present the first comprehensive comparison across major families of privacy preserving VL methods. Our analysis reveals substantial differences in privacy leakage between representations and highlights limitations in current design assumptions.