ImmersedPrivacy: Evaluating VLM Privacy Awareness in Physical Environments

A new study introduces ImmersedPrivacy, an interactive audio-visual evaluation framework designed to assess privacy awareness in Vision-Language Models (VLMs) deployed as autonomous cognitive cores for embodied assistants. Unlike digital chatbots, these agents operate in intimate spaces like homes and hospitals, where they can observe and manipulate privacy-sensitive information and artifacts. Current benchmarks are limited to unimodal, text-based representations that fail to capture real-world demands. ImmersedPrivacy uses a Unity-based simulator to create realistic physical environments and evaluates privacy awareness across three progressive tiers: identifying sensitive items in cluttered scenes, adapting to shifting social contexts, and resolving conflicts between privacy and task objectives. The framework aims to bridge the gap between existing evaluations and the physical grounding required for safe deployment. The study is published on arXiv under identifier 2605.05340.