Research

Learning Situated Awareness in the Real World

arXiv•February 18, 2026 ()•Chuhan Li, Ruilin Han, Joy Hsu, Yongyuan Liang, Rajiv Dhawan, Jiajun Wu, Ming-Hsuan Yang, Xin Eric Wang

Professional Abstract

"The paper introduces SAW-Bench (Situated Awareness in the Real World), a novel benchmark designed to evaluate egocentric situated awareness in multimodal foundation models (MFMs). Situated awareness is defined as the ability to relate oneself to the surrounding environment and to reason about possible actions based on that context. Traditional benchmarks have primarily focused on environment-centric spatial relations, which assess relationships among objects within a scene, neglecting the crucial observer-centric relationships that depend on the agent's viewpoint, pose, and motion. This oversight presents a significant gap in the evaluation of models intended to understand human-like perception and interaction with the environment. To address this issue, the authors developed SAW-Bench, which consists of 786 self-recorded videos captured using Ray-Ban Meta (Gen 2) smart glasses, showcasing a variety of indoor and outdoor environments. Accompanying these videos are over 2,071 human-annotated question-answer pairs that are structured to probe a model's observer-centric understanding through six distinct awareness tasks. The comprehensive evaluation conducted reveals a substantial performance gap of 37.66% between human participants and the best-performing MFM, Gemini 3 Flash. This gap underscores the limitations of current models in achieving human-like situational awareness. Further analysis indicates that while these models can leverage partial geometric cues present in egocentric videos, they frequently struggle to infer coherent camera geometry, resulting in systematic errors in spatial reasoning. The authors argue that SAW-Bench serves as a critical benchmark for assessing situated spatial intelligence, emphasizing the need for models to progress beyond mere passive observation to a more profound understanding of physically grounded, observer-centric dynamics. This research not only highlights the deficiencies in existing models but also sets the stage for future advancements in the field of artificial intelligence, particularly in enhancing the situational awareness capabilities of MFMs."

Technical Insights

1SAW-Bench is a new benchmark specifically targeting egocentric situated awareness in multimodal foundation models (MFMs).

2The benchmark includes 786 self-recorded videos from diverse environments, captured using Ray-Ban Meta smart glasses.

3Over 2,071 human-annotated question-answer pairs are included to assess observer-centric understanding across six awareness tasks.

4A significant performance gap of 37.66% was identified between human performance and that of the best-performing MFM, Gemini 3 Flash.

5The study reveals that current models can exploit partial geometric cues but often fail to establish coherent camera geometry.

6Systematic spatial reasoning errors were noted, indicating a critical area for improvement in model training.

7SAW-Bench aims to shift the focus from environment-centric evaluations to a more holistic understanding of observer-centric dynamics.

8The research underscores the importance of developing models that can reason contextually about their surroundings.

9The findings call for further advancements in artificial intelligence to enhance situational awareness capabilities in MFMs.