Embodied AI Reading Club | Vision-and-Language Navigation

В докладе представлен обзор различных задач и направлений области Vision-and-Language Navigation (VLN). Рассматриваются основные VLN бенчмарки, использование LLM и VLM для их решения и актуальные работы: 1. Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments () 2. REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments () 3. Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding () 4. Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments () 5. Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention () 6. CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation () 7. Memory-Maze: Scenario Driven Benchmark and Visual Language Navigation Model for Guiding Blind People () 8. LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action () 9. NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation () 10. BehAV: Behavioral Rule Guided Autonomy Using VLMs for Robot Navigation in Outdoor Scenes ()