Skip to content
ElderLab-York-UniversityPublic

About

Attentive Sensing for Autonomous Perception

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

Attentive Sensing for Autonomous Perception (ASAP)

For any visual system there is a tradeoff between Field-of-View (FoV) and resolution. A wide FoV is a huge advantage as it allows the system to detect potentially important events coming from many different directions. However a wide FoV comes at the expense of resolution, since photoreceptors must be spread over a wider portion of the visual field.

The human visual system has evolved a bipartite solution to this tradeoff. Central vision is served by roughly five million photoreceptive cones that provide high resolution sensation over a five degree FoV, while roughly one hundred million rods provide relatively low resolution vision over the remainder of the visual field. The effective resolution is extended by fast gaze-shifting mechanisms and a memory system that allows a form of integration over multiple fixations.

Traditional machine vision systems employ a single camera with relatively small FoV. However, applications such as autonomous mobile robot navigation require a wide FoV for obstacle detection and Simultaneous Localization and Mapping (SLAM), as well as high resolution to resolve important details (e.g. the face of a person of interest). These two requirements are conflicting and cannot be met optimally with a single camera.

ASAP integrates an array of wide-FoV cameras with a narrow-FoV camera that can be directed to objects of interest using specialized computer vision and control software. The system thus delivers detailed information from critical targets (e.g. human faces) in the full context of the wider scene.

Architecture

The architecture of ASAP went through a number of revisions during its development. In its latest iteration the system is implemented as a self-contained unit combining the camera suite, computing, connectivity and power delivery for all its components.

Demos

ASAP's wide-FoV and narrow-FoV streams

<iframe width="560" height="315" src="https://www.youtube.com/embed/5zow4WmyMsI?si=vL5z0in_ZKKiHPU3" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



Person tracking across wide-FoV cameras

<iframe width="560" height="315" src="https://www.youtube.com/embed/pX1uCBizPZs?si=zewBDBe3GtzqXuuW" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



WFSCameraTrack - Early version of ASAP with a single wide-FoV camera and a pan-tilt narror-FoV camera

<iframe src="https://player.vimeo.com/video/201032763?badge=0&autopause=0&player_id=0&app_id=58479" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write; encrypted-media; web-share" referrerpolicy="strict-origin-when-cross-origin" style="position:absolute;top:0;left:0;width:100%;height:100%;" title="WFSCameraTrack - Attentive Sensing for Dynamic Scene Analysis"></iframe>
<script src="https://player.vimeo.com/api/player.js"></script>



Superimposing WFSCameraTrack's narrow-FoV and wide-FoV streams

<iframe src="https://player.vimeo.com/video/201032702?badge=0&autopause=0&player_id=0&app_id=58479" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write; encrypted-media; web-share" referrerpolicy="strict-origin-when-cross-origin" style="position:absolute;top:0;left:0;width:100%;height:100%;" title="AttentiveSensorHighQual2-Square pixel video - Attentive Sensing for Dynamic Scene Analysis"></iframe>
<script src="https://player.vimeo.com/api/player.js"></script>



Publications

  • Perroni Filho, H., Trajcevski, A., Bhargava, K., Javed, N., & Elder, J. H. (2023, January). Attentive sensing for long-range face recognition. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) (pp. 613-622). IEEE. (link)
  • Trajcevski, A., Filho, H. P., Javed, N., Naheyan, T., Bhargava, K., & Elder, J. H. (2022, August). Sensorimotor System Design of Socially Intelligent Robots. In International Conference on Pattern Recognition (pp. 106-118). Cham: Springer Nature Switzerland. (link)
  • Dornaika, F., & Elder, J. H. (2012). Image registration for foveated panoramic sensing. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 8(2), 1-20. (link)
  • Elder, J. H., Prince, S. J., Hou, Y., Sizintsev, M., & Olevskiy, E. (2007). Pre-attentive and attentive detection of humans in wide-field scenes. International Journal of Computer Vision, 72(1), 47-66. (link)

About

Attentive Sensing for Autonomous Perception

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors