We present a few applications that is possible with smartphones when used with such smart helmets.
Here’s a video of some applications that we have built at CVIT and some that are currently in progress.
- Assistive Navigation through audio and haptic feedbacks
- Road crossing
- Environment related information, for example, warning obstacles in the path
- Provide product information
- Turn by turn navigation
- Memory Augmentation
- Face recognition, re-identification
- Visual logs
- Panoramic Image and Video Capture
- Stereo Image and Video Capture
- Video stabilization and hyperlapse
- Virtual tour
- Night vision
- Virtual and Augmented Reality
- Gesture Recognition
- Remote guidance and tutorials
- Lifelogging: Logs of physical activities
Technical details of few applications are presented here along with diagrams
Face recognition and re-identification
A large database of faces is created from the images on the device. These images are typically are of family members, friends and acquaintances. A close match from the database is found for each face in the person’s view. If a similar face is not present in the database, it is added with a new identity along with metadata like time, date and location. This feature can be selectively activated only when the user is having conversation with another person.
Video Stabilization and Hyperlapse
Video feed from the camera during physical activities tends to be very shaky since the helmet is worn on head. Such videos are very difficult to watch. Stabilisation and time lapse of such videos are usually created with dedicated expensive hardwares. A smartphone comes bundled with several useful sensors like accelerometer and gyroscope which can help us solve this issue. A stabilisation module with the help of available sensors can be used to estimate the camera pose with respect to the surrounding. A smooth camera path can then be estimated using gyroscope which is used to render the stabilised video. For longer video a hyperlapse summary can be created using stabilisation and keyframe selection. The resulting video is not only easy to watch and understand but also looks artistic and visually impressive.
Stereo Panoramic Image and Video Capture
With availability of several cameras, one application that naturally comes to mind is panoramic image and video capture. Each camera has different orientation and field of view. We get a panoramic image when the images from each camera are stitched together. If two front facing cameras are available, we can infer depth in the field of view just like humans do. Therefore, with arrays of cameras mounted on/with the helmet, a stereo panoramic image and video can be easily captured.
Virtual and Augmented Reality
Virtual Reality through smart phone (Google Card box, Oculus Gear VR) has become popular in recent times for gaming, watching videos etc. Our smart helmet can be extended as virtual reality headset by bringing the front module along with front facing cameras to the eye level. Apart from virtual reality use cases, the user can see the world view with augmented information in real time. This set applications make use of the cameras and sensors on the device and smartphone’s screen.
Infrared cameras can be used to extend the normal human vision capabilities. Thermography and night vision is one such application. IR camera module are available as extension for Raspberry pi which can be attached to the helmet. The user can watch the live night vision feed on the phone’s screen. This application is particularly useful for industrial usage to detect _unusual faulty equipments.