Tags

, , , , , , , , , , , , ,

I’ve been developing SaltwashAR, a Python Augmented Reality application, for many a moon. It works great on my PC – but what about on my face?

Let’s experience Augmented Reality with some cardboard glasses!

Stereoscopic images

If I am to port SaltwashAR to my face, the application will need to supply stereoscopic images – one image for my left eye to look at and one image for my right eye to look at:

AugmentedRealityGoogleCardboard_Stereoscopic

The two images of the equipment are almost identical, differing only by the perspective of the left and right eye. Stereoscopic images will provide an enhanced sense of depth to our AR experience.

But how do we provide these two images? From two webcams, spaced apart just as our eyes are!

AugmentedRealityGoogleCardboard_Webcams

Obviously it helps if both the webcams are of the same model, for image consistency. I am using the Logitech c270 Webcam (HD 720p).

Turns out I can use the OpenCV VideoCapture object to specify which webcam I want to capture an image from:

import cv2

# show images from two webcams
cv2.imshow("image1", cv2.VideoCapture(0).read()[1])
cv2.imshow("image2", cv2.VideoCapture(1).read()[1])
cv2.waitKey(8000)
cv2.destroyAllWindows()

The Python code is targeting the first and second USB webcams plugged into my PC.

In time, I will build stereoscopic images into SaltwashAR. But for now, I will simply run two instances to the application side by side, configured to different webcams:

AugmentedRealityGoogleCardboard_SaltwashARInstances

Smartphone

Okay, so I can render stereoscopic images of SaltwashAR on my PC. But unless I am going to strap a 1680 x 1050 resolution LG monitor to my face, I will need to display SaltwashAR on a much smaller device. A smartphone.

I will use a Remote Desktop app to render SaltwashAR from my Windows 10 Pro PC onto my Windows Phone 8.1 (Nokia Lumia 520 model). Gizmag provides details of How to remotely access your home PC from Windows Phone 8.1 ( I also had to grant remote access via my firewall software).

SaltwashAR is running in two windows on my PC, and my smartphone displays them in landscape mode:

AugmentedRealityGoogleCardboard_WindowsPhone_RemoteDesktop

Google Cardboard

I now have stereoscopic images of SaltwashAR rendered on my smartphone. But how should I place the smartphone in front of my eyes for an immersive AR experience?

I will use a Google Cardboard (V2.0 Specifications) headset, to view SaltwashAR on my smartphone.

AugmentedRealityGoogleCardboard_BrizTechVR

I have plumped for the BrizTechVR headset.

The smartphone can be placed inside the headset, and the two webcams can be fixed on top of the headset. The two webcams act as my eyes, streaming stereoscopic images of SaltwashAR to my smartphone – via my PC – for me to look at. Here’s what I saw:

AugmentedRealityGoogleCardboard_3DRobot

Each eye has its own perspective of a 3D robot, which has been rendered on a 2D marker. The robot is not really standing on my computer desk, but it sure feels like it when I spy her through my cardboard glasses!

The two webcams can be adjusted by hand, until the stereoscopic images converge.

Google Cardboard headsets are not only cheap, but the specification helps everyone to have a similar Virtual Reality or Augmented Reality experience (albeit with different smartphones and screen sizes).

Next steps

So what are the next steps for SaltwashAR with cardboard glasses? Here’s my checklist:

  • Build stereoscopic images into SaltwashAR (rather than running two instances of the application)
  • Ensure all the SaltwashAR features can work with cardboard glasses
  • Optimise the connection from PC to smartphone, to minimise network latency
  • Fix webcams to headset at eye level (rather than on top of headset)

SaltwashAR works great on a PC – now it’s coming to the face.

Ciao!

P.S.

Please check out the SaltwashAR Wiki for details on how to install and help develop the SaltwashAR Python Augmented Reality application.