Tags

, , , , , , , , , , , , ,

In my last post, Augmented Reality with Google Cardboard, I was able to experience Augmented Reality with some cardboard glasses.

In this post, I will make the following improvements:

  • Provide stereoscopic images from a new lightweight Augmented Reality application
  • Stream the images to an iPhone for a crisper resolution
  • Fix the webcams to the glasses at eye level

Augmented Reality application

I have created ArkwoodAR, a Python Augmented Reality application for Google Cardboard.

ArkwoodAR provides the stereoscopic images for Google Cardboard’s 3D experience:

augmentedrealitywithgooglecardboard_markii_stereoscopicimage

Simply place two webcams side by side – just like a set of human eyes – and plug them into your PC. ArkwoodAR will stitch the images from both webcams together:

# create stereoscopic image
stereo_image = np.concatenate((image_one, image_two), axis=1)

But what else does ArkwoodAR do? Well, to start with we will turn Google Cardboard into spy glasses!

augmentedrealitywithgooglecardboard_markii_spyvsspydrawing

All in the spirit of Spy vs. Spy

Now, before ArkwoodAR provides each stereoscopic image to Google Cardboard, it will inspect the image for a secret item. If the secret item is found, a green rectangle will be drawn around it.

The spy wearing the cardboard glasses begins to scan the room. Suddenly a green rectangle is displayed before his very eyes – the secret item has been found!

I have used OpenCV Template Matching in order to locate the secret item in the stereoscopic image:

# apply template matching
result = cv2.matchTemplate(image_gray, self.template, cv2.TM_CCOEFF_NORMED)

iPhone

An iPhone will be placed inside our cardboard glasses to display the stereoscopic images from our Augmented Reality application.

But how will we stream the images from the application running on our Windows 10 Pro PC onto our iPhone? Turns out the Remote Desktop app from the previous post – for connecting to my Windows Phone – is also available for iPhone and Android:

augmentedrealitywithgooglecardboard_markii_iphone

Google Cardboard

The iPhone has now been placed inside Google Cardboard – our cardboard glasses – and is displaying stereoscopic images from our Augmented Reality application.

Let’s fix the two webcams to our glasses:

augmentedrealitywithgooglecardboard_markii_headsetfront

Nice. I have used velcro to attach the webcams to the glasses. Indeed, the Logitech c270 webcams are a perfect design for our purpose, hanging as they do at eye level (I’ve also put a velcro strip on the first panel behind the webcam, just in case we want to attach them at a slightly higher level).

It goes without saying, but be sure to fix the ‘right eye’ webcam to the right-hand side of the glasses. And the ‘left eye’ webcam to the left-hand side of the glasses. There, I’ve said it.

The two webcams can now be adjusted by hand, until the stereoscopic images converge.

Here’s the rear of the glasses, where we press up our eyes and nose for a peek at the stereoscopic images:

augmentedrealitywithgooglecardboard_markii_headsetrear

Augmented Reality with Google Cardboard

Right, let’s check off what we have done so far:

  • Plugged two webcams into our PC
  • Produced stereoscopic images on our PC using ArkwoodAR (as well as inspecting each image for a secret item!)
  • Streamed the stereoscopic images to our iPhone
  • Put the iPhone in our cardboard glasses, and fixed the two webcams to the glasses at eye level

But do our spy glasses work? Let’s find out.

Here’s the secret item that we will be scanning the room for:

augmentedrealitywithgooglecardboard_markii_skull

Oh, it is the secret skull of Azabajia! Or something like that. It is a most valuable item.

Our spy puts on his Google Cardboard glasses and begins to scour the room. Here’s what he sees:

augmentedrealitywithgooglecardboard_markii_room

Just an unremarkable room. But wait:

augmentedrealitywithgooglecardboard_markii_roomwithskull

Wow! Our spy has found the secret item. The green rectangle before his very eyes has fed him the much sought-after treasure!

Next steps

It’s great fun being a spy, so long as the other spies don’t get me. But what else could we do with Augmented Reality on Google Cardboard?

ArkwoodAR can be developed to overlay the stereoscopic images with 3D graphics. It can provide audio, Artificial Intelligence, online data, as well as tons more computer vision. Imagine that.

But if you want all those things right now, why not install SaltwashAR, a Python Augmented Reality application that runs on your desktop.

Ciao!

P.S.

Sadly, the superb Spy vs. Spy is only for the old PS2:

augmentedrealitywithgooglecardboard_markii_spyvsspygame