Tags

, , , , , , , , , , , , ,

Imagine two webcams plugged into a PC – one webcam that shows what the left eye can see, and one webcam that shows what the right eye can see:

ArkwoodAR_StereoscopicImage_Robot

OpenCV computer vision detects a marker in each webcam image, and OpenGL graphics library draws a robot on each marker.

Notice how one robot is drawn from the perspective of the left eye, and the other robot is drawn from the perspective of the right eye. This gives our robot 3D depth!

With the two webcam images stitched together into a stereoscopic image, it can be streamed to a headset for a 3D Augmented Reality experience. My posts Augmented Reality with Google Cardboard and Augmented Reality with Google Cardboard (Mark II) did just that.

But how can we stitch the two webcam images together in OpenGL, whilst ensuring that we maintain a separate left eye and right eye view matrix? We’ll employ OpenGL Viewports, that’s how!

Okay, to start with we will capture frames from our webcams using OpenCV:

self.video_capture = cv2.VideoCapture(0)
self.current_frame = self.video_capture.read()[1]

The video capture is targeting device number 0 – which is the first webcam plugged into our PC. To target the second webcam we simply use device number 1.

Next, we get busy detecting a marker in each webcam image. We can’t draw our robots if the marker is not present in both images.

Now it is time to create a stereoscopic image with Viewports. We start by getting the measurements for the current OpenGL window:

window_half_width = glutGet(GLUT_WINDOW_WIDTH) / 2
window_height = glutGet(GLUT_WINDOW_HEIGHT)

We now know the measurement for half the width of the window, as well as the height of the window.

Let’s instantiate a Viewport for the left eye perspective:

glViewport(0, 0, window_half_width, window_height)

The first two parameters specify the x y coordinates at 0 – which is the top left of the window. The last two parameters extend our Viewport to half the width of the window and the full height of the window. Our Viewport is covering the left-hand side of the window.

With the Viewport in place, we can fill it with our ‘left eye’ webcam image and draw the robot at the correct angle on top of the marker.

Let’s now instantiate a Viewport for the right eye perspective:

glViewport(window_half_width, 0, window_half_width, window_height)

The first two parameters specify the x y coordinates at the top middle of the window. The last two parameters extend our Viewport to half the width of the window and the full height of the window. Our Viewport is covering the right-hand side of the window.

With the Viewport in place, we can fill it with our ‘right eye’ webcam image and draw the robot at the correct angle on top of the marker.

The OpenGL window is now rendering a stereoscopic image, just like the one at the top of this post. We can stream stereoscopic images at many frames a second to a headset for a 3D Augmented Reality experience.

Peering into my Google Cardboard glasses and shuffling my head, it is as if the robot is really standing at the edge of my desk!

But it’s not.

If you want to learn how to detect markers and render robots then try out SaltwashAR, a Python Augmented Reality application.

Ciao!