Tags

, , , , , , , , , , , , ,

A Shetland pony, on a penny-farthing, weaving gingerly through the crowded streets of Rio. Still it espouses the virtues of a northern climate, with the equal measure of cigar and megaphone to hairy-lipped mouth. Lush eyelashes, the sort a pop princess would swoon for. ‘Now, that,’ my Dutch lodger said, ‘is amazing.’

True. But I doubted he had ever seen such a spectacle.

Rendering video – frame by frame – onto an OpenGL cube is pretty special too. ‘Perhaps,’ the fat bastard replied, and popping a salted nut into his gob he teased, ‘Woo me.’

Okay, I will.

Sure, the OpenGL graphics library will render the video-laced cube. But it is OpenCV computer vision that will fetch the video frames. First, we instantiate an OpenCV video capture object using the Python programming language:

self.video_capture = cv2.VideoCapture()

Our Python class has a video_capture instance variable, to hold a reference to the video capture object.

What we need now is a handy class method that will return the next frame of video:

def _get_video_frame(self):

    # get latest frame from video
    success, frame = self.video_capture.read()
    if success: return frame

    if not self.video_capture.isOpened():
        self.video_capture.open('video/channel_one.mp4')
    else:
        self.video_capture.set(cv2.cv.CV_CAP_PROP_POS_FRAMES, 0)      

    return self.video_capture.read()[1]

We read the next frame of video and, if successful, it is returned to the calling code.

If we have not been able to read a frame of video, we need to find out why.

First, we check if the video file has been opened yet. If not, we load our chosen video (providing a relative path to the file).

If the video is already open, then the reason we are unable to read a frame is because the video has come to an end. So we set the frame position back to zero and start the video over again (we are looping the video).

Once we have sorted out the video, we read a frame and return it to the calling code.

Okay, next we need to shift from OpenCV to OpenGL, and create a texture of our video frame for rendering onto the cube:

# convert image to OpenGL texture format
tx_image = cv2.flip(frame, 0)
tx_image = Image.fromarray(tx_image)     
ix = tx_image.size[0]
iy = tx_image.size[1]
tx_image = tx_image.tobytes('raw', 'BGRX', 0, -1)
    
# create texture
texture_id = glGenTextures(1)
glBindTexture(GL_TEXTURE_2D, texture_id)
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR)
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR)
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, ix, iy, 0, GL_RGBA, GL_UNSIGNED_BYTE, tx_image)

First up, we need to convert our video frame to a format suitable for an OpenGL texture. Notice how we use OpenCV to flip the frame – otherwise it will be rendered upside down. PIL (Python Imaging Library) takes care of the rest of the conversion (incl. grabbing the image width ix and height iy).

Next, we create the OpenGL texture. The texture_id will let us access the texture later.

Right, time to draw our cube. Here’s one face of the cube with the texture applied:

glBindTexture(GL_TEXTURE_2D, texture_id)
glBegin(GL_POLYGON)
glTexCoord2fv([0.0, 0.0])
glVertex3fv([1.1, 1.1, -1.1])
glTexCoord2fv([1.0, 0.0])
glVertex3fv([0.0, 1.1, -1.1])
glTexCoord2fv([1.0, 1.0])
glVertex3fv([0.0, 1.1, 0.0])
glTexCoord2fv([0.0, 1.0])
glVertex3fv([1.1, 1.1, 0.0])
glEnd()

Notice the glBindTexture command using the texture_id to access the video frame texture.

Splendid. Let’s have a look at the OpenGL cube, with the video being rendered onto one of its faces:

ArkwoodAR_VideoOpenGL_Left

ArkwoodAR_VideoOpenGL_Centre

ArkwoodAR_VideoOpenGL_Right

The star of the show is a rather fetching skeleton.

‘Toadstools,’ Peters said, in his thick Dutch accent. ‘You were right, my little fungi. Playing video in OpenGL is rather special.’

‘Why, thank you!’

Ciao!

P.S. if you want to play about with OpenCV and OpenGL, why not try installing SaltwashAR, a Python Augmented Reality application.