, , , , , , , , , , , , ,

In my last post, Computer vision in virtual reality, my buddy Arkwood was able to play his favourite Commodore 64 retro computer game Commando on a TV screen in my virtual world. Not only that, but a speech bubble would appear at the side of the TV screen to tell Arkwood to collect the grenades.

Still he fails. He is so busy peering about for those grenades that he gets shot by an enemy soldier. Or a tank.

Not to worry, I told him. ‘I will help you learn how to play Commando like a master.’

I cranked open my C++ Microsoft Visual Studio application (with OpenGL graphics library and the Oculus SDK for Windows) and added a pause button to the virtual world.

Now, whenever the OpenCV computer vision code notices a grenade – in the pre-recorded video gameplay of Commando – and pops up a speech bubble to inform my chum, Arkwood can press the pause button. The video and audio will pause so that Arkwood can learn exactly where the grenades are and which enemies he has to dodge bullets from.

Here’s Arkwood learning how to play Commando and collect grenades using the pause button:

Works a treat. Now he fails not.

Adding the pause button to my C++ application was not too hard.

I created the pause and play buttons in Blender (the 3D creation suite) and exported it as an .obj file for my application. The application could then decide whether to render the pause or play button.

I used Axis-Aligned Bounding Box collision detection to determine whether one of my virtual hands was pressing the button.

Pausing the video was simply a case of avoiding retrieval of the next video frame. And the FMOD API has a handy setPaused method to put the brakes on my 3D audio.

Arkwood was in the corner, thinking. Then he said, ‘Maybe you could put some of my sexy videos on the TV screen, so I can pause them and learn better technique.’

All things debase.