Tags

, , , , , , , , , , , , ,

In my last post, Arkwood – my disgusting Belgian buddy – had a problem. Another problem that is, besides his BO, leering and general hideousness. It was regarding my Oculus Rift virtual reality headset.

‘I’m afraid that someone will stab me in the neck with a knife when I have the headset on!’

To ease his fears, I added an object into the virtual world that would let him see if anyone was creeping up behind him in the real world.

‘The webcam is trained against the computer room door. Now, when anyone slips in with a dagger, you will spy them on each face of that cube.’

oculusrift_cubecone_webcamtexture_sdk

But how does this all hang together. Here’s how…

The C++ code in the Oculus SDK for Windows ‘OculusRoomTiny(GL)’ Visual Studio project ‘Win32_GLAppUtil.h’ file:

  • Uses Assimp (Open Asset Import Library) to import a cube mesh and a cone mesh created in Blender (3D creation suite)
  • Uses OpenCV computer vision to obtain images from a webcam
  • Uses OpenGL graphics library to render the cube and cone into a virtual world, on the Oculus Rift virtual reality headset

Each image obtained from the webcam is added to the faces of the cube.

Arkwood was still hesitant about using the VR headset though.

‘Sometimes when I am exploring the virtual world, I forget to look at the cube!’

Well, don’t fuckin’ use the headset, I muttered to myself. But, alas, I relented.

‘Okay. I will make it more obvious to you in the virtual world, when someone is spotted on the webcam in the real world.’

I set about using OpenCV motion detection on the images the code was fetching from the webcam.

Mat WebcamImage;
bool IsMotion;

struct Webcam
{
	VideoCapture Capture; 
	bool IsThreadEnabled;
	int Threshold;

	Webcam() : IsThreadEnabled(true), Threshold(20000) {}

	void WebcamThread(){
		Capture.open(0);

		Ptr<BackgroundSubtractor> pMOG2 = createBackgroundSubtractorMOG2(20, 16, true);
		Mat mask;

		while (IsThreadEnabled) {
			Capture >> WebcamImage;

			pMOG2->apply(WebcamImage, mask, 0.001);

			cv::countNonZero(mask == 255) > Threshold ? IsMotion = true : IsMotion = false;
		}

		Capture.release();
	}
};

Webcam * webcam = new Webcam();
std::thread wt(&Webcam::WebcamThread, webcam);

We create a new instance of our Webcam struct and run a thread against its WebcamThread function (a thread will ensure we do not block the main program from rendering to the headset).

The WebcamThread function uses the OpenCV Background Subtractor on each image fetched from the webcam to determine the amount of motion detected in the webcam. If the amount of motion is above a threshold then the IsMotion variable is set to true, otherwise false.

When we come to render our objects into the virtual world, we only render the cone if the IsMotion variable is set to true:

for (int i = 0; i < numMeshes; ++i)
{
	if (i == 0 && !IsMotion){ continue; }

	Meshes[i]->Render(view, proj);
}

Plus, we need to be sure to stop our thread and tidy up our Webcam struct when we close down the program:

webcam->IsThreadEnabled = false;
wt.join();
delete webcam;

Great. With the code in place, let’s see what happens when the webcam detects little to no motion:

oculusrift_cubecone_webcammotionnotdetected_sdk

Only the cube has been rendered, with the webcam images on its faces.

But what happens if the webcam detects someone creeping up behind us:

oculusrift_cubecone_webcammotiondetected_sdk

Hurray! The cone has been rendered into the virtual world, to warn Arkwood that a gorilla is about to stab him to death with a sharp implement.

‘Are you happy now?’ I asked Arkwood.

He was shaking with terror. ‘Where the hell did that gorilla come from?’

I simply shrugged my shoulders. ‘Beats me. Maybe the zoo or something?’

Ciao!

Advertisements