Tags

, , , , , , , , , , , , ,

I was towelling down after a furious burst of yoga, when Monita said, ‘Your cube funny, it keep flickers.’

She was using ArkwoodAR, the Python Augmented Reality application for Google Cardboard. Holding the cardboard glasses up to her Hispanic eyes, she tried out each application feature.

First, the Secret Item feature:

ArkwoodAR_Caching_SecretItem

The Secret Item feature allows Monita to look around the room like a proper spy. If the secret item (in this case, a skull badge) is detected, a green rectangle is sketched about it. She has found her treasure!

Next, the Media Cube feature:

ArkwoodAR_Caching_MediaCube

The Media Cube feature puts a shape with 3D depth on a marker, right in front of her eyes. She can walk around each side of the cube. On one side of the cube, a video is playing of a skeleton in a chair.

But what does Monita mean, when she says, it keep flickers?

Well, she means that sometimes the Augmented Reality application fails to detect the badge, and the green rectangle flickers between being displayed and not being displayed. Similarly, the application sometimes fails to detect the marker, and the cube flickers between being displayed and not being displayed. It can lead to a very frustrating Augmented Reality experience.

‘Don’t worry,’ I told her, ‘I will add caching.’ She looked at me as if I had said that I would drink toilet water through a spiral straw.

Okay, here is the Python code for detecting the Media Cube and Secret Item features:

if self.media_cube:
    detection = self._manage_cache('mediacube', 
                        (self.media_cube.detect(image_one),
                            self.media_cube.detect(image_two)))

if self.secret_item:
    detection = self._manage_cache('secretitem', 
                        (self.secret_item.detect(image_one),
                            self.secret_item.detect(image_two)))

As you can see, the features call a detect method for each image supplied from the left-eye and right-eye webcams. But before we turn any detection into green rectangles and cubes, we put it through the cache:

# manage cache
def _manage_cache(self, cache_key, detection):

    # put detection in cache
    if all(detection):
        self.cache[cache_key] = detection

    # get detection from cache
    elif (cache_key in self.cache) and (self.cache[cache_key]):
        detection = self.cache[cache_key]
        self.cache[cache_key] = None

    # otherwise bin detection         
    else:
        detection = None  
          
    return detection

The cache is a simple Python dictionary of key value pairs: self.cache = {}

For the Media Cube feature, we pass the method a cache key of ‘mediacube’ and the result of our detection.

If our detection tuple has values, we stick them in the cache for later use.

But if our detection does not have values, we try grabbing them from the cache instead. If we find values, the cache is cleared out.

However, if our detection does not have values and neither does the cache, we simply set our detection to None.

With the _manage_cache method in place, we can always be sure of getting values from a cache on the first detection failure after success. The annoying flickering experience is removed, and the cube is more consistently rendered.

Of course, we could extend the cache to keep values for longer than one use. Problem is, too much caching and the cube begins to get out of sync with the marker and appears sluggish to the eye.

I added caching to ArkwoodAR, and asked Monita to give the cardboard glasses another go. ‘What do you think now?’ I asked her.

‘Hey, it better darling. But I bored of this skeleton in the chair. What about that blue movie, sí, under your bed?’

I told Monita that I had no idea what she was talking about.

Ciao!

P.S. check out the ArkwoodAR Wiki for more detail on using the Python Augmented Reality application for Google Cardboard.