Tags

, , , , , , , , , , , , ,

My 76-year-old neighbour, Mrs Elderflower, has a problem.

‘It’s my grandson, Timmy. He is afraid of the dark and will not sleep at night.’

I put a reassuring hand on her bony shoulder. ‘Don’t worry,’ I told her, ‘As luck would have it, I have just finished writing a children’s book that will ease the young lad into slumber.’

Everyone knows that a good book with a cup of cocoa is what’s needed at the end of a busy day. And my new book has an added advantage – it is an augmented reality book!

Here’s how it works…

A webcam is placed over the book. As each page of the book is turned, my Python code looks for the presence of glyphs. A glyph could look like this:

glyph_01

Or this:

glyph_02

Now, if the code detects a glyph, it will add a picture on top of it. Since each glyph is unique, we can add a different picture to each page of the book (or even different pictures on the same page).

Brilliant. All Mrs Elderflower need do is get her grandson to read my book from the computer screen, and watch the pages come to life!

The next day I asked her whether little Timmy had enjoyed my book. ‘I bet the young lad is sleeping as sound as a slug now,’ I remarked with confidence.

My elderly neighbour shot me down in flames. ‘You imbecile! My grandson may never sleep again!’

Maybe the story was a bit hard-hitting, what do you think:

All I can say is, if her grandson thought that scary, he should definitely steer clear of The Inscrutable Diaries Of Rodger Saltwash.

Ciao!

P.S.

My Python code uses OpenCV computer vision to detect glyphs in an image. All the code can be found in my post 3D Augmented Reality using OpenCV and Python, with the following amendments:

speech.py

import speech_recognition as sr
from threading import Thread
import time

class Speech:

    def __init__(self):
        self.recognizer = sr.Recognizer()
        self.text = None
         
    # create thread for capturing text
    def start(self):
        Thread(target=self._update_text, args=()).start()
 
    def _update_text(self):
        while True:
            self.text = None

            with sr.Microphone() as source:
                print "listening..."
                audio = self.recognizer.listen(source)

            try:
                self.text = self.recognizer.recognize_google(audio)
                print self.text
            except sr.UnknownValueError:
                print("Google Speech Recognition could not understand audio")
            except sr.RequestError:
                print("Could not request results from Google Speech Recognition service")
        
            time.sleep(5)

    # get the current text
    def get_text(self):
        return self.text

I have created a new Speech class, which can hear the reader of the book shouting “go away Dave”. Check out my post Whac-A-Mole with Augmented Reality (Mark II) for an explanation of how to employ the Python Speech Recognition library to convert speech to text.

Notice that I am running the _update_text method in a thread. I don’t want to block the book being rendered to the computer screen, waiting for my speech recognition to complete.

glyphs.py

This was formerly the main program – but now I have turned it into a function (replacing the while loop):

from speech import Speech

speech = Speech()
speech.start()

def detect_glyphs():

The only other tweak is to Stage 8: Substitute glyph. Now, if our Speech class tells us that the reader has shouted “go away Dave”, we turn the picture to black. Otherwise the picture of a devil will render as it did previously:

if speech.text == "go away Dave":
    substitute_image[:] = 0
else:
    for _ in range(glyph_rotation):
        substitute_image = rotate_image(substitute_image, 90)

Stage 9: Add effects is no longer required.

The main program

from glyphs import *

while True:
    try:
        detect_glyphs()
    except Exception as ex: 
        print(ex)

All that is left to do is loop and call our detect_glyphs function forever (or until the reader falls asleep).

I sure am glad to see Dave go!