My 76-year-old neighbour, Mrs Elderflower, has a problem.
‘It’s my grandson, Timmy. He is afraid of the dark and will not sleep at night.’
I put a reassuring hand on her bony shoulder. ‘Don’t worry,’ I told her, ‘As luck would have it, I have just finished writing a children’s book that will ease the young lad into slumber.’
Everyone knows that a good book with a cup of cocoa is what’s needed at the end of a busy day. And my new book has an added advantage – it is an augmented reality book!
Here’s how it works…
A webcam is placed over the book. As each page of the book is turned, my Python code looks for the presence of glyphs. A glyph could look like this:
Now, if the code detects a glyph, it will add a picture on top of it. Since each glyph is unique, we can add a different picture to each page of the book (or even different pictures on the same page).
Brilliant. All Mrs Elderflower need do is get her grandson to read my book from the computer screen, and watch the pages come to life!
The next day I asked her whether little Timmy had enjoyed my book. ‘I bet the young lad is sleeping as sound as a slug now,’ I remarked with confidence.
My elderly neighbour shot me down in flames. ‘You imbecile! My grandson may never sleep again!’
Maybe the story was a bit hard-hitting, what do you think:
All I can say is, if her grandson thought that scary, he should definitely steer clear of The Inscrutable Diaries Of Rodger Saltwash.
My Python code uses OpenCV computer vision to detect glyphs in an image. All the code can be found in my post 3D Augmented Reality using OpenCV and Python, with the following amendments:
import speech_recognition as sr from threading import Thread import time class Speech: def __init__(self): self.recognizer = sr.Recognizer() self.text = None # create thread for capturing text def start(self): Thread(target=self._update_text, args=()).start() def _update_text(self): while True: self.text = None with sr.Microphone() as source: print "listening..." audio = self.recognizer.listen(source) try: self.text = self.recognizer.recognize_google(audio) print self.text except sr.UnknownValueError: print("Google Speech Recognition could not understand audio") except sr.RequestError: print("Could not request results from Google Speech Recognition service") time.sleep(5) # get the current text def get_text(self): return self.text
I have created a new Speech class, which can hear the reader of the book shouting “go away Dave”. Check out my post Whac-A-Mole with Augmented Reality (Mark II) for an explanation of how to employ the Python Speech Recognition library to convert speech to text.
Notice that I am running the _update_text method in a thread. I don’t want to block the book being rendered to the computer screen, waiting for my speech recognition to complete.
This was formerly the main program – but now I have turned it into a function (replacing the while loop):
from speech import Speech speech = Speech() speech.start() def detect_glyphs():
The only other tweak is to Stage 8: Substitute glyph. Now, if our Speech class tells us that the reader has shouted “go away Dave”, we turn the picture to black. Otherwise the picture of a devil will render as it did previously:
if speech.text == "go away Dave": substitute_image[:] = 0 else: for _ in range(glyph_rotation): substitute_image = rotate_image(substitute_image, 90)
Stage 9: Add effects is no longer required.
The main program
from glyphs import * while True: try: detect_glyphs() except Exception as ex: print(ex)
All that is left to do is loop and call our detect_glyphs function forever (or until the reader falls asleep).
I sure am glad to see Dave go!