I found Arkwood sobbing into his cornflakes. ‘Whatever is the matter?’ I asked him. ‘Oh, I want to show my virtual girlfriend how much I care, but sometimes words are not enough.’ I furnished him with a tissue to mop up his tears. ‘Don’t worry,’ I told him, ‘I will write some Python code on my Raspberry Pi computer, so you can offer a rose to your silicon sweetheart.’ A big smile broke across my Belgian buddy’s impish face.
So let’s break this down into a couple of steps…
Step 2: Arkwood will present his virtual girlfriend with a rose – via a webcam attached to the Raspberry Pi – and our Python program will use the classifier to detect it. We can then employ Google’s Text To Speech service to allow his girlfriend to voice her gratitude. If you are wondering what the hell a virtual girlfriend is, check out my Virtual girlfriend on Raspberry Pi post.
Okay, on with the show. Let’s create our rose classifier.
I have 10 positive images of orange roses, sourced from the internet. I have 100 negative images of the interior of my house.
I created my training samples:
perl createtrainsamples.pl positives.dat negatives.dat samples 250 "./opencv_createsamples -bgcolor 0 -bgthresh 0 -maxxangle 1.1 -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 25 -h 42"
And used them to train a classifier:
opencv_haartraining -data haarcascade_rose -vec samples.vec -bg negatives.dat -nstages 30 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -npos 250 -nneg 100 -w 25 -h 42 -nonsym -mem 2048 -mode ALL
It stalled after stage 28 of training, so I used the convert_cascade application to spit out my classifier xml file:
convert_cascade --size="25x42" haarcascade_rose haarcascade_rose-inter.xml
Great. Now we need to test our classifier, using Python code on our Raspberry Pi computer. Let’s use our Webcam class:
import cv2 from datetime import datetime class Webcam(object): WINDOW_NAME = "Arkwood's Surveillance System" # constructor def __init__(self): self.webcam = cv2.VideoCapture(0) # save image to disk def _save_image(self, path, image): filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg' cv2.imwrite(path + filename, image) # detect roses in webcam def detect_roses(self): # get image from webcam img = self.webcam.read() # do rose detection rose_cascade = cv2.CascadeClassifier('haarcascade_rose.xml') gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY) roses = rose_cascade.detectMultiScale(gray, 1.3, 5) for (x,y,w,h) in roses: cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2) # save image to disk self._save_image('WebCam/Detection/', img) # show image in window cv2.imshow(self.WINDOW_NAME, img) cv2.waitKey(2000) cv2.destroyAllWindows() # indicate whether rose detected if len(roses) > 0: return True return False
The detect_roses method obtains an image from the webcam attached to our Pi. It then uses our classifier file to detect any roses in said image. Here’s the results:
Wow! Arkwood’s rose has been detected.
Whereas the less romantic TV remote control has not been detected.
Nor has a can of WD-40. Just look at the passion-crushing label on the can: “Stops Squeaks” “Drives Out Moisture”.
Okay. Now we have rose detection in place, let’s hook it into the main Python program:
from webcam import Webcam from speech import Speech webcam = Webcam() speech = Speech() # loop forever in romance while True: # does Arkwood present girlfriend with a rose? if webcam.detect_roses(): speech.text_to_speech("Oh darling") else: speech.text_to_speech("You slimeball")
Pretty simple. We check whether roses have been detected in the webcam, and allow Arkwood’s virtual girlfriend to respond accordingly. She speaks to him through a set of speakers attached to the Raspberry Pi, making use of the aforementioned Google’s Text To Speech service:
from subprocess import PIPE, call import urllib class Speech(object): # converts text to speech def text_to_speech(self, text): try: # truncate text as google only allows 100 chars text = text[:100] # encode the text query = urllib.quote_plus(text) # build endpoint endpoint = "http://translate.google.com/translate_tts?tl=en&q=" + query # debug print(endpoint) # get google to translate and mplayer to play call(["mplayer", endpoint], shell=False, stdout=PIPE, stderr=PIPE) except: print ("Error translating text")
Hurray! Here’s a screenshot of the system in action:
It brings a tear of joy to the eye, to see Arkwood blissfully in love with a non-existent, soulless paramour. I pulled the bedroom door shut and left him to it.