Peters, my obese Dutch lodger, had wanted to know how to post letters to Holland whilst sightseeing in London. I tried to help, but we had a falling out. I smacked his stupid face with a kitchen stool and rendered him unconscious. Later, we patched up our differences.
‘Maybe you could help me with the Highway Code,’ he ventured, ‘I never know which way round a roundabout I am supposed to go.’ No problem, I said.
In my previous post Pillar box detection on Google Street View I pointed a webcam at Google Street View and detected some pillar boxes at the side of the road. To start with, I created a classifier for pillar box detection using OpenCV. Then I wrote some Python code on my Raspberry Pi computer which would read each webcam image and attempt to detect pillar boxes, using said classifier. Finally, if pillar boxes were detected, I played some funky music through speakers.
Now I will attempt to detect road signs for roundabouts, using a similar approach to the one I took for pillar boxes, so that Peters can practice driving on the left-hand side.
Let’s create a haar cascade classifier for roundabout signs…
I have 22 positive images of roundabout signs. 19 of these images were sourced from the internet (15 photos, 4 drawings). 3 of these images were sourced from Google Street View (images used only for training, not testing).
I have 200 negative images of the interior of my house.
I created my training samples:
perl createtrainsamples.pl positives.dat negatives.dat samples 500 "./opencv_createsamples -bgcolor 0 -bgthresh 0 -maxxangle 1.1 -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 25 -h 26"
And used them to train a classifier:
opencv_haartraining -data haarcascade_roundabout -vec samples.vec -bg negatives.dat -nstages 30 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -npos 500 -nneg 200 -w 25 -h 26 -nonsym -mem 2048 -mode ALL
Training stalled at stage 20, so I used the convert_cascade application to spit out my classifier xml file:
convert_cascade --size="25x26" haarcascade_roundabout haarcascade_roundabout-inter.xml
Great. Now let’s have a look at the Python code that will use my roundabout sign classifier. First up, our Webcam class:
import cv2 from datetime import datetime class Webcam(object): WINDOW_NAME = "Roundabout Detection System" # constructor def __init__(self): self.webcam = cv2.VideoCapture(0) # save image to disk def _save_image(self, path, image): filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg' cv2.imwrite(path + filename, image) # detect roundabouts in webcam def detect_roundabouts(self): # get image from webcam img = self.webcam.read() # do roundabout detection roundabout_cascade = cv2.CascadeClassifier('haarcascade_roundabout.xml') gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY) roundabouts = roundabout_cascade.detectMultiScale(gray, scaleFactor=1.4, minNeighbors=6) for (x,y,w,h) in roundabouts: cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2) # save image to disk self._save_image('WebCam/Detection/', img) # show image in window cv2.imshow(self.WINDOW_NAME, img) cv2.waitKey(2000) cv2.destroyAllWindows() # indicate whether roundabouts detected if len(roundabouts) > 0: return True return False
The detect_roundabouts method obtains an image from the webcam attached to our Raspberry Pi. It then uses our classifier file to detect any roundabout signs in said image.
Here’s the main Python program that makes use of our Webcam class:
from webcam import Webcam from speech import Speech webcam = Webcam() speech = Speech() # wait until roundabouts detected while webcam.detect_roundabouts() == False: print ("no roundabouts found") # now tell Peters about highway code speech.text_to_speech("please steer left at approaching roundabout")
If roundabout signs have been detected, we use Google’s Text To Speech service to alert Peters through a set of speakers attached to the Raspberry Pi. Here’s our Speech class:
from subprocess import PIPE, call import urllib class Speech(object): # converts text to speech def text_to_speech(self, text): try: # truncate text as google only allows 100 chars text = text[:100] # encode the text query = urllib.quote_plus(text) # build endpoint endpoint = "http://translate.google.com/translate_tts?tl=en&q=" + query # debug print(endpoint) # get google to translate and mplayer to play call(["mplayer", endpoint], shell=False, stdout=PIPE, stderr=PIPE) except: print ("Error translating text")
Okay, time for a demo – let’s point the webcam at a monitor displaying Google Street View and see if it detects roundabout signs:
Magic! Two signs have been detected and the lovely Google fembot tells my rotund buddy that he should bear left. Our classifier still needs more training images, however, so as to avoid false-positives.
‘Okay,’ I said to Peters, who was munching on a Ginsters pasty, ‘It’s ready. Why don’t you have a play with Google Street View and get used to tackling roundabouts?’
Peters looked at me, astonished. ‘What for? I don’t have a car, and I have no intention of learning how to drive.’
Without a fuss, I strolled over to the ironing board and picked up the scorching iron. Peters spat out flakes of pastry as I plunged the hot piece of metal into his neck and burnt out his double chin. My lodger passed out. ‘That’s the last time I try to help you.’
Detecting road signs is a tricky enough business without having to suffer the blatant disregard for signage consistency, as demonstrated over at Orange Post.