Tags

, , , , , , , , , , , ,

Peters, my obese Dutch lodger, had wanted to know how to post letters to Holland whilst sightseeing in London. I tried to help, but we had a falling out. I smacked his stupid face with a kitchen stool and rendered him unconscious. Later, we patched up our differences.

‘Maybe you could help me with the Highway Code,’ he ventured, ‘I never know which way round a roundabout I am supposed to go.’ No problem, I said.

In my previous post Pillar box detection on Google Street View I pointed a webcam at Google Street View and detected some pillar boxes at the side of the road. To start with, I created a classifier for pillar box detection using OpenCV. Then I wrote some Python code on my Raspberry Pi computer which would read each webcam image and attempt to detect pillar boxes, using said classifier. Finally, if pillar boxes were detected, I played some funky music through speakers.

Now I will attempt to detect road signs for roundabouts, using a similar approach to the one I took for pillar boxes, so that Peters can practice driving on the left-hand side.

Let’s create a haar cascade classifier for roundabout signs…

I have 22 positive images of roundabout signs. 19 of these images were sourced from the internet (15 photos, 4 drawings). 3 of these images were sourced from Google Street View (images used only for training, not testing).

I have 200 negative images of the interior of my house.

I created my training samples:

perl createtrainsamples.pl positives.dat negatives.dat samples 500 "./opencv_createsamples  -bgcolor 0 -bgthresh 0 -maxxangle 1.1 -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 25 -h 26"

And used them to train a classifier:

opencv_haartraining -data haarcascade_roundabout -vec samples.vec -bg negatives.dat -nstages 30 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -npos 500 -nneg 200 -w 25 -h 26 -nonsym -mem 2048 -mode ALL

Training stalled at stage 20, so I used the convert_cascade application to spit out my classifier xml file:

convert_cascade --size="25x26" haarcascade_roundabout haarcascade_roundabout-inter.xml

Great. Now let’s have a look at the Python code that will use my roundabout sign classifier. First up, our Webcam class:

import cv2
from datetime import datetime

class Webcam(object):

    WINDOW_NAME = "Roundabout Detection System"

    # constructor
    def __init__(self):
        self.webcam = cv2.VideoCapture(0)
        
    # save image to disk
    def _save_image(self, path, image):
        filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg'
        cv2.imwrite(path + filename, image)

    # detect roundabouts in webcam
    def detect_roundabouts(self):

        # get image from webcam
        img = self.webcam.read()[1]
        
        # do roundabout detection
        roundabout_cascade = cv2.CascadeClassifier('haarcascade_roundabout.xml')

        gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
        roundabouts = roundabout_cascade.detectMultiScale(gray, scaleFactor=1.4, minNeighbors=6)

        for (x,y,w,h) in roundabouts:
            cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)

        # save image to disk
        self._save_image('WebCam/Detection/', img)

        # show image in window
        cv2.imshow(self.WINDOW_NAME, img)
        cv2.waitKey(2000)
        cv2.destroyAllWindows()
        
        # indicate whether roundabouts detected 
        if len(roundabouts) > 0:
            return True

        return False

The detect_roundabouts method obtains an image from the webcam attached to our Raspberry Pi. It then uses our classifier file to detect any roundabout signs in said image.

Here’s the main Python program that makes use of our Webcam class:

from webcam import Webcam
from speech import Speech

webcam = Webcam()
speech = Speech()

# wait until roundabouts detected
while webcam.detect_roundabouts() == False:
    print ("no roundabouts found")

# now tell Peters about highway code
speech.text_to_speech("please steer left at approaching roundabout")

If roundabout signs have been detected, we use Google’s Text To Speech service to alert Peters through a set of speakers attached to the Raspberry Pi. Here’s our Speech class:

from subprocess import PIPE, call
import urllib
 
class Speech(object):

    # converts text to speech
    def text_to_speech(self, text):
        try:
            # truncate text as google only allows 100 chars
            text = text[:100]
 
            # encode the text
            query = urllib.quote_plus(text)
 
            # build endpoint
            endpoint = "http://translate.google.com/translate_tts?tl=en&q=" + query
 
            # debug
            print(endpoint)
 
            # get google to translate and mplayer to play
            call(["mplayer", endpoint], shell=False, stdout=PIPE, stderr=PIPE)
        except:
            print ("Error translating text")

Okay, time for a demo – let’s point the webcam at a monitor displaying Google Street View and see if it detects roundabout signs:

roundaboutdetection_googlestreetview

Magic! Two signs have been detected and the lovely Google fembot tells my rotund buddy that he should bear left. Our classifier still needs more training images, however, so as to avoid false-positives.

‘Okay,’ I said to Peters, who was munching on a Ginsters pasty, ‘It’s ready. Why don’t you have a play with Google Street View and get used to tackling roundabouts?’

Peters looked at me, astonished. ‘What for? I don’t have a car, and I have no intention of learning how to drive.’

Without a fuss, I strolled over to the ironing board and picked up the scorching iron. Peters spat out flakes of pastry as I plunged the hot piece of metal into his neck and burnt out his double chin. My lodger passed out. ‘That’s the last time I try to help you.’

P.S.

Detecting road signs is a tricky enough business without having to suffer the blatant disregard for signage consistency, as demonstrated over at Orange Post.

Advertisements