Tags

, , , , , , , , , , , , ,

Peters wants to send a postcard to his parents when he visits London, so I told him to look out for the iconic red pillar boxes. ‘Just slip your card into the pillar box slot and the Royal Mail will ensure it gets delivered to Holland in good time.’ But which London streets are all those damn pillar boxes on? Let’s use OpenCV object detection and Google Street View to find out.

Now, as they say, a picture is worth a thousand words. Or in this case, a poorly sketched diagram is worth ten misspelled words. But it is better than nothing:

pillarbox_diagram

Okay, so here’s the plan. I create my own pillar box classifier using OpenCV – all the detail of how to do this is in my Guitar detection using OpenCV post. I then place the classifier xml file on my Raspberry Pi computer. The Python code on my Raspberry Pi obtains an image from the attached webcam and uses the classifier to determine if there are any pillar boxes in shot – if there are, it alerts us by playing some music through attached speakers. All that’s left to do is to point the webcam at a monitor which displays Google Street View, so that it can take snaps of each street we navigate down – my post Replay a Google Street View stroll can assist with a random jaunt through the streets of London. Easy-peasy.

Right, let’s create the pillar box classifier.

I have 10 positive images of pillar boxes, sourced from the internet. I have 100 negative images of the interior of my house.

I created my training samples:

perl createtrainsamples.pl positives.dat negatives.dat samples 250 "./opencv_createsamples  -bgcolor 0 -bgthresh 0 -maxxangle 1.1 -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 25 -h 74"

And used them to train a classifier:

opencv_haartraining -data haarcascade_pillarbox -vec samples.vec -bg negatives.dat -nstages 30 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -npos 250 -nneg 100 -w 25 -h 74 -sym -mem 2048 -mode ALL

Note that I am using the -sym parameter, as pillar boxes have vertical symmetry.

Training stalled at stage 19, so I used the convert_cascade application to spit out my classifier xml file:

convert_cascade --size="25x74" haarcascade_pillarbox haarcascade_pillarbox-inter.xml

Great. Now let’s have a look at the Python code that will use my pillar box classifier. First up, our Webcam class:

import cv2
from datetime import datetime

class Webcam(object):

    WINDOW_NAME = "Arkwood's Surveillance System"

    # constructor
    def __init__(self):
        self.webcam = cv2.VideoCapture(0)
        
    # save image to disk
    def _save_image(self, path, image):
        filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg'
        cv2.imwrite(path + filename, image)

    # detect pillarboxes in webcam
    def detect_pillarboxes(self):

        # get image from webcam
        img = self.webcam.read()[1]
        
        # do pillarbox detection
        pillarbox_cascade = cv2.CascadeClassifier('haarcascade_pillarbox.xml')

        gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
        pillarboxes = pillarbox_cascade.detectMultiScale(gray, 1.5, 6)

        for (x,y,w,h) in pillarboxes:
            cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)

        # save image to disk
        self._save_image('WebCam/Detection/', img)

        # show image in window
        cv2.imshow(self.WINDOW_NAME, img)
        cv2.waitKey(2000)
        cv2.destroyAllWindows()
        
        # indicate whether pillarboxes detected 
        if len(pillarboxes) > 0:
            return True

        return False

The detect_pillarboxes method obtains an image from the webcam attached to our Raspberry Pi. It then uses our classifier file to detect any pillar boxes in said image.

Here’s the main Python program that makes use of our Webcam class:

from webcam import Webcam
import pygame

webcam = Webcam()

# set up funk
pygame.mixer.init()
funk = pygame.mixer.Sound("73621__julezhaze__funk-git1.wav")

# wait until pillarboxes detected
while webcam.detect_pillarboxes() == False:
    print ("no pillar boxes found")

# now play funk
funk.play()

We can see that it will play some funky music through speakers attached to the Raspberry Pi if pillar boxes have been detected. It uses Pygame to play the music. I sourced the music from Freesound: https://www.freesound.org/people/Julezhaze/sounds/73621/

All that remains is to point our webcam at a monitor displaying Google Street View and wait until a pillar box is detected:

pillarboxdetection_googlestreetview

Hurray! It has detected a red pillar box and alerted us by playing some music. The classifier is far from robust though, and needs more training images to avoid false-positives – nevertheless, this is a promising start.

I told Peters the good news.

‘Hey, Peters! I’ll soon have a bunch of photos for you, of streets in London with pillar boxes. You’ll be able to send your postcard with ease.’

Peters gazed up at me, a KFC chicken drumstick in his greasy hand. ‘Screw it,’ he replied in his thick Dutch accent, ‘Have you seen the price of stamps? My parents can go fuck themselves.’ He slurped some cola through a plastic straw.

Calmly, I strolled to the kitchen and picked up a wooden stool, testing it for durability. I then strolled back through to the living room, where my smelly lodger was munching on his fast food, and smacked him across the head with said stool. It made a pleasing Thud sound, as his plump body fell limp.

Ciao!