Arkwood was delighted with my endeavours to accost the postman at his front door – at least, that is what I thought. So it came as a surprise when he told me this morning, ‘Fuckin’ piece of shite, your code. A car caused it to trigger’. Granted, I hadn’t accounted for an automobile engaging with the system, and so set about introducing face detection to the Python program, so that it only struck up conversation with humans.
The OpenCV Python Tutorials provided the code, which I wrapped into my Webcam class:
import cv2 from datetime import datetime class Webcam(object): WINDOW_NAME = "Arkwood's Surveillance System" # constructor def __init__(self): self.webcam = cv2.VideoCapture(0) # save image to disk def _save_image(self, path, image): filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg' cv2.imwrite(path + filename, image) # obtain changes between images def _delta(self, t0, t1, t2): d1 = cv2.absdiff(t2, t1) d2 = cv2.absdiff(t1, t0) return cv2.bitwise_and(d1, d2) # detect faces in webcam def detect_faces(self): # get image from webcam img = self.webcam.read() # do face/eye detection face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml') eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml') gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY) faces = face_cascade.detectMultiScale(gray, 1.3, 5) for (x,y,w,h) in faces: cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2) roi_gray = gray[y:y+h, x:x+w] roi_color = img[y:y+h, x:x+w] eyes = eye_cascade.detectMultiScale(roi_gray) for (ex,ey,ew,eh) in eyes: cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2) # save image to disk self._save_image('WebCam/Detection/', img) # show image in window cv2.imshow(self.WINDOW_NAME, img) cv2.waitKey(1000) # tidy and quit cv2.destroyAllWindows() if len(faces) == 0: return False return True # wait until motion is detected def detect_motion(self): # set motion threshold threshold = 170000 # hold three b/w images at any one time t_minus = cv2.cvtColor(self.webcam.read(), cv2.COLOR_RGB2GRAY) t = cv2.cvtColor(self.webcam.read(), cv2.COLOR_RGB2GRAY) t_plus = cv2.cvtColor(self.webcam.read(), cv2.COLOR_RGB2GRAY) # now let's loop until we detect some motion while True: # obtain the changes between our three images delta = self._delta(t_minus, t, t_plus) # display changes in surveillance window cv2.imshow(self.WINDOW_NAME, delta) cv2.waitKey(10) # obtain white pixel count i.e. where motion detected count = cv2.countNonZero(delta) # debug #print (count) # if the threshold has been breached, save some snaps to disk # and get the hell out of function... if (count > threshold): self._save_image('WebCam/Motion/', delta) self._save_image('WebCam/Photograph/', self.webcam.read()) cv2.destroyAllWindows() return True # ...otherise, let's handle a new snap t_minus = t t = t_plus t_plus = cv2.cvtColor(self.webcam.read(), cv2.COLOR_RGB2GRAY)
The Webcam class now has two public functions, detect_motion and detect_faces. detect_motion was discussed in my previous post, and is all to do with waiting until something moves in front of the webcam and triggers a threshold. detect_faces is the new function, which takes a snap from the webcam and determines whether the motion that triggered the threshold was a human or, say, vehicle. If it’s got a face and eyes then it’s a human, was my logic.
The rest of the program flows as in said previous post, using Google’s text to speech and speech to text services to converse with the visitor. Lovely.
from webcam import Webcam from speech import Speech webcam = Webcam() speech = Speech() # wait until motion detected at front door webcam.detect_motion() # if faces at front door if (webcam.detect_faces()): # ask visitor for identification speech.text_to_speech("State your name punk") # capture the visitor's reply visitor_name = speech.speech_to_text('/home/pi/PiAUISuite/VoiceCommand/speech-recog.sh') # ask visitor if postman speech.text_to_speech("Are you the postman " + visitor_name) # capture the visitor's reply is_postman = speech.speech_to_text('/home/pi/PiAUISuite/VoiceCommand/speech-recog.sh') # if postman, provide instruction if (is_postman == "yes"): speech.text_to_speech("Please leave the parcel at the back gate and leave") else: speech.text_to_speech("Fuck off")
Before installing the system to monitor my Belgian friend’s front door, I thought it best to take it for a test run. Click the image to see me playing the part of the postman and having my face clocked.
It worked a charm, so I took the gear round to Arkwood’s and set it up to wait for his first visitor. Rubbing my hands with glee, I trotted home and waited for my buddy to telephone with the good news.
The telephone rung a few hours later. Arkwood was in a rage.
‘A fuckin’ squirrel set the motion detection off! What a crock of crap.’
Hm. I hadn’t accounted for a furry rodent having a face and a set of eyes. Damn. Still, it gives me a good excuse to have a go at some facial recognition. Maybe I’ll even get a match on the postman without having to ask him for identification.