, , , , , , , , , , , , ,

PyMovieStudio is a Python application that lets you easily record, edit and screen a homemade movie! My last post has the detail.

But what about adding special effects to our movies, such as explosions? Wouldn’t it be great to blow up that pesky robot featured in the previous post.

In this post I will add object tracking to PyMovieStudio, so we know the exact coordinates of the killer robot. Armed with the coordinates, we can easily plant some bombs in the robot’s path and set them off!

Okay, so once the movie is edited, we enable Effects – this allows each frame of our movie to flow through the Tracking class:

import cv2

class Tracking:

    # initialise
    def __init__(self):
        # background subtraction
        self.backgroundSubtraction = cv2.BackgroundSubtractorMOG()

    # track frame
    def frame(self, frame):

        # apply background subtraction
        fgmask = self.backgroundSubtraction.apply(frame, learningRate=1.0/12)
        frame = cv2.cvtColor(fgmask, cv2.COLOR_GRAY2RGB)

        # get contours for objects in foreground
        contours = self._get_contours(fgmask)
        if not contours: return frame

        # get largest contour
        contour = sorted(contours, key=cv2.contourArea, reverse=True)[:1][0]

        # get centre coordinates of contour
        coord = self._get_contour_centroid(contour)

        # drawn coordinates
        self._draw_coordinates(frame, contour, coord)

        return frame

As you can see, the class initialises OpenCV Background Subtraction.

We apply background subtraction to each frame of our movie, to separate the foreground moving objects (i.e. our robot) from the background non-moving objects (i.e. our Lego policemen and the woodland hospital).

Here’s a frame of the movie:


And here’s how background subtraction has picked out the robot:


Great! Now we have our robot, let’s find its coordinates.

First, we use OpenCV Canny Edge Detection and Contours:

# get contours from image
def _get_contours(self, image):
    edges = cv2.Canny(image, 100, 200)
    contours, _ = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    return contours

The Tracking class’s _get_contours private method takes care of finding contours in our foreground objects.

Here’s our movie frame with canny edge detection:


Once we have sorted our contours and selected the largest one, let’s get its coordinates using OpenCV Moments:

# get contour centroid
def _get_contour_centroid(self, contour):
        M = cv2.moments(contour)
        cx = int(M['m10']/M['m00'])
        cy = int(M['m01']/M['m00'])
        return (cx, cy)
        return (0, 0)

The Tracking class’s _get_contour_centroid private method takes care of finding the coordinates for our largest contour.

Armed with the coordinates, we can use OpenCV Drawing Functions to draw the coordinates on the frame:

# draw coordinates on image
def _draw_coordinates(self, image, contour, coord):
    cv2.drawContours(image, [contour], -1, (0, 255, 0), 3)
    cv2.putText(image, "Co-ordinates {}".format(coord), (coord[0], coord[1]), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0))

The Tracking class’s _draw_contour private method takes care of drawing the contour and coordinates on the movie frame.

Here’s our movie frame with coordinates:


Splendid. We can now track our killer robot in the movie.

Here’s the movie with object tracking enabled:

The robot is tracked, and its coordinates exposed.

Notice that we get a lot of noise in our tracking as the camera angle changes, and on the final camera angle as it pans across the scene. Background subtraction works best when the scene does not change.

Armed with the robot’s coordinates, bombs can now be planted in its path and set off! PyMovieStudio runs in an OpenGL Graphics Library window, so we can add sophisticated special effects. But that’s for another day.

PyMovieStudio is free to use! To find out more about PyMovieStudio, check out its Wiki.