Real-time Pedestrian Detection using Python & OpenCV

Pedestrian detection or people detection is a very essential task in some areas such as surveillance systems, traffic control systems, etc. In this machine learning project, we are going to make a very simple pedestrian detection system using OpenCV.

OpenCV is an open-source library written in C/C++, but we can also use it in python. It is one of the most widely used libraries for computer vision tasks like face recognition, motion detection, object detection, etc.

OpenCV has a built-in pre-trained HOG + Linear SVM model to perform pedestrian detection.

HOG – Histogram of Oriented Gradients

The histogram of oriented gradients is a feature descriptor algorithm used in computer vision and image processing. Its main purpose is object detection. The essential thought behind the HOG descriptor is that local object shape and appearance within an image can be described by the distribution of intensity gradients or edge directions.

This algorithm checks the surrounding pixels of every single pixel using the sliding window technique. The goal is to check how darker is the current pixel compared to the surrounding pixels and then it draws an arrow showing the direction of the image getting darker for every pixel. These arrows are called Gradients.

Linear SVM -Linear Support Vector Machine

LSVM or Linear Support Vector Machine is a linear model for classification and regression problems. The idea behind SVM is that the algorithm creates a line or a hyperplane which separates the data into classes.

Challenges with HOG + L-SVM model

The HOG + L-SVM models are good for rapid prototyping. But it is not the best model from accuracy perspective. Sometimes it’s very hard to tune this model to achieve the best result.

Pedestrian Detection Project Prerequisites

OpenCV – V4.5

We will use OpenCV for performing all the image processing and object detection tasks.

Please run below command to install the OpenCV library and its essential packages.

pip install opencv-python opencv_contrib-python

Imutils – v0.5.4

Imutils is basically a helper package that consists of a series of convenience functions to make basic image processing functions such as rotation, resize, etc.

Please run below command to install the package

pip install imutils

Download Pedestrian Detection Python Code

Please download the source code of python opencv pedestrian detection project: Pedestrian Detection Project Code

Steps to Develop Pedestrian Detection project

  1. Import necessary packages and define the model
  2. Reading images/frames and preprocess
  3. Detect Pedestrians
  4. Draw rectangles around each detection in the frame
  5. Post-process output data to filter out the best results
  6. Detect real-time from Video

Step 1 – Import necessary packages and define the model:

Import the package required for pedestrian detection and start working on model

import cv2
import numpy as np
import imutils

# Histogram of Oriented Gradients Detector
HOGCV = cv2.HOGDescriptor()
HOGCV.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())

Explanation:

  • To extract necessary data from the image we’ll use the cv2.HOGDescriptor() method and set the descriptor object as HOGCV.
  • Using setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector()) method we initialise hog detection object with svm detection for detecting people.

Step 2 – Reading the image and Preprocess:

frame = cv2.imread("image.jpg")

width = frame.shape[1]
max_width = 600

if width > max_width:
  frame = imutils.resize(frame, width=max_width)

Explanation:

  • cv2.imread() function reads images from a given path.
  • If image width is greater than defined max_width then apply imutils.resize() function to resize the image to defined max_width.

Step 3 – Detect Pedestrians with OpenCV:

pedestrians, weights = HOGCV.detectMultiScale(frame, winStride=(4, 4), padding=(8, 8), scale=1.03)

pedestrians = np.array([[x, y, x + w, y + h] for (x, y, w, h) in pedestrians])

print(pedestrians)

Explanation:

  • detectMultiScale() detects the objects from the image and returns their x and y coordinate, height, and width.
  • The winStride parameter indicates the step size in both the x and y location of the sliding window.
  • The padding parameter indicates the number of pixels in both the x and y direction in which the sliding window ROI is “padded” prior to HOG feature extraction.
  • The scale parameter controls the factor in which our image is resized at each layer of the image pyramid.

sliding window example

Note: Smaller winStride will need more windows to be evaluated. It will give better accuracy but can cause performance issues.

image pyramid example

In the next line, we’ve processed the data to get the final rectangle coordinates.

Output:

[[354 152 424 291]
[301 141 397 332]
[360 116 474 343]
[171 87 286 318]]

Here we get 4 nested lists. That means 4 persons are detected.

Output may vary based on input image

Step 4 – Draw rectangles around each pedestrian detection in the frame:

Now we have all the detected pedestrian information. So let’s draw the bounding boxes.

count = 0

#  Draw bounding box over detected pedestrians
for x, y, w, h in pedestrians:
    cv2.rectangle(frame, (x, y), (w, h), (0, 0, 100), 2)
    cv2.rectangle(frame, (x, y - 20), (w,y), (0, 0, 255), -1)
    cv2.putText(frame, f'P{count}', (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255, 255, 255), 2)
    count += 1

cv2.imshow(“output”, frame)

Explanation:

  • Count parameter tracks how many objects are detected
  • Using the cv2.rectangle function we draw a bounding box around detected objects
  • cv2.putText function draws a text string in the frame

detections

Yes, now we have successfully detected pedestrians. But in the frame, there are 3 persons, but we get 4 detections. This is a problem.

To fix this, here is the next step in pedestrian detection.

Step 5 – Post-process output data to filter out the best results:

In the output frame, we can see that some of the bounding boxes overlap each other. If we can remove those overlapped boxes then we’ll have better output detection. So here we can apply non-maxima suppression and suppress bounding boxes that overlap with a significant threshold.

from imutils.object_detection import non_max_suppression

# apply non-maxima suppression to remove overlapped bounding boxes
pedestrians = non_max_suppression(pedestrians, probs=None, overlapThresh=0.5)

Explanation:

  • non_max_suppression is a function of the imutils package
  • Here the argument overlapThresh=0.5 means if any bounding box overlaps other boxes more than 50% then the box will be removed

filtered detection output

Now our pedestrian detection model can successfully detect exactly 3 persons in the frame.

So let’s make it a real-time pedestrian detection system.

Step 6 – Detect real-time pedestrian from Video:

cap = cv2.VideoCapture("video.mp4")
while True:
    _, frame = cap.read()

Explanation:

  • First, create a VideoCapture object and set it as cap.
  • You can replace “video.mp4” with your camera id (i.e 0) if you want to capture video from a webcam.
  • In the while loop, we read all the frames from our capture object.
  • And then we’ll go through the same process we did until now.

Pedestrian Detection OpenCV Output

pedestrian detection opencv output

Summary

In this machine learning project, we have developed pedestrian detection system using opencv and python. We applied a HOG descriptor and linear Support Vector Machine algorithm to perform all the tasks.

Through this project, we’ve learned about how a basic object detector works and how to apply it in real-time.

Leave a Reply

Your email address will not be published. Required fields are marked *