ESP32-CAM motion detection

Motion detection is the task of detecting when the scene in the ESP32 camera field of view changes all of a sudden.

This change may be caused by a lot of factors (an object moving, the camera itself moving, a light change...) and you may be interested in get notified when it happens.

For example, you may point your ESP32 camera to the door of your room and take a picture when the door opens.

In a scenario like this, you are not interested in localizing the motion (knowing where it happened in the frame), only that it happened.

Motion detection with PIR (infrared sensor)

Most tutorials on the web focus on human motion detection, so they equip the ESP32 with an external infrared sensor (a.k.a PIR, the one you find in home alarm systems) and take a photo when the PIR detects something.

If this setup works fine for you, go with it. It's easy, fast, low power and pretty accurate.

But the PIR approach has a few drawbacks:

1. you can only detect living beings: since it is based on infrared sensing, it can only detect when something "hot" is in its field of view (humans and animals, basically). If you want to detect a car passing, it won't work
2. it has a limited range: PIR sensors reach at most 10-15 meters. If you need to detect people walking on the street in front of your house at 30 meters, it won't work
3. it needs a clear line-of-sight: to detect infrared light, the PIR sensor needs no obstacles in-between itself and the moving object. If you put it behind a window to detect people outside your home, it won't work
4. it falsely triggers even when no motion happened: the PIR tecnique is actually a proxy for motion detection. The PIR sensor doesn't actually detects motion: it detects the presence of warm objects. For example, if a person comes into a room and lies down on the sofa, the PIR sensor will trigger for as long as the person doesn't leave the room

Motion detection without PIR (image based)

On the other hand, image motion detection can fulfill all the above cases because it performs motion detection on the camera frames, comparing each one with the previous looking for differences.

If a large portion of the image changed, it triggers.

Video motion detection has its drawbacks, nonetheless:

1. power-hungry: comparing each frame with the previous frame means the camera must be always on. While with the PIR sensor you can put the camera to sleep, now you have to continuously check each frame
2. insensitive to slow changes: to avoid false triggers, you will set a lower threshold on the image portion that need to change to detect motion (e.g. 10% of the frame). If something is moving slowly in your field of view such that it changes less than 10% of the frame, the algorithm will not pick it up.

Take some time to review the pros and cons of video motion detection now that you have a little more details.

You will also need to install the excellent JPEGDEC library from Larry Bank.

The ESPx way to do motion detection on the ESP32-CAM

The simple case for motion detection is to grab a frame and run the detection inside the main loop. You can configure a few parameters for the detection:

• pixel delta: how much should the current pixel differ from the previous frame to be considered in the change count? The higher this value, the less sensitive the algorithm is.
• smooth (from 0 to 1): when updating the background model, the algorithm makes a weighted average between the current frame and the previous frame. The higher this value, the more importance is given to the past frames.
• train epochs: before the model starts the detection, you can form the background model on a given number of frames. This helps the algorithm understand what the scene looks like and what instead makes a change.
• throttle: even though this is not a core part of motion detection, it still is very relevant. If something is moving in the frame, it will trigger the detection for as long as it moves (and a few frames later, until the background model stabilizes). You may only be interested in only getting a single event instead and mute the next ones for a given amount of time. This mute process is called throttling.
• threshold: how much percent of the image should change to be considered as a motion event?

Here's how you set these values in code. Everything is scoped under the motionx object.

// configure motion detection
// 1. configure smoothing factor
//    (the higher, the more past frames are taken into account)
motionx.smooth(0.8);
// 2. for the first N frames, just update the background
//    without running the detection
motionx.trainFor(10);
// 3. count in "motion" only pixels that changed value
//    by more than X
motionx.diffBy(10);
// 4. configure moving threshold to trigger a positive detection:
//    trigger when Y percent of pixels changed
motionx.threshold("30%");
// 5. drop motion events in rapid succession:
//    limit to at most one every N seconds
motionx.throttle("3s");

To feed frames to the motion detector, just grab one from the camera and call process(). If you don't know how to take frames from the camera, refer to ESP32-CAM: take first picture.

void loop() {
    auto frame = camx.grab();

    if (!motionx.process(frame)) {
        Serial.println(motionx.failure());
        return;
    }

    Serial.printf(
        "moving pixels %%=%.2f, detection time=%dms\n", 
        motionx.ratio(), motionx.stopwatch.millis()
    );

    // motionx is true if ratio >= threshold
    if (motionx) {
      Serial.println("Motion detected");
    }
}

Next steps

This post showcases the basic workflow to deal with motion detection from ESP32-CAM frames without any external PIR sensor. It gives you a lot of flexibility to accomodate the vast majority of use cases.

It leaves the actuation stage empty on purpose, so you can apply your specific logic:

• activate a relay
• send an MQTT message
• store the picture on a SD card

You can do whatever you want inside the if (motionx) block: it's that easy!

Complete sketch

/**
 * Perform motion detection on camera frames
 */
#include <JPEGDEC.h>
#include <espx.h>
#include <espx/camx.h>
#include <espx/motionx.h>


void setup() {
    delay(1000);
    Serial.begin(115200);
    Serial.println("Motionx example: motion detection");

    // configure camx through Serial Monitor
    camx.model.prompt();
    camx.pixformat.jpeg();
    camx.quality.high();
    camx.resolution.vga();

    // initialize camx,
    // enter endless loop on error
    camx.begin().raise();

    // configure motion detection
    // 1. configure smoothing factor
    //    (the higher, the more past frames are taken into account)
    motionx.smooth(0.8);
    // 2. for the first N frames, just update the background
    //    without running the detection
    motionx.trainFor(10);
    // 3. count in "motion" only pixels that changed value
    //    by more than X
    motionx.diffBy(10);
    // 4. configure moving threshold to trigger a positive detection:
    //    trigger when Y percent of pixels changed
    motionx.threshold("30%");
    // 5. drop motion events in rapid succession:
    //    limit to at most one every N seconds
    motionx.throttle("3s");
}


void loop() {
    auto frame = camx.grab();

    if (!motionx.process(frame)) {
        Serial.println(motionx.failure());
        return;
    }

    Serial.printf(
            "moving pixels %%=%.2f, detection time=%dms\n",
            motionx.ratio(), motionx.stopwatch.millis()
    );

    // motionx is true if ratio >= threshold
    if (motionx) {
      Serial.println("Motion detected");
    }
}