Fastest way to detect the non/least-changing pixels of successive images

问题

I want to find the pixels of a video stream that are static. This way I can detect logos and other non-moving items on my video stream. My idea behind the script is as follows:

collect a number of equally-sized and graysized frames in a list called previous
if a certain amount of frames is collected, call the function np.std
This function loops over all the x-and y-coordinates of a new image.
Calculate the standard deviation of the grayvalues for all the coordinates based on the grayvalues of the corresponding coordinates of all the frames

My script:

import math
import cv2
import numpy as np


video = cv2.VideoCapture(0)
previous = []
n_of_frames = 200

while True:
   ret, frame = video.read()
   if ret:
      cropped_img = frame[0:150, 0:500]
      gray = cv2.cvtColor(cropped_img, cv2.COLOR_BGR2GRAY)
      if len(previous) == n_of_frames:
         stdev_gray = np.std(previous, axis=2)
         previous = previous[1:]
         previous.append(gray)
      else:
         previous.append(gray)

      cv2.imshow('frame', frame)

      key = cv2.waitKey(1)
      if key == ord('q'):
         break

video.release()
cv2.destroyAllWindows()

This process is pretty slow and I am curious if there are faster ways to do this. I am open to Cython etc. Many many thanks in advance!

回答1:

An approach is to compare each frame-by-frame using cv2.bitwise_and(). The idea is that pixels in the previous frame must be present in the current frame to be a non-changing pixel. By iterating through the list of frames, all features in the scene must be present in the previous and current frame to be considered a non-moving item. So if we sequentially iterate through each frame, the last iteration will have shared features from all previous frames.

Using this set of frames captured once per second

We convert each frame to grayscale then cv2.bitwise_and() with the previous and current frame. The non-changing pixels of each successive iteration are highlighted in gray while changing pixels are black. The very last iteration should showcase pixels shared between all frames.

If instead you also thresholded each frame, you get a more pronounced result

import cv2
import glob

images = [cv2.imread(image, 0) for image in glob.glob("*.png")]

result = cv2.bitwise_and(images[0], images[1])
for image in images[2:]:
    result = cv2.bitwise_and(result, image)

cv2.imshow('result', result)
cv2.waitKey(0)

回答2:

It is possible to compute variance and standard deviation from sum and sum of squares.

VAR X = EX^2 - (EX)^2

See link https://en.wikipedia.org/wiki/Variance#Definition

Sum and Sum of squares can be updates sequentially by adding a new image and subtracting an image captures n_of_frames ago. Next compute a variance and take a square root to get standard deviation. Note that computation time does not depend on number of frames.

See the code

import math
import cv2
import numpy as np


video = cv2.VideoCapture(0)
previous = []
n_of_frames = 200

sum_of_frames = 0
sumsq_of_frames = 0

while True:
   ret, frame = video.read()
   if ret:
      cropped_img = frame[0:150, 0:500]
      gray = cv2.cvtColor(cropped_img, cv2.COLOR_BGR2GRAY)
      gray = gray.astype('f4')
      if len(previous) == n_of_frames:
         stdev_gray = np.sqrt(sumsq_of_frames / n_of_frames - np.square(sum_of_frames / n_of_frames))
         cv2.imshow('stdev_gray', stdev_gray * (1/255))
         sum_of_frames -= previous[0]
         sumsq_of_frames -=np.square(previous[0])
         previous.pop(0)
      previous.append(gray)
      sum_of_frames = sum_of_frames + gray
      sumsq_of_frames = sumsq_of_frames + np.square(gray)

      #cv2.imshow('frame', frame)

      key = cv2.waitKey(1)
      if key == ord('q'):
         break

video.release()
cv2.destroyAllWindows()

Result looks pretty awesome.

来源：https://stackoverflow.com/questions/58877724/fastest-way-to-detect-the-non-least-changing-pixels-of-successive-images

标签

python

numpy

OpenCV