I'm working on an AI project that involves object detection and action recognition on roughly 30 minute videos.
My pipeline is the following:
- determine crops using object detection model
- extract crops using Python and write them to disk as individual images for each frame.
- use action recognition model by inputting a series of the crops.
The models are fast but actual writing of the crops to disk is slow. Sure, using an SSD would speed it up but I'm sure ffmpeg would greatly speed it up.
Some of the challenges with the crops:
- the output size is always 128x128
- the input size is variable
- the crop moves on every frame
My process for extracting crops is simple using cv2.imwrite(output_crop_path, crop) in a for loop.
I've done experiments trying to use sndcommand and filter_complex. I tried this https://stackoverflow.com/a/67508233/4447761 but it outputs an image with black below the crop and the image gets wrapped around on the x axis.
