Python opencv mp4 processing on gpu

I am trying a to make a script that detects specific color in the frame using cv2. CPU version of the code runs same-ish time as the GPU version. Here is GPU snip:

import cv2import numpy as npimport timelower_yellow = np.array([20, 100, 100], dtype=np.uint8)upper_yellow = np.array([30, 255, 255], dtype=np.uint8)gpu_upper_yellow = cv2.cuda_GpuMat().upload(lower_yellow)gpu_upper_skin = cv2.cuda_GpuMat().upload(upper_yellow)cap = cv2.VideoCapture("path/to/any/video/file.mp4")start_time = time.time()frame_number = 1resolution = 3while cap.isOpened():    if frame_number % resolution == 0:        ret, frame = cap.read()        if not ret:            break        # hsv_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)        # masked_frame = get_masked_frame(hsv_frame, "skin")        gpu_frame = cv2.cuda_GpuMat()        gpu_frame.upload(frame)        hsv_frame = cv2.cuda.cvtColor(gpu_frame, cv2.COLOR_BGR2HSV)        masked_frame = cv2.cuda.inRange(hsv_frame, gpu_upper_yellow, gpu_upper_yellow ).download()        total_target_pixel = cv2.countNonZero(masked_frame)        # if total_target_pixel > threshold -> do something        frame_number += 1end_time = time.time()elapsed_time = end_time - start_timeprint(f"Elapsed time: {elapsed_time} seconds")

I do not see any improvements over CPU version of the code and checking the performance panel I can see that GPU is idle. Am I doing it wrong, or is such processing not possible on GPU?

I have correctly installed cudnn and cuda tool kits.