text_files/016_custom_video_source.txt

## Feed custom video source to `predict_batch` to DeGirum PySDK

This guide is for developers who want to learn how to configure a custom video generator and feed it to DeGirum PySDK to use it for model predictions. In this guide, we will walk you through the steps on creating your own custom video generator and pass it to model object's `predict_batch` function to perform prediction over it.


--------------------------------------------------------------------------------

### Simple predictions on video stream 

To begin with, lets take a look at how to run predictions on a simple video stream as is. For this, we will start with a simple frame generator function as below:

--------------------------------------------------------------------------------

# Define generator function to produce video frames
def frame_source(stream):
    while True:
      ret, frame = stream.read()
      if not ret:
         break # end of file
      yield frame

--------------------------------------------------------------------------------

We use this function to pass it to our `predict_batch` function as below:

--------------------------------------------------------------------------------

for result in model.predict_batch(frame_source(stream)):
    # Print raw results for each frame
    print(result)


--------------------------------------------------------------------------------

Putting it all together, lets run inference of a video source using simple video generator and `predict_batch` function:

--------------------------------------------------------------------------------

import degirum as dg
import cv2

# Declaring variables
# Set your model, inference host address, model zoo, and token in these variables.
your_model_name = "model-name"
your_host_address = "@local" # Can be dg.CLOUD, host:port, or dg.LOCAL
your_model_zoo = "degirum/hailo"
your_token = "<token>"

# Specify the video you will run inference on
your_video = "path/video.mp4"

# Loading a model
model = dg.load_model(
    model_name = your_model_name, 
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token 
    # optional parameters, such as overlay_show_probabilities = True
)

# Open your video file
stream = cv2.VideoCapture(your_video) 

# Define generator function to produce video frames
def frame_source(stream):
    while True:
      ret, frame = stream.read()
      if not ret:
         break # end of file
      yield frame

# Run predict_batch on stream of frames from video file
for result in model.predict_batch(frame_source(stream)):
    # Print raw results for each frame
    print(result)

# Release stream
stream.release()

--------------------------------------------------------------------------------

#### Modify video generator to use webcam or Rpi stream

In order to use webcam or an Rpi stream as video source, we will just modify our video source_path as below:

--------------------------------------------------------------------------------

##### Webcam input

--------------------------------------------------------------------------------

import degirum as dg
import cv2

# Declaring variables
# Set your model, inference host address, model zoo, and token in these variables.
your_model_name = "model-name"
your_host_address = "@local" # Can be dg.CLOUD, host:port, or dg.LOCAL
your_model_zoo = "degirum/hailo"
your_token = "<token>"


# Loading a model
model = dg.load_model(
    model_name = your_model_name, 
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token 
    # optional parameters, such as overlay_show_probabilities = True
)

# Open your video file
stream = cv2.VideoCapture(0) # Webcam source : 0 

# Define generator function to produce video frames
def frame_source(stream):
    while True:
      ret, frame = stream.read()
      if not ret:
         break # end of file
      yield frame

# Run predict_batch on stream of frames from video file
for result in model.predict_batch(frame_source(stream)):
    # Print raw results for each frame
    print(result)

# Release stream
stream.release()

--------------------------------------------------------------------------------

#### Rpi Camera using PiCamera Module
Similar to above example where we used OpenCV to capture frames from webcam, we can use `Picamera2` module to fetch frames from an Rpi Camera inside our frame generator function and pass that to `predict_batch`

--------------------------------------------------------------------------------

import cv2
import degirum as dg
import numpy as np
from picamera2 import Picamera2

your_model_name = "model-name"
your_host_address = "@local" # Can be dg.CLOUD, host:port, or dg.LOCAL
your_model_zoo = "degirum/hailo"
your_token = "<token>"

# Load the model
model = dg.load_model(
    model_name = your_model_name, 
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token 
    # optional parameters, such as overlay_show_probabilities = True
)

# Define frame generator using Picamera2
def picamera2_frame_generator():
    picam2 = Picamera2()
    picam2.configure(picam2.preview_configuration(main={"format": 'BGR888'}))
    picam2.start()
    try:
        while True:
            frame = picam2.capture_array()
            yield frame
    finally:
        picam2.stop()

# Run inference and display
for result in model.predict_batch(picamera2_frame_generator(rotate=True)):
    cv2.imshow("AI Inference PiCamera2", result.image_overlay)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cv2.destroyAllWindows()


--------------------------------------------------------------------------------

#### Transform your video source before predictions

Lets a take a look at how can you perform pre-processing or transformation on your input video source like rotation, resize, crop etc and perform model prediction over it. In this example, we will take camera input and rotate it 90' clockwise before passing it to `predict_batch` function.

--------------------------------------------------------------------------------

For this, we will modify our video generator as follows:

--------------------------------------------------------------------------------

def rotated_frame_generator(video_source):
    stream = cv2.VideoCapture(video_source)
    try:
        while True:
            ret, frame = stream.read()
            if not ret:
                break
            rotated_frame = cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE) # Rotate frame 90 degrees clockwise
            yield rotated_frame
    finally:
        stream.release()

--------------------------------------------------------------------------------

Putting it all together, it looks like this:

--------------------------------------------------------------------------------

import degirum as dg
import degirum_tools.video_support as vs
import cv2
import degirum_tools

hw_location = "@local"
model_zoo_url = "model_zoo_url"
model_name = "your_model_name"
video_source_path = 0 # Webcam source : 0
degirum_cloud_token = "<token>" # Optional, only needed for cloud inference

# Load the model
model = dg.load_model(
    model_name=model_name,
    inference_host_address=hw_location,
    zoo_url=model_zoo_url,
    token=degirum_cloud_token,
    overlay_color=(0, 255, 0)
)

# Define rotated frame generator
def rotated_frame_generator(video_source):
    stream = cv2.VideoCapture(video_source)
    try:
        while True:
            ret, frame = stream.read()
            if not ret:
                break
            rotated_frame = cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE)  # Rotate frame 90 degrees clockwise
            yield rotated_frame
    finally:
        stream.release()

# Run inference and display
for result in model.predict_batch(rotated_frame_generator(video_source_path)):
    cv2.imshow("AI Inference", result.image_overlay)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cv2.destroyAllWindows()


--------------------------------------------------------------------------------

#### Conclusion

In this guide, we have walked through the steps of setting up a custom video frame generator and pass it to model predictions using DeGirum PySDK. These steps allows you to modify and transform your video source feed and perform different operations such as rotation, crop etc and use the transformed video frames for model prediction. Such a process is useful especially when you want to modify your original source input for better accuracy on model predictions. 

--------------------------------------------------------------------------------