Spaces:
Sleeping
Sleeping
| ## Feed custom video source to `predict_batch` to DeGirum PySDK | |
| This guide is for developers who want to learn how to configure a custom video generator and feed it to DeGirum PySDK to use it for model predictions. In this guide, we will walk you through the steps on creating your own custom video generator and pass it to model object's `predict_batch` function to perform prediction over it. | |
| -------------------------------------------------------------------------------- | |
| ### Simple predictions on video stream | |
| To begin with, lets take a look at how to run predictions on a simple video stream as is. For this, we will start with a simple frame generator function as below: | |
| -------------------------------------------------------------------------------- | |
| # Define generator function to produce video frames | |
| def frame_source(stream): | |
| while True: | |
| ret, frame = stream.read() | |
| if not ret: | |
| break # end of file | |
| yield frame | |
| -------------------------------------------------------------------------------- | |
| We use this function to pass it to our `predict_batch` function as below: | |
| -------------------------------------------------------------------------------- | |
| for result in model.predict_batch(frame_source(stream)): | |
| # Print raw results for each frame | |
| print(result) | |
| -------------------------------------------------------------------------------- | |
| Putting it all together, lets run inference of a video source using simple video generator and `predict_batch` function: | |
| -------------------------------------------------------------------------------- | |
| import degirum as dg | |
| import cv2 | |
| # Declaring variables | |
| # Set your model, inference host address, model zoo, and token in these variables. | |
| your_model_name = "model-name" | |
| your_host_address = "@local" # Can be dg.CLOUD, host:port, or dg.LOCAL | |
| your_model_zoo = "degirum/hailo" | |
| your_token = "<token>" | |
| # Specify the video you will run inference on | |
| your_video = "path/video.mp4" | |
| # Loading a model | |
| model = dg.load_model( | |
| model_name = your_model_name, | |
| inference_host_address = your_host_address, | |
| zoo_url = your_model_zoo, | |
| token = your_token | |
| # optional parameters, such as overlay_show_probabilities = True | |
| ) | |
| # Open your video file | |
| stream = cv2.VideoCapture(your_video) | |
| # Define generator function to produce video frames | |
| def frame_source(stream): | |
| while True: | |
| ret, frame = stream.read() | |
| if not ret: | |
| break # end of file | |
| yield frame | |
| # Run predict_batch on stream of frames from video file | |
| for result in model.predict_batch(frame_source(stream)): | |
| # Print raw results for each frame | |
| print(result) | |
| # Release stream | |
| stream.release() | |
| -------------------------------------------------------------------------------- | |
| #### Modify video generator to use webcam or Rpi stream | |
| In order to use webcam or an Rpi stream as video source, we will just modify our video source_path as below: | |
| -------------------------------------------------------------------------------- | |
| ##### Webcam input | |
| -------------------------------------------------------------------------------- | |
| import degirum as dg | |
| import cv2 | |
| # Declaring variables | |
| # Set your model, inference host address, model zoo, and token in these variables. | |
| your_model_name = "model-name" | |
| your_host_address = "@local" # Can be dg.CLOUD, host:port, or dg.LOCAL | |
| your_model_zoo = "degirum/hailo" | |
| your_token = "<token>" | |
| # Loading a model | |
| model = dg.load_model( | |
| model_name = your_model_name, | |
| inference_host_address = your_host_address, | |
| zoo_url = your_model_zoo, | |
| token = your_token | |
| # optional parameters, such as overlay_show_probabilities = True | |
| ) | |
| # Open your video file | |
| stream = cv2.VideoCapture(0) # Webcam source : 0 | |
| # Define generator function to produce video frames | |
| def frame_source(stream): | |
| while True: | |
| ret, frame = stream.read() | |
| if not ret: | |
| break # end of file | |
| yield frame | |
| # Run predict_batch on stream of frames from video file | |
| for result in model.predict_batch(frame_source(stream)): | |
| # Print raw results for each frame | |
| print(result) | |
| # Release stream | |
| stream.release() | |
| -------------------------------------------------------------------------------- | |
| #### Rpi Camera using PiCamera Module | |
| Similar to above example where we used OpenCV to capture frames from webcam, we can use `Picamera2` module to fetch frames from an Rpi Camera inside our frame generator function and pass that to `predict_batch` | |
| -------------------------------------------------------------------------------- | |
| import cv2 | |
| import degirum as dg | |
| import numpy as np | |
| from picamera2 import Picamera2 | |
| your_model_name = "model-name" | |
| your_host_address = "@local" # Can be dg.CLOUD, host:port, or dg.LOCAL | |
| your_model_zoo = "degirum/hailo" | |
| your_token = "<token>" | |
| # Load the model | |
| model = dg.load_model( | |
| model_name = your_model_name, | |
| inference_host_address = your_host_address, | |
| zoo_url = your_model_zoo, | |
| token = your_token | |
| # optional parameters, such as overlay_show_probabilities = True | |
| ) | |
| # Define frame generator using Picamera2 | |
| def picamera2_frame_generator(): | |
| picam2 = Picamera2() | |
| picam2.configure(picam2.preview_configuration(main={"format": 'BGR888'})) | |
| picam2.start() | |
| try: | |
| while True: | |
| frame = picam2.capture_array() | |
| yield frame | |
| finally: | |
| picam2.stop() | |
| # Run inference and display | |
| for result in model.predict_batch(picamera2_frame_generator(rotate=True)): | |
| cv2.imshow("AI Inference PiCamera2", result.image_overlay) | |
| if cv2.waitKey(1) & 0xFF == ord("q"): | |
| break | |
| cv2.destroyAllWindows() | |
| -------------------------------------------------------------------------------- | |
| #### Transform your video source before predictions | |
| Lets a take a look at how can you perform pre-processing or transformation on your input video source like rotation, resize, crop etc and perform model prediction over it. In this example, we will take camera input and rotate it 90' clockwise before passing it to `predict_batch` function. | |
| -------------------------------------------------------------------------------- | |
| For this, we will modify our video generator as follows: | |
| -------------------------------------------------------------------------------- | |
| def rotated_frame_generator(video_source): | |
| stream = cv2.VideoCapture(video_source) | |
| try: | |
| while True: | |
| ret, frame = stream.read() | |
| if not ret: | |
| break | |
| rotated_frame = cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE) # Rotate frame 90 degrees clockwise | |
| yield rotated_frame | |
| finally: | |
| stream.release() | |
| -------------------------------------------------------------------------------- | |
| Putting it all together, it looks like this: | |
| -------------------------------------------------------------------------------- | |
| import degirum as dg | |
| import degirum_tools.video_support as vs | |
| import cv2 | |
| import degirum_tools | |
| hw_location = "@local" | |
| model_zoo_url = "model_zoo_url" | |
| model_name = "your_model_name" | |
| video_source_path = 0 # Webcam source : 0 | |
| degirum_cloud_token = "<token>" # Optional, only needed for cloud inference | |
| # Load the model | |
| model = dg.load_model( | |
| model_name=model_name, | |
| inference_host_address=hw_location, | |
| zoo_url=model_zoo_url, | |
| token=degirum_cloud_token, | |
| overlay_color=(0, 255, 0) | |
| ) | |
| # Define rotated frame generator | |
| def rotated_frame_generator(video_source): | |
| stream = cv2.VideoCapture(video_source) | |
| try: | |
| while True: | |
| ret, frame = stream.read() | |
| if not ret: | |
| break | |
| rotated_frame = cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE) # Rotate frame 90 degrees clockwise | |
| yield rotated_frame | |
| finally: | |
| stream.release() | |
| # Run inference and display | |
| for result in model.predict_batch(rotated_frame_generator(video_source_path)): | |
| cv2.imshow("AI Inference", result.image_overlay) | |
| if cv2.waitKey(1) & 0xFF == ord("q"): | |
| break | |
| cv2.destroyAllWindows() | |
| -------------------------------------------------------------------------------- | |
| #### Conclusion | |
| In this guide, we have walked through the steps of setting up a custom video frame generator and pass it to model predictions using DeGirum PySDK. These steps allows you to modify and transform your video source feed and perform different operations such as rotation, crop etc and use the transformed video frames for model prediction. Such a process is useful especially when you want to modify your original source input for better accuracy on model predictions. | |
| -------------------------------------------------------------------------------- | |