AidStream SDK

Introduction

AidStream SDK is a streaming media processing toolkit designed to build AI-based video analytics application and services. It enables developing the end-to-end accelerated applications with the simplified API encapsulation which eliminates the complexities of underlayer hardware-acceleration (GPU, VPU, NPU) technology.

AidStream is based on the concept of a pipeline, where developers create the predefined streaming data processing pipelines that incorporate AI model inference and other complex tasks, such as video encoding/decoding, video stream pushing/pulling, and video stream storage.

In a complete pipeline, data enter from an Input-Stream, and developers can use AidStream API to obtain the corresponding RGB data. After processing, the data is then fed into the Output-Stream. For example, callback functions can be inserted between the input and output ends to retrieve RGB data. Pre-processing, AI analytics and post-processing are performed within these callback functions before returning the processed data back to the pipeline. The processed RGB data then continues through the subsequent stages of the pipeline's output process.

The AidStream workflow is illustrated below:

Supported Combinations

Input-Stream and Output-Stream support different data types. A pipeline can only be successfully built when the combination of input and output streams is supported by AidStream. The currently supported combinations are shown in the table below:

Input / Output	RTSP (H.264)	RTSP (H.265)	File	Screen
RTSP H.264	✅	❌	✅	❌
RTSP H.265	✅	✅	✅	❌
File	✅	❌	✅	✅
MIPI Camera	✅	❌	✅	✅
USB Camera	✅	❌	✅	✅

• RTSP (H.264): RTSP stream encoded with H.264
• RTSP (H.265): RTSP stream encoded with H.265
• File: Local video file
• MIPI Camera: MIPI interface camera
• USB Camera: USB camera
• Screen: Device display

Quick Start

Installation

APLUX AidLux Dev Board

# AidStream SDK is pre-installed in the Linux system environment

APLUX Linux Container Dev Board

# AidStream SDK is pre-installed in the container image

• Third-party dev board: Please contact APLUX to obtain related software and installation instructions

API Documentation

• Python API
• C++ API

Example

Example programs are located in /usr/local/share/aidstream-gst/example/cxx/

Step 1: Edit the stream configuration file

The configuration file aidstream-gst.conf is located in /usr/local/share/aidstream-gst/conf/.
For configuration details, please refer to Stream Configuration File

Step 2: Compile the example code

• Include headers and link libraries

# Include the header

#include "aidstream.h"

# Link against the AidStream shared object library

$g++ example.cpp -o demo -L/usr/local/lib/ -laidstream-gst -lopencv_core -I/usr/local/include/aidlux/aidstream-gst/ -I/usr/local/lib/aidlux_opencv/include/opencv4/

AidStream SDK header location: /usr/local/include/aidlux/aidstream-gst/aidstream.h

AidStream SDK library location: /usr/local/lib/libaidstream-gst.so

• build

QCS8550 / QCS6490 AIBox

cmake ..
make

QCS6490

cmake -DV4L2=ON ..
make

Step 3: Run the Demo

After successful compilation and linking, four executable programs will be generated: demo, qnn_rtsp, rtsp, and start

# Quickly start a stream

start {stream id}

# Quickly start a stream with built-in algorithm
# The model is located at: /usr/local/share/aidstream-gst/example/datas/

qnn_rtsp {stream id}

# Quickly start multi-threaded push-pull streaming
# (This example uses 2 threads)

rtsp

Stream Configuration File

When using the interface int start_stream(string stream_id, GetImageCB &cb), you can configure the input and output streams using the configuration file
/usr/local/share/aidstream-gst/conf/aidstream-gst.conf.

Details:

• The configuration file must be in JSON format.
• Parameters marked as required must be specified.
• Parameter names are case-sensitive.

Parameter Name	Type	Required	Description
streamsId	string	Yes	Stream ID, uniquely identifies a set of stream configurations
inputType	string	Yes	The protocol type of the input stream. Supported types are: rtsp / rtmp / file. An invalid type or address will result in failure.
inputAddr	string	Yes	The address of the input stream.
outputType	string	Yes	The protocol type of the output stream. Supported types are: rtsp / wayland / empty. If an unsupported type is input, it will be set to empty, meaning the output stream will go to fakesink.
outputAddr	string	Yes	The address of the output stream.
decodeType	string	No	Hardware decoding format. Supported types are: H264 / H265
encodeType	string	No	Hardware encoding format. Supported types are: H264 / H265

Configuration File Example:

💡 Note

The input and output stream addresses can be customized as per the developer's requirements. The example below is for reference only.

{
"streams": [
    {
    "streamsId": "1",
    "properties": {
            "inputType": "rtsp",
            "inputAddr": "rtsp://admin:aidlux123@192.168.110.234:554/h264/ch1/main/av_stream",
            "outputType": "rtsp",
            "outputAddr": "rtsp://192.168.111.115:8554/aidstream-gst-rtsp-test-1",
            "decodeType": "H264",
            "encodeType": "H264"
            }
    },
    {
    "streamsId": "2",
    "properties": {
            "inputType": "file",
            "inputAddr": "./test_kobe.mp4",
            "outputType": "rtsp",
            "outputAddr": "rtsp://192.168.111.115:8554/aidstream-gst-rtsp-test-2",
            "decodeType": "H264",
            "encodeType": "H264"
            }
    },
    {
    "streamsId": "3",
    "properties": {
            "inputType": "rtmp",
            "inputAddr": "rtmp://192.168.111.115:1935/live/stream",
            "outputType": "rtsp",
            "outputAddr": "rtsp://192.168.111.115:8554/aidstream-gst-rtmp-test-3",
            "decodeType": "H264",
            "encodeType": "H264"
            }
    },
    {
    "streamsId": "4",
    "properties": {
            "inputType": "file",
            "inputAddr": "./test_kobe.mp4",
            "outputType": "wayland",
            "outputAddr": "",
            "decodeType": "H264",
            "encodeType": "H264"
            }
    },
    {                                                                                         
    "streamsId": "5",                                                                           
    "properties": {                                                                           
            "inputType": "rtsp",                                                              
            "inputAddr": "rtsp://admin:aidlux123@192.168.110.234:554/h264/ch1/main/av_stream",
            "outputType": "rtsp",
            "outputAddr": "rtsp://192.168.111.115:8554/aidstream-gst-rtsp-test-5",
            "decodeType": "H265",
            "encodeType": "H265"
            }                   
    }
]
}