AidLite C++ API

💡 Note

To use the AidLite-SDK C++ for development, please note the following:

• During compilation, include the header file located at /usr/local/include/aidlux/aidlite/aidlite.hpp.
• During linking, specify the library file located at /usr/local/lib/libaidlite.so.

Model Data Type enum DataType

For the AidLite SDK, different models from various frameworks are processed, each with its own input and output data types. When setting the input and output data types for a model in the usage flow described earlier, this enumeration is used.

Member Variable	Type	Value	Description
TYPE_DEFAULT	uint8_t	0	Invalid DataType
TYPE_UINT8	uint8_t	1	Unsigned byte data
TYPE_INT8	uint8_t	2	Byte data
TYPE_UINT32	uint8_t	3	Unsigned int32 integer
TYPE_FLOAT32	uint8_t	4	Float data
TYPE_INT32	uint8_t	5	Int32 data
TYPE_INT64	uint8_t	6	Int64 data

Inference Implementation Type enum ImplementType (Deprecated)

Member Variable	Type	Value	Description
TYPE_DEFAULT	uint8_t	0	Invalid ImplementType
TYPE_MMKV	uint8_t	1	Implemented via MMKV (not currently implemented, may be removed later)
TYPE_FAST	uint8_t	2	Implemented via IPC backend
TYPE_LOCAL	uint8_t	3	Implemented via local backend

💡 Note

Important: Starting from AidLite-SDK C++ version V2.0.4, ImplementType is deprecated.

Model Framework Type enum FrameworkType

As mentioned earlier, the AidLite SDK integrates multiple deep learning inference frameworks. Therefore, in the usage flow, it is necessary to specify which framework’s model is being used, requiring this framework type enumeration.

Member Variable	Type	Value	Description
TYPE_DEFAULT	uint8_t	0	Invalid FrameworkType
TYPE_SNPE	uint8_t	1	SNPE (DLC) model type
TYPE_TFLITE	uint8_t	2	TFLite model type
TYPE_RKNN	uint8_t	3	RKNN model type
TYPE_QNN	uint8_t	4	QNN model type
TYPE_SNPE2	uint8_t	5	SNPE2 (DLC) model type
TYPE_NCNN	uint8_t	6	NCNN model type
TYPE_MNN	uint8_t	7	MNN model type
TYPE_TNN	uint8_t	8	TNN model type
TYPE_PADDLE	uint8_t	9	Paddle model type

Acceleration Hardware Type enum AccelerateType

Each deep learning inference framework may support running on different acceleration devices (e.g., SNPE models on DSP devices, RKNN models on NPU devices). Thus, in the usage flow, it is necessary to specify which device the model is expected to run on, requiring this acceleration hardware enumeration.

Member Variable	Type	Value	Description
TYPE_DEFAULT	uint8_t	0	Invalid AccelerateType
TYPE_CPU	uint8_t	1	CPU acceleration
TYPE_GPU	uint8_t	2	GPU acceleration
TYPE_DSP	uint8_t	3	DSP acceleration
TYPE_NPU	uint8_t	4	NPU acceleration

Log Level enum LogLevel

The AidLite SDK provides an interface for setting logs (introduced later), which requires specifying the log level to be used, necessitating this log level enumeration.

Member Variable	Type	Value	Description
INFO	uint8_t	0	Information
WARNING	uint8_t	1	Warning
ERROR	uint8_t	2	Error
FATAL	uint8_t	3	Fatal error

Model Class class Model

As mentioned earlier, before creating an inference interpreter, detailed parameters related to the specific model need to be set. The Model class is primarily used to record the model’s file information, structural information, and related content during model execution.

Create Instance create_instance()

To set detailed model information, a model instance object is required first. This function is used to create a Model instance object. If the model framework (e.g., SNPE, TFLite) requires only one model file, this interface is used.

API	create_instance
Description	Constructs a Model type instance object by passing the model file’s path and name.
Parameters	model_path: Path and name of the model file.
Return Value	If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Model object.

// Create a model object using the inceptionv3_float32.dlc file in the current path; report an error if the return value is null
Model* model = Model::create_instance("./inceptionv3_float32.dlc");
if(model == nullptr){
    printf("Create model failed !\n");
    return EXIT_FAILURE;
}

Create Instance create_instance()

To set detailed model information, a model instance object is required first. This function is used to create a Model instance object. If the model framework (e.g., NCNN, TNN) involves two model files, this interface is used.

API	create_instance
Description	Constructs a Model type instance object by passing the model file’s path and name.
Parameters	model_struct_path: Path and name of the model structure file.
	model_weight_path: Path and name of the model weight parameters file.
Return Value	If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Model object.

// Create a model object using two NCNN model files in the current path; report an error if the return value is null
Model* model = Model::create_instance("./mobilenet_ssd_voc_ncnn.param", 
     "./mobilenet_ssd_voc_ncnn.bin");
if(model == nullptr){
    printf("Create model failed !\n");
    return EXIT_FAILURE;
}

Set Model Properties set_model_properties()

After successfully creating the model instance object, it is necessary to set the input and output data types and the shape information of the input and output tensors.

API	set_model_properties
Description	Sets the model’s properties, including input and output data shapes and data types.
Parameters	input_shapes: Array of input tensor shapes, in a 2D array structure.
	input_data_type: Input tensor data type, DataType enumeration type.
	output_shapes: Array of output tensor shapes, in a 2D array structure.
	output_data_type: Output tensor data type, DataType enumeration type.
Return Value	void

// Use the previously defined DataType enumeration; input and output shapes are 2D arrays
std::vector<std::vector<uint32_t>> input_shapes = {{1,299,299,3}};
std::vector<std::vector<uint32_t>> output_shapes = {{1,1001}};
model->set_model_properties(input_shapes, DataType::TYPE_FLOAT32, 
    output_shapes, DataType::TYPE_FLOAT32);

Get Model Path get_model_absolute_path()

If the model framework (e.g., SNPE, TFLite) involves only one model file, and the Model object is constructed without exceptions, the provided model file parameter will be converted to an absolute path.

API	get_model_absolute_path
Description	Retrieves the existing path of the model file.
Parameters	void
Return Value	The model file path string corresponding to the Model object.

Get Model Structure Path get_model_struct_absolute_path()

If the model framework (e.g., NCNN, TNN) involves two model files, and the Model object is constructed without exceptions, the provided model structure file parameter will be converted to an absolute path.

API	get_model_struct_absolute_path
Description	Retrieves the existing path of the model structure file.
Parameters	void
Return Value	The model structure file path string corresponding to the Model object.

Get Model Weight Path get_model_weight_absolute_path()

If the model framework (e.g., NCNN, TNN) involves two model files, and the Model object is constructed without exceptions, the provided model weight parameters file parameter will be converted to an absolute path.

API	get_model_weight_absolute_path
Description	Retrieves the existing path of the model weight parameters file.
Parameters	void
Return Value	The model weight parameters file path string corresponding to the Model object.

Configuration Class class Config

As mentioned earlier, before creating an inference interpreter, in addition to setting specific Model information, some configuration information for inference is also required. The Config class is used to record configuration options that need to be preset and will be used during runtime.

Create Instance create_instance()

To set runtime configuration information, a configuration instance object is required first. This function is used to create a Config instance object.

API	create_instance
Description	Constructs a Config class instance object.
Parameters	void
Return Value	If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Config object.

// Create a config instance object; report an error if the return value is null
Config* config = Config::create_instance();
if(config == nullptr){
    printf("Create config failed !\n");
    return EXIT_FAILURE;
}

Member Variables List

The Config object is used to set runtime configuration information, including the following parameters:

Member Variable	accelerate_type
Type	enum AccelerateType
Default Value	AccelerateType::TYPE_CPU
Description	Type of acceleration hardware.

Member Variable	implement_type (Deprecated)
Type	enum ImplementType
Default Value	ImplementType::TYPE_DEFAULT
Description	Distinguishes the underlying implementation method.

💡 Note

Important: Starting from AidLite-SDK C++ version V2.0.4, ImplementType is deprecated.

Member Variable	framework_type
Type	enum FrameworkType
Default Value	FrameworkType::TYPE_DEFAULT
Description	Type of underlying inference framework.

Member Variable	number_of_threads
Type	int8_t
Default Value	-1
Description	Number of threads, effective if greater than 0.

Member Variable	snpe_out_names
Type	String array
Default Value	Empty
Description	List of model output node names for FAST.

Member Variable	is_quantify_model
Type	int32_t
Default Value	0
Description	Whether it is a quantized model, 1 for quantized model for FAST.

Member Variable	fast_timeout
Type	int32_t
Default Value	-1
Description	Interface timeout in milliseconds, effective if greater than 0 for FAST.

config->framework_type = FrameworkType::TYPE_SNPE;
config->accelerate_type = AccelerateType::TYPE_CPU;
config->is_quantify_model = 0;
…
config->snpe_out_names.push_back("InceptionV3/Softmax");

Context Class class Context

Used to store the runtime context involved in the execution process, including the Model object and Config object. Additional runtime data may be extended in the future.

Constructor Context()

API	Context
Description	Constructs a Context instance object.
Parameters	model: Pointer to the Model instance object.
	config: Pointer to the Config instance object.
Return Value	Normal: Context instance object.

💡 Note

Important: The memory space pointed to by the model and config pointers must be allocated on the heap. After successfully constructing the Context, their lifecycles will be managed by the Context (when the Context is released, it will automatically release the model and config objects). Therefore, developers should not manually release their memory, as this will cause errors due to double freeing.

Get Model Member Variable get_model()

API	get_model
Description	Retrieves the Model object pointer managed by the context.
Parameters	void
Return Value	Pointer to the Model object.

Get Config Member Variable get_config()

API	get_config
Description	Retrieves the Config object pointer managed by the context.
Parameters	void
Return Value	Pointer to the Config object.

Interpreter Class class Interpreter

The Interpreter type object instance is the main entity for performing inference operations, used to execute the specific inference process. As mentioned in the previous sections, after creating the interpreter object, all operations are performed based on the interpreter object, making it the absolute core of the AidLite SDK.

Create Instance create_instance()

To perform inference-related operations, an inference interpreter is essential. This function is used to construct an instance of the inference interpreter.

API	create_instance
Description	Constructs an Interpreter type object using the data managed by the Context object.
Parameters	context: Pointer to the Context instance object.
Return Value	If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Interpreter object.

// Create an interpreter object using the Context object pointer; report an error if the return value is null
Interpreter* current_interpreter = Interpreter::create_instance(context);
if(current_interpreter == nullptr){
    printf("Create Interpreter failed !\n");
    return EXIT_FAILURE;
}

Initialize init()

After creating the interpreter object, some initialization operations (e.g., environment checks, resource construction) need to be performed.

API	init
Description	Performs the initialization operations required for inference.
Parameters	void
Return Value	0 indicates successful initialization; non-zero indicates failure.

// Initialize the interpreter; report an error if the return value is non-zero
int result = fast_interpreter->init();
if(result != EXIT_SUCCESS){
    printf("sample : interpreter->init() failed !\n");
    return EXIT_FAILURE;
}

Load Model load_model()

After the interpreter object completes initialization, the required model file can be loaded for the interpreter, completing the model loading process. Subsequent inference processes will use the loaded model resources.

API	load_model
Description	Completes model loading operations. Since the model file path is already set in the Model object, the model loading operation can be executed directly.
Parameters	void
Return Value	0 indicates successful model loading; non-zero indicates failure.

// Load the model for the interpreter; report an error if the return value is non-zero
int result = fast_interpreter->load_model();
if(result != EXIT_SUCCESS){
    printf("sample : interpreter->load_model() failed !\n");
    return EXIT_FAILURE;
}

Set Input Data set_input_tensor()

As mentioned in the flow introduction, before setting input data, different preprocessing operations are required for different models to adapt to the specific model.

API	set_input_tensor
Description	Sets the input tensor data required for inference.
Parameters	in_tensor_idx: Index value of the input tensor.
	input_data: Pointer to the storage address of the input tensor data.
Return Value	0 indicates successful setting of the input tensor; non-zero indicates failure.

// Set the input data for inference; report an error if the return value is non-zero
int result = fast_interpreter->set_input_tensor(0, input_tensor_data);
if(result != EXIT_SUCCESS){
    printf("interpreter->set_input_tensor() failed !\n");
    return EXIT_FAILURE;
}

Execute Inference invoke()

As mentioned in the flow introduction, after setting the input data, the next step is to execute the inference process on the input data.

API	invoke
Description	Executes the inference computation process.
Parameters	void
Return Value	0 indicates successful inference; non-zero indicates failure.

// Execute the inference operation; report an error if the return value is non-zero
int result = fast_interpreter->invoke();
if(result != EXIT_SUCCESS){
    printf("sample : interpreter->invoke() failed !\n");
    return EXIT_FAILURE;
}

Get Output Data get_output_tensor()

After inference is complete, the resulting data needs to be retrieved. As mentioned in the flow introduction, after retrieving the result data, it can be processed to determine if the results are correct.

API	get_output_tensor
Description	Retrieves the inference result data after successful inference.
Parameters	out_tensor_idx: Index value of the output tensor.
	output_data: Address of the pointer variable, the function resets the value of this pointer variable.
	output_length: Data size (in bytes) of the output tensor.
Return Value	0 indicates successful retrieval of the output tensor; non-zero indicates failure.

// Retrieve the inference result data; report an error if the return value is non-zero
float* out_data = nullptr;
uint32_t output_tensor_length = 0;
int result = fast_interpreter->get_output_tensor(0, 
   (void**)&out_data, &output_tensor_length);
if(result != EXIT_SUCCESS){
    printf("interpreter->get_output_tensor() failed !\n");
    return EXIT_FAILURE;
}

Resource Release destroy()

As mentioned earlier, the interpreter object requires initialization (init) and model loading operations. Correspondingly, the interpreter also needs to perform release operations to destroy previously created resources.

API	destroy
Description	Performs necessary release operations.
Parameters	void
Return Value	0 indicates successful release; non-zero indicates failure.

// Execute the interpreter release process; report an error if the return value is non-zero
int result = fast_interpreter->destroy();
if(result != EXIT_SUCCESS){
    printf("sample : interpreter->destroy() failed !\n");
    return EXIT_FAILURE;
}

Interpreter Builder Class class InterpreterBuilder

A unified creation function for Interpreter objects, used to create the required interpreter objects through this class.

Build Interpreter build_interpreter_from_path()

Builds an inference interpreter object, allowing different parameters to be provided. The simplest approach is to provide only the model file’s path and name. If the model framework (e.g., SNPE, TFLite) requires only one model file, this interface is used.

API	build_interpreter_from_path
Description	Directly creates the corresponding interpreter object using the model file’s path and name.
Parameters	path: Path and name of the model file.
Return Value	Returns a unique_ptr pointer to the Interpreter type. If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Interpreter object.

Build Interpreter build_interpreter_from_path()

Builds an inference interpreter object, allowing different parameters to be provided. The simplest approach is to provide only the model file’s path and name. If the model framework (e.g., NCNN, TNN) involves two model files, this interface is used.

API	build_interpreter_from_path
Description	Directly creates the corresponding interpreter object using the model file’s path and name.
Parameters	model_struct_path: Path and name of the model structure file.
	model_weight_path: Path and name of the model weight parameters file.
Return Value	Returns a unique_ptr pointer to the Interpreter type. If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Interpreter object.

Build Interpreter build_interpreter_from_model()

Builds an inference interpreter object. In addition to providing the model file path, a Model object can also be provided, allowing not only the model file path but also the input and output data types and shape information to be set.

API	build_interpreter_from_model
Description	Creates the corresponding interpreter object by passing a Model object. All Config-related parameters use default values.
Parameters	model: Model type object containing model-related data.
Return Value	Returns a unique_ptr pointer to the Interpreter type. If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Interpreter object.

💡 Note

Important: The memory space pointed to by the model pointer must be allocated on the heap. After successfully constructing the Interpreter, its lifecycle will be managed by the Interpreter (when the Interpreter is released, it will automatically release the model object). Therefore, developers should not manually release its memory, as this will cause errors due to double freeing.

Build Interpreter build_interpreter_from_model_and_config()

Builds an inference interpreter object. In addition to the methods mentioned above, both a Model object and a Config object can be provided, allowing not only model-related information but also additional runtime configuration parameters to be specified.

API	bathe_interpreter_from_model_and_config
Description	Creates the corresponding interpreter object by passing Model and Config objects.
Parameters	model: Model type object containing model-related data.
	config: Config type object containing configuration parameters.
Return Value	Returns a unique_ptr pointer to the Interpreter type. If nullptr, the function failed to construct the object; otherwise, it is a pointer to the Interpreter object.

💡 Note

Important: The memory space pointed to by the model and config pointers must be allocated on the heap. After successfully constructing the Interpreter, their lifecycles will be managed by the Interpreter (when the Interpreter is released, it will automatically release the model and config objects). Therefore, developers should not manually release their memory, as this will cause errors due to double freeing.

// Build an interpreter by passing Model and Config objects; report an error if the return value is null
std::unique_ptr<Interpreter>&& fast_interpreter = 
 InterpreterBuilder::build_interpreter_from_model_and_config(model, config);
if(fast_interpreter == nullptr){
    printf("build_interpreter failed !\n");
    return EXIT_FAILURE;
}

Other Methods

In addition to the inference-related interfaces described above, the AidLite-SDK also provides the following auxiliary interfaces.

Get SDK Version Information get_library_version()

API	get_library_version
Description	Retrieves version-related information for the current AidLite-SDK.
Parameters	void
Return Value	Version information string for the current AidLite-SDK.

Set Log Level set_log_level()

API	set_log_level
Description	Sets the minimum log level, outputting log data greater than or equal to this level. By default, logs at WARNING and above are printed.
Parameters	log_level: Value of the LogLevel enumeration type.
Return Value	Default return value is 0.

Log Output to Standard Terminal log_to_stderr()

API	log_to_stderr
Description	Sets log information to be output to the standard error terminal.
Parameters	void
Return Value	Default return value is 0.

Log Output to Text File log_to_file()

API	log_to_file
Description	Sets log information to be output to a specified text file.
Parameters	path_and_prefix: Path and prefix name for the log file.
	also_to_stderr: Indicates whether to also output logs to stderr terminal, default is false.
Return Value	0 indicates successful execution; non-zero indicates failure.

Get Latest Log Information last_log_msg()

API	last_log_msg
Description	Retrieves the latest log information for a specific log level, typically the latest error information.
Parameters	log_level: Value of the LogLevel enumeration type.
Return Value	Pointer to the storage address of the latest log information.