Introduction

As described in the Introduction, Inference DS is built upon the concept of nodes and the routing of messages between them via producers and consumers. This section explains how to configure nodes and routes for Inference DS via the user interface and via configuration files. Furthermore, it lists all nodes and describes how to create connections between them.

Nodes

Inputs

File Input Interface

The file input interface can be used to read images and point clouds from a given directory.

Video Capture Interface

The video capture interface can be used to read videos from files and from network streams. This node is based on the class "VideoCapture" of the OpenCV library.

Basler Blaze Plugin

The Basler Blaze plugin can be used to acquire point clouds from the Blaze time-of-flight camera. It depends on the python package harvesters that must be installed in addition to Inference DS.

IFM O2V1XX Plugin

The IFM O2V1XX plugin can be used to acquire images from IFM O2V cameras.

IFM O3D Plugin

The IFM O3D plugin can be used to acquire point clouds from the IFM O3D time-of-flight camera.

Sick Visionary-S Plugin

This plugin can be used to acquire range images from the Sick Visionary-S stereo camera.

Model Inference Node

The model inference node executes models from Deep Learning DS in the ONNX format.

A model node requires the path to the model.onnx file which contains the model structure and weights. Furthermore, specific parameters should be provided in the file parameters.json. An optional file class_labels.json can be provided including the mapping between local class label IDs and their ID on Deep Learning DS as well as their names.

It is optional to adopt certain runtime parameters of the network such as confidence thresholds.

Plugins

Plugins are nodes that provide individual functions such as grabbing frames via a specific SDK or applying processing steps in your pipeline. There are some standard plugins available as well. Furthermore, you can develop your own plugins and it is possible to configure these nodes in the user interface as well. Visit the section Plugin Development to get information on how to create your own plugin.

Point Cloud Processing Plugin

Plugin for processing point clouds from snapshot cameras (stereo, time-of-flight, etc.). This plugin normalizes the point cloud given the min. and max. range and creates a stacked image containing the range image and amplitude image. Furthermore, it offers an action to directly save the raw point clouds and the stacked image.

Outputs

File Output Interface

The file output interface can be used to save images and inference results in the given directory. Subdirectories will be created on every start of the node / Inference DS Core.

Video Writer Interface

The video writer interface can be used to write videos to files and send them as network streams. This node is based on the class "VideoWriter" of the OpenCV library.

Graphical Interface

Displays the inference results directly on the machine where the Inference DS Core is running. This requires the library OpenCV installed in the non-headless variant. This interface is intended mainly for local debugging.

Triggers

Triggers can be added to a variety of nodes in order to process fewer images. For instance, you can specify a trigger in a file output node in order to record only every 10th frame.

Frame Skip Trigger

Only processes every skip_framesth image.

Parameters:

  • skip_frame

Difference Trigger

Only processes the image if it differs from the last image by a threshold.

Parameters:

  • threshold [0,255]

Routing

The routing configuration is responsible for connections between producers and consumers of nodes. In the user interface, these connections can be done easily. Before a connection can be established, the nodes must be created, so that the producers and consumers have been initialized. To connect one node to another one, open the connections menu and select the desired source(s) or sink(s) respectively.

The routing is represented in the configuration file as a list of routing entries (source, sink) that define the data flow. Producers are entered as source and consumers are sinks in these entries. Producers and consumers are identified by their name (string). Each node can include consumers and producers. In many cases, the nodes have only one consumer or producer. In these cases, the names of the consumer or producer match the name of the node itself.

In this example, the “coin_model“ node receives inputs from a file input node called “coins“ before recording the results via a file output node:

routing:
- source: coins
  sink: coin_model
- source: coin_model
  sink: recording
YAML

It is recommended to create routing configurations that contain only parallel branches of independent data. That is, the results of one node should only be distributed to multiple other nodes in case these nodes do not modify the same data, e.g. two nodes that modify the inference results.