YoloSpatialDetectionNetwork

Spatial detection for the Yolo NN. It is similar to a combination of the YoloDetectionNetwork and SpatialLocationCalculator.

How to place it

Python

C++

pipeline = dai.Pipeline()
yoloSpatial = pipeline.create(dai.node.YoloSpatialDetectionNetwork)

dai::Pipeline pipeline;
auto yoloSpatial = pipeline.create<dai::node::YoloSpatialDetectionNetwork>();

Inputs and Outputs

               ┌───────────────────┐
input          │                   │       passthrough
──────────────►│-------------------├─────────────────►
               │     Yolo          │               out
               │     Spatial       ├─────────────────►
               │     Detection     │boundingBoxMapping
               │     Network       ├─────────────────►
inputDepth     │                   │  passthroughDepth
──────────────►│-------------------├─────────────────►
               └───────────────────┘

Message types

Usage

Python

C++

pipeline = dai.Pipeline()
yoloSpatial = pipeline.create(dai.node.YoloSpatialDetectionNetwork)
yoloSpatial.setBlobPath(nnBlobPath)

# Spatial detection specific parameters
yoloSpatial.setConfidenceThreshold(0.5)
yoloSpatial.input.setBlocking(False)
yoloSpatial.setBoundingBoxScaleFactor(0.5)
yoloSpatial.setDepthLowerThreshold(100) # Min 10 centimeters
yoloSpatial.setDepthUpperThreshold(5000) # Max 5 meters

# Yolo specific parameters
yoloSpatial.setNumClasses(80)
yoloSpatial.setCoordinateSize(4)
yoloSpatial.setAnchors([10,14, 23,27, 37,58, 81,82, 135,169, 344,319])
yoloSpatial.setAnchorMasks({ "side26": [1,2,3], "side13": [3,4,5] })
yoloSpatial.setIouThreshold(0.5)

dai::Pipeline pipeline;
auto yoloSpatial = pipeline.create<dai::node::YoloSpatialDetectionNetwork>();
yoloSpatial->setBlobPath(nnBlobPath);

// Spatial detection specific parameters
yoloSpatial->setConfidenceThreshold(0.5f);
yoloSpatial->input.setBlocking(false);
yoloSpatial->setBoundingBoxScaleFactor(0.5);
yoloSpatial->setDepthLowerThreshold(100); // Min 10 centimeters
yoloSpatial->setDepthUpperThreshold(5000); // Max 5 meters

// yolo specific parameters
yoloSpatial->setNumClasses(80);
yoloSpatial->setCoordinateSize(4);
yoloSpatial->setAnchors({10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319});
yoloSpatial->setAnchorMasks({{"side13", {3, 4, 5}}, {"side26", {1, 2, 3}}});
yoloSpatial->setIouThreshold(0.5f);

Examples of functionality

Spatial coordinate system

OAK camera uses right-handed (Cartesian) coordinate system for all spatial coordiantes.

../../_images/spatial-coordinates.png

More details (image source) can be found here.

Reference

Python

C++

class depthai.node.YoloSpatialDetectionNetwork

YoloSpatialDetectionNetwork node. (tiny)Yolov3/v4 based network with spatial location data.

class Connection

Connection between an Input and Output

class Id

Node identificator. Unique for every node on a single Pipeline

Properties

alias of depthai.SpatialDetectionNetworkProperties

property boundingBoxMapping

Outputs mapping of detected bounding boxes relative to depth map

Suitable for when displaying remapped bounding boxes on depth frame

getAnchorMasks(self: depthai.node.YoloSpatialDetectionNetwork) → Dict[str, List[int]]

Get anchor masks

getAnchors(self: depthai.node.YoloSpatialDetectionNetwork) → List[float]

Get anchors

getAssetManager(*args, **kwargs)

Overloaded function.

  1. getAssetManager(self: depthai.Node) -> depthai.AssetManager

Get node AssetManager as a const reference

  1. getAssetManager(self: depthai.Node) -> depthai.AssetManager

Get node AssetManager as a const reference

getConfidenceThreshold(self: depthai.node.DetectionNetwork) → float

Retrieves threshold at which to filter the rest of the detections.

Returns

Detection confidence

getCoordinateSize(self: depthai.node.YoloSpatialDetectionNetwork) → int

Get coordianate size

getInputRefs(*args, **kwargs)

Overloaded function.

  1. getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

Retrieves reference to node inputs

  1. getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

Retrieves reference to node inputs

getInputs(self: depthai.Node) → List[depthai.Node.Input]

Retrieves all nodes inputs

getIouThreshold(self: depthai.node.YoloSpatialDetectionNetwork) → float

Get Iou threshold

getName(self: depthai.Node) → str

Retrieves nodes name

getNumClasses(self: depthai.node.YoloSpatialDetectionNetwork) → int

Get num classes

getNumInferenceThreads(self: depthai.node.NeuralNetwork) → int

How many inference threads will be used to run the network

Returns

Number of threads, 0, 1 or 2. Zero means AUTO

getOutputRefs(*args, **kwargs)

Overloaded function.

  1. getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

Retrieves reference to node outputs

  1. getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

Retrieves reference to node outputs

getOutputs(self: depthai.Node) → List[depthai.Node.Output]

Retrieves all nodes outputs

getParentPipeline(*args, **kwargs)

Overloaded function.

  1. getParentPipeline(self: depthai.Node) -> depthai.Pipeline

  2. getParentPipeline(self: depthai.Node) -> depthai.Pipeline

property id

Id of node

property input

Input message with data to be inferred upon Default queue is blocking with size 5

property inputDepth

Input message with depth data used to retrieve spatial information about detected object Default queue is non-blocking with size 4

property inputs

Inputs mapped to network inputs. Useful for inferring from separate data sources Default input is non-blocking with queue size 1 and waits for messages

property out

Outputs ImgDetections message that carries parsed detection results.

property outNetwork

Outputs unparsed inference results.

property passthrough

Passthrough message on which the inference was performed.

Suitable for when input queue is set to non-blocking behavior.

property passthroughDepth

Passthrough message for depth frame on which the spatial location calculation was performed.

Suitable for when input queue is set to non-blocking behavior.

property passthroughs

Passthroughs which correspond to specified input

setAnchorMasks(self: depthai.node.YoloSpatialDetectionNetwork, anchorMasks: Dict[str, List[int]]) → None

Set anchor masks

setAnchors(self: depthai.node.YoloSpatialDetectionNetwork, anchors: List[float]) → None

Set anchors

setBlob(*args, **kwargs)

Overloaded function.

  1. setBlob(self: depthai.node.NeuralNetwork, blob: depthai.OpenVINO.Blob) -> None

Load network blob into assets and use once pipeline is started.

Parameter blob:

Network blob

  1. setBlob(self: depthai.node.NeuralNetwork, path: Path) -> None

Same functionality as the setBlobPath(). Load network blob into assets and use once pipeline is started.

Throws:

Error if file doesn’t exist or isn’t a valid network blob.

Parameter path:

Path to network blob

setBlobPath(self: depthai.node.NeuralNetwork, path: Path) → None

Load network blob into assets and use once pipeline is started.

Throws:

Error if file doesn’t exist or isn’t a valid network blob.

Parameter path:

Path to network blob

setBoundingBoxScaleFactor(self: depthai.node.SpatialDetectionNetwork, scaleFactor: float) → None

Specifies scale factor for detected bounding boxes.

Parameter scaleFactor:

Scale factor must be in the interval (0,1].

setConfidenceThreshold(self: depthai.node.DetectionNetwork, thresh: float) → None

Specifies confidence threshold at which to filter the rest of the detections.

Parameter thresh:

Detection confidence must be greater than specified threshold to be added to the list

setCoordinateSize(self: depthai.node.YoloSpatialDetectionNetwork, coordinates: int) → None

Set coordianate size

setDepthLowerThreshold(self: depthai.node.SpatialDetectionNetwork, lowerThreshold: int) → None

Specifies lower threshold in depth units (millimeter by default) for depth values which will used to calculate spatial data

Parameter lowerThreshold:

LowerThreshold must be in the interval [0,upperThreshold] and less than upperThreshold.

setDepthUpperThreshold(self: depthai.node.SpatialDetectionNetwork, upperThreshold: int) → None

Specifies upper threshold in depth units (millimeter by default) for depth values which will used to calculate spatial data

Parameter upperThreshold:

UpperThreshold must be in the interval (lowerThreshold,65535].

setIouThreshold(self: depthai.node.YoloSpatialDetectionNetwork, thresh: float) → None

Set Iou threshold

setNumClasses(self: depthai.node.YoloSpatialDetectionNetwork, numClasses: int) → None

Set num classes

setNumInferenceThreads(self: depthai.node.NeuralNetwork, numThreads: int) → None

How many threads should the node use to run the network.

Parameter numThreads:

Number of threads to dedicate to this node

setNumNCEPerInferenceThread(self: depthai.node.NeuralNetwork, numNCEPerThread: int) → None

How many Neural Compute Engines should a single thread use for inference

Parameter numNCEPerThread:

Number of NCE per thread

setNumPoolFrames(self: depthai.node.NeuralNetwork, numFrames: int) → None

Specifies how many frames will be available in the pool

Parameter numFrames:

How many frames will pool have

setSpatialCalculationAlgorithm(self: depthai.node.SpatialDetectionNetwork, calculationAlgorithm: depthai.SpatialLocationCalculatorAlgorithm) → None

Specifies spatial location calculator algorithm: Average/Min/Max

Parameter calculationAlgorithm:

Calculation algorithm.

property spatialLocationCalculatorOutput

Output of SpatialLocationCalculator node, which is used internally by SpatialDetectionNetwork. Suitable when extra information is required from SpatialLocationCalculator node, e.g. minimum, maximum distance.

class dai::node::YoloSpatialDetectionNetwork : public dai::NodeCRTP<SpatialDetectionNetwork, YoloSpatialDetectionNetwork, SpatialDetectionNetworkProperties>

YoloSpatialDetectionNetwork node. (tiny)Yolov3/v4 based network with spatial location data.

Public Functions

YoloSpatialDetectionNetwork(const std::shared_ptr<PipelineImpl> &par, int64_t nodeId)
void setNumClasses(const int numClasses)

Set num classes.

void setCoordinateSize(const int coordinates)

Set coordianate size.

void setAnchors(std::vector<float> anchors)

Set anchors.

void setAnchorMasks(std::map<std::string, std::vector<int>> anchorMasks)

Set anchor masks.

void setIouThreshold(float thresh)

Set Iou threshold.

int getNumClasses() const

Get num classes.

int getCoordinateSize() const

Get coordianate size.

std::vector<float> getAnchors() const

Get anchors.

std::map<std::string, std::vector<int>> getAnchorMasks() const

Get anchor masks.

float getIouThreshold() const

Get Iou threshold.

Got questions?

We’re always happy to help with code or other questions you might have.

Discord
Community Discord
forum
Discussion Forum
forum
Email Support