VideoEncoder

VideoEncoder node is used to encode ImgFrame into either H264, H265, or MJPEG streams. Only NV12 or GRAY8 (which gets converted to NV12) format is supported as an input. All codecs are lossy (except lossless MJPEG), for more information please see encoding quality docs.

Encoded bitstream (either MJPEG, H264, or H265) from the device can also be saved directly into .mp4 container with no computational overhead on the host computer. See demo here for more information.

Matroska

Besides ffmpeg and .mp4 video container (which is patent encumbered), you could also use the mkvmerge (see MKVToolNix for GUI usage) and .mkv video container to mux encoded stream into video file that is supported by all major video players (eg. VLC)

mkvmerge -o vid.mkv video.h265

How to place it

Python

C++

pipeline = dai.Pipeline()
encoder = pipeline.create(dai.node.VideoEncoder)

dai::Pipeline pipeline;
auto encoder = pipeline.create<dai::node::VideoEncoder>();

Inputs and Outputs

          ┌──────────────┐
          │              │
 input    │              │bitstream
─────────►│ VideoEncoder ├────────►
          │              │
          │              │
          └──────────────┘

Message types

Usage

Python

C++

pipeline = dai.Pipeline()

# Create ColorCamera beforehand
# Set H265 encoding for the ColorCamera video output
videoEncoder = pipeline.create(dai.node.VideoEncoder)
videoEncoder.setDefaultProfilePreset(cam.getFps(), dai.VideoEncoderProperties.Profile.H265_MAIN)

# Create MJPEG encoding for still images
stillEncoder = pipeline.create(dai.node.VideoEncoder)
stillEncoder.setDefaultProfilePreset(1, dai.VideoEncoderProperties.Profile.MJPEG)

cam.still.link(stillEncoder.input)
cam.video.link(videoEncoder.input)

dai::Pipeline pipeline;

// Create ColorCamera beforehand
// Set H265 encoding for the ColorCamera video output
auto videoEncoder = pipeline.create<dai::node::VideoEncoder>();
videoEncoder->setDefaultProfilePreset(cam->getFps(), dai::VideoEncoderProperties::Profile::H265_MAIN);

// Create MJPEG encoding for still images
stillEncoder = pipeline.create(dai.node.VideoEncoder);
stillEncoder->setDefaultProfilePreset(1, dai::VideoEncoderProperties::Profile::MJPEG);

cam->still.link(stillEncoder->input);
cam->video.link(videoEncoder->input);

Limitations

For H.264 / H.265 encoding, we have the following limits: - 248 million pixels/second limit for the encoder or 3840x2160 pixels at 30FPS. The resolution and frame rate can be divided into multiple streams - but the sum of all the pixels/second needs to be below 248 million. - Due to a HW constraint, video encoding can be done only on frames whose width values are multiples of 32. - 4096 pixel max width for a frame. - Maximum of 3 parallel encoding streams.

The MJPEG encoder is capable of 16384x8192 resolution at 500Mpixel/second. From our testing, we were able to encode 4K at 30FPS and 2x 800P at 55FPS.

Note the processing resources of the encoder are shared between H.26x and JPEG.

Reference

Python

C++

class depthai.node.VideoEncoder

VideoEncoder node. Encodes frames into MJPEG, H264 or H265.

class Connection

Connection between an Input and Output

class Id

Node identificator. Unique for every node on a single Pipeline

Properties

alias of depthai.VideoEncoderProperties

property bitstream

Outputs ImgFrame message that carries BITSTREAM encoded (MJPEG, H264 or H265) frame data.

getAssetManager(*args, **kwargs)

Overloaded function.

  1. getAssetManager(self: depthai.Node) -> depthai.AssetManager

Get node AssetManager as a const reference

  1. getAssetManager(self: depthai.Node) -> depthai.AssetManager

Get node AssetManager as a const reference

getBitrate(self: depthai.node.VideoEncoder) → int

Get bitrate in bps

getBitrateKbps(self: depthai.node.VideoEncoder) → int

Get bitrate in kbps

getFrameRate(self: depthai.node.VideoEncoder) → float

Get frame rate

getHeight(self: depthai.node.VideoEncoder) → int

Get input height

getInputRefs(*args, **kwargs)

Overloaded function.

  1. getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

Retrieves reference to node inputs

  1. getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

Retrieves reference to node inputs

getInputs(self: depthai.Node) → List[depthai.Node.Input]

Retrieves all nodes inputs

getKeyframeFrequency(self: depthai.node.VideoEncoder) → int

Get keyframe frequency

getLossless(self: depthai.node.VideoEncoder) → bool

Get lossless mode. Applies only when using [M]JPEG profile.

getMaxOutputFrameSize(self: depthai.node.VideoEncoder) → int
getName(self: depthai.Node) → str

Retrieves nodes name

getNumBFrames(self: depthai.node.VideoEncoder) → int

Get number of B frames

getNumFramesPool(self: depthai.node.VideoEncoder) → int

Get number of frames in pool

Returns

Number of pool frames

getOutputRefs(*args, **kwargs)

Overloaded function.

  1. getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

Retrieves reference to node outputs

  1. getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

Retrieves reference to node outputs

getOutputs(self: depthai.Node) → List[depthai.Node.Output]

Retrieves all nodes outputs

getParentPipeline(*args, **kwargs)

Overloaded function.

  1. getParentPipeline(self: depthai.Node) -> depthai.Pipeline

  2. getParentPipeline(self: depthai.Node) -> depthai.Pipeline

getProfile(self: depthai.node.VideoEncoder) → depthai.VideoEncoderProperties.Profile

Get profile

getQuality(self: depthai.node.VideoEncoder) → int

Get quality

getRateControlMode(self: depthai.node.VideoEncoder) → depthai.VideoEncoderProperties.RateControlMode

Get rate control mode

getSize(self: depthai.node.VideoEncoder) → Tuple[int, int]

Get input size

getWidth(self: depthai.node.VideoEncoder) → int

Get input width

property id

Id of node

property input

Input for NV12 ImgFrame to be encoded Default queue is blocking with size set by ‘setNumFramesPool’ (4).

setBitrate(self: depthai.node.VideoEncoder, bitrate: int) → None

Set output bitrate in bps, for CBR rate control mode. 0 for auto (based on frame size and FPS)

setBitrateKbps(self: depthai.node.VideoEncoder, bitrateKbps: int) → None

Set output bitrate in kbps, for CBR rate control mode. 0 for auto (based on frame size and FPS)

setDefaultProfilePreset(*args, **kwargs)

Overloaded function.

  1. setDefaultProfilePreset(self: depthai.node.VideoEncoder, fps: float, profile: depthai.VideoEncoderProperties.Profile) -> None

Sets a default preset based on specified frame rate and profile

Parameter fps:

Frame rate in frames per second

Parameter profile:

Encoding profile

  1. setDefaultProfilePreset(self: depthai.node.VideoEncoder, arg0: int, arg1: int, arg2: float, arg3: depthai.VideoEncoderProperties.Profile) -> None

Sets a default preset based on specified input size, frame rate and profile

Parameter width:

Input frame width

Parameter height:

Input frame height

Parameter fps:

Frame rate in frames per second

Parameter profile:

Encoding profile

  1. setDefaultProfilePreset(self: depthai.node.VideoEncoder, arg0: Tuple[int, int], arg1: float, arg2: depthai.VideoEncoderProperties.Profile) -> None

Sets a default preset based on specified input size, frame rate and profile

Parameter size:

Input frame size

Parameter fps:

Frame rate in frames per second

Parameter profile:

Encoding profile

setFrameRate(self: depthai.node.VideoEncoder, frameRate: float) → None

Sets expected frame rate

Parameter frameRate:

Frame rate in frames per second

setKeyframeFrequency(self: depthai.node.VideoEncoder, freq: int) → None

Set keyframe frequency. Every Nth frame a keyframe is inserted.

Applicable only to H264 and H265 profiles

Examples:

  • 30 FPS video, keyframe frequency: 30. Every 1s a keyframe will be inserted

  • 60 FPS video, keyframe frequency: 180. Every 3s a keyframe will be inserted

setLossless(self: depthai.node.VideoEncoder, arg0: bool) → None

Set lossless mode. Applies only to [M]JPEG profile

Parameter lossless:

True to enable lossless jpeg encoding, false otherwise

setMaxOutputFrameSize(self: depthai.node.VideoEncoder, maxFrameSize: int) → None

Specifies maximum output encoded frame size

setNumBFrames(self: depthai.node.VideoEncoder, numBFrames: int) → None

Set number of B frames to be inserted

setNumFramesPool(self: depthai.node.VideoEncoder, frames: int) → None

Set number of frames in pool

Parameter frames:

Number of pool frames

setProfile(*args, **kwargs)

Overloaded function.

  1. setProfile(self: depthai.node.VideoEncoder, profile: depthai.VideoEncoderProperties.Profile) -> None

Set encoding profile

  1. setProfile(self: depthai.node.VideoEncoder, arg0: Tuple[int, int], arg1: depthai.VideoEncoderProperties.Profile) -> None

Set encoding profile

  1. setProfile(self: depthai.node.VideoEncoder, arg0: int, arg1: int, arg2: depthai.VideoEncoderProperties.Profile) -> None

Set encoding profile

setQuality(self: depthai.node.VideoEncoder, quality: int) → None

Set quality

Parameter quality:

Value between 0-100%. Approximates quality

setRateControlMode(self: depthai.node.VideoEncoder, mode: depthai.VideoEncoderProperties.RateControlMode) → None

Set rate control mode

class dai::node::VideoEncoder : public dai::NodeCRTP<Node, VideoEncoder, VideoEncoderProperties>

VideoEncoder node. Encodes frames into MJPEG, H264 or H265.

Public Functions

VideoEncoder(const std::shared_ptr<PipelineImpl> &par, int64_t nodeId)
VideoEncoder(const std::shared_ptr<PipelineImpl> &par, int64_t nodeId, std::unique_ptr<Properties> props)
void setDefaultProfilePreset(float fps, Properties::Profile profile)

Sets a default preset based on specified frame rate and profile

Parameters

  • fps: Frame rate in frames per second

  • profile: Encoding profile

void setDefaultProfilePreset(int width, int height, float fps, Properties::Profile profile)

Sets a default preset based on specified input size, frame rate and profile

Parameters

  • width: Input frame width

  • height: Input frame height

  • fps: Frame rate in frames per second

  • profile: Encoding profile

void setDefaultProfilePreset(std::tuple<int, int> size, float fps, Properties::Profile profile)

Sets a default preset based on specified input size, frame rate and profile

Parameters

  • size: Input frame size

  • fps: Frame rate in frames per second

  • profile: Encoding profile

void setNumFramesPool(int frames)

Set number of frames in pool

Parameters

  • frames: Number of pool frames

int getNumFramesPool() const

Get number of frames in pool

Return

Number of pool frames

void setRateControlMode(Properties::RateControlMode mode)

Set rate control mode.

void setProfile(Properties::Profile profile)

Set encoding profile.

void setProfile(std::tuple<int, int> size, Properties::Profile profile)

Set encoding profile.

void setProfile(int width, int height, Properties::Profile profile)

Set encoding profile.

void setBitrate(int bitrate)

Set output bitrate in bps, for CBR rate control mode. 0 for auto (based on frame size and FPS)

void setBitrateKbps(int bitrateKbps)

Set output bitrate in kbps, for CBR rate control mode. 0 for auto (based on frame size and FPS)

void setKeyframeFrequency(int freq)

Set keyframe frequency. Every Nth frame a keyframe is inserted.

Applicable only to H264 and H265 profiles

Examples:

  • 30 FPS video, keyframe frequency: 30. Every 1s a keyframe will be inserted

  • 60 FPS video, keyframe frequency: 180. Every 3s a keyframe will be inserted

void setNumBFrames(int numBFrames)

Set number of B frames to be inserted.

void setQuality(int quality)

Set quality

Parameters

  • quality: Value between 0-100%. Approximates quality

void setLossless(bool lossless)

Set lossless mode. Applies only to [M]JPEG profile

Parameters

  • lossless: True to enable lossless jpeg encoding, false otherwise

void setFrameRate(float frameRate)

Sets expected frame rate

Parameters

  • frameRate: Frame rate in frames per second

void setMaxOutputFrameSize(int maxFrameSize)

Specifies maximum output encoded frame size

Properties::RateControlMode getRateControlMode() const

Get rate control mode.

Properties::Profile getProfile() const

Get profile.

int getBitrate() const

Get bitrate in bps.

int getBitrateKbps() const

Get bitrate in kbps.

int getKeyframeFrequency() const

Get keyframe frequency.

int getNumBFrames() const

Get number of B frames.

int getQuality() const

Get quality.

std::tuple<int, int> getSize() const

Get input size.

int getWidth() const

Get input width.

int getHeight() const

Get input height.

float getFrameRate() const

Get frame rate.

bool getLossless() const

Get lossless mode. Applies only when using [M]JPEG profile.

int getMaxOutputFrameSize() const

Public Members

Input input = {*this, "in", Input::Type::SReceiver, true, 4, true, {{DatatypeEnum::ImgFrame, true}}}

Input for NV12 ImgFrame to be encoded Default queue is blocking with size set by ‘setNumFramesPool’ (4).

Output bitstream = {*this, "bitstream", Output::Type::MSender, {{DatatypeEnum::ImgFrame, false}}}

Outputs ImgFrame message that carries BITSTREAM encoded (MJPEG, H264 or H265) frame data.

Public Static Attributes

static constexpr const char *NAME = "VideoEncoder"

Got questions?

We’re always happy to help with code or other questions you might have.

Discord
Community Discord
forum
Discussion Forum
forum
Email Support