VideoEncoder

VideoEncoder node is used to encode ImgFrame into either H264, H265, or MJPEG streams. Only NV12 or GRAY8 (which gets converted to NV12) format is supported as an input. All codecs are lossy (except lossless MJPEG), for more information please see encoding quality docs.

Encoded bitstream (either MJPEG, H264, or H265) from the device can also be saved directly into .mp4 container with no computational overhead on the host computer. See demo here for more information.

Matroska

Besides ffmpeg and .mp4 video container (which is patent encumbered), you could also use the mkvmerge (see MKVToolNix for GUI usage) and .mkv video container to mux encoded stream into video file that is supported by all major video players (eg. VLC)

mkvmerge -o vid.mkv video.h265

How to place it

Python

C++

pipeline = dai.Pipeline()
encoder = pipeline.create(dai.node.VideoEncoder)

dai::Pipeline pipeline;
auto encoder = pipeline.create<dai::node::VideoEncoder>();

Inputs and Outputs

          ┌──────────────┐
          │              │   bitstream
 input    │              ├────────►
─────────►│ VideoEncoder |
          │              │     out
          │              ├────────►
          └──────────────┘

Message types

Usage

Python

C++

pipeline = dai.Pipeline()

# Create ColorCamera beforehand
# Set H265 encoding for the ColorCamera video output
videoEncoder = pipeline.create(dai.node.VideoEncoder)
videoEncoder.setDefaultProfilePreset(cam.getFps(), dai.VideoEncoderProperties.Profile.H265_MAIN)
videoEncoder.setBitrateKbps(500) # 0.5 Mbps

# Create MJPEG encoding for still images
stillEncoder = pipeline.create(dai.node.VideoEncoder)
stillEncoder.setDefaultProfilePreset(1, dai.VideoEncoderProperties.Profile.MJPEG)
# stillEncoder.setLossless(True) # Lossless only for MJPEG
stillEncoder.setQuality(90) # 0-100, 100 being the best quality (not lossless though)

cam.still.link(stillEncoder.input)
cam.video.link(videoEncoder.input)

dai::Pipeline pipeline;

// Create ColorCamera beforehand
// Set H265 encoding for the ColorCamera video output
auto videoEncoder = pipeline.create<dai::node::VideoEncoder>();
videoEncoder->setDefaultProfilePreset(cam->getFps(), dai::VideoEncoderProperties::Profile::H265_MAIN);
videoEncoder->setBitrateKbps(500); // 0.5 Mbps

// Create MJPEG encoding for still images
stillEncoder = pipeline.create(dai.node.VideoEncoder);
stillEncoder->setDefaultProfilePreset(1, dai::VideoEncoderProperties::Profile::MJPEG);
// stillEncoder->setLossless(true); // Lossless only for MJPEG
stillEncoder->setQuality(90); // 0-100, 100 being the best quality (not lossless though)

cam->still.link(stillEncoder->input);
cam->video.link(videoEncoder->input);

Limitations

For H.264 / H.265 encoding, we have the following limits: - 248 million pixels/second (4K@30) limit for the encoder. The resolution and frame rate can be divided into multiple streams - but the sum of all the pixels/second needs to be below 248 million. - Due to a HW constraint, video encoding can be done only on frames whose width values are multiples of 32. - 4096 pixel max width for a frame.

The MJPEG encoder is capable of 16384x8192 resolution at 450 MPix/sec. From our testing, we were able to encode 4K at 30FPS and 2x 800P at 55FPS.

Note the processing resources of the encoder are shared between H.26x and JPEG.

Reference

Python

C++

class depthai.node.VideoEncoder
class Id

Node identificator. Unique for every node on a single Pipeline

getAssetManager(*args, **kwargs)

Overloaded function.

  1. getAssetManager(self: depthai.Node) -> depthai.AssetManager

  2. getAssetManager(self: depthai.Node) -> depthai.AssetManager

getBitrate(self: depthai.node.VideoEncoder) → int
getBitrateKbps(self: depthai.node.VideoEncoder) → int
getFrameRate(self: depthai.node.VideoEncoder) → float
getHeight(self: depthai.node.VideoEncoder) → int
getInputRefs(*args, **kwargs)

Overloaded function.

  1. getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

  2. getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

getInputs(self: depthai.Node) → List[depthai.Node.Input]
getKeyframeFrequency(self: depthai.node.VideoEncoder) → int
getLossless(self: depthai.node.VideoEncoder) → bool
getMaxOutputFrameSize(self: depthai.node.VideoEncoder) → int
getName(self: depthai.Node) → str
getNumBFrames(self: depthai.node.VideoEncoder) → int
getNumFramesPool(self: depthai.node.VideoEncoder) → int
getOutputRefs(*args, **kwargs)

Overloaded function.

  1. getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

  2. getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

getOutputs(self: depthai.Node) → List[depthai.Node.Output]
getParentPipeline(*args, **kwargs)

Overloaded function.

  1. getParentPipeline(self: depthai.Node) -> depthai.Pipeline

  2. getParentPipeline(self: depthai.Node) -> depthai.Pipeline

getProfile(self: depthai.node.VideoEncoder) → depthai.VideoEncoderProperties.Profile
getQuality(self: depthai.node.VideoEncoder) → int
getRateControlMode(self: depthai.node.VideoEncoder) → depthai.VideoEncoderProperties.RateControlMode
getSize(self: depthai.node.VideoEncoder) → Tuple[int, int]
getWidth(self: depthai.node.VideoEncoder) → int
setBitrate(self: depthai.node.VideoEncoder, bitrate: int) → None
setBitrateKbps(self: depthai.node.VideoEncoder, bitrateKbps: int) → None
setDefaultProfilePreset(*args, **kwargs)

Overloaded function.

  1. setDefaultProfilePreset(self: depthai.node.VideoEncoder, fps: float, profile: depthai.VideoEncoderProperties.Profile) -> None

  2. setDefaultProfilePreset(self: depthai.node.VideoEncoder, arg0: int, arg1: int, arg2: float, arg3: depthai.VideoEncoderProperties.Profile) -> None

  3. setDefaultProfilePreset(self: depthai.node.VideoEncoder, arg0: Tuple[int, int], arg1: float, arg2: depthai.VideoEncoderProperties.Profile) -> None

setFrameRate(self: depthai.node.VideoEncoder, frameRate: float) → None
setKeyframeFrequency(self: depthai.node.VideoEncoder, freq: int) → None
setLossless(self: depthai.node.VideoEncoder, arg0: bool) → None
setMaxOutputFrameSize(self: depthai.node.VideoEncoder, maxFrameSize: int) → None
setNumBFrames(self: depthai.node.VideoEncoder, numBFrames: int) → None
setNumFramesPool(self: depthai.node.VideoEncoder, frames: int) → None
setProfile(*args, **kwargs)

Overloaded function.

  1. setProfile(self: depthai.node.VideoEncoder, profile: depthai.VideoEncoderProperties.Profile) -> None

  2. setProfile(self: depthai.node.VideoEncoder, arg0: Tuple[int, int], arg1: depthai.VideoEncoderProperties.Profile) -> None

  3. setProfile(self: depthai.node.VideoEncoder, arg0: int, arg1: int, arg2: depthai.VideoEncoderProperties.Profile) -> None

setQuality(self: depthai.node.VideoEncoder, quality: int) → None
setRateControlMode(self: depthai.node.VideoEncoder, mode: depthai.VideoEncoderProperties.RateControlMode) → None
class dai::node::VideoEncoder : public dai::NodeCRTP<Node, VideoEncoder, VideoEncoderProperties>

VideoEncoder node. Encodes frames into MJPEG, H264 or H265.

Public Functions

VideoEncoder(const std::shared_ptr<PipelineImpl> &par, int64_t nodeId)
VideoEncoder(const std::shared_ptr<PipelineImpl> &par, int64_t nodeId, std::unique_ptr<Properties> props)
void setDefaultProfilePreset(float fps, Properties::Profile profile)

Sets a default preset based on specified frame rate and profile

Parameters

  • fps: Frame rate in frames per second

  • profile: Encoding profile

void setDefaultProfilePreset(int width, int height, float fps, Properties::Profile profile)

Sets a default preset based on specified input size, frame rate and profile

Parameters

  • width: Input frame width

  • height: Input frame height

  • fps: Frame rate in frames per second

  • profile: Encoding profile

void setDefaultProfilePreset(std::tuple<int, int> size, float fps, Properties::Profile profile)

Sets a default preset based on specified input size, frame rate and profile

Parameters

  • size: Input frame size

  • fps: Frame rate in frames per second

  • profile: Encoding profile

void setNumFramesPool(int frames)

Set number of frames in pool

Parameters

  • frames: Number of pool frames

int getNumFramesPool() const

Get number of frames in pool

Return

Number of pool frames

void setRateControlMode(Properties::RateControlMode mode)

Set rate control mode.

void setProfile(Properties::Profile profile)

Set encoding profile.

void setProfile(std::tuple<int, int> size, Properties::Profile profile)

Set encoding profile.

void setProfile(int width, int height, Properties::Profile profile)

Set encoding profile.

void setBitrate(int bitrate)

Set output bitrate in bps, for CBR rate control mode. 0 for auto (based on frame size and FPS)

void setBitrateKbps(int bitrateKbps)

Set output bitrate in kbps, for CBR rate control mode. 0 for auto (based on frame size and FPS)

void setKeyframeFrequency(int freq)

Set keyframe frequency. Every Nth frame a keyframe is inserted.

Applicable only to H264 and H265 profiles

Examples:

  • 30 FPS video, keyframe frequency: 30. Every 1s a keyframe will be inserted

  • 60 FPS video, keyframe frequency: 180. Every 3s a keyframe will be inserted

void setNumBFrames(int numBFrames)

Set number of B frames to be inserted.

void setQuality(int quality)

Set quality

Parameters

  • quality: Value between 0-100%. Approximates quality

void setLossless(bool lossless)

Set lossless mode. Applies only to [M]JPEG profile

Parameters

  • lossless: True to enable lossless jpeg encoding, false otherwise

void setFrameRate(float frameRate)

Sets expected frame rate

Parameters

  • frameRate: Frame rate in frames per second

void setMaxOutputFrameSize(int maxFrameSize)

Specifies maximum output encoded frame size

Properties::RateControlMode getRateControlMode() const

Get rate control mode.

Properties::Profile getProfile() const

Get profile.

int getBitrate() const

Get bitrate in bps.

int getBitrateKbps() const

Get bitrate in kbps.

int getKeyframeFrequency() const

Get keyframe frequency.

int getNumBFrames() const

Get number of B frames.

int getQuality() const

Get quality.

std::tuple<int, int> getSize() const

Get input size.

int getWidth() const

Get input width.

int getHeight() const

Get input height.

float getFrameRate() const

Get frame rate.

bool getLossless() const

Get lossless mode. Applies only when using [M]JPEG profile.

int getMaxOutputFrameSize() const

Public Members

Input input = {*this, "in", Input::Type::SReceiver, true, 4, true, {{DatatypeEnum::ImgFrame, true}}}

Input for NV12 ImgFrame to be encoded Default queue is blocking with size set by ‘setNumFramesPool’ (4).

Output bitstream = {*this, "bitstream", Output::Type::MSender, {{DatatypeEnum::ImgFrame, false}}}

Outputs ImgFrame message that carries BITSTREAM encoded (MJPEG, H264 or H265) frame data. Mutually exclusive with out.

Output out = {*this, "out", Output::Type::MSender, {{DatatypeEnum::EncodedFrame, false}}}

Outputs EncodedFrame message that carries encoded (MJPEG, H264 or H265) frame data. Mutually exclusive with bitstream.

Public Static Attributes

static constexpr const char *NAME = "VideoEncoder"

Got questions?

Head over to Discussion Forum for technical support or any other questions you might have.