VideoEncoder¶
VideoEncoder node is used to encode ImgFrame into either H264, H265, or MJPEG streams. Only NV12 or GRAY8 (which gets converted to NV12) format is supported as an input. All codecs are lossy (except lossless MJPEG), for more information please see encoding quality docs.
Encoded bitstream (either MJPEG, H264, or H265) from the device can also be saved directly into .mp4 container with no computational overhead on the host computer. See demo here for more information.
Matroska
Besides ffmpeg
and .mp4
video container (which is patent encumbered), you could also use the mkvmerge
(see MKVToolNix for GUI usage) and .mkv
video container
to mux encoded stream into video file that is supported by all major video players
(eg. VLC)
mkvmerge -o vid.mkv video.h265
How to place it¶
pipeline = dai.Pipeline()
encoder = pipeline.create(dai.node.VideoEncoder)
dai::Pipeline pipeline;
auto encoder = pipeline.create<dai::node::VideoEncoder>();
Inputs and Outputs¶
┌──────────────┐
│ │
input │ │bitstream
─────────►│ VideoEncoder ├────────►
│ │
│ │
└──────────────┘
Message types
Usage¶
pipeline = dai.Pipeline()
# Create ColorCamera beforehand
# Set H265 encoding for the ColorCamera video output
videoEncoder = pipeline.create(dai.node.VideoEncoder)
videoEncoder.setDefaultProfilePreset(cam.getFps(), dai.VideoEncoderProperties.Profile.H265_MAIN)
# Create MJPEG encoding for still images
stillEncoder = pipeline.create(dai.node.VideoEncoder)
stillEncoder.setDefaultProfilePreset(1, dai.VideoEncoderProperties.Profile.MJPEG)
cam.still.link(stillEncoder.input)
cam.video.link(videoEncoder.input)
dai::Pipeline pipeline;
// Create ColorCamera beforehand
// Set H265 encoding for the ColorCamera video output
auto videoEncoder = pipeline.create<dai::node::VideoEncoder>();
videoEncoder->setDefaultProfilePreset(cam->getFps(), dai::VideoEncoderProperties::Profile::H265_MAIN);
// Create MJPEG encoding for still images
stillEncoder = pipeline.create(dai.node.VideoEncoder);
stillEncoder->setDefaultProfilePreset(1, dai::VideoEncoderProperties::Profile::MJPEG);
cam->still.link(stillEncoder->input);
cam->video.link(videoEncoder->input);
Limitations¶
For H.264 / H.265 encoding, we have the following limits: - 248 million pixels/second limit for the encoder or 3840x2160 pixels at 30FPS. The resolution and frame rate can be divided into multiple streams - but the sum of all the pixels/second needs to be below 248 million. - Due to a HW constraint, video encoding can be done only on frames whose width values are multiples of 32. - 4096 pixel max width for a frame. - Maximum of 3 parallel encoding streams.
The MJPEG encoder is capable of 16384x8192 resolution at 500Mpixel/second. From our testing, we were able to encode 4K at 30FPS and 2x 800P at 55FPS.
Note the processing resources of the encoder are shared between H.26x and JPEG.
Examples of functionality¶
Reference¶
-
class
depthai.node.
VideoEncoder
VideoEncoder node. Encodes frames into MJPEG, H264 or H265.
-
class
Connection
Connection between an Input and Output
-
class
Id
Node identificator. Unique for every node on a single Pipeline
-
Properties
alias of
depthai.VideoEncoderProperties
-
property
bitstream
Outputs ImgFrame message that carries BITSTREAM encoded (MJPEG, H264 or H265) frame data.
-
getAssetManager
(*args, **kwargs) Overloaded function.
getAssetManager(self: depthai.Node) -> depthai.AssetManager
Get node AssetManager as a const reference
getAssetManager(self: depthai.Node) -> depthai.AssetManager
Get node AssetManager as a const reference
-
getBitrate
(self: depthai.node.VideoEncoder) → int Get bitrate in bps
-
getBitrateKbps
(self: depthai.node.VideoEncoder) → int Get bitrate in kbps
-
getFrameRate
(self: depthai.node.VideoEncoder) → float Get frame rate
-
getHeight
(self: depthai.node.VideoEncoder) → int Get input height
-
getInputRefs
(*args, **kwargs) Overloaded function.
getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]
Retrieves reference to node inputs
getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]
Retrieves reference to node inputs
-
getInputs
(self: depthai.Node) → List[depthai.Node.Input] Retrieves all nodes inputs
-
getKeyframeFrequency
(self: depthai.node.VideoEncoder) → int Get keyframe frequency
-
getLossless
(self: depthai.node.VideoEncoder) → bool Get lossless mode. Applies only when using [M]JPEG profile.
-
getMaxOutputFrameSize
(self: depthai.node.VideoEncoder) → int
-
getName
(self: depthai.Node) → str Retrieves nodes name
-
getNumBFrames
(self: depthai.node.VideoEncoder) → int Get number of B frames
-
getNumFramesPool
(self: depthai.node.VideoEncoder) → int Get number of frames in pool
- Returns
Number of pool frames
-
getOutputRefs
(*args, **kwargs) Overloaded function.
getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]
Retrieves reference to node outputs
getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]
Retrieves reference to node outputs
-
getOutputs
(self: depthai.Node) → List[depthai.Node.Output] Retrieves all nodes outputs
-
getParentPipeline
(*args, **kwargs) Overloaded function.
getParentPipeline(self: depthai.Node) -> depthai.Pipeline
getParentPipeline(self: depthai.Node) -> depthai.Pipeline
-
getProfile
(self: depthai.node.VideoEncoder) → depthai.VideoEncoderProperties.Profile Get profile
-
getQuality
(self: depthai.node.VideoEncoder) → int Get quality
-
getRateControlMode
(self: depthai.node.VideoEncoder) → depthai.VideoEncoderProperties.RateControlMode Get rate control mode
-
getSize
(self: depthai.node.VideoEncoder) → Tuple[int, int] Get input size
-
getWidth
(self: depthai.node.VideoEncoder) → int Get input width
-
property
id
Id of node
-
property
input
Input for NV12 ImgFrame to be encoded Default queue is blocking with size set by ‘setNumFramesPool’ (4).
-
setBitrate
(self: depthai.node.VideoEncoder, bitrate: int) → None Set output bitrate in bps, for CBR rate control mode. 0 for auto (based on frame size and FPS)
-
setBitrateKbps
(self: depthai.node.VideoEncoder, bitrateKbps: int) → None Set output bitrate in kbps, for CBR rate control mode. 0 for auto (based on frame size and FPS)
-
setDefaultProfilePreset
(*args, **kwargs) Overloaded function.
setDefaultProfilePreset(self: depthai.node.VideoEncoder, fps: float, profile: depthai.VideoEncoderProperties.Profile) -> None
Sets a default preset based on specified frame rate and profile
- Parameter
fps
: Frame rate in frames per second
- Parameter
profile
: Encoding profile
setDefaultProfilePreset(self: depthai.node.VideoEncoder, arg0: int, arg1: int, arg2: float, arg3: depthai.VideoEncoderProperties.Profile) -> None
Sets a default preset based on specified input size, frame rate and profile
- Parameter
width
: Input frame width
- Parameter
height
: Input frame height
- Parameter
fps
: Frame rate in frames per second
- Parameter
profile
: Encoding profile
setDefaultProfilePreset(self: depthai.node.VideoEncoder, arg0: Tuple[int, int], arg1: float, arg2: depthai.VideoEncoderProperties.Profile) -> None
Sets a default preset based on specified input size, frame rate and profile
- Parameter
size
: Input frame size
- Parameter
fps
: Frame rate in frames per second
- Parameter
profile
: Encoding profile
-
setFrameRate
(self: depthai.node.VideoEncoder, frameRate: float) → None Sets expected frame rate
- Parameter
frameRate
: Frame rate in frames per second
- Parameter
-
setKeyframeFrequency
(self: depthai.node.VideoEncoder, freq: int) → None Set keyframe frequency. Every Nth frame a keyframe is inserted.
Applicable only to H264 and H265 profiles
Examples:
30 FPS video, keyframe frequency: 30. Every 1s a keyframe will be inserted
60 FPS video, keyframe frequency: 180. Every 3s a keyframe will be inserted
-
setLossless
(self: depthai.node.VideoEncoder, arg0: bool) → None Set lossless mode. Applies only to [M]JPEG profile
- Parameter
lossless
: True to enable lossless jpeg encoding, false otherwise
- Parameter
-
setMaxOutputFrameSize
(self: depthai.node.VideoEncoder, maxFrameSize: int) → None Specifies maximum output encoded frame size
-
setNumBFrames
(self: depthai.node.VideoEncoder, numBFrames: int) → None Set number of B frames to be inserted
-
setNumFramesPool
(self: depthai.node.VideoEncoder, frames: int) → None Set number of frames in pool
- Parameter
frames
: Number of pool frames
- Parameter
-
setProfile
(*args, **kwargs) Overloaded function.
setProfile(self: depthai.node.VideoEncoder, profile: depthai.VideoEncoderProperties.Profile) -> None
Set encoding profile
setProfile(self: depthai.node.VideoEncoder, arg0: Tuple[int, int], arg1: depthai.VideoEncoderProperties.Profile) -> None
Set encoding profile
setProfile(self: depthai.node.VideoEncoder, arg0: int, arg1: int, arg2: depthai.VideoEncoderProperties.Profile) -> None
Set encoding profile
-
setQuality
(self: depthai.node.VideoEncoder, quality: int) → None Set quality
- Parameter
quality
: Value between 0-100%. Approximates quality
- Parameter
-
setRateControlMode
(self: depthai.node.VideoEncoder, mode: depthai.VideoEncoderProperties.RateControlMode) → None Set rate control mode
-
class
-
class
dai::node
::
VideoEncoder
: public dai::NodeCRTP<Node, VideoEncoder, VideoEncoderProperties>¶ VideoEncoder node. Encodes frames into MJPEG, H264 or H265.
Public Functions
-
void
setDefaultProfilePreset
(float fps, Properties::Profile profile)¶ Sets a default preset based on specified frame rate and profile
- Parameters
fps
: Frame rate in frames per secondprofile
: Encoding profile
-
void
setDefaultProfilePreset
(int width, int height, float fps, Properties::Profile profile)¶ Sets a default preset based on specified input size, frame rate and profile
- Parameters
width
: Input frame widthheight
: Input frame heightfps
: Frame rate in frames per secondprofile
: Encoding profile
-
void
setDefaultProfilePreset
(std::tuple<int, int> size, float fps, Properties::Profile profile)¶ Sets a default preset based on specified input size, frame rate and profile
- Parameters
size
: Input frame sizefps
: Frame rate in frames per secondprofile
: Encoding profile
-
void
setNumFramesPool
(int frames)¶ Set number of frames in pool
- Parameters
frames
: Number of pool frames
-
int
getNumFramesPool
() const¶ Get number of frames in pool
- Return
Number of pool frames
-
void
setRateControlMode
(Properties::RateControlMode mode)¶ Set rate control mode.
-
void
setProfile
(Properties::Profile profile)¶ Set encoding profile.
-
void
setProfile
(std::tuple<int, int> size, Properties::Profile profile)¶ Set encoding profile.
-
void
setProfile
(int width, int height, Properties::Profile profile)¶ Set encoding profile.
-
void
setBitrate
(int bitrate)¶ Set output bitrate in bps, for CBR rate control mode. 0 for auto (based on frame size and FPS)
-
void
setBitrateKbps
(int bitrateKbps)¶ Set output bitrate in kbps, for CBR rate control mode. 0 for auto (based on frame size and FPS)
-
void
setKeyframeFrequency
(int freq)¶ Set keyframe frequency. Every Nth frame a keyframe is inserted.
Applicable only to H264 and H265 profiles
Examples:
30 FPS video, keyframe frequency: 30. Every 1s a keyframe will be inserted
60 FPS video, keyframe frequency: 180. Every 3s a keyframe will be inserted
-
void
setNumBFrames
(int numBFrames)¶ Set number of B frames to be inserted.
-
void
setQuality
(int quality)¶ Set quality
- Parameters
quality
: Value between 0-100%. Approximates quality
-
void
setLossless
(bool lossless)¶ Set lossless mode. Applies only to [M]JPEG profile
- Parameters
lossless
: True to enable lossless jpeg encoding, false otherwise
-
void
setFrameRate
(float frameRate)¶ Sets expected frame rate
- Parameters
frameRate
: Frame rate in frames per second
-
void
setMaxOutputFrameSize
(int maxFrameSize)¶ Specifies maximum output encoded frame size
-
Properties::RateControlMode
getRateControlMode
() const¶ Get rate control mode.
-
Properties::Profile
getProfile
() const¶ Get profile.
-
int
getBitrate
() const¶ Get bitrate in bps.
-
int
getBitrateKbps
() const¶ Get bitrate in kbps.
-
int
getKeyframeFrequency
() const¶ Get keyframe frequency.
-
int
getNumBFrames
() const¶ Get number of B frames.
-
int
getQuality
() const¶ Get quality.
-
std::tuple<int, int>
getSize
() const¶ Get input size.
-
int
getWidth
() const¶ Get input width.
-
int
getHeight
() const¶ Get input height.
-
float
getFrameRate
() const¶ Get frame rate.
-
bool
getLossless
() const¶ Get lossless mode. Applies only when using [M]JPEG profile.
-
int
getMaxOutputFrameSize
() const¶
Public Members
-
Input
input
= {*this, "in", Input::Type::SReceiver, true, 4, true, {{DatatypeEnum::ImgFrame, true}}}¶ Input for NV12 ImgFrame to be encoded Default queue is blocking with size set by ‘setNumFramesPool’ (4).
-
Output
bitstream
= {*this, "bitstream", Output::Type::MSender, {{DatatypeEnum::ImgFrame, false}}}¶ Outputs ImgFrame message that carries BITSTREAM encoded (MJPEG, H264 or H265) frame data.
Public Static Attributes
-
static constexpr const char *
NAME
= "VideoEncoder"¶
-
void
Got questions?
We’re always happy to help with code or other questions you might have.