ColorCamera¶

ColorCamera node is a source of image frames. You can control in at runtime with the InputControl and InputConfig.

How to place it¶

pipeline = dai.Pipeline()
cam = pipeline.create(dai.node.ColorCamera)

dai::Pipeline pipeline;
auto cam = pipeline.create<dai::node::ColorCamera>();

Inputs and Outputs¶

                       ColorCamera node
               ┌──────────────────────────────┐
               │   ┌─────────────┐            │
               │   │    Image    │ raw        │     raw
               │   │    Sensor   │---┬--------├────────►
               │   └────▲────────┘   |        │
               │        │   ┌--------┘        │
               │      ┌─┴───▼─┐               │     isp
inputControl   │      │       │-------┬-------├────────►
──────────────►│------│  ISP  │ ┌─────▼────┐  │   video
               │      │       │ |          |--├────────►
               │      └───────┘ │   Image  │  │   still
inputConfig    │                │   Post-  │--├────────►
──────────────►│----------------|Processing│  │ preview
               │                │          │--├────────►
               │                └──────────┘  │
               └──────────────────────────────┘

Message types

inputConfig - ImageManipConfig
inputControl - CameraControl
raw - ImgFrame - RAW10 bayer data. Demo code for unpacking here
isp - ImgFrame - YUV420 planar (same as YU12/IYUV/I420)
still - ImgFrame - NV12, suitable for bigger size frames. The image gets created when a capture event is sent to the ColorCamera, so it’s like taking a photo
preview - ImgFrame - RGB (or BGR planar/interleaved if configured), mostly suited for small size previews and to feed the image into NeuralNetwork
video - ImgFrame - NV12, suitable for bigger size frames

ISP (image signal processor) is used for bayer transformation, demosaicing, noise reduction, and other image enhancements. It interacts with the 3A algorithms: auto-focus, auto-exposure, and auto-white-balance, which are handling image sensor adjustments such as exposure time, sensitivity (ISO), and lens position (if the camera module has a motorized lens) at runtime. Click here for more information.

Image Post-Processing converts YUV420 planar frames from the ISP into video/preview/still frames.

still (when a capture is triggered) and isp work at the max camera resolution, while video and preview are limited to max 4K (3840 x 2160) resolution, which is cropped from isp. For IMX378 (12MP), the post-processing works like this:

┌─────┐   Cropping to   ┌─────────┐  Downscaling   ┌──────────┐
│ ISP ├────────────────►│  video  ├───────────────►│ preview  │
└─────┘  max 3840x2160  └─────────┘  and cropping  └──────────┘

The image above is the isp output from the ColorCamera (12MP resolution from IMX378). If you aren’t downscaling ISP, the video output is cropped to 4k (max 3840x2160 due to the limitation of the video output) as represented by the blue rectangle. The Yellow rectangle represents a cropped preview output when the preview size is set to a 1:1 aspect ratio (eg. when using a 300x300 preview size for the MobileNet-SSD NN model) because the preview output is derived from the video output.

Usage¶

pipeline = dai.Pipeline()
cam = pipeline.create(dai.node.ColorCamera)
cam.setPreviewSize(300, 300)
cam.setBoardSocket(dai.CameraBoardSocket.RGB)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam.setInterleaved(False)
cam.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)

dai::Pipeline pipeline;
auto cam = pipeline.create<dai::node::ColorCamera>();
cam->setPreviewSize(300, 300);
cam->setBoardSocket(dai::CameraBoardSocket::RGB);
cam->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1080_P);
cam->setInterleaved(false);
cam->setColorOrder(dai::ColorCameraProperties::ColorOrder::RGB);

Limitations¶

Here are known camera limitations for the RVC2:

ISP can process about 600 MP/s, and about 500 MP/s when the pipeline is also running NNs and video encoder in parallel
3A algorithms can process about 200..250 FPS overall (for all camera streams). This is a current limitation of our implementation, and we have plans for a workaround to run 3A algorithms on every Xth frame, no ETA yet
ISP Scaling numerator value can be 1..16 and denominator value 1..32 for both vertical and horizontal scaling. So you can downscale eg. 12MP (4056x3040) only to resolutions calculated here

Examples of functionality¶

Reference¶

class depthai.node.ColorCamera

ColorCamera node. For use with color sensors.

class Connection: Connection between an Input and Output

class Id: Node identificator. Unique for every node on a single Pipeline

Properties: alias of depthai.ColorCameraProperties

property frameEvent

Outputs metadata-only ImgFrame message as an early indicator of an incoming frame.

It’s sent on the MIPI SoF (start-of-frame) event, just after the exposure of the current frame has finished and before the exposure for next frame starts. Could be used to synchronize various processes with camera capture. Fields populated: camera id, sequence number, timestamp

getAssetManager(*args, **kwargs)

Overloaded function.

getAssetManager(self: depthai.Node) -> depthai.AssetManager

Get node AssetManager as a const reference

getAssetManager(self: depthai.Node) -> depthai.AssetManager

Get node AssetManager as a const reference

getBoardSocket(self: depthai.node.ColorCamera) → depthai.CameraBoardSocket

Retrieves which board socket to use

Returns: Board socket to use

getCamId(self: depthai.node.ColorCamera) → int

getCamera(self: depthai.node.ColorCamera) → str

Retrieves which camera to use by name

Returns: Name of the camera to use

getColorOrder(self: depthai.node.ColorCamera) → depthai.ColorCameraProperties.ColorOrder: Get color order of preview output frames. RGB or BGR

getFp16(self: depthai.node.ColorCamera) → bool: Get fp16 (0..255) data of preview output frames

getFps(self: depthai.node.ColorCamera) → float

Get rate at which camera should produce frames

Returns: Rate in frames per second

getFrameEventFilter(self: depthai.node.ColorCamera) → List[depthai.FrameEvent]

getImageOrientation(self: depthai.node.ColorCamera) → depthai.CameraImageOrientation: Get camera image orientation

getInputRefs(*args, **kwargs)

Overloaded function.

getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

Retrieves reference to node inputs

getInputRefs(self: depthai.Node) -> List[depthai.Node.Input]

Retrieves reference to node inputs

getInputs(self: depthai.Node) → List[depthai.Node.Input]: Retrieves all nodes inputs

getInterleaved(self: depthai.node.ColorCamera) → bool: Get planar or interleaved data of preview output frames

getIspHeight(self: depthai.node.ColorCamera) → int: Get ‘isp’ output height

getIspNumFramesPool(self: depthai.node.ColorCamera) → int: Get number of frames in isp pool

getIspSize(self: depthai.node.ColorCamera) → Tuple[int, int]: Get ‘isp’ output resolution as size, after scaling

getIspWidth(self: depthai.node.ColorCamera) → int: Get ‘isp’ output width

getName(self: depthai.Node) → str: Retrieves nodes name

getOutputRefs(*args, **kwargs)

Overloaded function.

getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

Retrieves reference to node outputs

getOutputRefs(self: depthai.Node) -> List[depthai.Node.Output]

Retrieves reference to node outputs

getOutputs(self: depthai.Node) → List[depthai.Node.Output]: Retrieves all nodes outputs

getParentPipeline(*args, **kwargs)

Overloaded function.

getParentPipeline(self: depthai.Node) -> depthai.Pipeline
getParentPipeline(self: depthai.Node) -> depthai.Pipeline

getPreviewHeight(self: depthai.node.ColorCamera) → int: Get preview height

getPreviewKeepAspectRatio(self: depthai.node.ColorCamera) → bool

Got questions?

We’re always happy to help with code or other questions you might have.

ColorCamera¶

How to place it¶

Inputs and Outputs¶

Usage¶

Limitations¶

Examples of functionality¶

Reference¶

Got questions?

Community Discord

Discussion Forum

Email Support