Script JSON communication

This example shows how Script node can communicate with the outside world (host computer) using JSON serialization by sending a Buffer message. Similarly, you could use SPIIn and SPIOut to send JSON between MCU and Script node via SPI.

How would this be useful:

2-way communication between Script node and host/MCU could be used to eg. alter the flow of an application.

For example, we might want to use 2 different NN models on the device, one until noon and the other from noon till midnight. Host (eg. RPi) could check the current time and when it’s noon, it would send a simple message to the Script node which would start forwarding all frames to the other NeuralNetwork node (which has different NN model).

What it does:

Host creates a dictionary, serializes it, sends it to the Script node. Script node receives the Buffer message, deserializes the dictionary, changes values a bit, serializes the dictionary again and sends it to the host, which deserializes the changed dictionary and prints the new values.

Demo

~/depthai-python/examples/Script$ python3 script_json_communication.py
dict {'one': 1, 'foo': 'bar'}
[14442C1041B7EFD000] [3.496] [Script(1)] [warning] Original: {'one': 1, 'foo': 'bar'}
[14442C1041B7EFD000] [3.496] [Script(1)] [warning] Changed: {'one': 2, 'foo': 'baz'}
changedDict {'one': 2, 'foo': 'baz'}

Setup

Please run the install script to download all required dependencies. Please note that this script must be ran from git context, so you have to download the depthai-python repository first and then run the script

git clone https://github.com/luxonis/depthai-python.git
cd depthai-python/examples
python3 install_requirements.py

For additional information, please follow installation guide

Source code

Python

C++

Also available on GitHub

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
#!/usr/bin/env python3
import depthai as dai
import json

pipeline = dai.Pipeline()

xin = pipeline.create(dai.node.XLinkIn)
xin.setStreamName('in')

script = pipeline.create(dai.node.Script)
xin.out.link(script.inputs['in'])
script.setScript("""
    import json

    # Receive bytes from the host
    data = node.io['in'].get().getData()
    jsonStr = str(data, 'utf-8')
    dict = json.loads(jsonStr)

    # Change initial dictionary a bit
    dict['one'] += 1
    dict['foo'] = "baz"

    b = Buffer(30)
    b.setData(json.dumps(dict).encode('utf-8'))
    node.io['out'].send(b)
""")

xout = pipeline.create(dai.node.XLinkOut)
xout.setStreamName('out')
script.outputs['out'].link(xout.input)

# Connect to device with pipeline
with dai.Device(pipeline) as device:
    # This dict will be serialized (JSON), sent to device (Script node),
    # edited a bit and sent back to the host
    dict = {'one':1, 'foo': 'bar'}
    print('dict', dict)
    data = json.dumps(dict).encode('utf-8')
    buffer = dai.Buffer()
    buffer.setData(list(data))
    device.getInputQueue("in").send(buffer)

    # Wait for the script to send the changed dictionary back
    jsonData = device.getOutputQueue("out").get()
    jsonText = str(jsonData.getData(), 'utf-8')
    changedDict = json.loads(jsonText)
    print('changedDict', changedDict)

Also available on GitHub

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
#include <chrono>
#include <iostream>
#include <thread>

// Includes common necessary includes for development using depthai library
#include "depthai/depthai.hpp"

// Include nlohmann json
#include "nlohmann/json.hpp"

int main() {
    using namespace std;

    dai::Pipeline pipeline;

    auto xin = pipeline.create<dai::node::XLinkIn>();
    xin->setStreamName("in");

    auto script = pipeline.create<dai::node::Script>();
    xin->out.link(script->inputs["in"]);
    script->setScript(R"(
        import json

        # Receive bytes from the host
        data = node.io['in'].get().getData()
        jsonStr = str(data, 'utf-8')
        dict = json.loads(jsonStr)

        # Change initial dictionary a bit
        dict['one'] += 1
        dict['foo'] = "baz"

        b = Buffer(30)
        b.setData(json.dumps(dict).encode('utf-8'))
        node.io['out'].send(b)
    )");

    auto xout = pipeline.create<dai::node::XLinkOut>();
    xout->setStreamName("out");
    script->outputs["out"].link(xout->input);

    // Connect to device with pipeline
    dai::Device device(pipeline);

    // This dict will be serialized (JSON), sent to device (Script node),
    // edited a bit and sent back to the host
    nlohmann::json dict{{"one", 1}, {"foo", "bar"}};
    cout << "dict: " << dict << "\n";
    auto buffer = dai::Buffer();
    auto data = dict.dump();
    buffer.setData({data.begin(), data.end()});
    device.getInputQueue("in")->send(buffer);

    // Wait for the script to send the changed dictionary back
    auto jsonData = device.getOutputQueue("out")->get<dai::Buffer>();
    auto changedDict = nlohmann::json::parse(jsonData->getData());
    cout << "changedDict: " << changedDict << "\n";
    const nlohmann::json expectedDict{{"one", 2}, {"foo", "baz"}};
    if(expectedDict != changedDict) return 1;
    return 0;
}

Got questions?

Head over to Discussion Forum for technical support or any other questions you might have.