Script Engine SDK

The Script Engine runs Go programs on each ControlBird node. A program is ordinary Go, compiled with TinyGo to WebAssembly. It receives a client for the Store, so it can read and write entity data, react to field changes and timers, and query the Historian for time-series data. Programs are authored and tested in the Plugin Editor app, and they start when you run them or when their node becomes the leader.

How a program runs

Every program is an entity. The engine compiles the Source field with TinyGo, runs it, and records the outcome in the program's status fields.

1. Create a Program entity with New program in the Plugin Editor.
2. Edit the Source field with Go that declares package main and func Run(ctx context.Context, client *store.Client) error. Run either does its work and returns, or registers handlers and returns.
3. Set the Args field with the input for the run (or leave it empty).
4. Click Run, or write any value to Execute, to start a Trigger program. A Leader program starts by itself on the leader node.
5. Read LastExecutionStatus, LastExecutionTime, LastError and CompileError to observe the result.
6. Review the program's own log for everything it printed.

The first run after a change to Source compiles it. A compile takes about a minute, so expect a pause before the first run of edited code. See Limits for what TinyGo does differently from standard Go.

Program entities

The Script Engine recognizes two entity types. A Program holds source code. A RuleChain compiles a visual graph to Go and always runs as a Leader program.

Program fields

Writing a program

A program is one Go source file in package main that defines Run. The engine passes it a context and a *store.Client connected to the node's Store. A program that registers no handlers does its work in Run and finishes when Run returns: nil finishes the run as Success, and an error finishes it as Error. The rest of this page covers programs that stay registered and react to events.

package main

import (
    "context"
    "fmt"

    "github.com/controlbird/platform/sdk/store"
)

func Run(ctx context.Context, client *store.Client) error {
    nameField, err := client.FieldType(ctx, "Name")
    if err != nil {
        return err
    }
    boiler, err := client.ResolvePath(ctx, "Root/Plant/Boiler")
    if err != nil {
        return err
    }
    result, err := client.Read(ctx, boiler, store.Path(nameField))
    if err != nil {
        return err
    }
    name, err := result.Value.String()
    if err != nil {
        return err
    }
    fmt.Println("Entity name:", name)
    return nil
}

Value types

The Store is strongly typed. Build the value you write with the constructor for its type, and read a value back with the matching accessor, such as Int(), Float(), String() or Bool(). An accessor returns store.ErrValueKind when the value holds a different type.

Store client

Field and entity types are referenced by resolved type handles. Resolve them once with FieldType and EntityType, then pass the handles to read and write operations.

Type resolution

nameField, err := client.FieldType(ctx, "Name")
deviceType, err := client.EntityType(ctx, "Device")

Reading and writing fields

Read a field, then unwrap the typed value. Pass several fields to store.Path to follow entity references in one round trip, for example the name of an entity's parent.

result, err := client.Read(ctx, entityID, store.Path(nameField))
name, err := result.Value.String()

parent, err := client.Read(ctx, entityID, store.Path(parentField, nameField))
parentName, err := parent.Value.String()

Write a value with the constructor for its type.

err = client.Write(ctx, entityID, store.Path(statusField), store.Int(1))
err = client.Write(ctx, entityID, store.Path(temperatureField), store.Float(23.5))

Creating and deleting entities

newDevice, err := client.Create(ctx, deviceType, nil, "NewDevice")
fmt.Println("Created device:", newDevice)
err = client.Delete(ctx, newDevice)

Finding entities with CEL filters

Find returns the IDs of entities of a given type that match a CEL filter expression. Related calls include ResolvePath and EntityTypes.

sensorType, err := client.EntityType(ctx, "Sensor")
filter := "Status == 1"
activeSensors, err := client.Find(ctx, sensorType, &filter)

for _, sensorID := range activeSensors {
    fmt.Println("Active sensor:", sensorID)
}

Client operations

Historian queries

Query recorded time-series data with client.QueryHistory. Time bounds are Unix seconds, and Limit is between 1 and 1024. Each returned record carries a timestamp in Unix seconds, entity ID, field path, the typed value, the writer name, and context.

now := time.Now().Unix()
records, err := client.QueryHistory(ctx, store.HistoryQuery{
    TableName: "temperature_table",
    StartTime: now - 3600,
    EndTime:   now,
    Limit:     100,
})
if err != nil {
    return err
}

for _, record := range records {
    temperature, err := record.Value.Float()
    if err != nil {
        return err
    }
    fmt.Println(record.Timestamp, temperature)
}

Program log

Everything a program prints, with fmt.Println, the log package or standard error, is written to the program's own log file on the node that ran it, and so is every error a handler returns, as a line that starts with handler error:. Logs rotate at a fixed size so they never grow without bound.

fmt.Println("Program started")
fmt.Println("Active sensors:", len(activeSensors))

Input and result with Args

A program keeps no state between runs except what it writes to the Store. Variables that handlers share live as long as the run, and a restart or a change of leader starts them over. The Args field carries data in and out of a run. The value current when Execute is written, or when a Leader run starts, is the run's input. When the run finishes as Success, the result it sets is written back to Args. A failed run, or one that sets no result, leaves Args unchanged.

package main

import (
    "context"
    "fmt"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
)

func Run(ctx context.Context, client *store.Client) error {
    input := program.Input(ctx)
    fmt.Println("Input:", string(input))
    program.SetResult(ctx, []byte(`{"status":"done"}`))
    return nil
}

Starting programs

Trigger programs

A Trigger program runs once per Execute write. Write the field from the Plugin Editor's Run button or from any service or program. Each write uses the Source, SourceLanguage and Args current at that moment. Runs of one program execute one at a time, in the order written. Up to 64 runs can wait, and a write beyond that finishes as Error "execution queue full". A run interrupted by a change of leader finishes as Error "run interrupted by failover" and is not run again.

A Trigger run that registers no handlers finishes when Run returns. It can also register handlers, for example to wait for a reply, as shown under Waiting for a reply. A run that never ends holds the queue behind it.

Leader programs

A Leader program starts one run when its node becomes the leader and when Enabled turns true on the leader, and it ignores Execute. Use it for programs that react to events. Programs run on the leader node only, so a program runs once across the cluster regardless of node count. If the leader changes during a run, the new leader finishes the interrupted run as Error "run interrupted by failover" and starts the program again from the beginning, so Run registers its handlers again and anything that happened during the change is not replayed.

A Leader run lasts while at least one handler is registered. A run that returns from Run with nothing registered finishes as Success. A run that ends, with or without an error, stays stopped until the next leadership term, a change to Enabled, or an edit to Source, SourceLanguage or StartMode. A run that panics, calls os.Exit, runs out of memory or falls too far behind its notifications is restarted, as described under Limits.

Handlers and events

Import github.com/controlbird/platform/services/plugin-engine/program to register handlers. Each registration returns a handle, and a handler receives the same ctx as Run.

How events are delivered

• One handler runs at a time. No handler starts before the previous one returns, so handlers share variables without locks.
• The notifications of one subscription arrive in the order the Store sent them.
• Events are delivered in turns. A turn first calls the handlers of the timers that had fired when it began, then those of the notifications that had arrived by then. Anything that arrives during a turn waits for the next one. Beyond that, timers and notifications have no relative order.
• A repeating timer that falls behind does not queue up its missed ticks: at most one call is pending per timer.
• Once program.Cancel returns, the handler is never called again, even when an event for it was already waiting.
• Handlers registered on identical NotifyConfig values share one subscription and run in the order they were registered.
• The context a handler receives is never cancelled. Return from the handler to finish its work, and use program.WithTimeout to bound a slow Store call.

When a run ends

A run ends at the first of these:

• Run returns an error.
• A handler calls program.Stop.
• No handler is registered after an event returns, including when Run returns with nothing registered and when the last handler is cancelled.
• The run is cancelled: the program is disabled, deleted or edited, the node stops leading, or ExecutionTimeoutMs passes.
• A handler panics, or the run falls too far behind its notifications (see Falling behind).

The result a run sets with program.SetResult is written to Args when the run ends as Success: Run returns with nothing registered, the last handler is cancelled, or program.Stop(nil) is called. An error, a panic or a failed run writes no result.

Errors in handlers

A handler that returns an error does not end the run. The engine writes handler error: followed by the error text to the program log and calls the next handler. The error does not appear in LastError, which describes how the last run finished. To end the run and put an error in LastError, call program.Stop(err) with a non-nil error. A panic ends the run.

Periodic work

Register program.Every to repeat a task. The handler runs once per interval, one interval after Run registers it.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
)

func Run(ctx context.Context, client *store.Client) error {
    sensorType, err := client.EntityType(ctx, "Sensor")
    if err != nil {
        return err
    }
    _, err = program.Every(time.Minute, func(ctx context.Context) error {
        sensors, err := client.Find(ctx, sensorType, nil)
        if err != nil {
            return err
        }
        fmt.Println("Sensors:", len(sensors))
        return nil
    })
    return err
}

Scheduled work

There is no cron registration. Parse a schedule with github.com/robfig/cron/v3 and register program.After for its next time. The handler registers the next one before it does its work. A Leader program runs once across the cluster, so the schedule fires once across the cluster.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
    "github.com/robfig/cron/v3"
)

func Run(ctx context.Context, client *store.Client) error {
    schedule, err := cron.ParseStandard("*/5 * * * *")
    if err != nil {
        return err
    }
    var fire program.TimerHandler
    fire = func(ctx context.Context) error {
        if _, err := program.After(time.Until(schedule.Next(time.Now())), fire); err != nil {
            return err
        }
        fmt.Println("Every five minutes")
        return nil
    }
    _, err = program.After(time.Until(schedule.Next(time.Now())), fire)
    return err
}

Time zones

Schedules evaluate in UTC. To schedule in a named time zone, import time/tzdata and start the spec with CRON_TZ=. Without the import, cron.ParseStandard returns the error "provided bad location", and a call to time.LoadLocation does not compile. The import adds the time zone database to the program, so import it only in programs that need it.

package main

import (
    "context"
    "fmt"
    "time"
    _ "time/tzdata"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
    "github.com/robfig/cron/v3"
)

func Run(ctx context.Context, client *store.Client) error {
    schedule, err := cron.ParseStandard("CRON_TZ=Europe/Berlin 0 6 * * *")
    if err != nil {
        return err
    }
    var fire program.TimerHandler
    fire = func(ctx context.Context) error {
        if _, err := program.After(time.Until(schedule.Next(time.Now())), fire); err != nil {
            return err
        }
        fmt.Println("Six o'clock in Berlin")
        return nil
    }
    _, err = program.After(time.Until(schedule.Next(time.Now())), fire)
    return err
}

Reacting to field changes

Register program.OnNotification to be called when a field changes. Set TriggerOnChange to true to be notified only when the value changes, or false to be notified on every write. The handler is not called for the value that is current at registration. For a NotifyEntity subscription, Registration.Initial returns that value, followed by the value of each Context path, before any notification arrives. For the other target kinds it is empty.

package main

import (
    "context"
    "fmt"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
)

func Run(ctx context.Context, client *store.Client) error {
    sensor, err := client.ResolvePath(ctx, "Root/Plant/Boiler/Sensor")
    if err != nil {
        return err
    }
    temperatureField, err := client.FieldType(ctx, "Temperature")
    if err != nil {
        return err
    }
    registration, err := program.OnNotification(ctx, store.NotifyConfig{
        TargetKind:      store.NotifyEntity,
        EntityID:        sensor,
        FieldType:       temperatureField,
        TriggerOnChange: true,
    }, func(ctx context.Context, notification store.Notification) error {
        if notification.Current.Value == nil {
            return nil
        }
        temperature, err := notification.Current.Value.Float()
        if err != nil {
            return err
        }
        fmt.Println("Temperature:", temperature)
        return nil
    })
    if err != nil {
        return err
    }
    if len(registration.Initial) > 0 {
        starting, err := registration.Initial[0].Value.Float()
        if err != nil {
            return err
        }
        fmt.Println("Starting temperature:", starting)
    }
    return nil
}

A handler registered again on an identical NotifyConfig in the same run can, for target kinds other than NotifyEntity, receive notifications of writes made shortly before it registered. For a NotifyEntity subscription it never receives a value that Initial already reflects.

Waiting for a reply

A Trigger run can wait for something to happen. Register the handler that receives the answer, register a timeout with program.After, and end the run from whichever fires first. The reply handler sets the result and stops with success. The timeout stops with an error, which becomes LastError.

package main

import (
    "context"
    "errors"
    "time"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
)

func Run(ctx context.Context, client *store.Client) error {
    device, err := client.ResolvePath(ctx, "Root/Devices/Gateway")
    if err != nil {
        return err
    }
    replyField, err := client.FieldType(ctx, "Reply")
    if err != nil {
        return err
    }
    _, err = program.OnNotification(ctx, store.NotifyConfig{
        TargetKind: store.NotifyEntity,
        EntityID:   device,
        FieldType:  replyField,
    }, func(ctx context.Context, notification store.Notification) error {
        if notification.Current.Value == nil {
            return nil
        }
        reply, err := notification.Current.Value.String()
        if err != nil {
            return err
        }
        program.SetResult(ctx, []byte(reply))
        program.Stop(nil)
        return nil
    })
    if err != nil {
        return err
    }
    _, err = program.After(10*time.Second, func(ctx context.Context) error {
        program.Stop(errors.New("no reply after 10 seconds"))
        return nil
    })
    return err
}

Cancelling handlers

Keep the handle a registration returns to cancel it later. Cancelling the last handler ends the run as Success.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
)

func Run(ctx context.Context, client *store.Client) error {
    var polling program.Handle
    attempts := 0
    polling, err := program.Every(5*time.Second, func(ctx context.Context) error {
        attempts++
        fmt.Println("Attempt", attempts)
        if attempts == 3 {
            return program.Cancel(ctx, polling)
        }
        return nil
    })
    return err
}

Bounding Store calls

A Store call has a default time limit of one second. program.WithTimeout and program.WithDeadline can only shorten it, for the calls made with the context they return. Use them in place of context.WithTimeout, which a program cannot call.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/controlbird/platform/sdk/store"
    "github.com/controlbird/platform/services/plugin-engine/program"
)

func Run(ctx context.Context, client *store.Client) error {
    boiler, err := client.ResolvePath(ctx, "Root/Plant/Boiler")
    if err != nil {
        return err
    }
    statusField, err := client.FieldType(ctx, "Status")
    if err != nil {
        return err
    }
    _, err = program.Every(time.Minute, func(ctx context.Context) error {
        bounded := program.WithTimeout(ctx, 200*time.Millisecond)
        result, err := client.Read(bounded, boiler, store.Path(statusField))
        if err != nil {
            return err
        }
        status, err := result.Value.Int()
        if err != nil {
            return err
        }
        fmt.Println("Status:", status)
        return nil
    })
    return err
}

Stopping and limiting runs

Setting Enabled to false cancels the active run, which finishes as Error "run cancelled", and drops queued runs. Execute writes while the program is disabled start nothing. A run that is still active after ExecutionTimeoutMs is cancelled and finishes as Error "run timed out". A handler that never returns holds its run until the run is cancelled, and the engine stops it a short time after the cancel, even when it is only looping.

Limits

Programs are Go compiled with TinyGo 0.42.0 to WebAssembly (wasip1). TinyGo supports most of Go, with the differences and limits below.

Compiling

A compile takes about a minute, and one that runs past 5 minutes is stopped and finishes the run as Error. A compile error names the file, line and column, and appears in the program's CompileError and LastError fields.

No goroutines or blocking calls

A program has no goroutines, and nothing in it can wait for another task. The compiler refuses each construct below, names its position, and tells you what to use instead. The diagnostics appear even when the source has other type errors.

A select with a default, sync.Mutex, sync/atomic, time.Now and context.WithValue work as in any Go program. time.Sleep works too, but it holds the whole run until it returns, so no other handler runs meanwhile. A cancelled run wakes it early.

32-bit int, uint and uintptr

int, uint and uintptr are 32-bit. A conversion to any of them from int64, uint64, uint32, time.Duration, float32 or float64, or from a named type over one of these such as store.EntityType, is a compile error that names its position. Keep such values as int64 or uint64. Constants are exempt.

This does not compile:

seconds := int(elapsed.Seconds())

Keep the value 64-bit instead:

seconds := int64(elapsed.Seconds())

No recover

recover() does not work, so a program cannot catch a panic. A panic ends the run, and the run finishes as Error with the panic message in LastError. A program that calls os.Exit also finishes as Error, whatever the exit code.

Restarts

A long-running program, which is a Program with StartModeLeader and every RuleChain, restarts when its run ends by a panic, by os.Exit, by running out of memory or by falling too far behind its notifications. A run that returns, with or without an error, is not restarted. The first restart comes after 1 second, and the delay doubles up to 60 seconds. The delay returns to 1 second once a run has stayed up for 10 minutes. Disabling or deleting the program, or the node ceasing to lead, cancels the restart, and editing the program cancels it and starts the edited program at once. A Trigger program is not restarted; each Execute write starts a new run.

Falling behind

The engine does not merge or drop notifications. They wait for their handler, up to 4 MiB for the whole run. A run that falls further behind, or whose subscription the Store ends, finishes as Error "store: subscription queue overflow". A Leader program then restarts, and its Run registers again and reads the current values from Registration.Initial. Keep handlers short: a handler that takes longer than the notifications take to arrive makes the run fall behind.

Memory and size

A run has 128 MiB of memory for its heap, stack and runtime together. A run that needs more ends with fatal error: out of memory in LastError. A single large allocation can fail well before the total reaches 128 MiB, so build large buffers in pieces. Handlers run on a 256 KiB stack, so deep recursion also ends the run.

A registration beyond the subscription or timer limit returns an error from program.OnNotification, program.Every or program.After.

Imports

google.golang.org/protobuf cannot be imported. github.com/robfig/cron/v3 is available for scheduling, alongside the ControlBird SDK and the standard library packages that TinyGo supports. A program carries no time zone data unless it imports time/tzdata.