Protocols

Open Pectus comprises of 3 major components: Engine, Aggregator and Frontend. The protocols that connect them are described in the following.

Frontend/Aggregator Protocol 

The frontend/aggregator protocol is primarily REST-based, but includes push messages via WebSocket.

REST 

The REST protocol contains these interactions:

        sequenceDiagram
    participant F as Frontend
    participant A as Aggregator

    Note right of A: REST

    Note over F, A: Dashboard
    F ->> A: GET process_units
    A -->> F: 
    F ->> A: GET recent_batch_jobs
    A -->> F: 

    Note over F, A: Unit Details

    F ->> A: GET process_unit/{id}/command_examples
    A -->> F: 
    F ->> A: GET process_unit/{id}/process_diagram
    A -->> F: 
    F ->> A: GET process_unit/{id}/plot_configuration
    A -->> F: 

    loop Initial data and on push message METHOD
    F ->> A: GET process_unit/{id}/method
    A -->> F: 
    end
    loop Initial data and on push message RUN_LOG
    F ->> A: GET process_unit/{id}/run_log
    A -->> F: 
    end
    loop polling
    F ->> A: GET process_unit/{id}/process_values
    A -->> F: 
    end

    Note over F, A: Command window
    alt Execute command
    F ->> A: POST process_unit/{id}/execute_command
    A -->> F: 
    end

    Note over F, A: Method Editor
    alt Save method
    F ->> A: POST process_unit/{id}/method
    A -->> F: 
    end

    Note over F, A: Run controls
    alt Start/Pause/Hold/Stop
    F ->> A: POST process_unit/{id}/execute_command
    A -->> F: 
    end

Fig. 5 Frontend/Aggregator REST interactions.

WebSocket 

The WebSocket interactions are all push messages that instruct the frontend to request updates of a certain kind.

The available messages are:

RUN_LOG
METHOD
CONTROL_STATE
ERROR_LOG
PROCESS_UNITS

The frontend knows the relevant REST endpoints that correspond to each type of message.

        sequenceDiagram
    participant F as Frontend
    participant A as Aggregator
    Note right of A: WebSocket
    A ->> F: publish(message)
    F -->> A: -

Fig. 6 Frontend/Aggregator WebSocket interactions.

Engine/Aggregator Protocol 

The engine/aggregator protocol is primarily WebSocket-based, but includes REST messages for registration and health check.

This protocol ensures that the aggregator is kept updated on engines (process units) and their states. It also allows users of the frontend to control connected engines.

Note

This diagram shows a simplified overview. The actual protocol is more elaborate due to its built-in error recovery. This allows it to support temporarily disconnected engines as well as updating and/or restarting the aggregator during active runs. This is documented in Error Recovery.

        sequenceDiagram
    participant A as Aggregator
    participant E as Engine

    Note right of E: Web Socket
    E ->> A: connect
    A -->> E: 

    E ->> A: UodInfoMsg
    A -->> E: 
    E ->> A: TagsUpdatedMsg
    A -->> E: 

    loop Every second
    E ->> A: TagsUpdatedMsg
    A -->> E: 
    end

    alt User: Run command
    A ->> E: InvokeCommandMsg
    E -->> A: 
    end

    alt User: Change run state
    A ->> E: InvokeCommandMsg
    E -->> A: 
    end

Fig. 7 Engine/Aggregator protocol overview.

State Diagrams 

This section documents the important states and state changes in openpectus.

Note

Note on transition naming:

# Lower case transitions (e.g. “register ok”) denote some action in the system. # Capitalized transitions (e.g. “Start”) denote a specific command being executed.

Engine States 

When an engine is started, it automatically connects to the hardware specified in its UOD and to the aggregator URL specified as command line argument. It cannot function properly if either of these connections are unavailable on startup (though the error recovery features will continuously attempt to recover).

Once both connections are in place, the engine is in state Connected. This means that:

# Engine is ready to receive commands or run a method. # The scan cycle loop is started so tag values are continuously read from hardware # Engine is displayed in the frontend dashboard as a process unit with status Ready # Engine details can be viewed in frontend details, including real-time updated values of its configured tags.

This state is also referred to as Steady State (as opposed to states such as starting/initializing/connecting/reconnecting).

Avoid using the term Running the describe Engine state because is ambiguous. It might mean that the engine is Connected/in Steady State, or it could mean that a method is running. The term Connected is used here and in the diagrams to refer to an engine in Steady State.

           stateDiagram-v2    
      [*] --> Connected: Connected to hardware and Aggregator
         note right of Connected: aka Steady State, Stopped
        Connected --> Method_Running: Start
      Method_Running --> Method_Error: Method instruction fail
      Method_Error --> Method_Running: Acknowledge and Retry
      Method_Error --> Connected: Stop
      Method_Running --> Connected: Stop
      Method_Running --> Method_Complete
      Method_Complete --> Connected

Fig. 8 Engine state diagram.

Aggregator States 

The aggregator manages a number of engines and tracks the state of each one.

Note

Persistence of run state is based changes on the run_id system tag. If no longer set, all collected data is saved as a complete run.

        stateDiagram-v2    
    Engine_Unknown --> Engine_Registered: [engine registers]
    Engine_Registered --> Engine_Connected: [ws connect]
    Engine_Connected --> Engine_Registered: [ws disconnect, other]
    Engine_Connected --> Engine_Unknown: [ws disconnect, aggregator refuse]
    Engine_Connected --> Engine_Running: [Start]
    Engine_Running --> Engine_Complete: [Stop]
    Engine_Running --> Received_EngineData: [Engine sends data]
    note right of Received_EngineData: Create and persist BatchJobProcessValueData
    Received_EngineData --> Engine_Running: [auto]

Fig. 9 Aggregator state diagram.

Deployment Diagram 

        C4Deployment

Deployment_Node(userpc, "User pc", "Web Browser"){
    Container(spa, "OpenPectus Frontend", "Single Page Application,  JavaScript and Angular", "Allows access to Open Pectus via a web browser.")
}

Deployment_Node(azureContainer, "Azure private cloud", "Docker"){
    Container(aggregator, "Open Pectus Aggregator", "Python, FastAPI")
}

Deployment_Node(enginePc, "Lab pc", "Laboratory pc connected to equipment"){
    Container(engine, "Open Pectus Engine", "Python")
}

Rel(spa, aggregator, "Loads data", "json/HTTPS")    
Rel(aggregator, spa, "Notifications", "json/WSS")    

Rel(engine, aggregator, "Sends data", "json/HTTPS")
Rel(aggregator, engine, "Sends commands", "json/WSS")

UpdateRelStyle(spa, aggregator, $textColor="white", $offsetY="-40", $offsetX="-50")
UpdateRelStyle(aggregator, spa, $textColor="white", $offsetY="20", $offsetX="-50")

UpdateRelStyle(engine, aggregator, $textColor="white", $offsetY="-35", $offsetX="-110")
UpdateRelStyle(aggregator, engine, $textColor="white", $offsetY="50")

Fig. 10 Deployment diagram.

CSV File Format 

It is possible to export data from a concluded run to a CSV file. The CSV file includes metadata and time series data for all tags defined in the Unit Operation Definition. An example of a CSV file generated for a run using the built-in demo_uod.py UOD is given in Listing 3.

A download link is available in the frontend user interface. It is also possible to download the CSV file using the OpenAPI /api/recent_runs/{run_id}/csv_file endpoint.

Listing 3 Example of exported CSV file. Filename: RecentRun-c87d65e2-7e1a-4477-aa89-b4f56db75773.csv

# Recent Run Id,c87d65e2-7e1a-4477-aa89-b4f56db75773
# Engine Id,LC...50_DemoUod
# Engine Computer name,LC...50
# Engine Version,0.1.13
# Engine Hardware tring,"ErrorRecoveryDecorator(state=ErrorRecoveryState.OK,decorated=DemoHardware)"
# Uod author nam,Demo Author
# Uod author email,demo@openpectus.org
# Uod file name,C:\Users\...\openpectus/engine/configuration/demo_uod.py
# Aggregator Computer name,AZR-PECTUS-PRD
# Aggregator Version,0.1.13
# Starting Time (UTC),2025-01-09 20:15:02.926612
# Ending Time (UTC),2025-01-09 20:16:07.110109
# Contributors,E..L

Run Time [s],FT01 [L/h],TestInt,TestFloat [kg],TestString,Category,FT02 [L/h],Time [s],Reset,System State,CmdWithRegexArgs [dm2],TestPercentage [%]
0.0,10,42,9.87,test,Rising,10.795927561732627,2.3903110027313232,N/A,Stopped,34.87,34.87
5.34400000000096,12,42,9.87,test,Rising,12.552170854700819,7.742166519165039,N/A,Stopped,34.87,34.87

OpenAPI 

The OpenAPI specification is listed below. It is also available at the /docs endpoint on the Aggregator e.g. http://localhost:9800/docs.

GET /api/process_unit/{unit_id}

Get Unit

Parameters:

unit_id (string)

Example request:

GET /api/process_unit/{unit_id} HTTP/1.1
Host: example.com

Status Codes: