openpectus.engine.hardware_recovery

Attributes

logger

Classes

ErrorRecoveryState

Create a collection of name/value pairs.

ErrorRecoveryConfig

ErrorRecoveryDecorator

Implements error recovery as a decorator around concrete hardware, without coupling it to Engine.

Module Contents

openpectus.engine.hardware_recovery.logger
class openpectus.engine.hardware_recovery.ErrorRecoveryState(*args, **kwds)

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

  • attribute access:

>>> Color.RED
<Color.RED: 1>

  • value lookup:

>>> Color(1)
<Color.RED: 1>

  • name lookup:

>>> Color['RED']
<Color.RED: 1>

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

Disconnected = 0

No connection attempt yet

OK = 1

No problem with the hardware connection

Issue = 2

There may be an issue. A few errors detected. Trying to recover by waiting for success reads/writes while masking errors.

Reconnect = 3

There is a connection an issue. Trying to recover by reconnecting while still masking errors.

Error = 4

Connection is lost and reconnection has not been successful. Trying to recover by reconnecting. Errors are no longer masked.

class openpectus.engine.hardware_recovery.ErrorRecoveryConfig
reconnect_timeout_seconds = 10

Number of seconds in state Issue before changing to state Reconnect

error_timeout_seconds = 18000

Number of seconds in state Reconnect before changing to state Error

only_write_modified_values = True

Set to True to only write register values that have changes since last successful write

class openpectus.engine.hardware_recovery.ErrorRecoveryDecorator(decorated, config, connection_status_tag)

Bases: openpectus.engine.hardware.HardwareLayerBase

Implements error recovery as a decorator around concrete hardware, without coupling it to Engine.

Error recovery has the 5 states defined in ErrorRecoveryState. It is configured using the time thresholds defined in ErrorRecoveryConfig.

Once connected, state is OK. This state is kept until a read or write error occurs (read(), write() or one of the batch variants raise HardwareLayerException). Such an error causes state to transition to Issue. In this state, if a success read/write occurs, state transitions back to OK. If no success read/write occurs within a duration of issue_timeout_seconds, state is set to Reconnect.

In state Reconnect, reconnect attempts are started. If successful, state is set to OK. If not successful within a duration of error_timeout_seconds, state is set to Error.

While in states Issue and Reconnect, any read and write errors are masked by returning last-known-good values for reads and caching values for writes. This means that the Engine (and the user) will not notice the connection loss.

The one exception to this is uod commands. We have to assume that a uod command cannot execute correctly without hardware connection so we have to fail uod commands. (A possible improvement would be to require uod commands to consider the connection and fail with predefined exception types).

In state Error reconnect attempts continue but errors are no longer masked. The Connection Status tag is set to ‘Disconnected’. If reconnect is successful, state is set to OK and Connection Status is set to Connected.

The consequence of no longer masking errors, is that the Engine will enter the “paused on error” state where the user can decide whether to continue or not.

Note: It is unlikely but possible that errors occur so soon after successful connection that no value is yet available as last-known-good. In this case, a read returns a None value is returned and the error is logged.

Reconnect note: The default implementation uses the tick() method to detect and execute reconnect. If this takes too long and hurts engine timing, the hardware should instead implement its reconnect via threading.

Parameters:
__str__()
Return type:

str

decorated
config
connection_status_tag
connect_error_callback: Callable[[Exception], None] | None = None
error_callback: Callable[[], None] | None = None
reconnect_callback: Callable[[], None] | None = None
reconnecting_callback: Callable[[], None] | None = None
reconnected_callback: Callable[[], None] | None = None
last_known_good_reads: dict[str, Any]
pending_writes: dict[openpectus.engine.hardware.Register, Any]
last_success_writes: dict[str, Any]
state: ErrorRecoveryState
last_success_read_write
last_success_connect
last_state_reconnect_time

The time of the last transition to state Reconnect

reconnect_count = 0
reconnect_tick = -1
reconnect_backoff_ticks = [5, 20, 100, 300, 1200, 18000]
get_recovery_state()
Return type:

ErrorRecoveryState

property is_connected: bool

Returns a value indicating whether there is an active connection to the hardware.

Return type:

bool

property registers
success_read()
success_write(values, registers)
Parameters:
filter_write_values(values, registers)

Filteres out register values that do not need to be written because they have not changed since the last time the were written.

Parameters:
Return type:

tuple[Sequence[Any], Sequence[openpectus.engine.hardware.Register]]

error_read_write()
_is_backoff_tick(tick)
Parameters:

tick (int)

Return type:

bool

tick()

Invoked on each tick by engine.

_update_connection_status()
on_ok()
on_issue()
on_error()
on_reconnect()
on_reconnecting()
on_reconnected()
read(r)

Read single register value. Abstract method.

Parameters:

r (openpectus.engine.hardware.Register)

Return type:

Any

read_batch(registers)

Read batch of register values. Override to provide efficient implementation. Virtual method.

Parameters:

registers (Sequence[openpectus.engine.hardware.Register])

Return type:

list[Any]

_get_last_known_good_values(registers)
Parameters:

registers (Sequence[openpectus.engine.hardware.Register])

Return type:

list[Any]

write(value, r)

Write single register value. Abstract method.

Parameters:
Return type:

None

write_batch(values, registers)

Write batch of register values. Override to provide efficient implementation. Virtual method.

Parameters:
connect()

Connect to hardware. Throw HardwareLayerException on error. Virtual method. Implementations must call base method on completed connect.

disconnect()

Disconnect from hardware. Throw HardwareLayerException on error. Virtual method. Implementations must call base method on completed disconnect.

_write_pending_values(except_names=[])
Parameters:

except_names (list[str])

_setup_decorated_method_forwards()

Set up method forwards to avoid the decorator breaking commands that are implemented as methods on the concrete hardware