Usage

Key components

• Address
  • An immutable representation of a modbus address which can handle many ways to specify a modbus address.
• Field
  • Maps a set of registers into a usable value
    • Can result in a Long, Double, String or List<String> depending on the used mapping expression.
    • Extensive expression language is available that allows referencing other Fields in the same Block to do calculations (does PEMDAS).
• Block
  • A group of Fields that all use the same kind (for example: All Fields only use Holding Registers) of registers of a device.
• SchemaDevice
  • All the Blocks and Fields for a specific logical device.
• ModbusDevice
  • The abstraction that allows a SchemaDevice to connect a physical Modbus system.
  • Several ModbusDevice implementations have been made to wrap an existing Modbus library to make it usable in this toolkit.

Modbus is slow

The basic problem with real devices providing data over Modbus is that they are very very slow.

I have found that reading a single register from my heatpump takes about 500ms and that reading a block of 125 registers in a single request also takes about 500ms.

So combining multiple sets of registers in fewer modbus requests is makes it all faster.

I have also found that requesting part of the registers that belong to a logical value will (in some systems) result either in garbage data or read errors.

Things take into consideration in the design of this toolkit:

• Always fetch the registers for a single field in the same modbus request.
  • Some groups of fields even must be fetched in a combination in SunSpec
    • The “sync” type of a group (as used in Model 704)
• Only fetch what is needed.
  • Most of the time you do not need everything: you must indicate what you want.
• Try to combine as many of the needed registers into a few as possible modbus requests.
  • Which may include fetching registers you did not ask for.
• Some values will never change (i.e. are immutable)
  • Only read those the first time and then never again.
• Remember read errors
  • No need to retry a read error
  • Optimize the fetching around these errors

Basic workflow

So the base workflow of this library has been chosen als follows:

• Obtain an instance of SchemaDevice

  • This has one or more Blocks and each has Fields
  • Each Field has an expression.
  • The expression dictates the ReturnType of the specific Field.

• Connect to the actual Modbus Device using one of the supported Modbus implementations.

• Link these two together

  • schemaDevice.connect(modbusDevice)

• Indicate to the SchemaDevice instance which of the fields you need.

And then as often as you like (and your hardware supports!):

• Tell it to do update
  • Now the library will retrieve the needed modbus register values and remember them. This also includes when these were retrieved !
• For each field you can now get the actual value

Add the main library to your project

<dependency>
  <groupId>nl.basjes.modbus</groupId>
    <artifactId>modbus-schema-device</artifactId>
  <version>0.6.0</version>
</dependency>

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>nl.basjes.modbus</groupId>
      <artifactId>modbus-schema-bom</artifactId>
      <version>0.6.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  <dependencies>
<dependencyManagement>

<dependency>
  <groupId>nl.basjes.modbus</groupId>
  <artifactId>modbus-schema-device</artifactId>
</dependency>

Creating a SchemaDevice

You can obtain a SchemaDevice using one of these forms:

• Create it manually using code
• Reading it form a Yaml file
• Code created by converting the Yaml file into code.
• Generated by code in some other way (like with SunSpec)

This defines a single Block with a single “Name” Field in code:

val schemaDevice = SchemaDevice()

val block = Block.builder()
  .schemaDevice(schemaDevice)        // The Block is automatically linked to the SchemaDevice
  .id("Block 1")                     // The ID under which the Field can be retrieved
  .description("The first block")    // A human-readable description
  .build()

Field.builder()
  .block(block)                      // The Field is automatically linked to the Block
  .id("Name")                        // The ID under which the Field can be retrieved
  .description("The name Field")     // A human-readable description
  .expression("utf8(hr:00000 # 12)") // The expression that maps registers into the desired output
  .unit("")                          // You can label the output with a unit (for human use) 
                                     //   like "Volt", "Meters", "Seconds", etc.
  .immutable(false)                  // If a field NEVER changes value then set this to true. 
                                     // - Used by the Modbus Optimizer to skip this in later retrievals.
  .system(false)                     // If a field is not a user level usable value set this to true 
                                     //   (for example a scaling factor).
                                     // - Used by the code generator to hide these fields.
  .build()

require(schemaDevice.initialize()) { "Unable to initialize schema device" }

SchemaDevice schemaDevice = new SchemaDevice();

Block block = Block.builder()
  .schemaDevice(schemaDevice)        // The Block is automatically linked to the SchemaDevice
  .id("Block 1")                     // The ID under which the Field can be retrieved
  .description("The first block")    // A human-readable description
  .build();

Field field = Field.builder()
  .block(block)                      // The Field is automatically linked to the Block
  .id("Name")                        // The ID under which the Field can be retrieved
  .description("The name Field")     // A human-readable description
  .expression("utf8(hr:00000 # 12)") // The expression that maps registers into the desired output
  .unit("")                          // You can label the output with a unit (for human use) 
                                     //   like "Volt", "Meters", "Seconds", etc.
  .immutable(false)                  // If a field NEVER changes value then set this to true. 
                                     // - Used by the Modbus Optimizer to skip this in later retrievals.
  .system(false)                     // If a field is not a user level usable value set this to true 
                                     //   (for example a scaling factor).
                                     // - Used by the code generator to hide these fields.
  .build();

if (!schemaDevice.initialize()) {
    throw new IllegalStateException("Unable to initialize schema device");
}

The same can be put in a Yaml file like this:

# $schema: https://modbus.basjes.nl/v1/ModbusSchema.json
description: 'Demo based on a SunSpec device schema'
schemaFeatureLevel: 1

blocks:
  - id: 'Block 1'
    description: 'The first block'
    fields:
      - id: 'Name'                    # The ID under which the Field can be retrieved
        description: 'The name Field' # A human readable description
        expression: 'utf8(hr:0 # 12)' # The expression that maps registers into the desired output
        #  unit: ""                   # You can label the output with a unit (for human use)
                                      #   like "Volt", "Meters", "Seconds", etc.
        #  immutable: false           # If a field NEVER changes value then set this to true.
                                      # - Used by the Modbus Optimizer to skip this in later retrievals.
        #  system: false              # If a field is not a user level usable value set this to true

which you then need to load in your application:

val schema = File("example.yaml").readText(Charsets.UTF_8)

SchemaDevice schemaDevice = YamlLoaderKt.toSchemaDevice(new File("example.yaml"));

Connect to the actual Modbus Device

Some example code in Kotlin for the currently available modbus libraries

val hostname = "modbus.iot.basjes.nl"
val port     = MODBUS_STANDARD_TCP_PORT
val unitId   = 1

val connectionString = "modbus-tcp:tcp://$hostname:$port?unit-identifier=$unitId"

print("Connecting...")
ModbusDevicePlc4j(connectionString).use { modbusDevice ->
  println(" done")

  ...

}

val hostname = "modbus.iot.basjes.nl"
val port     = MODBUS_STANDARD_TCP_PORT
val unitId   = 1

val master: AbstractModbusMaster = ModbusTCPMaster(hostname, port)
try {
  print("Connecting...")
  master.connect()
  println(" done")
  val modbusDevice: ModbusDevice = ModbusDeviceJ2Mod(master, unitId)

  ...
  
} catch (e: Exception) {
  throw ModbusException("Unable to connect to the master", e)
} finally {
  master.disconnect()
}

val hostname = "modbus.iot.basjes.nl"
val port     = MODBUS_STANDARD_TCP_PORT
val unitId   = 1

val configBuilder = NettyClientTransportConfig.Builder()
configBuilder.hostname      = hostname
configBuilder.port          = port

val transport = NettyTcpClientTransport(configBuilder.build())
val client = ModbusTcpClient.create(transport)

try {
  print("Connecting...")
  client.connect()
  println(" done")

  val modbusDevice: ModbusDevice = ModbusDeviceDigitalPetri(client, unitId)

  ...  

} catch (e: Exception) {
  println(" FAILED")
  throw ModbusException("Unable to connect to the master", e)
} finally {
  println("Disconnecting...")
  client.disconnect()
}

Link these two together

schemaDevice.connect(modbusDevice)

schemaDevice.connect(modbusDevice);

Get the needed Fields

Then you need to indicate which fields you need.

val name = schemaDevice["Block 1"]["Name"] ?: return // Cannot continue

name.need()

Block block = schemaDevice.getBlock("Block 1");
if (block == null) {
    return; // Cannot continue
}

Field name = block.getField("Name");
if (name == null) {
    return; // Cannot continue
}

name.need();

Retrieve the data from the physical system

At this point we have a schema, are connected to the physical device and have indicated the “Name” field is needed.

Now we tell the system to retrieve all register values that are needed.

If we simply do this then all fields that need to be updated will be updated.

schemaDevice.update()

schemaDevice.update();

We can also specify a maximum age and this has the potential to reduce the number of modbus requests because this will only update those registers for which the value is too old.

Only update the needed registers for which there is no value available or the available value is more than 5000 ms old.

schemaDevice.update(5000)

schemaDevice.update(5000);

Work with the meaningful values

Each Field has an expression that dictates what the return type of that Field is. Depending on the indicated return type a different field need to be used to get the value. This strong typing was done to avoid any internal conversions of the data and thus reduce the loss of accuracy.

Note that these will return a null in all of these cases:

• No data has been retrieved yet for this Field.
• There was a read error in one of the registers of this Field.
• You are accessing the wrong value property for this Field (i.e. use .longValue for a Field that returns a String )

In one of my projects I used a library which had a separate function for each of those types and the code to use it became this. This looks like it can be combined but because of the differences in the underlying types it could not.

when(it.returnType) {
    DOUBLE ->        it.doubleValue                ?.let { value -> point.addField(label, value) }
    LONG ->          it.longValue                  ?.let { value -> point.addField(label, value) }
    STRING ->        it.stringValue                ?.let { value -> point.addField(label, value) }
    STRINGLIST ->    it.stringListValue?.toString()?.let { value -> point.addField(label, value) }
    UNKNOWN -> TODO()
    BOOLEAN -> TODO()
}

Rinse and repeat

From here your application will probably do repeated cycles of schemaDevice.update() followed by extracting the needed values.