Yaml format

You can create the schema for your device!

A simple example of the end result

# $schema: https://modbus.basjes.nl/v2/ModbusSchema.json
description: 'A very simple demo schema'
schemaFeatureLevel: 2

blocks:
  - id: 'Block 1'
    description: 'The first block'
    fields:
      - id: 'Name'
        description: 'The name Field'
        # If a field NEVER changes value then set this to true
#        immutable: true
        # If a field is not a user level usable value set this to true (for example a scaling factor)
#        system: true
        expression: 'utf8(hr:0 # 12)'

      - id: 'Flag'
        description: 'The flag Field'
        # If a field NEVER changes value then set this to true
#        immutable: true
        # If a field is not a user level usable value set this to true (for example a scaling factor)
#        system: true
        expression: 'boolean(c:0)'

tests:
  - id: 'Just to demo the test capability'
    input:
      - firstAddress: 'hr:0'
        rawValues: |2-
          # --------------------------------------
          # The name is here
          4e69 656c 7320 4261 736a 6573 0000 0000 0000 0000 
          0000 0000

      - firstAddress: 'c:0'
        rawValues: |2-
          # --------------------------------------
          # The flag is here
          0

    blocks:
      - id:          'Block 1'
        expected:
          'Name':    [ 'Niels Basjes' ]
          'Flag':    [ 'false' ]

Adding tests is really easy

It is really easy to add tests if you have a schema yaml and a real device to get real data from.

  1. Define all the Blocks and Fields for your schema.
  2. Now load that in the library to create a SchemaDevice.
  3. Connect to a ModbusDevice instance with a real device behind it.
  4. Get the register and discrete values for all the defined fields
schemaDevice.updateAll()
  1. Based on all currently loaded register values generate the test
schemaDevice.createTestsUsingCurrentRealData()
  1. Simply print the schema with all the tests included
println(schemaDevice.toYaml())

Or simply start with this kotlin script which does all that.

AddTestsToSchema.main.kts
#!/usr/bin/env kotlin

// Include the needed libraries
@file:DependsOn("nl.basjes.modbus:modbus-api-j2mod:0.14.0")
@file:DependsOn("nl.basjes.modbus:modbus-schema-device:0.14.0")

// Regular Kotlin import statements
import com.ghgande.j2mod.modbus.facade.ModbusTCPMaster
import nl.basjes.modbus.device.api.MODBUS_STANDARD_TCP_PORT
import nl.basjes.modbus.device.j2mod.ModbusDeviceJ2Mod
import nl.basjes.modbus.schema.toSchemaDevice
import nl.basjes.modbus.schema.toTable
import nl.basjes.modbus.schema.toYaml
import java.io.File

// The hostname to connect to
val modbusHost        = "modbus.iot.basjes.nl"
val modbusPort        = MODBUS_STANDARD_TCP_PORT
val modbusUnit        = 1

print("Modbus: Connecting...")
val modbusMaster = ModbusTCPMaster(modbusHost, modbusPort)
modbusMaster.connect()
ModbusDeviceJ2Mod(modbusMaster, modbusUnit). use { modbusDevice ->
    println(" done")

    // Read the schema from a file
    val schema = File("minimal.yaml").readText(Charsets.UTF_8)

    // Convert that into a SchemaDevice with all the defined mappings (Blocks and Fields)
    val device = schema.toSchemaDevice()

    // Connect this Schema Device to the physical device
    device.connect(modbusDevice)

    // Real Modbus devices are so very slow that we need to indicate which need to be kept up to date
    // Because we are testing the entire schema we say we need all
    device.needAll()

    // Now we tell the system all fields must be made up-to-date, and we consider values
    // less than 5 seconds old to be "new enough".
    device.update(5000) // << Here the modbus calls are done.

    // Output the results as a table ( "----" is a not retrieved register, "xxxx" is a read error )
    println(device.toTable(onlyUseFullFields = false, includeRawDataAndMappings = true))

    // Now create a NEW test scenario from the currently available values and add that to the
    // schema definition.
    device.createTestsUsingCurrentRealData()

    // And print the entire thing as a reusable schema yaml (including the added tests).
    println("#-------------- BEGIN: MODBUS SCHEMA IN YAML FORMAT --------------")
    println(device.toYaml())
    println("#-------------- END: MODBUS SCHEMA IN YAML FORMAT ----------------")
}

Yaml File Format explained

# $schema: https://modbus.basjes.nl/v2/ModbusSchema.json
description: 'A very simple demo schema'
schemaFeatureLevel: 2
maxRegistersPerModbusRequest: 125
  • # $schema: https://modbus.basjes.nl/v2/ModbusSchema.json
    • A reference to a JsonSchema that is used to validate and document the yaml file. Many editors support this to aid you in writing a valid yaml structure.
  • description: 'A very simple demo schema'
    • A human readable description for what this is the schema
  • schemaFeatureLevel: 2
    • The allowed functionality in this schema. Version 1 only allowed registers, Version 2 added booleans (coils and discrete inputs)
  • maxRegistersPerModbusRequest: 125
    • [OPTIONAL] If the device does not allow getting the full 125 registers per request you can limit that here.

Block

blocks:
- id: 'Block 1'
  description: 'The first block'
  fields:

You can have multiple blocks. Each block must have a unique id and has its own set of Fields. See it as a namespace which bundled logical values together.

NOTE: A field can only use other fields in their expression from the SAME block.

Field

    - id: 'Name'
      description: 'The name Field'
      # If a field NEVER changes value then set this to true
#        immutable: true
        # If a field is not a user level usable value set this to true (for example a scaling factor)
#        system: true
      expression: 'utf8(hr:0 # 12)'
      unit: 'W'

A block has 1 or more Fields.

  • id
    • Each Field must have a unique id and an expression that determines the way the value for this Field is obtained.
  • description
    • A human-readable explanation what this field is.
  • expression
    • The content of the expression is explained on the other pages under schema on this site.
  • unit
    • If the returned value has a unit (like Watt, Volt or Celsius) you can use this to add a human readable unit to the Field. This can be used downstream to show in a dashboard.
  • immutable
    • If the value of a Field will NEVER change (during the runtime of the application) then set immutable: true because that will aid in optimizing the queries done to the actual device.
  • system
    • If the value is something that should not be considered a meaningful value by itself then set it to system: true. The effect is that in many places (like code generation and GraphQL) the Field it will be hidden. This is used in the SunSpec scaling factors (which are intermediate fields needed to calculate a value) and padding fields (which are useless).
  • fetchGroup
    • Don’t use this because it will hurt you. Only needed in very rare cases.
    • It is possible to force multiple Fields to always be part of the same Modbus Query. This requires the registers of those fields to be directly next to each other.

Tests

tests:
- id: 'Just to demo the test capability'

Optionally a schema yaml can contain tests. Each test validates the relation between specific input registers/discretes and the output values for specific fields.

  • id
    • The ID of a test will be used to name the test code when generating code.

Test Input

  input:
    - firstAddress: 'hr:0'
      rawValues: |2-
      # --------------------------------------
      # The name is here
      4e69 656c 7320 4261 736a 6573 0000 0000 0000 0000
      0000 0000

    - firstAddress: 'c:0'
      rawValues: |2-
      # --------------------------------------
      # The flag is here
      0 0 1 1 0

The input section contains the input modbus values. Multiple blocks of register values can be provided and these will all be merged together as input for the tests.

  • firstAddress
    • The address of the first value in the list of provided values.
  • rawValues
    • A space/newline separated list of raw modbus input values.
    • In case of Registers these will be groups of 4 hex letters each representing a single register.
      • Values you may find:
        • [0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]: Normal 4 digit hex values.
        • ----: No value available.
        • xxxx or XXXX: A Modbus Read error is returned when these are requested.
    • In case of Discretes these will be groups of 1 letter each representing a single discrete. Values:
      • Values you may find:
        • 0: A 0 is returned.
        • 1: A 1 is returned.
        • -: No value available.
        • x or X: A Modbus Read error is returned when these are requested.

NOTE: When the tests are generated from a read device the lowercase x (xxxx or x) means this was a Soft read error and when you see an upper case X (XXXX or X) it means this was a Hard read error. Check the page on fetching for more information about this.

Test Expectations

  blocks:
    - id:          'Block 1'
      expected:
        'Name':    [ 'Niels Basjes' ]
        'Flag':    [ 'false' ]

The last part is specifying the expectations that must be correct. It is not required to have all blocks and/or fields specified.

For each Field which you want to verify you can add an expected value.

Note the expected value is an array !

  • An empty array [] means no value should be returned.
  • An array can have
    • a number [ 42 ] or [ 42.42 ]
    • a boolean (as String) [ 'true' ] or [ 'false' ]
    • a list of Strings [ 'one', 'two' ]