XML Parser - Operators

In the XML parser, we have several operators which need to be used at different levels.

Top-level operators:

  • _vars - allows you to define variable which can be used later on in the script. The parser will basically replace the vars throughout the script with the value you set. For example, this:

    _vars:
    root: /response/result

    Means "${root}" will be replaced with "/response/result" throughout the script.

  • _tags - device tags. There is also a _tags section under metric, but more on that below. The "_tags" section at the top level is used to define device tags, such as "os.version" or "model". For example:
    _tags:
    "os.name":
    _constant: "panos"
    "os.version":
    _text: "/response/result/system/sw-version"
    "vendor":
    _constant: "paloaltonetworks"
    "model":
    _text: "/response/result/system/model"
  • _metrics - only allowed in monitoring scripts. Metrics are defined under this section. Each metric is under its own "-" section. It looks like this:
    _metrics:
    -
    _value.double:
    _count: "${root}/entry"
    _tags:
    "im.name":
    _constant: "routes-usage"

    A few things to note:

    • In case no metrics were parsed (due to missing data or incorrect parsing) then the command will fail; that is, unless some dynamic variables were parsed by the command. 
      • By using _optional_metrics instead of _metrics, one could override this behavior and not fail the command when no metrics are parsed.
    • the "_value" operator is used to determine the type of metric (more on this below). 
    • The "_tags" section here is not the same as the one at the root level (described above). Rather, these are tags to attach to the metric.
  • _omit - drop certain lines from the input:

    _omit:
      - _top: 2
      - _bottom: 3
      - _regex: "regex"

    Means omit 2 lines from the top, then three lines from the bottom, the lines matching "regex". The order matters - the omitting will be executed according to the order. Older versions didn't use the "-" at the beginning of each operator line. In that case, it only supported _top and _bottom, no _regex.

     

Value-level operators:

Metrics can be either DOUBLE (a number) or COMPLEX (essentially a JSON structure):

  • _value.double - this is what you use for a DOUBLE metric. Underneath it you should simply say what the value is. For example:
    _value.double:
        _constant: "1"

    Which would set the value to "1". Of course, that's not terribly useful so you'll rarely see "_constant" used for _value.double. Instead you'll see one of the other operators listed further below. 
  • _value.complex - this is for a COMPLEX metric. There are two JSON structures for complex metrics - one is a simple value such as this:
    {"value": "myval"} — note that the key is always "value" in this case

    The other is a JSON array, like this:
    [ {"key1": "someval", "key2": "someotherval"} , {"key1": "anotherval", "key2": "anotherotherval"} ]
    NOTE: there are two items in the above array. In each, there are two keys, each key has a value. The keys MUST match in the different items - a given item cannot have more or less, or different, keys compared to the others in the array. 

    To make the XML parser to generate a JSON array, you must use the _groups operator (described below) as well as "_value.complex-array" operator (see more below).

When grabbing a value (for the double metric, for a temporary variable (discussed further below) or for a value in a JSON structure), you can use these operators: 

  • _text - the equivalent of "innerText" in many programming languages. It takes the element we're currently in and extracts just the text from it (without the XML element). For example, the text of this:
    <name>indeni</name>
    Is simply "indeni".
     
  • _attribute - the _text operator doesn't support attributes (this is due to how the XML object model is represented in programming languages). So instead, use this:
    _attribute:
                _name: "name"
     Will extract "john" from <element name="john">blah blah<element>

  • _count - provide this operator with an xpath and it will return the number of elements it found matching the xpath.
  • _sum - similar to _count, except it treats each element as a number and sum's all the numbers (it ignores non-numeric element values).
  • _name - the name of the element. Useful for getting the name of the current element we're in by doing "_name: ."
  • _constant - refers to a string. The string to the right of this operator will be used as is, no xpath evaluation or calculation of any kind.

In order to iterate over multiple elements which match a certain xpath (for example, a list of users, or a list of disks), use the _groups section. For example:

_metrics:
    -
        _groups:
            ${root}/job:                    # This will iterate over each element called "job".
                _value.complex:
                    id:
                        _text: "id"
                    status:
                        _text: "status"
                    result:
                        _text: "result"
                    details:
                        _text: "details"
                _tags:
                    "im.name":
                        _constant: "jobs"
          _value: complex-array			   # Tells the parser to treat this as a JSON array - multiple objects, each has the id, status, result and details keys

Lastly, there's the _transform block. This allows you to translate the contents you've pulled from the XML to the format the metrics need. You use _awk to do this. For example:

Transform with double metric, no groups
_metrics:
    -
        _tags:
            "im.name":
                _constant: "uptime-seconds"
            "live-config":
               _constant: "true"
            "display-name":
                _constant: "Uptime"
            "im.dstype.displayType":
                _constant: "seconds"
        _temp:
            "uptime":
                _text: "/response/result/system/uptime"
        _transform:
            _value.double: | 
               {
                   # 230 days, 16:57:34
                split("${temp.uptime}", vals, " ")
                if (arraylen(vals) == 3  && vals[2] == "days,") {
                    # 230 days, 16:57:34
                    days = vals[1]
                    split(vals[3], timevals, ":")
                    hours = timevals[1]
                    minutes = timevals[2]
                    seconds = timevals[3]
                    uptime = (days * 3600 * 24) + (hours * 3600) + (minutes * 60) + seconds
                    print uptime
                }
               }
Transform with double metrics, with groups
_metrics:
    -
        _groups:
            ${root}/*:
                _temp:
                    status: 
                        _text: "status"
                _tags:
                    name:
                        _text: "name"
                    "im.name":
                        _constant: "ntp-server-state"
                    "live-config":
                       _constant: "true"
                    "display-name":
                        _constant: "NTP Servers - State"
                    "im.dstype.displayType":
                        _constant: "state"
                    "im.identity-tags":
                        _constant: "name"
        _transform:
            _value.double: |
                {
                    if ("${temp.status}" == "synched") {print "1.0"} else {print "0.0"}
                }

If you want to try and grab information from a parent element, use ancestor, like here:

_metrics:
    -
        _groups:
            ${root}/data-processors/*/second/cpu-load-average/entry:
                _value.double:
                    _text: value
                _temp:
                    coreid:
                        _text: "coreid"
                    dpid:
                        _name: ancestor::node()[3]
                _tags:
                    "im.name":
                        _constant: "cpu-usage"
                    "live-config":
                       _constant: "true"
                    "display-name":
                        _constant: "CPU Usage"
                    "im.dstype.displayType":
                        _constant: "percentage"
                    "im.identity-tags":
                        _constant: "cpu-id"
                    "cpu-is-avg":
                        _constant: "false"
        _transform:
            _tags:
                "cpu-id": |
                    {
                        print "${temp.dpid}: ${temp.coreid}"
                    }