The Count metric.

The Count metric is a simple metric that just counts the number of tokens in the target text.

Configuration

Configuration for the parameters of the Count metric:

title: "CountConfig"
type: "object"
$defs:
  TokenizerSpec:
    type: "object"
    properties:
      name:
        type: "string"
        description: "Name of the tokenizer to be used."
      config:
        type: "object"
        description: >
          "Any additional configuration that tokenizer needs. "
          "This will be in the form specified by the JSONSchema for that tokenizer."
    required:
      - "name"
      - "config"
properties:
  tokenizer:
    $ref: "#/$defs/TokenizerSpec"

Data

Accepted data format of the Count metric:

title: "CountData"
type: "object"
properties:
  target:
    type: "string"
    description: "Input text to evaluate."
required:
  - "target"

Results

Format of the results of the Count metric:

title: "CountResult"
type: "object"
$defs:
  OverallResult:
    type: "object"
    properties:
      value:
        type: "number"
        description: "Average number of tokens per target text."
    required:
      - "value"
  ExampleResult:
    type: "object"
    properties:
      value:
        type: "number"
        description: "Number of tokens in a single target text."
    required:
      - "value"
properties:
  overall:
    $ref: "#/$defs/OverallResult"
  examples:
    type: "array"
    items:
      $ref: "#/$defs/ExampleResult"
required:
  - "overall"
  - "examples"