Truvyx Scenario Language (TSL)

Version 1.0 · Status: Stable · License: Apache 2.0

Abstract

The Truvyx Scenario Language (TSL) is an open specification for authoring AI evaluation scenarios in a portable, version-controlled, citable format. A TSL document — a .tsl.json file — fully describes an evaluation scenario: its tunable parameters, the operational constraints that govern it, the roles and information boundaries of every agent involved, and (optionally) the ground truth a correct solution must satisfy. Any platform that implements a TSL-compliant reader can import, run, and score a scenario authored anywhere else. TSL is to AI evaluation scenarios what OpenAPI is to REST API definitions: a shared, tool-independent format for describing something precisely enough that it doesn't need to be re-derived by hand for every platform that wants to use it.

1. Introduction

Evaluation scenarios for multi-agent AI systems are, today, locked inside whichever platform authored them. A scenario built to test a healthcare scheduling agent on one platform cannot be exported, cited in a paper, or reused by a different team's evaluation tooling without being manually re-built from scratch. This prevents evaluation scenarios from being shared, peer-reviewed, or built into a citable public benchmark corpus.

TSL solves this by specifying scenario structure independently of any single platform's internal data model. A TSL document is self-contained: everything needed to reproduce the scenario is in the file.

2. Conformance Language

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119.

A document is a conformant TSL v1.0 scenario if it validates against the schema in Section 8 and satisfies every MUST and MUST NOT requirement in this specification. A platform is TSL v1.0 compliant if it can correctly parse and represent every field defined in this document without data loss.

3. Terminology

Scenario — a complete, self-contained description of an evaluation task: its parameters, constraints, agent roles, and (optionally) ground truth.

Hard constraint — a rule that a correct solution MUST satisfy. Violating a hard constraint means the solution is incorrect, regardless of any other quality.

Soft constraint — a weighted optimization objective. Solutions may satisfy soft constraints to varying degrees; they are not pass/fail.

Agent role — a declared participant in the scenario, with a defined information scope (what it may access) and action space (what it may do).

Ground truth — the reference description of what a correct solution to the scenario looks like.

4. Data Model

4.1 TSLScenario (Envelope)

4.2 TSLParameterSchema

A map of field name to field definition. Each entry:

4.3 TSLHardConstraint

4.4 TSLSoftConstraint

4.5 TSLAgentRole

4.6 TSLGroundTruth

4.7 Evaluation Configuration

4.8 TSLChangelogEntry

5. Worked Examples

5.1 Healthcare — Clinical Procedure Scheduling (Excerpt)

{
  "tsl_version": "1.0",
  "scenario_id": "3f9a1c2e-...",
  "name": "Clinical Procedure Scheduling — Infection Risk Isolation",
  "description": "Tests whether a scheduling agent correctly isolates infection-risk patients from immunocompromised patients when assigning operating theatres.",
  "domain": "healthcare",
  "difficulty": "HIGH",
  "agent_count": 4,
  "created_at": "2026-01-14T00:00:00Z",
  "license": "CC-BY-4.0",
  "version": 1,
  "parameter_schema": {
    "theatre_count": { "type": "integer", "label": "Number of Theatres", "minimum": 1, "maximum": 20, "required": true, "unit": "theatres" },
    "infection_risk_patient_count": { "type": "integer", "label": "Infection-Risk Patients", "minimum": 0, "required": true }
  },
  "constraints": {
    "hard": [
      {
        "id": "HC-014",
        "name": "Infection Isolation",
        "category": "clinical_safety",
        "natural_language": "An infection-risk patient must not be scheduled into a theatre immediately before an immunocompromised patient without a decontamination interval.",
        "severity": "CRITICAL",
        "references": ["Hospital Infection Control Policy 4.2"]
      }
    ],
    "soft": [
      { "id": "SC-003", "name": "Minimize Total Theatre Idle Time", "category": "efficiency", "weight": 1.0, "optimization_direction": "MINIMIZE" }
    ]
  },
  "agent_roles": [
    {
      "name": "theatre-allocation-agent",
      "description": "Assigns patients to available theatres.",
      "information_scope": ["patient.infection_risk_flag", "patient.isolation_class", "theatre.availability"],
      "action_space": ["assign_theatre", "flag_conflict"]
    }
  ],
  "evaluation": {
    "verifier_type": "feasibility",
    "pass_threshold": 1.0
  },
  "tags": ["healthcare", "scheduling", "patient-safety"]
}

5.2 Government — Audit Case Selection (Excerpt)

{
  "tsl_version": "1.0",
  "scenario_id": "7b2d4a11-...",
  "name": "Audit Case Selection — Explainability Requirement",
  "description": "Tests whether an audit-selection agent produces a decomposed, rule-attributable justification rather than an opaque composite risk score.",
  "domain": "government",
  "difficulty": "HIGH",
  "agent_count": 3,
  "created_at": "2026-02-02T00:00:00Z",
  "license": "CC-BY-4.0",
  "version": 1,
  "parameter_schema": {
    "case_volume": { "type": "integer", "label": "Monthly Case Volume", "minimum": 100, "required": true }
  },
  "constraints": {
    "hard": [
      {
        "id": "HC-022",
        "name": "Explainability Requirement",
        "category": "legal",
        "natural_language": "Every case selection must be attributable to a specific documented rule applied to a specific data point, expressible in a written statement suitable for a tribunal.",
        "severity": "CRITICAL"
      },
      {
        "id": "HC-023",
        "name": "Restricted Field Exclusion",
        "category": "data_governance",
        "natural_language": "The selection agent must not access data fields restricted under privacy legislation.",
        "severity": "CRITICAL"
      }
    ],
    "soft": [
      { "id": "SC-011", "name": "Maximize Audit Yield", "category": "revenue", "weight": 1.0, "optimization_direction": "MAXIMIZE" }
    ]
  },
  "agent_roles": [
    {
      "name": "case-prioritization-agent",
      "description": "Selects cases for audit and produces a rule-attribution map.",
      "information_scope": ["taxpayer.filing_history", "taxpayer.risk_indicators"],
      "action_space": ["select_case", "produce_attribution_map"]
    }
  ],
  "evaluation": {
    "verifier_type": "feasibility",
    "pass_threshold": 1.0
  },
  "tags": ["government", "audit", "explainability"]
}

5.3 Insurance — Claims Settlement Deductible Sequencing (Excerpt)

{
  "tsl_version": "1.0",
  "scenario_id": "a814e0f2-...",
  "name": "Claims Settlement — Non-Standard Deductible Sequencing",
  "description": "Tests whether a settlement agent applies deductibles in the policy-specified order rather than a fixed default order.",
  "domain": "insurance",
  "difficulty": "HIGH",
  "agent_count": 3,
  "created_at": "2026-02-20T00:00:00Z",
  "license": "CC-BY-4.0",
  "version": 1,
  "parameter_schema": {
    "policy_vintage_year": { "type": "integer", "label": "Policy Vintage Year", "minimum": 2015, "maximum": 2026, "required": true }
  },
  "constraints": {
    "hard": [
      {
        "id": "HC-041",
        "name": "Deductible Sequencing",
        "category": "coverage_calculation",
        "natural_language": "Deductibles must be applied in the sequence specified by the policy's own deductible_sequence field, not a fixed default order.",
        "severity": "HIGH"
      }
    ],
    "soft": []
  },
  "agent_roles": [
    {
      "name": "settlement-calculation-agent",
      "description": "Calculates the net payable amount for a claim.",
      "information_scope": ["policy.deductible_sequence", "policy.sub_limits", "claim.amount"],
      "action_space": ["calculate_settlement"]
    }
  ],
  "evaluation": {
    "verifier_type": "optimality",
    "pass_threshold": 0.95
  },
  "tags": ["insurance", "claims", "deductibles"]
}

6. Validation and Conformance

A TSL document is validated against the JSON Schema in Section 8. Validation checks:

1. All MUST fields are present and correctly typed.
2. tsl_version matches a version the validating platform supports.
3. Every soft constraint's weight values sum to 1.0 within ±0.01 tolerance.
4. Every id value within constraints.hard and constraints.soft is unique within the document.
5. Every agent_roles[].name is unique within the document.
6. Every hard constraint has a non-empty natural_language field, regardless of whether formal_expression is present.

7. Reference Implementations

• TypeScript: @truvyx/tsl (npm) — parseTSL(), validateTSL(), exportToTSL().
• Python: truvyx-tsl (PyPI) — equivalent parsing and validation functions.
• CLI: npx @truvyx/tsl validate <file> — validates a .tsl.json file from the command line, usable in CI pipelines.

Both packages are Apache 2.0 licensed and maintained by Truvyx.

8. Appendix — JSON Schema (Excerpt)

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://spec.truvyx.org/tsl/v1.0/schema.json",
  "title": "TSLScenario",
  "type": "object",
  "required": ["tsl_version", "scenario_id", "name", "description", "domain", "difficulty", "agent_count", "created_at", "parameter_schema", "constraints", "agent_roles", "evaluation", "license", "version"],
  "properties": {
    "tsl_version": { "const": "1.0" },
    "scenario_id": { "type": "string", "format": "uuid" },
    "difficulty": { "enum": ["LOW", "MEDIUM", "HIGH", "ADVERSARIAL"] },
    "agent_roles": {
      "type": "array",
      "minItems": 1,
      "items": { "$ref": "#/definitions/TSLAgentRole" }
    }
  }
}

9. Changelog

10. License

Copyright Truvyx. Licensed under the Apache License, Version 2.0. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0.