[Breaking Change] Misleading error message: Runtime exception: ERROR: Schema violation: Data does not match any schemas from 'oneOf'

[konrad-jamrozik]

• [ServiceBus]: Api version 2023-01-01-preview to support data disaster recovery azure-rest-api-specs#25268
  • https://github.com/Azure/azure-rest-api-specs/pull/25268/checks?check_run_id=16913653149
  • https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3094298&view=logs&j=8f628bfe-7d8d-5bf3-e141-37106862c4f5&t=723dfc11-d28f-5d74-45f1-663cde6746b4
  • Reported by Shubham Chaudhari
  • ERROR: Schema violation: Data does not match any schemas from 'oneOf'

Related work:

• [Breaking Change] ERROR: Schema violation: Data does not match any schemas from 'oneOf' #6036

Investigation

Error message in the check run says:

"new":"https://github.com/Azure/azure-rest-api-specs/blob/0bac3d9045a1610fe2c5ede033df1df55b612525/specification/servicebus/resource-manager/Microsoft.ServiceBus/preview/2023-01-01-preview/namespace-preview.json",
"old":"https://github.com/Azure/azure-rest-api-specs/blob/main/specification/servicebus/resource-manager/Microsoft.ServiceBus/stable/2021-11-01/namespace-preview.json",
"details":"Command failed: node "/mnt/vss/_work/_tasks/AzureApiValidation_5654d05d-82c1-48da-ad8f-161b817f6d41/0.0.59/common/temp/node_modules/.pnpm/https://github.com/[email protected]/node_modules/autorest/dist/app.js" --v2 --input-file=specification/servicebus/resource-manager/Microsoft.ServiceBus/preview/2023-01-01-preview/namespace-preview.json --output-artifact=swagger-document.json --output-artifact=swagger-document.map --output-file=new --output-folder=/tmp\nERROR: Schema violation: Data does not match any schemas from 'oneOf'\n - file:///mnt/vss/_work/1/azure-rest-api-specs/specification/servicebus/resource-manager/Microsoft.ServiceBus/preview/2023-01-01-preview/namespace-preview.json:347:10 ($.paths["/subscriptions/subscriptionId/resourceGroups/resourceGroupName/providers/Microsoft.ServiceBus/namespaces/namespaceName/failover"].post["x-ms-examples"].NamespaceFailOver)\nFATAL: swagger-document/individual/schema-validator - FAILED\nFATAL: Error: [OperationAbortedException] Error occurred. Exiting.\nProcess() cancelled due to e"

Notably it says:

ERROR: Schema violation: Data does not match any schemas from 'oneOf'\n - file:///mnt/vss/_work/1/azure-rest-api-specs/specification/servicebus/resource-manager/Microsoft.ServiceBus/preview/2023-01-01-preview/namespace-preview.json:347:10 ($.paths["/subscriptions/subscriptionId/resourceGroups/resourceGroupName/providers/Microsoft.ServiceBus/namespaces/namespaceName/failover"].post["x-ms-examples"].NamespaceFailOver

In the new spec file we can see:

    "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ServiceBus/namespaces/{namespaceName}/failover": {
      "post": {
        "tags": [
          "Namespaces Geo DR Failover"
        ],
        "operationId": "Namespaces_Failover",
        "x-ms-examples": {
          "NamespaceFailOver": {
            "$ref": "./examples/NameSpaces/SBNamespaceFailover.json"
          }
        },

The 347:10 location given in error message points to "NamespaceFailOver": {.

In the old spec file we can see

    "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ServiceBus/namespaces/{namespaceName}/privateEndpointConnections": {
      "get": {
        "tags": [
          "Namespaces PrivateEndpointConnections"
        ],
        "operationId": "PrivateEndpointConnections_List",
        "x-ms-examples": {
          "NameSpaceCreate": {
            "$ref": "./examples/NameSpaces/PrivateEndPointConnectionList.json"
          }
        },

Looks like in the case of old spec the x-ms-examples had an entry for NameSpaceCreate and there was SBNameSpaceCreate.json example.

However, in the case of the new spec the x-ms-examples has an entry for NamespaceFailOver while there is SBNamespaceFailover.json example. The problem here appears to be the casing mismatch: NamespaceFailOver vs NamespaceFailover.

If this is not the root-cause, then maybe the contents of the SBNamespaceFailover.json does not adhere to the schema.

If none of the above are the root-cause, then an investigation by OpenAPI spec SME would help here.

Update 2023-09-20 11:59 PM PST

See this Teams discussion

Reference

• x-ms-examples in AutoRest docs
• x-ms-examples in specs repo docs
  • x-ms-examples schema
• Stack Overflow question about the error message

[konrad-jamrozik]

For analysis of runtime error buckets, see:

• [Breaking Change] Runtime exceptions aggregate #6169 (comment)

[konrad-jamrozik]

Addressed by:

• Pull Request 500754: Implement support for pretty-printing runtime errors originating from AutoRest invocation from within OAD (breaking change detector)

[konrad-jamrozik]

New Teams discussion here.

[konrad-jamrozik]

This is how one can read the AutoRest runtime messages now:

The above comes from:
https://github.com/Azure/azure-rest-api-specs/pull/28080/checks?check_run_id=22780562664