Nullable types

[serganch]

Hey everyone!

I've been checking out Kompendium for a while now and have to say, it's pretty cool and seems to be the most advanced option for Ktor. I'm especially excited about the "complex scalars" feature.

But there's one thing that's been confusing me. It's how nullable fields get handled in the schema.

Let's say we have this class:

data class Data(val field: String?)

Kompendium turns it into this schema:

{
  "Data": {
    "type": "object",
    "properties": {
      "field": {
        "oneOf": [
          {},
          {
            "type": "string"
          }
        ]
      }
    },
    "required": []
  }
}

So, this basically says "field can be an empty object or a string." Plus, it's implied to be optional since it's not in the required array.

I was pretty surprised by this. If you compare it to what the Ktor plugin for IDEA generates:

    Data:
      type: "object"
      properties:
        field:
          type: "string"

This seems much more straightforward and makes sense to me.

I dug into Kompendium's source code and found out that this behavior with nullable types isn't a bug, but quite contrary it's a tested feature. According to the tests, a nullable type should be described as:

"oneOf": [
  {
    "type": "null"
  },
  {
    "type": "string"
  }
]

But in my case, it's:

"oneOf": [
  {},
  {
    "type": "string"
  }
]

The missing "type": "null" does seem to be a bug, causing all sorts of confusion for openapi visualizers like redoc and scalar, and for client code generators too.

However, even with the intended "type": "null" behavior, I don't get why this is necessary. Nullability already means the field is optional, which is indicated by it not being in required. And vice versa if it's not nullable, it's automatically in required. So why the need for oneOf? It makes the schema super complex and heavy. In my case, all fields in all data classes are nullable, and reading the documentation generated that way is a mind-blowing experience I must tell you.

I found a test in Kompendium's code called "Nullable fields do not lead to doom," which tests this scenario. Can someone explain the reasoning behind this decision and what kind of "doom" the authors wanted to avoid? And is there a way to make this behavior configurable?

Thanks a ton for any insights!

Sergey

[brizzbuzz]

thanks for the detailed post! TBH Kompendium has not been getting a lot of attention from me as I've been out of the Kotlin ecosystem for a while now :(

It certainly seems weird that one of the oneOf responses would just be {}... perhaps it has something to do with the settings in the kotlinx-serialzation configuration you're using? that is a pretty common gotcha in kompendium.

Nullability already means the field is optional, which is indicated by it not being in required

This is, to my understanding, not strictly true. I think certain languages (javascript?) make a distinction between a field being present in a dictionary and set to null vs. not being included at all in the payload. My understanding is that this is the semantic difference between a required nullable field vs a non-required field when it comes to openapi specs.

[brizzbuzz]

per the json-schema spec (kompendium uses OpenAPI 3.1, where schema objects are json schema compliant)

In JSON a property with value null is not equivalent to the property not being present. This fails because null is not of type "string", it's of type "null"

https://json-schema.org/understanding-json-schema/reference/object#required

[brizzbuzz]

https://json-schema.org/understanding-json-schema/reference/null

[serganch]

Hi Ryan!

Thank you for the quick reply! I understand your point (thank you for the spec!) and admit that you're formally correct.
However, in Kotlin like in many (if not all) other strictly-typed languages there is just no means to distinguish the absence of a field and a null value (those both effectively mean the same thing). I personally wouldn't mind oneOf provided the UI handled it sensibly.

However it does not. This is what I see in redoc:

Let's imagine you have dozens of fields like that.

Instead I want it to be like this but without the "required" mark

And although it might not be 100% accurate in terms of the json schema spec, it's concise, readable and what's most important still absolutely workable in terms of client-server interaction. I mean both sides will unambiguously handle any piece of data compliant with this schema unless server responds with explicit nulls (and the client is JS), which is hardly a viable case.

Unfortunately, Kompendium doesn't seem to provide a way of achieving this

[brizzbuzz]

I could see adding some way for users to mark that they want Kompendium to effectively treat all nullables as not required. Can you open an issue for that feature request? no idea when I'll get around to implementing it, but it seems reasonable to me as opt-in / non-default behavior.

[serganch]

It would be great, Ryan! Here it is #650
Best regards!

[brizzbuzz]

yea i was able to replicate this by using this serializer settings

  install(ContentNegotiation) {
    json(Json {
      serializersModule = KompendiumSerializersModule.module
      encodeDefaults = false
      explicitNulls = false
    })
  }

but if you flip encodeDefaults to true then it shows as expected

[serganch]

Thank you for pointing at encodeDefaults. It works!