Wrong 'Content-Length' in response?

[Dev-XYS]

I am not entirely sure this problem really exists, but I have found that the response for textDocument/completion seems to give a wrong Content-Length. However, using racket-langserver with VSCode is all right, but I'm not sure whether VSCode uses the Content-Length field or just reads a complete JSON object for parsing responses.

The way to reproduce the issue:

• Operating system: macOS 12.2.1
• Racket: v8.3 [cs]
• Put the following text into a file requests.txt (contains 4 requests: initialize, didOpen, didChange (inserts a character), completion (requests completion) ). Use CRLF.

Content-Length: 0

{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":7404,"clientInfo":{"name":"Visual Studio Code","version":"1.64.2"},"locale":"en-us","rootPath":"/","rootUri":"file:///","capabilities":{"workspace":{"applyEdit":true,"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"],"failureHandling":"textOnlyTransactional","normalizesLineEndings":true,"changeAnnotationSupport":{"groupsOnLabel":true}},"didChangeConfiguration":{"dynamicRegistration":true},"didChangeWatchedFiles":{"dynamicRegistration":true},"symbol":{"dynamicRegistration":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"tagSupport":{"valueSet":[1]}},"codeLens":{"refreshSupport":true},"executeCommand":{"dynamicRegistration":true},"configuration":true,"workspaceFolders":true,"semanticTokens":{"refreshSupport":true},"fileOperations":{"dynamicRegistration":true,"didCreate":true,"didRename":true,"didDelete":true,"willCreate":true,"willRename":true,"willDelete":true}},"textDocument":{"publishDiagnostics":{"relatedInformation":true,"versionSupport":false,"tagSupport":{"valueSet":[1,2]},"codeDescriptionSupport":true,"dataSupport":true},"synchronization":{"dynamicRegistration":true,"willSave":true,"willSaveWaitUntil":true,"didSave":true},"completion":{"dynamicRegistration":true,"contextSupport":true,"completionItem":{"snippetSupport":true,"commitCharactersSupport":true,"documentationFormat":["markdown","plaintext"],"deprecatedSupport":true,"preselectSupport":true,"tagSupport":{"valueSet":[1]},"insertReplaceSupport":true,"resolveSupport":{"properties":["documentation","detail","additionalTextEdits"]},"insertTextModeSupport":{"valueSet":[1,2]}},"completionItemKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]}},"hover":{"dynamicRegistration":true,"contentFormat":["markdown","plaintext"]},"signatureHelp":{"dynamicRegistration":true,"signatureInformation":{"documentationFormat":["markdown","plaintext"],"parameterInformation":{"labelOffsetSupport":true},"activeParameterSupport":true},"contextSupport":true},"definition":{"dynamicRegistration":true,"linkSupport":true},"references":{"dynamicRegistration":true},"documentHighlight":{"dynamicRegistration":true},"documentSymbol":{"dynamicRegistration":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSupport":true,"tagSupport":{"valueSet":[1]},"labelSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupport":true,"disabledSupport":true,"dataSupport":true,"resolveSupport":{"properties":["edit"]},"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}},"honorsChangeAnnotations":false},"codeLens":{"dynamicRegistration":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"onTypeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true,"prepareSupportDefaultBehavior":1,"honorsChangeAnnotations":true},"documentLink":{"dynamicRegistration":true,"tooltipSupport":true},"typeDefinition":{"dynamicRegistration":true,"linkSupport":true},"implementation":{"dynamicRegistration":true,"linkSupport":true},"colorProvider":{"dynamicRegistration":true},"foldingRange":{"dynamicRegistration":true,"rangeLimit":5000,"lineFoldingOnly":true},"declaration":{"dynamicRegistration":true,"linkSupport":true},"selectionRange":{"dynamicRegistration":true},"callHierarchy":{"dynamicRegistration":true},"semanticTokens":{"dynamicRegistration":true,"tokenTypes":["namespace","type","class","enum","interface","struct","typeParameter","parameter","variable","property","enumMember","event","function","method","macro","keyword","modifier","comment","string","number","regexp","operator"],"tokenModifiers":["declaration","definition","readonly","static","deprecated","abstract","async","modification","documentation","defaultLibrary"],"formats":["relative"],"requests":{"range":true,"full":{"delta":true}},"multilineTokenSupport":false,"overlappingTokenSupport":false},"linkedEditingRange":{"dynamicRegistration":true}},"window":{"showMessage":{"messageActionItem":{"additionalPropertiesSupport":true}},"showDocument":{"support":true},"workDoneProgress":true},"general":{"regularExpressions":{"engine":"ECMAScript","version":"ES2020"},"markdown":{"parser":"marked","version":"1.1.0"}}},"trace":"off","workspaceFolders":[{"uri":"file:///","name":"Demo"}]}
}Content-Length: 0

{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///a",
"text":"#lang racket\n\n(define i 0)\n(display )\n"}}
}Content-Length: 0

{"jsonrpc":"2.0","method":"textDocument/didChange","params":{
    "textDocument": {
        "uri": "file:///a"
    },
    "contentChanges": [
        {
            "range": {
                "start": {
                    "line": 3,
                    "character": 9
                },
                "end": {
                    "line": 3,
                    "character": 9
                }
            },
            "rangeLength": 0,
            "text": "i"
        }
    ]
}}Content-Length: 0

{"jsonrpc":"2.0","id":2,"method":"textDocument/completion","params":{"textDocument":{"uri":"file:///a"},
"position":{"line":3,"character":10},"context":{"triggerKind":1}}
}

• Run command (cat requests.txt && cat) | racket -l racket-langserver. Wait for enough time that all four requests are processed, then Ctrl-C to terminate.

It's the fourth request that gives the wrong Content-Length (72071), but the actual length of the JSON object is 72066. I've looked into msg-io.rkt, where Content-Length is computed, but I cannot figure out what's going wrong in the code.

[dannypsnl]

To save someone's time, this is in msg-io.rkt

(define (display-message msg [out (current-output-port)])
  (when (verbose-io?)
    (eprintf "\nresp = ~v\n" msg))
  (define null-port (open-output-nowhere))
  (write-json msg null-port)
  (define content-length (file-position null-port))
  (fprintf out "Content-Length: ~a\r\n\r\n" content-length)
  (write-json msg out))

Idea is writing message to nowhere port then rely on file-position as content-length.

The following simple test shows write a string works normally

(define out (open-output-nowhere))
(fprintf out "123")
(file-position out) ; 3

[Dev-XYS]

@dannypsnl Thank you for pointing it out! Just out of curiosity, were you able to replicate the issue?

[dannypsnl]

Yes, I have meet this problem before, and it became more frequently in 8.4

[Dev-XYS]

I have figured out the problem, and it's my fault. The Content-Length field should be the length of the body in bytes, instead of Unicode characters. I was counting the number of Unicode characters in the response.

[dannypsnl]

But it still shouldn't be 0?

[Dev-XYS]

Yes. But it's not 0. The 0s in my original post are in the input, not output from the language server. racket-langserver disregards the Content-Length of input and reads a complete JSON object instead, so using 0 is safe here.

[jryans]

@Dev-XYS Should this issue be closed? It sounds like you worked out what was happening…?

[Dev-XYS]

Yes. I will close it now.