[C#] Flight DoExchange server is incompatible with C++/PyArrow client

[adamreeve]

Describe the bug, including details regarding any error messages, version, and platform.

A .NET Flight Server with a DoExchange method is incompatible with a PyArrow Flight client. The C# implementation expects the first message received from the data stream to contain both the descriptor and the schema, but the client first sends the descriptor with an empty data_header, followed by another message with the schema.

This results in an error like this in the C# server:

fail: Grpc.AspNetCore.Server.ServerCallHandler[6]
      Error when executing service method 'DoExchange'.
      System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'length')
         at Google.FlatBuffers.ByteBuffer.GetUint(Int32 offset)
         at Google.FlatBuffers.ByteBuffer.GetInt(Int32 offset)
         at Apache.Arrow.Flatbuf.Message.GetRootAsMessage(ByteBuffer _bb, Message obj)
         at Apache.Arrow.Flatbuf.Message.GetRootAsMessage(ByteBuffer _bb)
         at Apache.Arrow.Flight.Internal.RecordBatchReaderImplementation.ReadSchemaAsync(CancellationToken cancellationToken)
         at Apache.Arrow.Flight.Internal.RecordBatchReaderImplementation.ReadFlightDescriptor()
         at ExchangeServer.Server.DoExchange(FlightServerRecordBatchStreamReader requestStream, FlightServerRecordBatchStreamWriter responseStream, ServerCallContext context) in /home/adam/dev/flight-exchange/DotnetExample/ExchangeServer/Server.cs:line 15
         at Grpc.Shared.Server.DuplexStreamingServerMethodInvoker`3.Invoke(HttpContext httpContext, ServerCallContext serverCallContext, IAsyncStreamReader`1 requestStream, IServerStreamWriter`1 responseStream)
         at Grpc.Shared.Server.DuplexStreamingServerMethodInvoker`3.Invoke(HttpContext httpContext, ServerCallContext serverCallContext, IAsyncStreamReader`1 requestStream, IServerStreamWriter`1 responseStream)
         at Grpc.AspNetCore.Server.Internal.CallHandlers.DuplexStreamingServerCallHandler`3.HandleCallAsyncCore(HttpContext httpContext, HttpContextServerCallContext serverCallContext)
         at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.<HandleCallAsync>g__AwaitHandleCall|8_0(HttpContextServerCallContext serverCallContext, Method`2 method, Task handleCall)

DoExchange with a .NET client and .NET server does work, and using a PyArrow server and .NET client also works.

Component(s)

C#

[adamreeve]

take

[adamreeve]

I have a fix for this (adamreeve@8910d69) but don't see a way to easily add a test. I think it's worth setting up C# Flight integration tests first (#44361).

[CurtHagenlocher]

Issue resolved by pull request 44424
#44424