Skip to content

Commit

Permalink
Default ensure_memory_alignment to false, mention in docs
Browse files Browse the repository at this point in the history
  • Loading branch information
EnricoMi committed Mar 7, 2025
1 parent ef2f9a9 commit 82fa7f2
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 4 deletions.
5 changes: 3 additions & 2 deletions cpp/src/arrow/ipc/options.h
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,9 @@ struct ARROW_EXPORT IpcReadOptions {

/// \brief Whether to align incoming data if mis-aligned
///
/// Received mis-aligned data is copied to aligned memory locations.
bool ensure_memory_alignment = true;
/// Received mis-aligned data is copied to aligned memory locations allocated via the
/// MemoryPool configured as \ref arrow::ipc::IpcReadOptions::memory_pool.
bool ensure_memory_alignment = false;

/// \brief Options to control caching behavior when pre-buffering is requested
///
Expand Down
13 changes: 11 additions & 2 deletions docs/source/cpp/flight.rst
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,16 @@ Memory management
-----------------

Flight tries to reuse allocations made by gRPC to avoid redundant
data copies. However, this means that those allocations may not
data copies. Experience shows that Arrow data in those allocations are not
found at aligned addresses. Some Flight client use cases may require `data alignment`_,
which can be enforced by setting :member:`arrow::ipc::IpcReadOptions::ensure_memory_alignment`
to ``true``. This uses the :member:`arrow::ipc::IpcReadOptions::memory_pool`
to a allocate memory with aligned addresses, but only for mis-alligned data.
However, this creates data copies of your data recieved via Flight.

.. note:: Ensuring memory alignment is not supported for non-CPU Flight data.

Unless gRPC data are copied as described above, allocations made by gRPC may not
be tracked by the Arrow memory pool, and that memory usage behavior,
such as whether free memory is returned to the system, is dependent
on the allocator that gRPC uses (usually the system allocator).
Expand Down Expand Up @@ -361,5 +370,5 @@ Closing unresponsive connections
.. _ARROW-15764: https://issues.apache.org/jira/browse/ARROW-15764
.. _ARROW-16697: https://issues.apache.org/jira/browse/ARROW-16697
.. _ARROW-6062: https://issues.apache.org/jira/browse/ARROW-6062

.. _data alignment: https://arrow.apache.org/docs/format/Columnar.html#buffer-alignment-and-padding
.. _gRPC: https://grpc.io/

0 comments on commit 82fa7f2

Please sign in to comment.