Issue #8099: Add binary:join/2 to stdlib

[onno-vos-dev]

See linked issue for details.

[github-actions]

CT Test Results

    2 files     96 suites   1h 7m 38s ⏱️
2 163 tests 2 115 ✅ 48 💤 0 ❌
2 523 runs  2 473 ✅ 50 💤 0 ❌

Results for commit 7a3bd5c.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• Complete CT logs (Download Logs)
• HTML Documentation (Download HTML Docs)
• Windows Installer

// Erlang/OTP Github Action Bot

[onno-vos-dev]

Not sure if it'll make it into OTP 27 or not but I'm assuming that RC is still open 👍

[seriyps]

as an option

Suggested change

	join(T, Separator, H).
	lists:foldl(fun(Element, Acc) -> <<Acc/binary, Separator/binary, Element/binary>> end, H, T).

[mikpe]

Would that not lose some performance though? Performance being the main reason for this implementation.

[onno-vos-dev]

There's very little change performance wise in this suggestion nor do I see the big difference in reduction of complexity? Care to elaborate why you'd prefer this option? 😄

[seriyps]

Just a will to replace generic code patterns with standard library functions (to use high-order-functions instead of manual recursion).

[onno-vos-dev]

Squashed to get rid of some of the commit noise

[paulo-ferraz-oliveira]

Fwiw, there was an attempt at this a few years back, that ended up not making it due to a core team decision. It's possible, though, that stuff's changed since then.

[josevalim]

@bjorng @jhogberg @michalmuskala

I understand why starting with the empty string makes the compiler use the new "private append" optimisation. However I wonder if this could be generalized?

While H cannot be privately appended because it comes from an external function, the next iteration of join/3 can be privately appended, because no one uses the intermediate binary. Therefore, for those "closed loops", would it make sense to have a bit in the binary that tells when to private append or not? Generally speaking, everything after the first iteration would be privately appended. This would allow private append to happen in more situations, although I am not aware of the costs of reserving one extra bit for binaries.

PS: I may be completely off mark here.

[bjorng]

I understand why starting with the empty string makes the compiler use the new "private append" optimisation. However I wonder if this could be generalized?

While H cannot be privately appended because it comes from an external function, the next iteration of join/3 can be privately appended, because no one uses the intermediate binary.

The private append operation is faster because it does fewer tests and less working than general append operation. Therefore, the private append must only be used with a binary that has been specially prepared.

However, it should be possible for the compiler to generalize the optimization. If the call looks like:

join(T, Separator, H)

the compiler could rewrite it to:

    Acc = <<>>,
    join(T, Separator, <<Acc/binary, H/binary>>);

Not sure that this code pattern is common enough to make the optimization worthwhile to implement, though. It would not be needed if the clause is rewritten to:

join([_ | _]=List, Separator) when is_binary(Separator) ->
    join(List, Separator, <<>>);

[bjorng]

After the first release candidate, we generally focus on bug fixes and polishing of features already included or planned for the release. To ensure that Erlang/OTP 27 will be as good as it possibly can be, we need to minimize the time we spend on things not to be included in the release. Therefore, we will not review this pull request until after OTP 27 has been released. If we have not came back to it before September, feel free to remind us.

[bjorng]

Now we are back working on Erlang/OTP 28. OTP Technical Board will have to it approve this addition to the binary module. It is likely that it will be approved within a week or two.

Meanwhile, I've pushed a commit with suggested clean ups and proper error handling. Please review. If you approve, please rebase on the latest master and squash into one commit.

[onno-vos-dev]

Now we are back working on Erlang/OTP 28. OTP Technical Board will have to it approve this addition to the binary module. It is likely that it will be approved within a week or two.

Meanwhile, I've pushed a commit with suggested clean ups and proper error handling. Please review. If you approve, please rebase on the latest master and squash into one commit.

Thank you! Squashed and rebased now 👍

[bjorng]

Thanks! Added to our daily builds.

[bjorng]

Thanks for the pull request.