Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix attributes protobuf encoding #609

Closed

Conversation

albertored
Copy link
Contributor

Attributes with array values are wrongly encoded by exporter:

1> otel_otlp_common:to_attributes(#{
        bin_list => [<<"a">>, <<"b">>],
        int_list => [1, 2],
        float_list => [1.2, 3.4],
        bool_list => [true, false]
    }).

[#{key => <<"int_list">>,
   value => #{value => {string_value,<<1,2>>}}},
 #{key => <<"float_list">>,
   value =>
       #{value =>
             {array_value,#{values =>
                                [#{value => {double_value,1.2}},
                                 #{value => {double_value,3.4}}]}}}},
 #{key => <<"bool_list">>,
   value =>
       #{value =>
             {array_value,#{values =>
                                [#{value => {string_value,<<"true">>}},
                                 #{value => {string_value,<<"false">>}}]}}}},
 #{key => <<"bin_list">>,
   value => #{value => {string_value,<<"ab">>}}}]

See specification about attributes

At the moment I choose to emit an error for nested arrays and omit them from the export but not sure it is the best option

@albertored albertored requested a review from a team August 22, 2023 17:20
@codecov
Copy link

codecov bot commented Aug 22, 2023

Codecov Report

Patch coverage: 92.59% and project coverage change: +0.31% 🎉

Comparison is base (23c49ec) 72.26% compared to head (358bee6) 72.58%.
Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #609      +/-   ##
==========================================
+ Coverage   72.26%   72.58%   +0.31%     
==========================================
  Files          60       60              
  Lines        1904     1922      +18     
==========================================
+ Hits         1376     1395      +19     
+ Misses        528      527       -1     
Flag Coverage Δ
api 68.06% <ø> (ø)
elixir 18.31% <ø> (ø)
erlang 73.89% <92.59%> (+0.31%) ⬆️
exporter 69.49% <92.59%> (+2.69%) ⬆️
sdk 78.13% <ø> (ø)
zipkin 54.16% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...ps/opentelemetry_exporter/src/otel_otlp_common.erl 83.63% <92.59%> (+10.66%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@albertored albertored force-pushed the fix-attributes-encoding branch from d380416 to 1492640 Compare August 23, 2023 07:13
@albertored albertored mentioned this pull request Aug 23, 2023
@albertored albertored requested a review from tsloughter August 25, 2023 21:17
end
end;
to_any_value(Value) ->
#{value => {string_value, to_binary(io_lib:format("~p", [Value]))}}.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this used now instead of unicode:characters_to_binary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly, if using ~p I would imagine wanting ~tp as well at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a new addition but only old code that changed position. It is the catch-all clause for values that do not fall into the attributes allowed types.

Be aware that it's not calling list_to_binary but only a private to_binary function (already defined in the old code) that is exactly unicode:characters_to_binary

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see now. Got confused using the diff.

But the use of unicode:character_to_binary to tell if a list is a string is still removed, right? That is what I noticed and lead me to misread this code as new.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it has been removed, that part was the bug.

Attribute values are correctly typed to not be charlist because in that case they are totally undistinguishable from list of integers (that is an allowed attribute value).

So I removed the conversion of charlist to string_value

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One reason to construct an attributes record manually is to use in multiple signals, so then it'd somehow have to know if you aren't using any signal (traces, metrics, logs) to know if it should no-op.

Having the option to pass just a map/list instead of the record does at least give the user the ability to pass attributes only to a single signal in a call like with_span and have them ignored.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can accept both raw maps/lists or attributes record and document that if a record is used the validation and truncation overhead is executed also in a noop case. If a user want to be sure to not add overhead when not needed then he should use maps/lists.

But he main point is that I would like to only save attributes in the validated record form so that exporters downstream do not need to re-validate attributes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we have a path forward on this? hehe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, sorry, too many open branches and I forgot about this one, I'll take a look in the next few days

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Closing in favor of #633

@albertored albertored requested review from tsloughter and ferd August 26, 2023 09:21
@albertored albertored marked this pull request as draft August 28, 2023 16:13
@albertored albertored closed this Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants