otel transform processor

[howan36]

Component(s)

No response

Describe the issue you're reporting

my logic:

- context: log
            conditions:
              - resource.attributes["filter"] == "dcdn" 
            statements:
              - set(resource.attributes["log.type"],"dcdn-waf") where Substring(body,20,3) == "waf"
              - set(resource.attributes["service.name"],"dcdn-waf") where Substring(body,20,3) == "waf"

as my understanding, the conditions and the where clause are 'and' relationship, but some logs not meet Substring(body,20,3) == "waf", and only meet resource.attributes["filter"] == "dcdn" , the set was executed.

[atoulme]

You're going to need to provide a reproducible test case. But I'd wager something else is going on here. I think you have multiple logs under your resource. If at least one log matches, your resource is transformed, so even if logs don't match, their resource is still modified all the same.

[howan36]

{
"level": "debug",
"ts": "2024-12-31T06:02:46.319Z",
"caller": "[email protected]/parser.go:35",
"msg": "TransformContext after statement execution",
"kind": "processor",
"name": "transform",
"pipeline": "logs",
"statement": "set(resource.attributes["service.name"],"dcdn-waf") where Substring(body,20,3) == "waf"",
"condition matched": false,
"TransformContext": {
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.26.0",
"log.type": "dcdn",
"filter": "dcdn",
"service.name": "dcdn"
},
"dropped_attribute_count": 0
},
"scope": {
"attributes": {},
"dropped_attribute_count": 0,
"name": "opentelemetry.sdk._logs._internal",
"version": ""
},
"log_record": {
"attributes": {
"code.filepath": "/app/sls-sync/sync-data.py",
"code.function": "process",
"code.lineno": 114
},
"body": "{"topic": "dcdn-edge-rtlog", "source": "log_service", "body_bytes_sent": "2", "client_ip": "112.115.191.xx", "content_type": "application/json", "domain": "xxxx.xxx-system.cn", "hit_info": "-,WAF|-,DYNAMIC|CHARGE|NOTLAST", "http_range": "-", "method": "POST", "proxy_ip": "-", "refer_domain": "res.app.ikea.cn", "refer_param": "", "refer_protocol": "https", "refer_uri": "/", "remote_ip": "112.115.191.xx", "remote_port": "61939", "request_size": "244", "request_time": "64", "response_size": "291", "return_code": "200", "scheme": "https", "sent_http_content_range": "-", "server_addr": "182.243.xxx", "server_port": "443", "unixtime": "1735353420", "uri": "/api/83/envelope/", "uri_param": "sentry_key=163a9e2bc1554bxxxx&sentry_version=7&sentry_client=sentry.javascript.vue%2F7.119.2", "user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 18_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 IKEAiOS/4.12.0Version/16", "user_info": "-", "uuid": "b6f31ba717353534202042654e", "via_info": "cache7.l2et15-7[9,0], cache7.l2et15-7[9,0], ens-cache19.cn6846[61,0]", "xforwordfor": "112.115.191.205", "owner_id": "1672019695948159"}",
"dropped_attribute_count": 0,
"flags": 0,
"observed_time_unix_nano": 1735624965016763400,
"severity_number": 13,
"severity_text": "WARN",
"span_id": "0000000000000000",
"time_unix_nano": 1735624965016753200,
"trace_id": "00000000000000000000000000000000"
},
"cache": {}
}
}

[howan36]

this is my collector logs, there are only body field of log_record, no others, and it's string type

[howan36]

i want to use resource.attributes["filter"] as the first condition, when it meets, and is waf key words condition, then to set resource attributes

[github-actions]

Pinging code owners for processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself. For example, comment '/label priority:p2 -needs-triaged' to set the priority and remove the needs-triaged label.