Strimzi
Error when accessing Topic using principal.builder.class #6460
-
|
I am using my own user principle class using principal.builder.class (logic as discussed on other thread is to CN=[email protected],O= Test Bank ,DC=test,DC=com use only [email protected] The kafka cluster is up and running and looking at error logic of custom principal is working: 2022-03-03 08:43:25,857 INFO Principal = User: uklonvd907100.uk.db.com is Denied Operation = Describe from host = 127.0.0.6 on resource = Topic:LITERAL:kafka-cert-topic for request = Metadata with resourceRefCount = 1 (kafka.authorizer.logger) [data-plane-kafka-request-handler-3] Topic yaml: apiVersion: kafka.strimzi.io/v1beta2 user Yaml: apiVersion: kafka.strimzi.io/v1beta1 what is going wrong any idea? |
Beta Was this translation helpful? Give feedback.
Replies: 10 comments 21 replies
-
|
Sorry for formatting of yaml. no clue why its all crumble up when i submit. let me see if I can upload the file directly |
Beta Was this translation helpful? Give feedback.
-
|
TBH, without the formatting it is hard to say. It is hard to read the YAMLs, but also no way to say if the YAML is correct or not. But you are using TLS client authentication. So the User Operator will expect the usernames to be prefixed with |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the formatting earlier. Trying to upload from my personal laptop, As suggested I removed the authentication section from the KafkaUser resource: my manifest look like but still getting same error: 2022-03-03 08:43:25,857 INFO Principal = User: uklonvd907100.uk.db.com is Denied Operation = Describe from host = 127.0.0.6 on resource = Topic:LITERAL:kafka-cert-topic for request = Metadata with resourceRefCount = 1 (kafka.authorizer.logger) [data-plane-kafka-request-handler-3] |
Beta Was this translation helpful? Give feedback.
-
|
TBH I never use the - resource:
type: topic
name: my-topic
patternType: literal
operation: All
host: "*"gives you the rights for |
Beta Was this translation helpful? Give feedback.
-
|
Hi @scholzj sorry should have pasted the correct log on my first post. I am indeed trying to access my-topic Apologies. Now I have updated the principal builder to give the user the right name with the CN= prefix ... the yaml used are as attached but still seeing same error: 2022-03-03 13:48:50,633 INFO Principal = User:CN=uklonvd907100.uk.db.com is Denied Operation = Describe from host = 127.0.0.6 on resource = Topic:LITERAL:my-topic for request = Metadata with resourceRefCount = 1 (kafka.authorizer.logger) [data-plane-kafka-request-handler-7] |
Beta Was this translation helpful? Give feedback.
-
|
Ok. I guess the next step would be to use the Kafka APIs and find out how the ACLs have been setup in Kafka itself. |
Beta Was this translation helpful? Give feedback.
-
|
sorry, not done this in past, any help or any documentation as what need to be done ? ACL is one bit which is holding us from going Live, so any help would be much appreciated |
Beta Was this translation helpful? Give feedback.
-
|
Not really. The Kafka documentation would be the source for this. There is a Javadoc for the Admin API which can do it. There is also the |
Beta Was this translation helpful? Give feedback.
-
|
@scholzj when I use below configs for broker , type: custom for authentication ... I am assuming this is how the setup need to be while we set custom class, will the KafkaBrokerConfigurationBuilder.java be using the below code? Just looking I may need to set few listener config to make the broker authentication to work? |
Beta Was this translation helpful? Give feedback.
-
|
I don't think you need it -> there is no check for the principal builder class and the authentication type. Plus you said that the principal builder class was working. |
Beta Was this translation helpful? Give feedback.
-
|
@scholzj Let me clear the air - As I am not able to authorize I am trying various options : So far I have tried two config
Error: |
Beta Was this translation helpful? Give feedback.
-
|
Did you checked the ACLs as I suggested? |
Beta Was this translation helpful? Give feedback.
-
|
Sorry No. but will try that now. I wasn't sure if the config I was using was right. as you pointed out principal builder class was working for me so I will set config back to how it was and check the ACLS. authorization: listeners: name: plain |
Beta Was this translation helpful? Give feedback.
-
|
@scholzj I am trying to run the ACL as you suggested since I dont have full access, I am trying to run kubectl exec as below kubectl exec -it my-cluster-kafka-2 -- /opt/kafka/bin/kafka-acls.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list --topic my-topic here my-cluster-kafka-bootstrap:9092 is running a Plain listener. Not sure what I am missing on the command? is there any alternative way to run commands on a pod?. Getting below error: Error while executing ACL command: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeAcls |
Beta Was this translation helpful? Give feedback.
-
|
Ideally you should never run these things from the Kafka pod itself but always from some other pod (started specifically for running the utility). I do not really know the tool -> but I guess you will need to do some setting up to get it working. You still need to authenticate and authorize for this. So to debug the ACLs issue, you probably need to setup your user as a super user so that it can actually list the ACLs in the first place. |
Beta Was this translation helpful? Give feedback.
-
|
@scholzj thanks for the reply. I will try both ways. your response above - "POD started specifically for running the utility" - we start the container with base image of kafka you meant isnt? |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, something like this: |
Beta Was this translation helpful? Give feedback.
-
|
Just noticed - this error on LOG of kafka. does this mean anything more :-) I have not created user - User:my-cluster-kafka 2022-03-08T11:03:32.757351651Z2022-03-08 11:03:32,757 INFO Principal = User:my-cluster-kafka is Denied Operation = ClusterAction from host = 240.0.26.97 on resource = Cluster:LITERAL:kafka-cluster for request = UpdateMetadata with resourceRefCount = 1 (kafka.authorizer.logger) [control-plane-kafka-request-handler-0] |
Beta Was this translation helpful? Give feedback.
-
Well, that suggests that your Principal Builder class does not work properly. This is one of the users used internally by Strimzi -> these need to have the original user names. Otherwise the cluster will not sync and the operators will not work. |
Beta Was this translation helpful? Give feedback.
-
|
The original username follows the original Principal Builder rules. |
Beta Was this translation helpful? Give feedback.
-
|
(i.e. in this case I guess |
Beta Was this translation helpful? Give feedback.
-
|
Strange - I updated the custom class to return CN=my-cluster-kafka , still same error. 2022-03-08T11:56:33.439839177Z2022-03-08 11:56:33,439 INFO Principal = User:CN=my-cluster-kafka is Denied Operation = ClusterAction from host = 240.0.26.2 on resource = Cluster:LITERAL:kafka-cluster for request = UpdateMetadata with resourceRefCount = 1 (kafka.authorizer.logger) [control-plane-kafka-request-handler-0] Let me check the DefaultKafkaPrincipalBuilder which I think is used if I am not overriding the class with custom one. think I am missing something obvious in my class and strimzi users are not set correctly. One Q - when you call DefaultKafkaPrincipalBuilder (assuming this is class you using in strimzi) do you pass value for either of this KerberosShortNamer , SslPrincipalMapper ?? .. |
Beta Was this translation helpful? Give feedback.
-
|
I guess my guess was wrong then ... checking the code, it has the organization as well -> |
Beta Was this translation helpful? Give feedback.
-
|
thank you that explains. I exactly know what to do now. If it fails, will trouble you again (alert in advance) |
Beta Was this translation helpful? Give feedback.
-
|
@scholzj CN=my-cluster-kafka,O=io.strimzi is this principal instance of X500Principal? do you know at all? |
Beta Was this translation helpful? Give feedback.
-
|
No idea. |
Beta Was this translation helpful? Give feedback.
-
|
I thought creation of user with , and = are not allowed? as regex error comes when we try @scholzj .. just curious |
Beta Was this translation helpful? Give feedback.
-
You mean when you create a KafkaUser? That is true -> but these are internal users, they do not have a matching KafkaUser resource. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for all the support @scholzj Made it to work. if any one is interested : code which worked for ssl make sure this is added to so that strimzi internal acl and functions are not affected. if (!(principal instanceof X500Principal) || principal == KafkaPrincipal.ANONYMOUS) { |
Beta Was this translation helpful? Give feedback.
-
|
Great, glad it works. |
Beta Was this translation helpful? Give feedback.
Thanks for all the support @scholzj Made it to work.
if any one is interested : code which worked for ssl make sure this is added to so that strimzi internal acl and functions are not affected.
if (!(principal instanceof X500Principal) || principal == KafkaPrincipal.ANONYMOUS) {
return new KafkaPrincipal(KafkaPrincipal.USER_TYPE, principal.getName());
}