feat: client telemetry

[port8080]

No description provided.

[port8080]

I hacked the endpoint to prod, and saw the attached json show up under ds-hub-telemetry. NB that I will need to apply https://github.com/huggingface-internal/infra/pull/1631 to see the records as parsed timestamps and counts instead of strings
logs on the client

{"timestamp":"2024-12-13T00:53:37.183852Z","level":"INFO","fields":{"action":"clean","repo_id":"Some(\"675b6a8fa4c359344add6b23\")","file_name":"None","file_size_count":11489,"new_bytes_count":11489,"start_ts":"2024-12-13T00:53:36.793427+00:00","end_processing_ts":"2024-12-13T00:53:37.183780+00:00"},"filename":"/Users/ajitb/hf/xet-core/data/src/clean.rs","line_number":227}

and logs sent to telemetry endpoint at https://huggingface.co/api/telemetry/xet/cli attached below

record1.json

[Hugoch]

@port8080 After some discussions with @ylow @XciD @julien-c , we would prefer to be extra careful when implementing client-side telemetry. We advocate for keeping server-side metrics. I understand that client-side may be useful for local cache hits assessment, if you really want to keep it let's make it opt-in for testing people but not the default 😄

[port8080]

Moving fwd with this after #142 and a discussion between @rajatarya and @XciD

[ylow]

Will this implode if we update the JWT contents?

[port8080]

@ylow JWT parsing failure will cause repo_id to be None but the code should not explode.
I agree that copying pub struct TokenClaim violates DRY and is clunky, but do not see a way around it for now.

[port8080]

I ran it e2e with the hash based approach and it works.

• note 67859b5f82c688377fe022d0 is my repo "port8080/test-xet-sagemaker"
• command run was HF_HUB_ENABLE_TELEMETRY=1 python ~/Desktop/test-poc.py
• kibbana query is from ds-hub-telemetry | where url == "/api/telemetry/xet/cli" with the output being
  record.json

[jgodlew]

Code looks good to me!