[NEW] keyspace_expiration_misses stats

[proost]

The problem/use-case that the feature addresses

In large-scale caching systems, passive expiration of keys can cause unexpected cache misses, resulting in unnecessary performance degradation. The current keyspace_misses metric in Valkey tracks all key misses, but it does not distinguish between misses due to expired keys and misses due to non-existent keys.

This lack of distinction makes it difficult for users to identify whether their cache misses are primarily caused by TTL settings or application-level key access patterns. Without visibility into passive expiration misses, users cannot optimize their TTL values effectively or adjust eviction policies to reduce unnecessary misses.

The keyspace_expiration_misses metric will provide better observability by tracking how many cache misses are specifically due to passive expiration. This will help users:

• Identify whether TTL values are too short or too long.
• Understand how frequently keys expire before they are accessed again.
• Optimize TTL configurations to reduce cache misses and improve cache hit ratios.

Description of the feature

The feature introduces a new metric called keyspace_expiration_misses, which counts the number of key misses caused specifically by passive expiration.

The new metric will be tracked in the lookupKey() function in the Valkey codebase. When a key is found to be expired during a lookup, the keyspace_expiration_misses counter will be incremented. This will allow users to distinguish between expiration misses and other types of misses when inspecting Valkey's INFO stats output.

Example output from the INFO stats command after the feature is implemented:

# Stats
keyspace_hits: 1000000
keyspace_misses: 200000
keyspace_expiration_misses: 150000

Key changes in Valkey codebase:

• Modify lookupKey() to increment keyspace_expiration_misses.
• Update INFO stats output to include the new metric.

Additional information

Potential Next Steps:
Once keyspace_expiration_misses is implemented, a future enhancement could include tracking the distribution expiration gap (the time difference between now and the expiration time when a key is passively expired). This would provide deeper insights into whether keys are being accessed just after they expire or long after they are no longer relevant.