Retain expired services? If yes, why and how?

[farshidtz]

In the current version, the expired services are removed from the catalog. Instead, it may be useful to retain them permanently.

Keeping expired services in the same catalog may add unnecessary overhead to filtering and increase the size of index query responses.

Some ideas:

1. Keep them in a separate catalog exposed at a different endpoint.
2. Keep them in a separate catalog and retrieve when queried specifically. E.g. ?expired=true shall respond with expired items only.
3. Add them to index only if explicitly asked. E.g. ?include_expired=true. This will scale poorly if it requires reading everything from DB and filtering them.
4. Store them separately and add them to the index if explicitly queried. However, it will not be possible to sort all items without reading all from the DB.

[farshidtz]

I prefer the second option. What do you think @rohitadeshmukh13 ?

[rohitadeshmukh13]

Totally agree with you about retaining the expired services instead of deleting them.
Regarding the implementation, off the top of my head, how about a mix-up of options 2 and 3 - 'Keep them in a separate catalog' and 'Add (append) them to index only if explicitly asked. E.g. ?include_expired=true'?

[farshidtz]

That's option 4 :)

[raphaelahrens]

Why would it be useful to keep the expired services?
I think this should first be discussed, before you can discuss how it should be implemented.
It could be useful to discover that a service is down.

If it is useful would you also add additional metadata, like if the service expired or if it was deleted?

[farshidtz]

Why would it be useful to keep the expired services?
I think this should first be discussed, before you can discuss how it should be implemented.
It could be useful to discover that a service is down.

If it is useful would you also add additional metadata, like if the service expired or if it was deleted?

The use-case is from a project, @rohitadeshmukh13 can explain better.

[rohitadeshmukh13]

Yes, this was being discussed in a project, but it's not a requirement yet.
As you rightly said, this could be useful to know what all services 'expired' and at what time.
If we all think this could be useful or would be needed at some point, we can discuss this further - regardless of whether this becomes a requirement in the project I'm working on or not.