Cache expiration recommendations

[bogdanciuca]

It is easy to implement cache expiration on top of fastcache by caching values with marshaled deadlines and verifying deadlines after reading these values from the cache.

While I completely agree with this statement, this only handles "cache hit" expiration. How would you recommend implementing a scheduler, that cleans up the cache once in a while?

The main issue is that I don't see a way to "iterate over all cache keys", neither a way to "delete all by value" (e.g. by expiration) in order to implement this.

Currently, it seems that the only possible way to achieve this is by creating an additional data structure (e.g. thread-safe map[string]int with tokenKey -> expirationVal or reversed, map[int][]string with expirationKey -> []tokenVal) in order to have some sort of iteration over keys / expirations.

However, this adds extra complexity (has to be kept in sync, so basically holding a cache for another cache) and memory (duplicating the keys), especially in cases with large keys (I'm using ~1000 char JWTs as keys, no need for value in this scenario, just expiration, which is usually quite big, e.g up to 24 hours).

@valyala Looking forward to hear you take on this, I would really like to use fastcache in our products.

[valyala]

There is much simpler approach - to maintain two caches and swap them periodically while resetting the oldest one. This approach is used by VictoriaMetrics - see https://github.com/VictoriaMetrics/VictoriaMetrics/blob/7d73623c6989a27a297358453a963d75c4ceedc1/lib/workingsetcache/cache.go

[bogdanciuca]

@valyala Thanks for sharing!

However, looking at the code and comments it seems that the solution doesn't fit the described use case.

// The cache evicts inactive entries after the given expireDuration.
// Recently accessed entries survive expireDuration.

If my understanding is correct, this refers only to inactive entries. I don't realIy care about inactive/recently accessed, but rather need a strict expiration, which can be custom / set per entry and can vary from 1 minute to 24 hours. The scheduler should evict expired entries, even though they have been recently accessed.

[bogdanciuca]

I pretty much went for Ristretto, due to its broad feature set.

However, I think it would be nice to have a way to iterate over cached keys. This would enable various flows, but not sure how this fits on fastcache's roadmap. Would it make sense to open a Feature request?

Thanks!

[valyala]

I think it would be nice to have a way to iterate over cached keys. This would enable various flows, but not sure how this fits on fastcache's roadmap. Would it make sense to open a Feature request?

@bogdanciuca , there is a pull request for this - see #3

[bogdanciuca]

I see it has not been merged because of performance issues and you previously recommended forking. Is there no other solution to have this functionality in fastcache?

[valyala]

There were no clear needs for visiting all the cache entries since the #3 has been proposed, so it is unlikely it will be merged soon.

Could you share practical use cases for this solution? Probably there are better solutions for some of them.

[bogdanciuca]

I'll revisit our use cases and share some more details.

[bogdanciuca]

@valyala So I ended up dropping ristretto due to slow sets and switched to bigcache (need for TTL, even though it's not customisable per entry, still a good tradeoff).

So now I have a Cache interface, implemented by 2 cache types, one based on fastcache and the other based on bigcache. I'm using fastcache for persistent data types and bigcache for data types that require a TTL (which varies per data type, e.g. 1min, 5min, 24h).

So I guess my point here is that I could've used just fastcache, if there would've been a TTL feature, which is quite a must for caches. I also see more recent interest on this, see #44. Any suggestions?