I have a standard scenario where you have multiple parallel requests trying to access the same key in Redis based cache.
When this key is expired the requesting process notifies some external worker that it needs to be recomputed (the worker might possibly be on another server). The worker recomputes it and updates the cache.
When the cache is hot, everything is fine because I can keep serving the stale data from the cache until the new value is recomputed.
The problem is when the cache is cold and there is no data in Redis to serve yet. The requesting process needs to wait until the value is generated by the external worker. I can’t use a cache warm-up in this case because only the keys that are requested should be cached because of limited cache size.
So the question is how can I make PHP requests to wait until the computed value is available in Redis? Or what would be the common solution in this case?
The possible solutions I have already though about:
The Redis blpop command probably would not work because the value being recomputed is not in a list and feels a bit like a workaround.
Maybe it is possible to implement some kind of file based lock? However web app and worker are on separate servers and NFS does not support file locks for example.
The only possible working solution I could think of is to have an infinite while loop that pings Redis every X miliseconds with some Y max wait time. However, is this really a good and practical solution? Because I am not a fan of having infinite loops in supposedly short-living web requests. Besides, hundreds of requests would potentially be running infinite loops and waiting while the value is being recomputed.