-
Notifications
You must be signed in to change notification settings - Fork 4
Added invalidation helpers + ::cache meta to defcached and cached
#10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
9b24046 to
57fd5aa
Compare
src/fmnoise/coldbrew.clj
Outdated
| (.build cache-builder (cache-loader cache-fn)) | ||
| (.build cache-builder)))) | ||
|
|
||
| (defn get-cache-keys |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cache-keys maybe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cache-keys still sounds a bit too general to me — feels like it’s missing the ‘action’ part a function name usually has.
But after chatting with ChatGPT, I can see that cache-keys actually fits more with the Clojure style — like meta, keys, and so on 🤔
| [^Cache cache key] | ||
| (.invalidate cache key)) | ||
|
|
||
| (defn invalidate-keys |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking if invalidate-all could be better name to mimic native call, but feels like all is a bit misleading because it's not "all" but listed ones 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that is why I named it this way - invalidate-all looks too bad for my liking😅🙈 Of course, we can have, for example, invalidate-all with 2 arities - cache + cache keys. Then it will be a bit more logical, but still...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm now thinking maybe we should allow passing caching function as an argument 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean we're passing cache here, but what if we'll pass function, and this functions will extract cache using the implementation details
| ~@body) | ||
| ~cache-options))) | ||
| (defn ~@fdefn (~fname ~@cache-key)) | ||
| (alter-meta! (var ~name) merge (meta ~fname) ~(meta name))))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
interesting, does alter-meta! fit better than vary-meta here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I mentioned in PM, looks like vary-meta returns a new object with altered meta - it doesn't mutate the initial def, so it did nothing. Meta passed to defcache persisted without vary-meta help.
Unless I missed some corner case, or something else... Because this angle of interaction with meta is a bit new to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, vary-meta returns new object, same as with-meta, while alter-meta! does meta mutation, but it works on var reference level rather than the object level, so I'm not sure it will work good with 2 different approaches - object meta association on caching function using with-meta and then on var reference using alter-meta!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also I can't remember which kind of meta did we pass on that function and for the sake of what 😅
probably just normal function meta non-related to caching at all
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't remember whether I tested all possibilities, but as I remember, in this case, I could have used (alter-meta! (var ~name) merge (meta ~fname)) because even without name, meta persisted well.
| (let [options (meta f) | ||
| condition-fn (-> f meta :when) | ||
| cache (build-cache options (when-not condition-fn f))] | ||
| ^{::cache cache} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like it's pretty much an implementation detail, so maybe we could add a helper for getting cache from the function using the meta? 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a bit tricky, but I did it=) Question is how to name it... too many caches in naming🤔 get-defcache-cache or defcache-cache or defcache-cache-object or ......
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, there is a cached function too, so I need to add a helper for both defcache and cached🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
now I feel like maybe storing cache in function meta/var meta is a bit messy way 🤔
maybe we could have caches registry in atom or volatile, which could allow us to have more tooling for monitoring caches size 🤔
and we could probably use defcached fully qualified function name for the cache key in the registry 🤔
but I'm not sure how should we put the cache into registry when using cached 🤔
maybe we should just add something like cache-id for manipulating the cache, and that would allow us to get rid of cached->cache or anything like that, allowing just to pass cache-id when creating cache and then using it with cache-keys and invalidate-keys 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Personally, I prefer meta because:
- It exists on the same level of execution - just a part of the result
- There's no extra complexity with cache IDs, storage layers, etc.
- I was able to link it to the source caching object with minimal changes, and having access to that caching object is all I need right now.
Yes, integrating this meta into defcached and accessing it can be a bit tricky, but it works. And for the end user, it’s just a cached->cache or defcached->cached.
That said, if you have a clear idea for a different approach, I’d be really interested to see how it turns out =)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
meta is convenient, but it's way more complicated than it seems, as it could be attached to the object, but only of certain types, and passed along with it, and also to refs (vars, namespaces etc) which has completely different semantics and can contain objects of any type including nil 🤷🏽♂️
so, more straightforward way could be giving cache meaningful id, if you'll ever need to work with it and some clear way to access cache by id
| (cond-lookup cache key condition-fn f args) | ||
| (lookup cache key)))))) | ||
|
|
||
| (defn cached-cache [cached-fn] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds like oily-oil 🙈
I'd try something like cached->cache or fn->cache
| (defn ~@fdefn (~fname ~@cache-key)) | ||
| (alter-meta! (var ~name) merge (meta ~fname) ~(meta name))))) | ||
|
|
||
| (defmacro defcached-cache [defcache-fn] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a bit weird name, and I don't fully understand what's the use-case
but the use-case if probably related to the way we store meta - on function object or var ref, which adds more confusion 😞
| ~@body) | ||
| ~cache-options))) | ||
| (defn ~@fdefn (~fname ~@cache-key)) | ||
| (alter-meta! (var ~name) merge (meta ~fname) ~(meta name))))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also I can't remember which kind of meta did we pass on that function and for the sake of what 😅
probably just normal function meta non-related to caching at all
| [^Cache cache key] | ||
| (.invalidate cache key)) | ||
|
|
||
| (defn invalidate-keys |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm now thinking maybe we should allow passing caching function as an argument 🤔
| (let [options (meta f) | ||
| condition-fn (-> f meta :when) | ||
| cache (build-cache options (when-not condition-fn f))] | ||
| ^{::cache cache} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
now I feel like maybe storing cache in function meta/var meta is a bit messy way 🤔
maybe we could have caches registry in atom or volatile, which could allow us to have more tooling for monitoring caches size 🤔
and we could probably use defcached fully qualified function name for the cache key in the registry 🤔
but I'm not sure how should we put the cache into registry when using cached 🤔
maybe we should just add something like cache-id for manipulating the cache, and that would allow us to get rid of cached->cache or anything like that, allowing just to pass cache-id when creating cache and then using it with cache-keys and invalidate-keys 🤔
| (fn [~@cache-key] | ||
| ~@body) | ||
| ~cache-options))) | ||
| (defn ~@fdefn (~fname ~@cache-key)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe should apply cache metadata taken from (meta ~fname) to this function object rather than applying it to var?
something like
(def ~name
(with-meta
(fn ~@(when docstring [docstring])
~@(when prepost [prepost])
[~@args]
(~fname ~@cache-key))
(meta ~fname)))))my idea here is to always put cache meta on function object, so when we need to get cache, we can use only function instead of 2 approaches: function and var
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks interesting!🤔 I didn't want to fight with all those variables, but it doesn't look as bad as I imagined.
I'm busy with other stuff this week, but on the next week I should be able to return into this pr.
No description provided.