Skip to content

Conversation

@Aeonax
Copy link

@Aeonax Aeonax commented Mar 28, 2025

No description provided.

@Aeonax Aeonax force-pushed the invalidation-helpers branch from 9b24046 to 57fd5aa Compare March 28, 2025 12:03
(.build cache-builder (cache-loader cache-fn))
(.build cache-builder))))

(defn get-cache-keys
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache-keys maybe?

Copy link
Author

@Aeonax Aeonax Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache-keys still sounds a bit too general to me — feels like it’s missing the ‘action’ part a function name usually has.

But after chatting with ChatGPT, I can see that cache-keys actually fits more with the Clojure style — like meta, keys, and so on 🤔

[^Cache cache key]
(.invalidate cache key))

(defn invalidate-keys
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking if invalidate-all could be better name to mimic native call, but feels like all is a bit misleading because it's not "all" but listed ones 🤔

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that is why I named it this way - invalidate-all looks too bad for my liking😅🙈 Of course, we can have, for example, invalidate-all with 2 arities - cache + cache keys. Then it will be a bit more logical, but still...

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm now thinking maybe we should allow passing caching function as an argument 🤔

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean we're passing cache here, but what if we'll pass function, and this functions will extract cache using the implementation details

~@body)
~cache-options)))
(defn ~@fdefn (~fname ~@cache-key))
(alter-meta! (var ~name) merge (meta ~fname) ~(meta name)))))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting, does alter-meta! fit better than vary-meta here?

Copy link
Author

@Aeonax Aeonax Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mentioned in PM, looks like vary-meta returns a new object with altered meta - it doesn't mutate the initial def, so it did nothing. Meta passed to defcache persisted without vary-meta help.
Unless I missed some corner case, or something else... Because this angle of interaction with meta is a bit new to me.

Copy link
Owner

@fmnoise fmnoise Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, vary-meta returns new object, same as with-meta, while alter-meta! does meta mutation, but it works on var reference level rather than the object level, so I'm not sure it will work good with 2 different approaches - object meta association on caching function using with-meta and then on var reference using alter-meta!

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also I can't remember which kind of meta did we pass on that function and for the sake of what 😅
probably just normal function meta non-related to caching at all

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember whether I tested all possibilities, but as I remember, in this case, I could have used (alter-meta! (var ~name) merge (meta ~fname)) because even without name, meta persisted well.

(let [options (meta f)
condition-fn (-> f meta :when)
cache (build-cache options (when-not condition-fn f))]
^{::cache cache}
Copy link
Owner

@fmnoise fmnoise Apr 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like it's pretty much an implementation detail, so maybe we could add a helper for getting cache from the function using the meta? 🤔

Copy link
Author

@Aeonax Aeonax Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit tricky, but I did it=) Question is how to name it... too many caches in naming🤔 get-defcache-cache or defcache-cache or defcache-cache-object or ......

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, there is a cached function too, so I need to add a helper for both defcache and cached🤔

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now I feel like maybe storing cache in function meta/var meta is a bit messy way 🤔
maybe we could have caches registry in atom or volatile, which could allow us to have more tooling for monitoring caches size 🤔
and we could probably use defcached fully qualified function name for the cache key in the registry 🤔
but I'm not sure how should we put the cache into registry when using cached 🤔
maybe we should just add something like cache-id for manipulating the cache, and that would allow us to get rid of cached->cache or anything like that, allowing just to pass cache-id when creating cache and then using it with cache-keys and invalidate-keys 🤔

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I prefer meta because:

  • It exists on the same level of execution - just a part of the result
  • There's no extra complexity with cache IDs, storage layers, etc.
  • I was able to link it to the source caching object with minimal changes, and having access to that caching object is all I need right now.

Yes, integrating this meta into defcached and accessing it can be a bit tricky, but it works. And for the end user, it’s just a cached->cache or defcached->cached.


That said, if you have a clear idea for a different approach, I’d be really interested to see how it turns out =)

Copy link
Owner

@fmnoise fmnoise May 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meta is convenient, but it's way more complicated than it seems, as it could be attached to the object, but only of certain types, and passed along with it, and also to refs (vars, namespaces etc) which has completely different semantics and can contain objects of any type including nil 🤷🏽‍♂️
so, more straightforward way could be giving cache meaningful id, if you'll ever need to work with it and some clear way to access cache by id

@Aeonax Aeonax requested a review from fmnoise April 16, 2025 08:50
(cond-lookup cache key condition-fn f args)
(lookup cache key))))))

(defn cached-cache [cached-fn]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds like oily-oil 🙈
I'd try something like cached->cache or fn->cache

(defn ~@fdefn (~fname ~@cache-key))
(alter-meta! (var ~name) merge (meta ~fname) ~(meta name)))))

(defmacro defcached-cache [defcache-fn]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit weird name, and I don't fully understand what's the use-case
but the use-case if probably related to the way we store meta - on function object or var ref, which adds more confusion 😞

~@body)
~cache-options)))
(defn ~@fdefn (~fname ~@cache-key))
(alter-meta! (var ~name) merge (meta ~fname) ~(meta name)))))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also I can't remember which kind of meta did we pass on that function and for the sake of what 😅
probably just normal function meta non-related to caching at all

[^Cache cache key]
(.invalidate cache key))

(defn invalidate-keys
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm now thinking maybe we should allow passing caching function as an argument 🤔

(let [options (meta f)
condition-fn (-> f meta :when)
cache (build-cache options (when-not condition-fn f))]
^{::cache cache}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now I feel like maybe storing cache in function meta/var meta is a bit messy way 🤔
maybe we could have caches registry in atom or volatile, which could allow us to have more tooling for monitoring caches size 🤔
and we could probably use defcached fully qualified function name for the cache key in the registry 🤔
but I'm not sure how should we put the cache into registry when using cached 🤔
maybe we should just add something like cache-id for manipulating the cache, and that would allow us to get rid of cached->cache or anything like that, allowing just to pass cache-id when creating cache and then using it with cache-keys and invalidate-keys 🤔

(fn [~@cache-key]
~@body)
~cache-options)))
(defn ~@fdefn (~fname ~@cache-key))
Copy link
Owner

@fmnoise fmnoise May 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe should apply cache metadata taken from (meta ~fname) to this function object rather than applying it to var?

something like

 (def ~name
         (with-meta
           (fn ~@(when docstring [docstring])
             ~@(when prepost [prepost])
             [~@args]
             (~fname ~@cache-key))
           (meta ~fname)))))

my idea here is to always put cache meta on function object, so when we need to get cache, we can use only function instead of 2 approaches: function and var

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks interesting!🤔 I didn't want to fight with all those variables, but it doesn't look as bad as I imagined.

I'm busy with other stuff this week, but on the next week I should be able to return into this pr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants