Optimize accessing fields through memoization cache

UserCurt commented

2025-10-11 03:36:01 +00:00

(Migrated from codeberg.org)

An mpath expression or expression array can have repeated elements. The current implementation will try to access the field multiple times, which can be slow particularly with some custom class implementations. Implementation of memorization can improve this by reducing the number of times mpath needs to access fields. This is most appropriate in mpath.get, but it is also applicable in mpath.resolve as sometimes it needs to get values to resolve the mpath. Note that when get calls resolve, resolve should share the memoization object for further efficiency gains.

A few options for passing the cache between resolve and get:

Have a second output to mpath.resolve for the memoization cache
Have mpath.resolve return mpath.ConcretePointer which inherits mpath.Pointer with an additional property for the cache.

Within get, there may need to be a method that is able to identify common elements within sub-mpaths.

There may be some edge case considerations when it comes to chunked atomic index operations. Consider mpath.resolve(S, ["/a/b<1>/c/x"; "/a/b<1>/c/y"]). Here, /c/x is a single indexOp and so is /c/y.

An mpath expression or expression array can have repeated elements. The current implementation will try to access the field multiple times, which can be slow particularly with some custom class implementations. Implementation of memorization can improve this by reducing the number of times mpath needs to access fields. This is most appropriate in `mpath.get`, but it is also applicable in `mpath.resolve` as sometimes it needs to get values to resolve the mpath. Note that when `get` calls `resolve`, `resolve` should share the memoization object for further efficiency gains. A few options for passing the cache between `resolve` and `get`: - Have a second output to `mpath.resolve` for the memoization cache - Have `mpath.resolve` return `mpath.ConcretePointer` which inherits `mpath.Pointer` with an additional property for the cache. Within `get`, there may need to be a method that is able to identify common elements within sub-mpaths. There may be some edge case considerations when it comes to chunked atomic index operations. Consider `mpath.resolve(S, ["/a/b<1>/c/x"; "/a/b<1>/c/y"]`). Here, `/c/x` is a single indexOp and so is `/c/y`.

UserCurt commented

2025-10-11 11:15:38 +00:00

(Migrated from codeberg.org)

Having resolve return mpath.ConcretePointer rather than mpath.Pointer could help with mpath.exist (issue #8) not having to do extra work after running mpath.resolve as it could also store whether the path exists (in a specific struct). It may also be helpful for recursive dot operation (issue #17) in ways that are unclear at this time.

Having resolve return `mpath.ConcretePointer` rather than `mpath.Pointer` could help with `mpath.exist` (issue #8) not having to do extra work after running `mpath.resolve` as it could also store whether the path exists (in a specific struct). It may also be helpful for recursive dot operation (issue #17) in ways that are unclear at this time.

UserCurt commented

2026-07-31 22:57:02 +00:00

Owner

Commit 7ff390f integrated get with the shared traversal engine, removing the second data fetch which was main issue brought forward by this issue. So this source is considered obsolete.

Though a similar idea persists in issue #47 tracking related write-back for mutation operations.

Commit `7ff390f` integrated `get` with the shared traversal engine, removing the second data fetch which was main issue brought forward by this issue. So this source is considered obsolete. Though a similar idea persists in issue #47 tracking related write-back for mutation operations.

UserCurt closed this issue

2026-07-31 22:57:02 +00:00

Rows
Columns

Optimize accessing fields through memoization cache #6