Add filtering to dot operations with field matching #33

New issue

Open

opened 2026-07-02 08:46:27 +00:00 by UserCurt · 0 comments

UserCurt commented

2026-07-02 08:46:27 +00:00

(Migrated from codeberg.org)

Matching children can be too permissive in some circumstances as operations like DotGlob or later-defined DotRegex (#4) can only select by name. This issue proposes adding filtering syntax to select a subset of matched names provided a condition. This would give more control to the user in selecting fields based on more than just the field name itself.

The notation is not yet settled, but they would likely come directly after the field-matching operation, like in /*[FILTER]. Some inspiration can be taken from syntax of other similar tools like XPath [____], JSONPath [?____], JMESPath [? ____ ].

Ideas for conditions

The available conditions is not yet clear, but it can include things such as:

/*[type:struct], /*[type:table], /*[type:object], /*[type:CLASSNAME]: Select fields with values of a given type
/*[type:container], /*[type:noncontainer]: Select fields whether or not they are container types
/*[has:/X], /*[has:PATH]: Selects fields that have a child or given descendant chain. There may be some nuance where only some branches of the descendant chain are valid.
/*[has:>]/X/Y/Z: Selects fields that have the given descendant chain
/*[value > 0], /*[(value<-3) || (value>3)], /*[value(1) < value(2)]: Select fields that satisfy some expression
/*[has:/X && value(/X) > 0]: Some combined conditions may require short-circuiting to avoid errors
/*[size(value, 2) == 3], /*[numel(value) > 2], /*[~isscalar(value)]: The expression could include function calls

The examples above only uses value for the currently selected child, but there may be benefits to being able to select parents, ascendants, or descendants. For example:

/users/*[value(/age)>=18]/name would select users' names of those at least 18 years old.
/{Susan,Jane}/Children/*[value(/Height)>=value(/../../Height)] would select children of Susan and Jane who are taller than their mother

Open questions

What a full suite of features could entail, even if not initially supported
The exact syntax used
Whether . should be required or implicit
Whether filtering should be supported with DotExact, for example /X[/Z > 0]/Y
How it interacts with nested matching dot operations, for example [has:/*/X]
How this interacts with non-traversable branch handling in #32
Bundle boundary may be forced directly after some filters because MPath may need the value to evaluate the condition, but this may not always be true

Example

One use case is to select valid children only.

data = struct(...
  A = struct(X = 1), ...
  B = 2, ...
  C = struct(X = 3)
);
mpath.get(data, "/*/X")
mpath.get(data, "/*[has:/X]/X")

Without filtering, * matches A, B, and C, then /X is attempted on each branch. The /B/X branch is non-traversable because B is not a container. The filter [has:/X] would exclude B from the list of fields because it does not satisfy the condition of having a child X.

This would make it so users would not need to rely on MissingPolicy to filter in-post. See #32 regarding traversing paths through non-containers.

Matching children can be too permissive in some circumstances as operations like `DotGlob` or later-defined `DotRegex` (#4) can only select by name. This issue proposes adding filtering syntax to select a subset of matched names provided a condition. This would give more control to the user in selecting fields based on more than just the field name itself. The notation is not yet settled, but they would likely come directly after the field-matching operation, like in `/*[FILTER]`. Some inspiration can be taken from syntax of other similar tools like XPath `[____]`, JSONPath `[?____]`, JMESPath `[? ____ ]`. ### Ideas for conditions The available conditions is not yet clear, but it can include things such as: - `/*[type:struct]`, `/*[type:table]`, `/*[type:object]`, `/*[type:CLASSNAME]`: Select fields with values of a given type - `/*[type:container]`, `/*[type:noncontainer]`: Select fields whether or not they are container types - `/*[has:/X]`, `/*[has:PATH]`: Selects fields that have a child or given descendant chain. There may be some nuance where only some branches of the descendant chain are valid. - `/*[has:>]/X/Y/Z`: Selects fields that have the given descendant chain - `/*[value > 0]`, `/*[(value<-3) || (value>3)]`, `/*[value(1) < value(2)]`: Select fields that satisfy some expression - `/*[has:/X && value(/X) > 0]`: Some combined conditions may require short-circuiting to avoid errors - `/*[size(value, 2) == 3]`, `/*[numel(value) > 2]`, `/*[~isscalar(value)]`: The expression could include function calls The examples above only uses `value` for the currently selected child, but there may be benefits to being able to select parents, ascendants, or descendants. For example: - `/users/*[value(/age)>=18]/name` would select users' names of those at least 18 years old. - `/{Susan,Jane}/Children/*[value(/Height)>=value(/../../Height)]` would select children of Susan and Jane who are taller than their mother ### Open questions - What a full suite of features could entail, even if not initially supported - The exact syntax used - Whether `.` should be required or implicit - Whether filtering should be supported with `DotExact`, for example `/X[/Z > 0]/Y` - How it interacts with nested matching dot operations, for example `[has:/*/X]` - How this interacts with non-traversable branch handling in #32 - Bundle boundary may be forced directly after some filters because MPath may need the value to evaluate the condition, but this may not always be true ### Example One use case is to select valid children only. ```matlab data = struct(... A = struct(X = 1), ... B = 2, ... C = struct(X = 3) ); mpath.get(data, "/*/X") mpath.get(data, "/*[has:/X]/X") ``` Without filtering, `*` matches `A`, `B`, and `C`, then `/X` is attempted on each branch. The `/B/X` branch is non-traversable because `B` is not a container. The filter `[has:/X]` would exclude `B` from the list of fields because it does not satisfy the condition of having a child `X`. This would make it so users would not need to rely on `MissingPolicy` to filter in-post. See #32 regarding traversing paths through non-containers.