Define behavior when paths continue through non-container values #32

New issue

Closed

opened 2026-07-02 01:45:06 +00:00 by UserCurt · 1 comment

UserCurt commented

2026-07-02 01:45:06 +00:00

(Migrated from codeberg.org)

Behavior is currently defined when a path continues through a container value but the requested child is missing:

MissingPolicy="omit": omit that missing branch from the resolved result
MissingPolicy="missing": replace the would-be path with missing mpath.Pointer
MissingPolicy="retain": keep the would-be path
MissingPolicy="error": throw an error

Behavior is less clearly defined when a path tries to continue through a non-container value.

Compare:

data = struct(...
  A = struct(), ...
  B = 1, ...
  C = struct(X=2) ...
);
mpath.resolve(data, "/{A,C}/X", MissingPolicy=...)  % A is a container type
mpath.resolve(data, "/{B,C}/X", MissingPolicy=...)  % B is not a container type

A is a container type so hypothetically could have had the child X but did not in this circumstance. In contrast, B is not a container type so could not have ever had the child X to begin with. A is traversable, while B is not. In some sense, the path including B is ill-defined because the path is "physically" impossible.

The design question is whether the behavior for non-traversable paths should be controlled by MissingPolicy, or whether non-traversable continuation deserves a separate policy.

Using the same MissingPolicy treatment would keep the API simpler. Having retain include would-be non-traversable paths can help with debugging. However, it can be misleading as one may think /B/X is a possible path even though it cannot exist while A is a non-container. Additionally, mpath.set with MissingPolicy="create" will require a set of traversable paths and require the more strict interpretation where non-traversable paths give an error.

Introduction of a new parallel policy for handling non-traversable paths (e.g., NonTraversablePolicy) could distinguish this case from ordinary missing children. Possible values would be similar to MissingPolicy and could include "error", "omit", "missing", and "retain". This certainly adds complexity and possible confusion to the API, and users may need to remember to modify both policies in tandem.

Alternatively, an orthogonal policy could be introduced (e.g., NonTraversablePolicy) could act as a modifier to MissingPolicy and could include "error" and "allow". Perhaps a "missing" could be valuable in some circumstances: Using MissingPolicy="retain" with NonTraversablePolicy="missing" would keep the shape while making non-traversable paths missing but non-existing but traversable paths still retained.

Regardless of the solution, mpath.set will need an additional constructibility check (when enabled), either ignoring or erroring when candidate paths are not possible due to non-traversability.

It is worth highlighting that non-container values are valid terminal results (/B), but only become problematic when later path operations require them to act as parents (/B/X).

This also relates to whether field-matching dot operations (e.g., DotGlob and DotRegex in the future) should match children locally or look ahead to the remaining path. For example, mpath.resolve(data, "/*/X") could be interpreted in two ways:

local matching: * matches all children, then /X is attempted on each branch (current implementation)
lookahead matching: * matches only children for which the remaining descendant path /X can be traversed

The lookahead approach would automatically exclude non-containers. The current implementation gets around the need for lookahead through MissingPolicy. This is not perfect, as indicated by this issue. Future support for filtering (#33) could improve the situation by making lookahead explicit: /*[has:/X]/X would have * match only fields that have the child X).

Behavior is currently defined when a path continues through a container value but the requested child is missing: - `MissingPolicy="omit"`: omit that missing branch from the resolved result - `MissingPolicy="missing"`: replace the would-be path with missing `mpath.Pointer` - `MissingPolicy="retain"`: keep the would-be path - `MissingPolicy="error"`: throw an error Behavior is less clearly defined when a path tries to continue through a non-container value. Compare: ```matlab data = struct(... A = struct(), ... B = 1, ... C = struct(X=2) ... ); mpath.resolve(data, "/{A,C}/X", MissingPolicy=...) % A is a container type mpath.resolve(data, "/{B,C}/X", MissingPolicy=...) % B is not a container type ``` `A` is a container type so hypothetically could have had the child `X` but did not in this circumstance. In contrast, `B` is not a container type so could not have ever had the child `X` to begin with. `A` is traversable, while `B` is not. In some sense, the path including `B` is ill-defined because the path is "physically" impossible. The design question is whether the behavior for non-traversable paths should be controlled by `MissingPolicy`, or whether non-traversable continuation deserves a separate policy. Using the same `MissingPolicy` treatment would keep the API simpler. Having `retain` include would-be non-traversable paths can help with debugging. However, it can be misleading as one may think `/B/X` is a possible path even though it cannot exist while `A` is a non-container. Additionally, `mpath.set` with `MissingPolicy="create"` will require a set of traversable paths and require the more strict interpretation where non-traversable paths give an error. Introduction of a new parallel policy for handling non-traversable paths (e.g., `NonTraversablePolicy`) could distinguish this case from ordinary missing children. Possible values would be similar to `MissingPolicy` and could include `"error"`, `"omit"`, `"missing"`, and `"retain"`. This certainly adds complexity and possible confusion to the API, and users may need to remember to modify both policies in tandem. Alternatively, an orthogonal policy could be introduced (e.g., `NonTraversablePolicy`) could act as a modifier to `MissingPolicy` and could include `"error"` and `"allow"`. Perhaps a `"missing"` could be valuable in some circumstances: Using `MissingPolicy="retain"` with `NonTraversablePolicy="missing"` would keep the shape while making non-traversable paths missing but non-existing but traversable paths still retained. Regardless of the solution, `mpath.set` will need an additional constructibility check (when enabled), either ignoring or erroring when candidate paths are not possible due to non-traversability. It is worth highlighting that non-container values are valid terminal results (`/B`), but only become problematic when later path operations require them to act as parents (`/B/X`). --- This also relates to whether field-matching dot operations (e.g., `DotGlob` and `DotRegex` in the future) should match children locally or look ahead to the remaining path. For example, `mpath.resolve(data, "/*/X")` could be interpreted in two ways: - local matching: `*` matches all children, then `/X` is attempted on each branch (current implementation) - lookahead matching: `*` matches only children for which the remaining descendant path `/X` can be traversed The lookahead approach would automatically exclude non-containers. The current implementation gets around the need for lookahead through `MissingPolicy`. This is not perfect, as indicated by this issue. Future support for filtering (#33) could improve the situation by making lookahead explicit: `/*[has:/X]/X` would have `*` match only fields that have the child `X`).

UserCurt commented

2026-07-12 14:31:48 +00:00

(Migrated from codeberg.org)

Addressed by af551a2 primarily, and also 2cbf33f for a small follow-up on yet-widely supported DotIndex.

Traversal through a non-container now uses the existing missing-policy model rather than introducing a separate non-traversable policy.

From a user’s perspective, resolving a branching path should be consistent in that when some branches not valid from non-container items should not give an error under MissingPolicy for anything but "error". So it was decided to have matching members of non-containers are treated just like failing to match any members, acting like a missing path.

Addressed by `af551a2` primarily, and also `2cbf33f` for a small follow-up on yet-widely supported `DotIndex`. Traversal through a non-container now uses the existing missing-policy model rather than introducing a separate non-traversable policy. From a user’s perspective, resolving a branching path should be consistent in that when some branches not valid from non-container items should not give an error under `MissingPolicy` for anything but `"error"`. So it was decided to have matching members of non-containers are treated just like failing to match any members, acting like a missing path.