A CAD frame as the observation a Monty sensor module reads.
This is the one place the two worlds meet, and it is deliberately thin: the
frame supplies exact points, and Monty's own
Monty.Frameworks.Models.SensorModules.ObservationProcessor fits the surface,
finds the principal curvatures and assembles the message. Nothing here
re-implements a percept — a node trained on a CAD part has to be the percept a
run would have made, or the graph being built is a graph of something else.
The observation
Monty's sensor modules never see a depth image: the transform chain turns one
into "semantic_3d" — one row per pixel, [x, y, z, semantic_id] — and that
is what they read. Here the points come from ray casting the exact B-rep, so
"semantic_3d" is exact rather than an un-projection of a depth buffer, and a
pixel that met nothing carries 0.0, Monty's background.
"rgba" is shading: a B-rep has no colour, so the image is lit by one light
from the upper left of the camera. A "hsv" feature read from it says how the
surface turns, not what it is painted; a file with no appearance has no colour
to perceive, and inventing one would be worse than saying so.
The features a node keeps
node_features/1 merges the morphological and non-morphological features, the
same way GraphLM.select_features_to_use/2 does. Those are the columns a graph
memory stores per node: hsv, pose_vectors, pose_fully_defined,
principal_curvatures_log, and whatever else the sensor was asked for.
Summary
Types
A Monty observation: the map the sensor modules read.
Functions
The percept: Monty's own processor over this frame.
The features a graph node keeps from a message — morphological and the rest,
merged, the way GraphLM.select_features_to_use/2 merges them.
The observation a Monty sensor module reads from this frame.
Types
Functions
@spec message(Quiddity.Frame.t(), keyword()) :: {:ok, Monty.Cmp.Message.t()}
The percept: Monty's own processor over this frame.
Answers {:ok, %Monty.Cmp.Message{}}, whose location is the centre pixel's
point, whose morphological_features carry the surface normal and whether the
pose is defined, and whose non_morphological_features carry the depths,
colour and curvatures the sensor asked for.
A frame that met nothing still makes a message — Monty's processor does that,
with on_object 0.0 and process_features_in_lm false — because "the sensor
is over empty space" is a fact a model is allowed to act on. A caller that
wants only real percepts filters on on_object, the way Training.nodes/2
does.
@spec node_features(Monty.Cmp.Message.t()) :: %{optional(String.t()) => term()}
The features a graph node keeps from a message — morphological and the rest,
merged, the way GraphLM.select_features_to_use/2 merges them.
Keys are strings and values are what Monty stores: a three-element list for a pose vector, a two-element list for curvatures, a float for a defined pose.
@spec observation(Quiddity.Frame.t(), keyword()) :: observation()
The observation a Monty sensor module reads from this frame.
Options: :semantic_id (what an on-surface pixel is called, default 1).