Add sgn.health, reporting a run's lifecycle to a process supervisor:
ready once the graph is running, an alive heartbeat as it strides, and
stopping as the run loop exits (at end-of-stream or on failure)
Pipeline.run(health=...) accepts a HealthReporter or a sequence of them;
None (the default) selects reporters from the environment, and health=[]
disables reporting. Custom transports subclass sgn.health.HealthReporter
Add sgn.health.Systemd, speaking sd_notify(3) (READY=1, WATCHDOG=1,
STOPPING=1, each with a STATUS= line) for Type=notify services;
enabled when NOTIFY_SOCKET is set, with the heartbeat interval derived
from WATCHDOG_USEC
Add sgn.health.HTTPHealth, serving GET /readyz and GET /healthz (200
or 503 with a JSON snapshot) on the HTTPControl server when the run
happens inside one, or on a server started for the run alone; enabled by
the SGN_HEALTH_PORT environment variable, with SGN_HEALTH_HOST and
SGN_HEALTH_MAX_AGE
HTTPControl accepts per-instance host, port and signals overrides,
and registry_file=None skips writing the registry file
Add sgn.errors with PipelineError and PipelineEOSError, both
re-exported from the top-level package
Add ParallelizeBase.eos_error and Pipeline.eos_errors(), exposing the
failure a worker stashed before its element still ended its stream at EOS
Pipeline.run() raises PipelineEOSError after the graph drains when an
element reached EOS only after absorbing an unrecoverable error (e.g. a
resource source whose reconnect budget ran out), so the process exits
abnormally instead of reporting a clean end of stream; sinks have flushed
by then, and each element's error is logged
Worker output queues are drained while joining, so a worker blocked in
put() on a full bounded queue is released to reach its stop check instead
of wedging the join
Thread-mode workers are daemon threads, so a worker that cannot be joined no
longer keeps the interpreter alive after a pipeline error
Add ElementLike.use_threads, the instance-level opt-in for dispatching an
element's pad callbacks onto the pipeline thread pool, and
ElementLike.executor_pads, selecting which of its pad types dispatch
("src", "snk", "inl")
Add PadLike.uses_executor, whether an active executor would run this pad's
callback
Add sgn.validator.PadConstraint, recording the pad topology a validator
enforces; the validator decorators attach it to the decorated method as
pad_constraint (stacked decorators merge), so tools can inspect an
element's expected pad structure without instantiating it
Add the TESTPOINT_FIELDS environment variable, a space-separated list of
the frame fields to display, in order
Add the SGN_FULL_TRACEBACK environment variable, restoring the untrimmed
traceback on a failed run
Warn when a requested thread pool will sit idle, because no pad opts into
dispatch or an element's executor_pads selects no working pad type
TestpointDisplay.padspec now defaults to ".*" (all pads)
Dispatching a pad callback onto the thread pool now requires
use_threads=True on the instance as well as thread_safe = True on the
class; elements that dispatched on thread_safe alone now run on the
event-loop thread
use_threads=True without thread_safe raises ValueError, and invalid
executor_pads tokens are rejected at construction
ElementLike subclasses must redeclare use_threads or executor_pads with
an annotation to change the class default; a bare class attribute raises
TypeError
InputPull narrows executor_pads to {"src"}
A failed run logs a one-line ERROR headline naming the element, pad, and
callback, and trims sgn and asyncio plumbing frames from the traceback; the
exception itself propagates unchanged
A StopIteration or StopAsyncIteration escaping a pad callback becomes a
RuntimeError naming the original type, instead of the interpreter's bare
RuntimeError: coroutine raised StopIteration
Faster graph execution: the topological schedule is cached per graph,
UniqueID hashes are cached at construction, and non-dispatching pads are
awaited inline instead of each being wrapped in a task
Custom Pad.__call__ implementations must not suspend except via _dispatch
The testpoint latency column is headed latency [s], and fields that
cannot be formatted are logged at DEBUG level
Worker-backed elements nested inside a composed element are now detected by
Parallelize.needs_parallelization(), so their workers start; the run
previously hung
Fix a hang when a thread_safe element leaked a StopIteration on Python
<= 3.11: it is now converted before crossing an asyncio Future, whose
set_exception left the awaited future never completed
Add a latency field option to the testpoint frame display, showing the
frame's latency in seconds via its latency() method (available on frame
types that provide one, e.g. sgn-ts TSFrame)
Add opt-in thread-pool dispatch of pad callbacks via Pipeline.run(threaded=...)
for elements that set thread_safe = True. threaded accepts a pool size
or an existing thread-based Executor. This composes with subprocess
parallelization: threaded is now forwarded through the automatic
Parallelize path, so thread_safe elements still dispatch onto the thread
pool in the main process while Parallelize* elements run in workers.
Pad callbacks (new/pull/internal) are now invoked positionally rather
than by keyword (e.g. call(pad) instead of call(pad=...)), required for
thread-pool dispatch. Subclasses that renamed the first new/pull
parameter away from pad/frame are unaffected as long as they accept it
positionally.
Exceptions raised by a pipeline run inside an already-running event loop
(e.g. in Jupyter) are re-raised on the calling thread instead of being
silently swallowed
Add sgn.control.Slot, a thread-safe single-value mailbox with set(),
get() and take(), holding the HTTP control slot values and available
to downstream packages registering extra control routes
Add HTTPControl.bound_port exposing the port the control server actually
bound to
Add gap_eos_scope option ("all_inputs", the historical behavior, or
"per_output") to CallableTransform, controlling whether an output
frame's gap/EOS state aggregates over all inputs or only the inputs mapped
to that output
Add Parallelize.cleanup_all_shm() for best-effort cleanup of all tracked
shared-memory segments
Add sgn.profile.reset_mem_baseline() to reset the memory-profiling
baseline; memory-diff stats are reported separately with signed values
Rework the HTTP control mailboxes: HTTPControl.post_queues/get_queues
are now post_slots/get_slots holding Slot values instead of
queue.Queue, posting is atomic, the WSGI server is shut down on context
exit, and HTTPControl.port is no longer mutated to the bound port (use
bound_port); HTTPControlSinkElement.queuesize was removed
connect() no longer implicitly links multiple disjoint-named source pads
to a single sink pad and raises ValueError instead; pass an explicit
link_map
SinkPad.link() raises ValueError when the pad is already linked, and
pad-validation asserts were promoted to real exceptions that also fire
under python -O
Exceptions raised in pad callbacks propagate as the original exception
object with a note naming the pad, instead of being reconstructed from the
message
Pipeline creates its event loop in run() (and closes it afterwards)
instead of calling asyncio.get_event_loop() at construction
StatsSource tracks its collection interval with a monotonic clock and
marks frames emitted on off-interval ticks as gaps
CallableTransform.from_combinations() no longer accepts the ignored
source_pad_names parameter
PadSelection.pad_names is a frozenset, and selecting zero pad names
raises ValueError
Composed element classes no longer accept source_pad_names/
sink_pad_names constructor arguments (they are derived from the
composition), and ambiguous or unknown update_pad_names entries raise
SGNLOGLEVEL parsing splits on the last colon, so logger names may
contain colons
The testpoint display bootstraps lazily on first use, so importing sgn
never constructs rich objects at module load
Parallelize* worker parameters whose names collide with
framework-reserved attribute names raise ValueError instead of silently
shadowing framework state
CollectSink collects payloads carried on the EOS frame itself, and
skip_empty=True also skips frames whose extracted payload is None
UniqueID equality compares ids with a type guard instead of comparing
hashes, eliminating false equality with unrelated objects
Static pad declarations inherited from element base classes are recognized
by class validation, so intermediate base classes with dynamic pads
disallowed no longer fail
SinkElement.mark_eos() raises ValueError for pads that are not sink
pads of the element instead of silently recording a bogus entry
Memory profiling cumulative stats are taken from the current snapshot
rather than the previous one
Add sgn.testpoint: live terminal diagnostics for running pipelines,
activated via environment variables without code changes —
TESTPOINT=frame shows a live table of frame info flowing through pads,
TESTPOINT=exec shows per-pad execution-time statistics;
TESTPOINT_REGEX selects pads and TESTPOINT_FIELDS selects columns.
Requires the rich package, imported only when a testpoint is enabled
Visualize composed elements: Pipeline.to_graph(), to_dot() and
visualize() gained an expand_composed parameter (default True)
rendering composed elements as cluster subgraphs showing their internal
structure
Add link() for explicit pad-map linking inside composed-element
_build() hooks; insert(), connect() and link() return the
composition so calls can be chained
Add an internal_elements read-only property to composed elements
Warn when two internal elements expose boundary source pads with the same
short name, and support mapping one boundary sink name to multiple
internal sink pads via update_pad_names
Rework composition: build composed elements by subclassing
ComposedSourceElement/ComposedTransformElement/ComposedSinkElement
and implementing _build(); the internal_elements/internal_links
constructor arguments were removed, Compose no longer subclasses
Graph, and Compose.build_link_dict() was removed
Add on_startup() hook on elements for setup that runs after the
pipeline is fully constructed but before the first frame, called in
graph order
Support name in pipeline membership tests for elements and pads
Support building reusable composed elements by subclassing
ComposedSourceElement, ComposedTransformElement, and
ComposedSinkElement, as an alternative to the Compose builder
Add Compose class for composing multiple elements into reusable units
Support for ComposedSourceElement, ComposedTransformElement,
ComposedSinkElement via .as_source(), .as_transform(), .as_sink()
Add class-level fixed/extra pad configuration for elements via
static_sink_pads, static_source_pads, allow_dynamic_sink_pads,
allow_dynamic_source_pads
Add validation decorators for element reuse
Add CollectSink flag to keep frames where data is None