Application API
This page documents the public application protocols in flashdreams.api_v2
and the runtime-owned value types those protocols exchange. For implementation
workflows, start with Create a demo or
Integrate model adapter for a demo.
Application and session
flashdreams.api_v2.application.IApplication
Bases: ABC
One application, for as long as the process runs.
Parses its own arguments and holds whatever its sessions share, such as a
checkpoint or a compiled pipeline. It outlives every session it creates, so
that shared state is loaded once here and released in :meth:close.
An application module implements this and :class:ISession. The runtime
creates everything else and passes it in.
init
abstractmethod
init(commandline_args: Sequence[str]) -> None
Parse application arguments and start preparation.
Source code in flashdreams/flashdreams/api_v2/application.py
25 26 27 28 | |
default_client_window_mode
default_client_window_mode() -> str | None
Return the preferred CLI mode when --mode is omitted.
Asked before :meth:init, so this must not load models or acquire
resources. Return an installed client-window mode name, or None
to retain the runtime's MP4 default. An explicit --mode wins.
Source code in flashdreams/flashdreams/api_v2/application.py
30 31 32 33 34 35 36 37 | |
resolve_client_window_mode
resolve_client_window_mode(mode: str) -> str
Return the installed window mode implementing the requested output.
An application may select a registered mode that adds its input source
while preserving the requested presentation backend. The runtime still
validates arguments and creates the window. Called before :meth:init;
do not load models or acquire resources here. Help and application
preload skip this hook.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mode
|
str
|
Explicit CLI mode, or the resolved application/runtime default. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Installed mode name. The default returns |
Raises:
| Type | Description |
|---|---|
ValueError
|
The application cannot support the requested output. |
Source code in flashdreams/flashdreams/api_v2/application.py
39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | |
session_desc
session_desc() -> SessionDesc | None
Return the description of a session this application would generate.
A caller has to describe a session before there is one to describe, and
only the application knows what its model was trained for. Asked before
:meth:init, so describing a session costs nothing.
Returns:
| Type | Description |
|---|---|
SessionDesc | None
|
The session to create when nobody asks for another, or |
SessionDesc | None
|
the default, from an application that generates whatever it is |
SessionDesc | None
|
asked for. Its caller describes the session instead. |
Source code in flashdreams/flashdreams/api_v2/application.py
59 60 61 62 63 64 65 66 67 68 69 70 71 | |
create_session
abstractmethod
create_session(session_desc: SessionDesc) -> ISession
Create one isolated, uninitialized session for session_desc.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session_desc
|
SessionDesc
|
Session the runtime is asking for. |
required |
Returns:
| Type | Description |
|---|---|
ISession
|
A session for |
ISession
|
actually produce. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The application cannot honour |
Source code in flashdreams/flashdreams/api_v2/application.py
73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | |
close
close() -> None
Release whatever the application holds.
Not abstract, and does nothing by default, so an application with nothing to release does not implement it.
Source code in flashdreams/flashdreams/api_v2/application.py
89 90 91 92 93 94 95 | |
flashdreams.api_v2.session.ISession
Bases: ABC
One application run with a model loop and a UI loop.
Register the model loop in :meth:init. If no UI loop is registered,
the runtime uses :class:BlitModelOutputToScreenLoop.
parallel_context
property
parallel_context: ParallelContext | None
Return the model mesh before initialization, or None for a local session.
The runtime synchronizes model steps and inputs across this world. Only rank zero owns a client window and output sinks.
session_desc
abstractmethod
property
session_desc: SessionDesc
Return the description used to configure the runtime.
init
abstractmethod
init() -> None
Initialize state and register this session's loops.
A model loop is required. Registering a UI loop is optional; without one
the runtime uses :class:BlitModelOutputToScreenLoop.
Raises:
| Type | Description |
|---|---|
RuntimeError
|
No model loop was registered by the time the runtime asks for the loops. |
Source code in flashdreams/flashdreams/api_v2/session.py
65 66 67 68 69 70 71 72 73 74 75 76 | |
register_ui_loop
register_ui_loop(loop_type: type[IUILoop[Any]], *, state: Any = None, **kwargs: Any) -> IUILoop[Any]
Create and register the UI loop, and return it.
Omit this call to use the default UI. The loop is paced at the session's
frames_per_second_for_ui.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
loop_type
|
type[IUILoop[Any]]
|
UI loop class to instantiate. |
required |
state
|
Any
|
State the loop owns. Unlike a model loop, a UI loop is allowed to hold none. |
None
|
**kwargs
|
Any
|
Passed to |
{}
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
The runtime already took the loops, or a UI loop was registered already. |
TypeError
|
|
Source code in flashdreams/flashdreams/api_v2/session.py
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | |
register_model_loop
register_model_loop(loop_type: type[IModelLoop[Any]], *, state: Any, **kwargs: Any) -> IModelLoop[Any]
Create and register the model loop, and return it.
The loop is paced at the session's frames_per_second_for_step.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
loop_type
|
type[IModelLoop[Any]]
|
Model loop class to instantiate. |
required |
state
|
Any
|
State the loop owns. Required, since a model loop with no state has nothing to generate from. |
required |
**kwargs
|
Any
|
Passed to |
{}
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
The runtime already took the loops, or a model loop was registered already. |
TypeError
|
|
Source code in flashdreams/flashdreams/api_v2/session.py
129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | |
close
close() -> None
Release resources owned by the session.
Source code in flashdreams/flashdreams/api_v2/session.py
185 186 187 | |
Model and UI loops
flashdreams.api_v2.loop.IModelLoop
Bases: ILoop[StateT], ABC
Loop that generates model results on the model thread.
:meth:ILoop.step must return list[StepResult] here, one entry per
channel, with every channel reporting the same frame_count. An empty
list means the step produced no presentable output.
Every admitted rank executes the same model step and ordered collectives.
The integration owns TP/CP partitioning and must gather all CP shards needed
for publication before the presenting rank returns non-empty results.
Workers return [] only after all required collectives, including gathers.
Gather before decoding when it needs the complete latent/view sequence;
decoding may run only on the presenting rank when the model permits it.
TP reductions that already replicate a complete value need no extra gather.
A run whose steps all present nothing still ends when the model loop does.
Returning a bare :class:StepResult or None raises :class:TypeError.
inference_state
property
inference_state: ModelInferenceState
Return whether this model loop has started, is running, or finished.
step
abstractmethod
step(step_index: int, events: UserInputEvents) -> list[StepResult]
We expect IModelLoop's to return a list of StepResults, one result per channel.
Source code in flashdreams/flashdreams/api_v2/loop.py
325 326 327 328 | |
flashdreams.api_v2.loop.IUILoop
Bases: ILoop[StateT], ABC
Loop whose output is sent to the client window.
:meth:ILoop.step must return list[StepResult] here: one frame to
present, or an empty list to present nothing this step. A list longer than
one raises :class:TypeError. Model frames to draw come from
:meth:presented_model_frame and :meth:presented_model_frames rather
than from the model loop directly.
:meth:has_pending_model_frames and :attr:presented_model_frame_count
report whether more model frames are waiting and how many have been
selected.
model_inference_state
property
model_inference_state: ModelInferenceState
Return whether model inference has started, is running, or finished.
presented_model_frame_count
property
presented_model_frame_count: int
Return how many model frames have been selected in this generation.
step
abstractmethod
step(step_index: int, events: UserInputEvents) -> list[StepResult]
Return zero or one :class:StepResult to present this tick.
Source code in flashdreams/flashdreams/api_v2/loop.py
504 505 506 507 | |
register_session_ui_loop_objects
register_session_ui_loop_objects(*, session_desc: SessionDesc, presentation_manager: PresentationManager) -> None
Store UI objects supplied when this loop is registered with a session.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session_desc
|
SessionDesc
|
Description of the session this UI presents. |
required |
presentation_manager
|
PresentationManager
|
Buffer containing model frames. |
required |
Source code in flashdreams/flashdreams/api_v2/loop.py
509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 | |
request_new_session
request_new_session(session_desc: SessionDesc) -> None
Ask the runtime to replace this session after the current UI step.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session_desc
|
SessionDesc
|
Fully resolved description for the replacement session. |
required |
Source code in flashdreams/flashdreams/api_v2/loop.py
539 540 541 542 543 544 545 546 | |
request_hide_cursor
request_hide_cursor(hide_cursor: bool) -> None
Request that the client window show or hide its cursor.
Source code in flashdreams/flashdreams/api_v2/loop.py
548 549 550 551 552 553 | |
request_lock_cursor_to_window
request_lock_cursor_to_window(lock_cursor_to_window: bool) -> None
Request that the client window release or capture pointer motion.
Source code in flashdreams/flashdreams/api_v2/loop.py
555 556 557 558 559 560 561 562 | |
request_new_window_size
request_new_window_size(new_window_size: tuple[int, int]) -> None
Request a positive downstream client-window width and height.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
new_window_size
|
tuple[int, int]
|
Requested |
required |
Raises:
| Type | Description |
|---|---|
TypeError
|
The size is not a pair of integers. |
ValueError
|
Either dimension is not positive. |
Source code in flashdreams/flashdreams/api_v2/loop.py
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 | |
request_links
request_links(links: tuple[ClientWindowLink, ...]) -> None
Replace up to sixteen clickable regions of the output content.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
links
|
tuple[ClientWindowLink, ...]
|
Immutable regions; an empty tuple clears existing links. |
required |
Raises:
| Type | Description |
|---|---|
TypeError
|
The request is not a tuple of |
ValueError
|
More than sixteen regions are supplied. |
Source code in flashdreams/flashdreams/api_v2/loop.py
588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 | |
flush_ui_loop_requests
flush_ui_loop_requests() -> UILoopRequests | None
return UI loop requests and clear the internal state of requests.
Source code in flashdreams/flashdreams/api_v2/loop.py
612 613 614 615 616 | |
presented_model_frame
presented_model_frame(channel_index: int = 0) -> Tensor | None
Return the current frame from one model-result channel.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
channel_index
|
int
|
Channel to read, indexed as the model loop returned them. |
0
|
Returns:
| Type | Description |
|---|---|
Tensor | None
|
A |
Tensor | None
|
before the first model result has been presented. |
Raises:
| Type | Description |
|---|---|
IndexError
|
The presented result has no such channel. |
ValueError
|
The presentation stream is on a different CUDA device from the presented result. |
Source code in flashdreams/flashdreams/api_v2/loop.py
618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 | |
presented_model_frames
presented_model_frames() -> tuple[Tensor, ...]
Return the current frame from every model-result channel.
Returns:
| Type | Description |
|---|---|
Tensor
|
One |
...
|
empty tuple before the first model result has been presented. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The presentation stream is on a different CUDA device from a presented result. |
Source code in flashdreams/flashdreams/api_v2/loop.py
640 641 642 643 644 645 646 647 648 649 650 651 652 | |
has_pending_model_frames
has_pending_model_frames() -> bool
Return whether another model frame is ready to present.
Source code in flashdreams/flashdreams/api_v2/loop.py
660 661 662 663 | |
flashdreams.runtime_v2.imgui_ui_loop.ImGuiUILoop
ImGuiUILoop(*, renderer: _ImGuiUIRenderer | None = None, width: int | None = None, height: int | None = None, cuda_device: str | None = None)
Bases: IUILoop[_StateT], ABC, Generic[_StateT]
Render immediate Dear ImGui controls over an optional model frame.
Subclass this and implement :meth:step_ui. The imgui argument exposes
imgui_bundle.imgui and adds an image-like pixel convenience form:
imgui.image(key, pixels, size=(width, height)).
Configure an ImGui loop without eagerly creating GPU resources.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | |
resize_ui_loop
resize_ui_loop(width: int, height: int) -> None
Resize the ImGui renderer.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
49 50 51 | |
get_ui_loop_size
get_ui_loop_size() -> tuple[int, int]
Return the current ImGui render-target size.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
53 54 55 | |
step_ui
abstractmethod
step_ui(imgui: Any, step_index: int, events: UserInputEvents) -> Tensor | None
Draw one immediate-mode UI frame and return its optional back buffer.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
57 58 59 60 61 62 63 64 65 | |
step
step(step_index: int, events: UserInputEvents) -> list[StepResult]
Render and composite one ImGui frame.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | |
reset
reset() -> None
Reset renderer state after a session reset event.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
92 93 94 | |
close
close() -> None
Release the renderer.
Source code in flashdreams/flashdreams/runtime_v2/imgui_ui_loop.py
96 97 98 | |
Session description
flashdreams.runtime_v2.session_desc.SessionDesc
dataclass
SessionDesc(*, output_layout: VideoTensorLayout = tchw, backpressure_mode: BackpressureMode = BLOCK, presentation_mode: PresentationMode = CONTINUOUS, frames_per_second_for_ui: int = 30, frames_per_second_for_step: int = 30, video_width: int = 1280, video_height: int = 720, metadata: dict[str, Any] = dict())
Description of a session, passed to create one and to open a window on it.
The runtime fills this in to ask an application for a session, and the
session reports back what it resolved to. The same description then
configures the client window through OutputSink.open.
output_layout
class-attribute
instance-attribute
output_layout: VideoTensorLayout = VideoTensorLayout.tchw
Declared tensor layout for generated video results.
backpressure_mode
class-attribute
instance-attribute
backpressure_mode: BackpressureMode = BackpressureMode.BLOCK
What the model thread does when its output queue is full.
presentation_mode
class-attribute
instance-attribute
presentation_mode: PresentationMode = PresentationMode.CONTINUOUS
What the UI loop does when no new model frame is ready.
frames_per_second_for_ui
class-attribute
instance-attribute
frames_per_second_for_ui: int = 30
Rate to poll input and tick the UI, in frames per second.
frames_per_second_for_step
class-attribute
instance-attribute
frames_per_second_for_step: int = 30
Initial video rate and maximum model-loop iterations per second.
video_width
class-attribute
instance-attribute
video_width: int = 1280
Initial output video width in pixels for ClientWindow.
video_height
class-attribute
instance-attribute
video_height: int = 720
Initial output video height in pixels for ClientWindow.
metadata
class-attribute
instance-attribute
metadata: dict[str, Any] = field(default_factory=dict)
Runtime and application extension values, including reserved runtime keys.
flashdreams.runtime_v2.session_desc.BackpressureMode
Bases: Enum
What the model thread does when its presentation chunk queue is full.
BLOCK
class-attribute
instance-attribute
BLOCK = 'block'
Wait when the presentation queue is full.
DROP_OLDEST
class-attribute
instance-attribute
DROP_OLDEST = 'drop_oldest'
Drop the oldest queued model chunk.
flashdreams.runtime_v2.session_desc.PresentationMode
Bases: Enum
Control what the UI loop does when no new model frame is ready.
ON_DEMAND
class-attribute
instance-attribute
ON_DEMAND = 'on_demand'
Present only when there is a new model frame and it is selected.
CONTINUOUS
class-attribute
instance-attribute
CONTINUOUS = 'continuous'
Present every UI tick, reusing the newest model frame when necessary.