The dataflow model

ipyflow’s features – staleness detection, reactive scheduling, slicing, memoization – all read from a single in-memory structure: a dataflow graph whose nodes are symbols and whose edges are the dependencies between them. This page describes the objects that make up that model. Their programmatic surface is cataloged in Data model.

ipyflow builds the graph by using pyccolo to instrument executing code: the AST of each cell is rewritten to insert tracing hooks, and as the code runs those hooks observe every assignment, usage, and mutation. Nothing about your code changes – the instrumentation is transparent.

Timestamp

A Timestamp is a (cell_num, stmt_num) pair naming a single point in execution:

  • cell_num – the cell execution counter, which increments every time any cell runs (so it strictly increases across the session).

  • stmt_num – the 0-indexed statement within that cell execution.

Timestamps are how ipyflow reasons about “before” and “after”. A symbol is stale when one of its dependencies carries a newer timestamp than the symbol itself – that comparison is the heart of staleness detection.

run_cell("a = 1")            # a updated at cell 1
run_cell("b = a + 1")        # b updated at cell 2
run_cell("assert timestamp(a).cell_num == 1")
run_cell("assert timestamp(b).cell_num == 2")
# Re-running a gives it a newer timestamp than b, so b is now stale.
run_cell("a = 2")
run_cell("assert timestamp(a).cell_num > timestamp(b).cell_num")
run_cell("assert lift(b).is_waiting")

Symbol

A Symbol is a node in the graph – a variable binding (or an attribute/subscript member). Each symbol tracks:

  • its name (readable_name) and a reference to the actual Python obj;

  • its timestamp, the point at which it was last written;

  • its parents – the symbols it was computed from (incoming edges);

  • its children – the symbols computed from it (outgoing edges);

  • the scope that contains it.

parents and children are exactly what the deps / users helpers expose, filtered to user-visible symbols. Following them transitively yields the backward and forward slices.

An important subtlety is aliasing: several symbols can bind the same object. ipyflow indexes symbols by object id (aliases: obj_id -> {Symbol}) so that a mutation to an object is attributed to every name pointing at it.

Scope and Namespace

A Scope is a symbol table mapping names to symbols. Scopes nest the way Python’s do:

  • the global scope holds top-level notebook variables;

  • a function scope holds a function’s locals;

  • a namespace scope holds an object’s attributes or items.

That last kind is a Namespace – a scope attached to a particular object (keyed by obj_id) that represents members like df.columns, config["key"], or obj.attr. Namespaces are created lazily, the first time a member is accessed, and they are what let ipyflow track dependencies below the variable level: a cell that reads x[0] depends on the x[0] member symbol, not on all of x, so an unrelated change to x[1] does not make it stale.

Cell and Statement

A Cell represents a notebook code cell. It records the frontend cell_id, the execution counter cell_ctr at which it ran, its content, its position in the notebook (used for in-order scheduling), and its edges to other cells – derived from the symbol edges above. Cells are the unit the UI and reactive scheduler operate on.

A Statement is the finer-grained analog: one AST statement, with its own timestamp and symbol-derived edges. Statement-level edges are what make statement slicing (%flow slice --stmt) more precise than cell slicing.

Putting it together

When a cell runs, the tracer creates or updates a Symbol for each assignment and records an edge for each usage, stamping everything with the current Timestamp. The central NotebookFlow singleton owns this state – the scopes, the alias index, the namespaces, and the per-session settings – and answers the questions the frontend and the public API ask of it: what is stale?, what must re-run?, what code reproduces this value?

The last of those – turning the graph into runnable code – is program slicing, covered next in Program slicing.