# Helmsman: when a research candidate becomes infrastructure

> The design of an explicit promotion boundary: frozen definitions, concurrent requests, durable evidence gates, and the limits of a deployed research workspace.

Published: 2026-09-23  
Work period: July 2026  
Product: [Helmsman](https://adityadalmia.com/projects/helmsman/)  
Canonical HTML: https://adityadalmia.com/blog/helmsman-promotion-boundary/

Experiments are cheap to create. Operational commitments are expensive to understand later.

A research workspace can accumulate hundreds of variations on an idea. If each variation immediately acquires a monitor, a schedule and deployment state, the operator inherits an infrastructure inventory before deciding which experiments deserve attention. If the experiments remain disconnected files, the opposite problem appears: carrying one forward means reconstructing its identity and evidence by hand.

That boundary shaped the operator workspace of Helmsman, the private research platform I’m building with collaborators. My work has involved the workspace’s design and integration with the surrounding research and execution systems, with collaborators and AI-assisted implementation. The interesting engineering is in the contracts between those systems.

The [earlier workspace article](https://adityadalmia.com/blog/helmsman-control-plane/) introduces the workspace. Here I want to follow one consequential action: turning a research definition into a monitor specification without silently turning it into a running or validated system.

As of this article’s preparation on September 22, 2026, the Helmsman workspace has a reachable, authenticated web interface. We are working with mock data for now. The implementation described below has source and historical test receipts; the read-only inspection for this article did not exercise promotion or establish real-data readiness.

## Keep the experiment cheaper than the commitment

Structure Lab, accessible inside the workspace, lets us make a definition explicit before evaluating it. A definition describes how to recognise a pattern in price observations. For an invented example, a price might move from $100 to $103, return to $101, and rise again. The definition specifies which parts of that sequence count as an event. Those numbers illustrate the question, not a result or a rule used by our system.

We split the representation into two tiers.

The research tier holds candidate definitions and evaluation records. Registering a candidate gives it a durable identity. It does not create a monitor or deploy anything. Automated searches and human sketches can use the same registry without filling the operator workspace with every variation they generate.

The operational tier holds a **thesis** and a **monitor**. A thesis is a research claim with context and an explanation of what would disprove it—for example, a claim about what tends to happen after that illustrative $100 price sequence. A monitor is the specification of what to observe in support of the claim.

Composer is the explicit crossing between those tiers. We rejected making every candidate a monitor: the lifecycle machinery would become the cost of merely asking a research question. We also avoided retyping a candidate into an unrelated monitoring object. Promotion carries its frozen configuration identity forward.

\[Diagram: Illustrative path from research definition to monitor specification — Registration gives a candidate identity. Explicit promotion creates operational records. Compilation and deployment remain separate steps. Research definition Register a frozen candidate Identity + evidence Review, then explicitly promote Thesis + monitor specification Evidence gate when required Separate compilation and deployment Illustrative architecture, not a product screenshot. Neither registration nor promotion executes the detector.]

## Give the definition an identity the UI cannot rewrite

The registry computes a SHA-256 hash over canonical JSON: object keys are sorted recursively, with a consistent serialization. The API owns that calculation. Asking a Python worker and a JavaScript interface to independently serialize and hash the same configuration would introduce another compatibility contract, including number formatting. Centralising the operation removes that disagreement from the clients.

Registering an identical configuration returns the existing definition. Database triggers reject updates and deletes to definition rows; a changed configuration becomes a new row, with an optional parent reference recording where it came from.

The hash identifies the configuration under that serialization contract. It does not certify that the idea is useful, and it does not identify every surrounding label or note. That distinction matters when a familiar name sits above a different experiment.

There was another, less obvious identity problem. The existing narrative compiler updates monitors using the thesis and agent name. Reusing its naming scheme for a promoted structure definition would let a later narrative compilation overwrite the frozen condition. Composer instead gives the structure monitor a distinct name incorporating the definition hash. The two compilation paths can coexist without one silently replacing the other’s specification.

## Walk through one promotion

Consider an operator investigating the invented price sequence above. They adjust a definition in the lab, register it, and inspect the evaluation evidence associated with that identity. In Compose, they select the definition, the market being observed and the observation interval, then write the claim and what would invalidate it.

The preview asks the API for derived fields such as the agent name and schedule. The browser does not independently reconstruct them. The write request checks the definition, evidence and duplicate status again; a preview is information for the operator, not permission to skip validation later.

If the request is accepted, the operational database receives a thesis and a monitor specification. The monitor begins with deployment state `spec` and promotion stage `log_only`.

Those are separate axes. Deployment state describes how far the specification has travelled through preparation and deployment. Promotion stage describes the intended operating stage. An object can exist as a specification without being deployed; creating it at `log_only` does not start an observation process either. Compilation and deployment remain subsequent actions.

This distinction also determined a feature we left out. An executable-flow preview sounded helpful, but the available compilation endpoint created a change bundle and updated stored state. Calling it just to paint a preview would make viewing the form a mutation. The implemented preview therefore shows the condition and server-derived schedule. A fuller preview needs a genuinely side-effect-free rendering contract first.

## An acknowledgement must survive the click

The difficult case is an interesting candidate without adequate evaluation evidence. Refusing to record it at all would obstruct investigation. Allowing it to move forward without retaining that uncertainty would make a warning disappear as soon as the dialog closed.

Composer requires an explicit acknowledgement in that case. Acceptance creates the observation-stage specification **and a persistent decision gate** naming the new thesis and monitor. The later stage-transition handler consults that gate before allowing advancement to `paper`.

Paper trading means simulation: recording an imaginary $100 purchase and $103 sale without committing those dollars. The stage name describes an intended operating mode; it is not evidence that a complete simulator is running. The gate prevents the transition in the implemented handler until a human resolves the blocking decision.

There is a practical limitation to this mechanism. The existing decision register matches accepted gates through object identifiers contained in its blocking text. Composer writes those identifiers deliberately. This is a useful integration with an existing contract, but it is less robust than a dedicated relational policy model. Arbitrary prose about an idea is not guaranteed to match its monitor.

The inspected handler also refuses advancement into real-money stages outright. Neither the acknowledgement nor favourable evaluation evidence removes that separate restriction.

One regression explains why the evidence classification deserves its own tests. A candidate could have favourable results on data used during its development and only thin evidence on later data. Treating the first result as sufficient merely because some later observations existed would overstate what had been tested.

The correction distinguishes the evidence used for ranking from the evidence needed to waive acknowledgement. Tests cover absent records, development-period evidence, insufficient later evidence and conflicting assessments. The engineering lesson is about preserving the population a conclusion came from. A summary value and a flag saying “later data exists” cannot safely stand in for that relationship.

## Put the boundary inside the transaction

A disabled button cannot protect against a retry or two requests arriving together.

The implemented promotion identity combines the frozen definition, the market and the observation interval. A quick lookup catches an existing promotion. The handler then begins a database transaction, acquires a transaction-scoped advisory lock for that identity and repeats the lookup under the lock. A duplicate returns a conflict with a pointer to the existing monitor.

The second lookup is consequential: without it, two requests can both pass the initial check and create two operational objects for the same candidate. The lock coordinates callers using this promotion path; it is not a substitute for reviewing every possible writer to the monitor table.

Inside the transaction, the handler creates the thesis, monitor, any required decision gate, and two entity-creation events. A failure rolls those writes back together. This keeps the newly created objects and their creation records within the same commit boundary.

Exercising the workflow exposed a smaller bug that typechecking could not find. Both creation events initially used PostgreSQL’s transaction timestamp. They therefore had the same time, and the event view’s UUID tie-breaker could display the monitor before its parent thesis. The implementation changed those inserts to use the wall-clock timestamp function. That fixes the observed ordering problem for this operation; it does not establish a universal causal clock.

The transaction’s limits are equally important. The frozen definition and evaluation evidence are read from a separate research database before the operational transaction. This is not a distributed snapshot or a transaction spanning both stores. The append-only definition stabilises the configuration reference, but evaluation evidence can continue to accumulate.

There is also a separate promotion-summary event written by generic middleware after the handler commits. That middleware documents a crash window between the mutation and its summary event. The two entity-creation events have the stronger atomicity guarantee; it would be incorrect to extend that guarantee to the entire event history.

## Verify behaviour, then qualify availability

The July 9 Composer build report records a development workflow from registration through promotion, deployment-state transition and flow compilation. It also records requests rejected for missing acknowledgement and duplicate promotion. Those are historical integration receipts, not actions repeated against the deployed application for this article.

The focused evidence-classifier tests were rerun while preparing this draft. All seven passed. They exercise classification logic without contacting a database or invoking a model. They do not prove database locking under concurrent load, current deployment permissions or a complete running research workflow.

Release evidence has its own scope. July deployment records establish that the web and API services were released and inspected. The local infrastructure definitions reviewed for this article pin a July 19 application revision, while the source checkout contains later changes. A release pin is desired-state evidence; it does not identify the currently running process by itself.

The September 22 read-only check established successful sign-in and a rendered Structure lab, including a chart and the Lab, Evidence and Compose navigation. The web health probe returned successfully, but its implementation is deliberately independent of backend dependencies. None of those observations establishes that every action behind the interface works.

The chart’s source control displayed a database selection. That label does not establish whether the stored records are mocked, current or complete. Our current status remains mock-data operation. Moving to real data is the next intended step, with fresh ingestion and workflow receipts still needed before describing it as achieved.

## What I would carry into the next iteration

The boundary is implemented; the wider operating claims still need evidence. Current live registration and promotion were not tested, detector execution was not triggered, and later operating stages remain separate work. A useful next verification would follow an authorised candidate across current data ingestion, persisted evidence, promotion, compilation and execution, keeping a receipt for each transition.

I would also strengthen the places where the current implementation relies on convention: typed relationships for evidence gates, and an explicit treatment of the post-commit summary-event gap. Those are improvements to pursue, not guarantees the system already provides.

The part I want to preserve is the shape of the decision. A candidate can stay cheap to create and easy to discard. When we decide it deserves operational attention, Helmsman gives that commitment an identity, a specification and a record of the uncertainty carried forward.