Developer documentation

Build against the specification.

A key, the Python SDK, and the loop that turns one executable task definition into a training signal, a sealed evaluation, and a release decision you can hand to someone who was not in the room.

01

Get an API key

One workspace and one key per email. Issued immediately during the beta.

Or from a terminal
curl -X POST https://formalengines.com/api/signups -H 'content-type: application/json' -d '{"email":"you@example.com"}'

Same endpoint, same response: 201 with the key, 409 if the email already has a workspace.

02

Install the SDK

Python 3.12 or newer. The base client is httpx, Pydantic, and the shared contracts.

Install
pip install formal-engines --extra-index-url https://formalengines.com/pypi/simple/

Wheels are served from this domain. formal-engines is not on public PyPI yet, so the extra index is required.

Publishing environments
pip install 'formal-engines[authoring]' --extra-index-url https://formalengines.com/pypi/simple/

The authoring extra adds environment packaging and local validation. It is imported only when you callspecifications.publish(path=…), as the quickstart below does.

03

Quickstart

Publish a specification, train against it, and export the release evidence.

Python
import os

from formal_engines import FormalEngines
from formal_engines.types import Gate, ModelRef, TrainingPlan

# The client reads both of these from the environment; export them in your shell instead if you prefer.
os.environ["FORMAL_ENGINES_API_KEY"] = "fe_live_your_key"
os.environ["FORMAL_ENGINES_BASE_URL"] = "https://formalengines.com/api"

fe = FormalEngines()

# Package a local verifiers environment and publish it as an immutable specification version.
spec = fe.specifications.publish(
    path="./environments/support_agent",
    train_split="train",
    eval_split="sealed_eval",
)

# Baseline: score the untrained model on the sealed split.
baseline = fe.evaluations.run(
    specification=spec,
    model=ModelRef(name="Qwen/Qwen3.5-4B"),
    split="sealed_eval",
).wait()

# GRPO against the same specification that defines the reward.
training = fe.training.start(
    TrainingPlan(
        specification=spec,
        base_model="Qwen/Qwen3.5-4B",
        algorithm={"type": "grpo"},
        max_steps=100,
        batch_size=128,
        rollouts_per_example=8,
    )
)

checkpoint = training.wait().best_checkpoint

# Candidate: the trained checkpoint, scored on the split training never saw.
candidate = fe.evaluations.run(
    specification=spec, model=checkpoint, split="sealed_eval"
).wait()

# Gates turn the two runs into a recorded pass or fail.
decision = fe.releases.decide(
    candidate=candidate,
    baseline=baseline,
    gates=[
        Gate.metric("task_success", minimum=0.82),
        Gate.regression("policy_violation", maximum_delta=0.0),
    ],
)

# Signed bundle: specification version, run ids, metrics, gate results.
decision.export("release-evidence.json")

publish packages the directory, creates the version, validates it, and publishes it in one call. Both run handles expose .wait(), .events(), .metrics(), and.cancel(); polling, cursors, retries, and idempotency stay inside the SDK.

04

How it works

Versions, runs, sealed splits, gates, evidence.

A specification is an executable environment — task state, tools, permitted actions, constraints, and the verifiers that decide what counts as success — published as an immutable version. Training and evaluation both address that version by id, so a reward and a score always come from the same definition of correct. Candidate checkpoints are scored on a sealed split that training never sees, which is the only thing that makes the resulting number worth quoting. Gates state the release condition as data: an absolute floor on a metric, or a no-regression bound measured against the baseline run. A release decision applies those gates to a candidate and baseline pair and freezes the outcome into an evidence bundle — specification version, run ids, metrics, gate results, and an HMAC-SHA256 signature over the canonical payload — so the record survives the argument about whether the model got better.

The product story

05

API at a glance

Seven resource groups under one base URL.

ResourcePurpose
/v1/specificationsCreate a specification, add versions, validate one, publish it. A published version is immutable and is what every run refers to.
/v1/evaluation-runsScore a model or checkpoint on one split of a version. Metrics and traces hang off the run.
/v1/training-runsStart post-training against a specification, then read metrics, logs, and checkpoints.
/v1/runs/{run_id}/eventsOne server-sent event feed for both run types. Events carry a monotonic sequence, so reconnecting with the last cursor loses nothing and repeats nothing.
/v1/release-decisionsApply gates to a candidate and baseline pair, record the verdict, and export the evidence file.
/v1/evidence-bundlesRetrieve the signed bundle behind a decision.
/v1/artifactsContent-addressed transfer of packaged environments. specifications.publish() uses it for you.
Base URL
https://formalengines.com/api
Auth
Authorization: Bearer fe_live_… on every request. Keys are workspace-scoped.
Idempotency
Every mutating request accepts an Idempotency-Key. Replaying a key returns the original run instead of starting a second one.

06

Private beta

What you can expect from the shared compute pool right now.

Formal Engines is in invite-scale private beta. A signup gives you a workspace and a key straight away, but every run is scheduled on a small shared beta compute pool: training jobs queue, and large ones may not be admitted at all. If you need dedicated capacity, a particular accelerator, or a model size beyond a few billion parameters, write to us with the task and the shape of the workload before you start. Your specifications, traces, and checkpoints stay in your workspace.

hello@formalengines.com