Skip to content

architecture.md

Latest commit

 

History

History
182 lines (134 loc) · 11.6 KB

architecture.md

File metadata and controls

182 lines (134 loc) · 11.6 KB
Raw

komet-node: Architecture

Overview

komet-node is a local Stellar testnet whose execution engine is the K formal semantics of Soroban. It decodes incoming Stellar transactions into K steps and executes them through the compiled semantics.

The split between Python and K follows one rule: K does everything that is part of the Soroban/Stellar protocol, and Python does only what K structurally cannot. The compiled K semantics are a one-shot interpreter — one invocation runs one request to completion and exits, with no network, no memory between invocations, and no decoder for Stellar's binary XDR format. Python supplies exactly those three missing pieces: it is the long-running process that holds the HTTP socket, it keeps the world state on disk between invocations, and it decodes the XDR envelope (and parses uploaded wasm). Everything else — RPC method dispatch, receipt bookkeeping, ledger-sequence accounting, status determination, and JSON-RPC response formatting — runs inside the K semantics (node.md).

flowchart TB
    client(["Stellar client"])

    subgraph py["Python — StellarRpcServer process (long-running)"]
        direction TB
        server["server.py — StellarRpcServer<br/>HTTP/JSON-RPC, owns the socket"]
        encoder["transaction.py — TransactionEncoder<br/>XDR → request envelope<br/>(+ kasmer steps for wasm uploads)"]
        interp["interpreter.py — NodeInterpreter<br/>runs the LLVM interpreter"]
        server -->|"1 — build envelope"| encoder
        server -->|"2 — run(envelope)"| interp
    end

    subgraph ksem["K semantics — LLVM backend (kdist/node.md + soroban-semantics)"]
        node["dispatch RPC method · run steps via KASMER<br/>update bookkeeping · format response"]
    end

    subgraph iodir["io dir (on disk)"]
        direction LR
        state[("state.kore")]
        meta[("metadata.json")]
        txs[("receipts/ · traces/")]
    end

    client -->|"JSON-RPC (base64 XDR)"| server
    interp -->|"request.json + state.kore"| node
    node -->|"response.json"| interp
    node <-->|"read / write"| meta
    node <-->|"read / write"| txs
    interp <-->|"round-trip"| state
    server -->|"JSON-RPC response"| client
Loading

Components

server.py — StellarRpcServer

StellarRpcServer is the long-running process around the semantics: a plain http.server.HTTPServer that keeps the HTTP socket open and the state files on disk across requests — the networking and persistence the one-shot K interpreter has no notion of. It receives JSON-RPC requests, uses TransactionEncoder to turn a transaction into a request envelope, hands the envelope to NodeInterpreter, and returns the response.json the semantics produced. It holds no ledger counter, receipt store, or response-formatting logic — those live in K.

handle_rpc(method, params, id) is the dispatch entry point and is usable without the HTTP layer (scripts, tests).

→ Detailed documentation

The server implements six RPC methods — getHealth, getNetwork, getLatestLedger, sendTransaction, getTransaction, and the komet-specific traceTransaction extension — and the K semantics answer all of them. JSON-RPC framing (single calls, batch arrays, notifications) is handled in Python; the semantics see one request envelope per invocation.

sendTransaction always returns PENDING and clients poll getTransaction for the result — matching the Stellar RPC async pattern even though the transaction executes synchronously. See server.md for details.

transaction.py — TransactionEncoder

TransactionEncoder decodes Stellar's binary XDR transaction format, which K cannot parse. It turns a stellar_sdk transaction envelope into a JSON request envelope containing the RPC method, the transaction hash, the envelope XDR, and the decoded operations as JSON "steps". For the one case K cannot consume as JSON — a wasm upload, whose ModuleDecl has no JSON form — it produces the kasmer steps in K-AST form for direct injection into the <program> cell.

→ Detailed documentation

interpreter.py — NodeInterpreter

NodeInterpreter runs request envelopes through the K semantics. It builds the initial configuration, feeds a request envelope to the LLVM interpreter against state.kore, and persists the resulting state. It knows nothing about Stellar. It performs no whole-configuration kast↔kore conversions — the initial config is built directly in KORE and request steps are spliced into the <program> cell at the KORE level.

→ Detailed documentation

kdist/node.md — K Semantics

node.md is the K module compiled into the LLVM binary. It implements the whole RPC layer on the K side: it reads request.json, dispatches on the method field, reads and updates metadata.json and the per-transaction receipts/ files, executes transaction steps via KASMER, and writes the JSON-RPC response.json. KASMER is the Komet execution harness whose Steps — setAccount, deployContract, callTx, uploadWasm — carry out the Soroban operations a transaction decodes into.

→ Detailed documentation

The io-dir

All of the server's input and output artifacts live in one directory, the io dir (set by --io-dir; when omitted, a fresh temporary directory). Its contents fall into two groups: the files and directories that persist across requests and restarts and together hold the chain, and a few transient files that the server and the semantics rewrite on every request to pass data to each other.

Path Lifetime Written by Contents
state.kore persistent NodeInterpreter the full K world-state configuration — accounts, contract code (including uploaded wasm ModuleDecls), contract storage, ledger metadata — serialized in KORE. Read before each run and rewritten after a successful one.
metadata.json persistent the K semantics {"latest_ledger": N} — the server ledger counter, bumped by 1 per committed transaction.
receipts/receipt_<hash>.json persistent the semantics (on success) or the server (on failure) one stored receipt per transaction, keyed by tx hash, answering getTransaction. Each is {status, ledger, createdAt, envelopeXdr, resultXdr, resultMetaXdr}.
traces/trace_<hash>.jsonl persistent the semantics one execution trace per transaction, keyed by tx hash — the instruction-level records, one JSON object per line. traceTransaction returns this file's contents.
requests/request_<n>.json persistent the server an archive of each incoming JSON-RPC request, numbered by a monotonic counter, kept for debugging.
request.json transient the server the request envelope for the call in flight (method, id, now, and method-specific fields). The semantics remove it once they respond.
response.json transient the semantics the JSON-RPC response ({jsonrpc, id, result}) for the most recent call. The server reads it back; it is absent when a transaction gets stuck.

Receipts, traces, and request archives are split into one file per item — keyed by tx hash, or numbered — so that no single file grows without bound as the chain advances. The server creates the receipts/, traces/, and requests/ directories before the semantics run, because the K file-system hooks open files with POSIX open(), which does not create parent directories.

The world state stays in KORE (rather than a JSON snapshot) because an uploaded wasm module is a ModuleDecl that the semantics cannot reconstruct from bytes — only wasm2kast (Python) can produce it. The receipts and the ledger counter, by contrast, are plain data and live in JSON files, which the semantics read and write directly via the file-system hooks.

flowchart TB
    boot(["server start"]) --> exists{"state.kore exists?"}
    exists -->|"no"| init["empty_config() builds the idle K config in KORE<br/>write state.kore · seed metadata.json {latest_ledger: 0}<br/>create receipts/ traces/ requests/"]
    exists -->|"yes"| reuse["use existing state.kore<br/>seed metadata.json if missing · ensure artifact dirs exist"]
    init --> ready(["ready for requests"])
    reuse --> ready

    ready --> tx(["transaction submitted"])
    tx --> run["semantics run the decoded steps<br/>(trace appended to traces/trace_&lt;hash&gt;.jsonl)"]
    run --> stuck{"steps completed<br/>without getting stuck?"}
    stuck -->|"yes — SUCCESS"| ok["write receipts/receipt_&lt;hash&gt;.json<br/>bump latest_ledger → metadata.json<br/>write response.json · persist new state.kore"]
    stuck -->|"no — FAILED"| fail["no response.json<br/>state.kore and ledger left unchanged<br/>server writes a FAILED receipts/receipt_&lt;hash&gt;.json"]
    ok --> ready
    fail --> ready
Loading

Because the persistent artifacts live on disk, the server can be stopped and restarted without losing the world state, the ledger counter, or the stored receipts. To resume a session, point --io-dir at a directory holding a saved state.kore (its sidecar files and receipts are read if present). To start fresh, point it at an empty directory.

Request flow (end to end)

sequenceDiagram
    autonumber
    actor Client
    participant Server as StellarRpcServer
    participant Enc as TransactionEncoder
    participant Interp as NodeInterpreter
    participant K as node.md (K semantics)
    participant FS as io dir files

    Note over Client,FS: Submit
    Client->>Server: sendTransaction { transaction: base64 XDR }
    Server->>Enc: build_tx_request(...)
    Enc-->>Server: request envelope (+ program_steps if wasm upload)
    Server->>Interp: run(state_file, io_dir, envelope, program_steps)
    Interp->>FS: write request.json
    Note right of Interp: wasm only — splice upload steps into the program cell, in KORE
    Interp->>K: llvm_interpret on state.kore
    Note over K: insert-handleRequestFile → dispatch → dispatchMethod
    K->>FS: run steps (trace → traces/trace_&lt;hash&gt;.jsonl)
    K->>FS: write receipts/receipt_&lt;hash&gt;.json (SUCCESS)
    K->>FS: bump latest_ledger in metadata.json
    K->>FS: write response.json { status: PENDING, ... }
    K-->>Interp: updated configuration
    Interp->>FS: persist new state.kore
    Interp-->>Server: response.json (verbatim)
    Server-->>Client: { hash, status: PENDING, latestLedger, ... }

    Note over Client,FS: Poll for the result
    Client->>Server: getTransaction { hash }
    Server->>Interp: run(read-only envelope)
    Interp->>K: llvm_interpret on state.kore
    K->>FS: read receipts/receipt_&lt;hash&gt;.json
    K-->>Interp: response.json
    Interp-->>Server: response.json
    Server-->>Client: { status: SUCCESS, ledger, createdAt, envelopeXdr, ... }
Loading

Dependencies

Dependency Role
komet Soroban K semantics, SorobanDefinition, SCValue dataclasses, kasmer step types
pyk K toolchain Python bindings: llvm_interpret, kdist, KORE parsing/prelude, kast_to_kore (only for the wasm module)
pykwasm Wasm → K AST conversion (wasm2kast), used for the wasm-upload step
stellar_sdk Stellar transaction types, XDR encoding/decoding, TransactionEnvelope

What's not yet implemented

  • resultXdr / resultMetaXdr in getTransaction responses (contract return values)
  • simulateTransaction (dry-run without state mutation)
  • getEvents, getLedgerEntries, getFeeStats and other read-only RPC methods
  • ExtendFootprintTTL and RestoreFootprint operations
  • SCVec / SCMap contract-argument types in the request encoder (scval_to_json)