F# hot reload: Edit-and-Continue delta emission behind --test:HotReloadDeltas#19941
Draft
NatElkins wants to merge 143 commits into
Draft
F# hot reload: Edit-and-Continue delta emission behind --test:HotReloadDeltas#19941NatElkins wants to merge 143 commits into
NatElkins wants to merge 143 commits into
Conversation
…eaps, row indexing, EncLog writer) Ported verbatim from the hot-reload prototype branch onto current main: - ILDeltaHandles.fs: typed handles for EnC delta metadata rows - ILMetadataHeaps.fs: heap accounting shared by baseline and delta passes - ILRowIndexing.fs: row index helpers for delta table emission - ILEncLogWriter.fs: IEncLogWriter abstraction recording EncLog/EncMap entries - ilbinary.fs: additive table/coded-index definitions required by EnC emission - EnvironmentHelpers.fs: FSHARP_HOTRELOAD_* environment variable helpers - Caches.fs: qualify System.Guid to avoid shadowing from new helpers All additions are dormant without the hot reload flag; no behavior change for normal compilation.
…ble name-map state - Generated/GeneratedNames.fs: central helpers for synthesized member/type names - Generated/CompilerGeneratedNameMapState.fs: capture/replay of name-generator state so successive hot reload generations synthesize identical names - CompilerGlobalState.fs: thread name-map state through NiceNameGenerator - IlxGen.fs: route compiler-generated local names through a single freshIlxName helper (replaces direct IlxGenNiceNameGenerator.FreshCompilerGeneratedName call sites) and add IlxGenEnvSnapshot/snapshotIlxGenEnv/restoreIlxGenEnv to capture the codegen environment a delta generation must replay - IlxGen.fsi: expose the snapshot API 3-way merge against current main verified: all 10 freshIlxName sites intact, no new upstream name-generator call sites bypass the helper, net diff vs main matches the prototype footprint exactly (+85/-49).
- TypedTree/SynthesizedTypeMaps.fs: stable maps for synthesized (closure/ state-machine) types across generations - TypedTree/TypedTreeDiff.fs: semantic diff between baseline and updated typed trees - binding/tycon snapshots, FNV-1a digests, rude-edit classification (signature changes, constraint changes, mutable-field toggles, type layout changes) - HotReload/DefinitionMap.fs: maps baseline definitions to updated symbols Static port validation against current main: every TypedTreeOps symbol consumed by TypedTreeDiff.fs resolves in the refactored namespace (upstream split TypedTreeOps.fs into TypedTreeOps.*.fs with [<AutoOpen>] modules inside namespace FSharp.Compiler.TypedTreeOps, so consumer opens are unchanged).
- ilwrite.fs/.fsi: thread IEncLogWriter through metadata emission; expose ILTokenMappings, MetadataHeapSizes, MetadataSnapshot and WriteILBinaryInMemoryWithArtifacts so a compilation can capture the token maps and heap layout a later delta generation must build on; markerForUnicodeBytes made public for delta #US emission (ECMA-335 II.24.2.4) - ilwritepdb.fs: portable PDB hooks for baseline capture - ILBaselineReader.fs: pure F# reader recovering baseline metadata state (row counts, heap sizes, GUID heap start) from emitted images - HotReloadBaseline.fs: baseline snapshot assembly for hot reload sessions - HotReloadPdb.fs: PDB delta support shared across generations 3-way merged cleanly against current main; upstream MethodDefKey/Codebuf changes (compareILTypes etc.) retained. Flag-off emission path is unchanged: full builds use the no-op EncLog writer (createNullEncLogWriter).
EnC delta (#~, #Strings, #US, #Blob, #GUID) construction, ported verbatim: - FSharpSymbolChanges.fs: symbol-level change set consumed by the writer - IlxDeltaStreams.fs: per-generation heap/stream builders (delta-local offsets, generation-aligned Blob/US heaps) - FSharpDefinitionIndex.fs: definition-to-token index over the baseline - DeltaMetadataEncoding.fs / DeltaMetadataTypes.fs / DeltaMetadataTables.fs: ECMA-335 II.22/II.24 row encodings, coded indices, table models - DeltaTableLayout.fs / DeltaIndexSizing.fs: row layout and wide/narrow index sizing for delta images - DeltaMetadataSerializer.fs: serializes EncLog/EncMap-ordered tables - DeltaMetadataSrmWriter.fs: System.Reflection.Metadata-based parity writer (FSHARP_HOTRELOAD_COMPARE_SRM_METADATA) - FSharpDeltaMetadataWriter.fs: top-level per-generation metadata delta writer (EncId/EncBaseId chaining)
- SymbolMatcher.fs: matches baseline symbols to updated tree symbols - HotReloadAccessorTypes.fs: synthesized accessor type support - IlxDeltaEmitter.fs: per-generation IL emission for added/updated methods, async/state-machine @hotreload type synthesis, rude-edit guarded (HotReloadUnsupportedEditException instead of failwith on unresolvable references) - HotReloadState.fs: per-session mutable state (generation chain, name maps) - DeltaBuilder.fs: orchestrates diff -> symbol matching -> emission - HotReloadCapabilities.fs: capability flags (Baseline, AddMethodToExistingType, ...) - HotReloadContracts.fs: cross-layer contracts for tooling integration - RudeEditDiagnostics.fs: rude-edit kinds and diagnostic mapping - EditAndContinueLanguageService.fs: Roslyn-shaped EnC entry point; rejects all rude edits before invoking the emitter - Adapters.fs: adapter layer between FCS session API and EnC service
- CompilerOptions.fs: --enable:hotreloaddeltas (baseline capture emission) and --enable:hotreloadhook (synthesized-name replay only) - CompilerConfig.fs/.fsi: emitCaptureArtifacts config and emit-hook wiring - CompilerEmitHookState.fs / CompilerEmitHookBootstrap.fs / HotReloadEmitHook.fs: pluggable emit hook capturing baseline artifacts (token maps, metadata snapshot, PDB) during normal fsc emission - fsc.fs: invoke the emit hook around main4-main6 binary emission - fsi.fs: qualify System.Guid (shadowing from new compiler-level opens) - FSComp.txt: fscHotReloadRequiresDebugInfo (2026), fscHotReloadIncompatibleWithOptimization (2027) - IDs verified free on current main; xlf regeneration deferred to a follow-up commit (requires a build with /p:UpdateXlfOnBuild=true) Flag-off compilations bypass the hook entirely (null EncLog writer).
- service.fs/.fsi: session surface the IDE/dotnet-watch layer consumes: FSharpChecker.StartHotReloadSession (2 overloads), EmitHotReloadDelta (2 overloads), EndHotReloadSession, HotReloadSessionActive, HotReloadCapabilities; plus FSharpHotReloadError, FSharpHotReloadDelta, FSharpAddedOrChangedMethodInfo, FSharpHotReloadCapability/-ies types - FSharpProjectSnapshot.fs: snapshot support for session baselines - FSharpCheckerResults.fs: expose typed-tree access needed by the differ (sessions require keepAssemblyContents=true) One textual conflict resolved in the service.fs open block (upstream added FSharp.Compiler.Caches at the same spot the prototype added ILDynamicAssemblyWriter/ILPdbWriter; kept all three). Upstream's new single-argument DiagnosticSink signature is unaffected - ported code implements no DiagnosticsLogger. Net diff vs main matches the prototype footprint (+927 service-layer lines).
- tests/FSharp.Compiler.Service.Tests/HotReload/ (18 files): unit tests for TypedTreeDiff, delta metadata writer, coded indices, ILBaselineReader, portable PDB reader, SRM parity, rude edits, thread safety, generated names, error/edge paths (258 tests on the prototype branch) - tests/FSharp.Compiler.ComponentTests/HotReload/ (15 files): integration tests - baseline capture, delta emission, runtime MetadataUpdater ApplyUpdate scenarios, mdv validation, PDB generations (101 tests) - FSharpWorkspace.fs: workspace plumbing used by hot reload session tests - HotReload.runsettings sets DOTNET_MODIFIABLE_ASSEMBLIES=debug and COMPlus_ForceEnc=1 for ApplyUpdate runs; NOTE: repo moved to Microsoft.Testing.Platform since the prototype - verify RunSettingsFilePath is still honored when running these suites - mdv-based tests honor FSHARP_HOTRELOAD_MDV_PATH Test fsproj compile items re-applied onto the MTP-migrated project files (anchors verified; 3-way merge clean).
…hook, release note - tests/scripts/: hot-reload-verify.sh (full pipeline gate), hot-reload-demo-smoke.sh (HOTRELOAD_SMOKE_RUNTIME_APPLY runtime apply), metadata coupling/parity and plugin-boundary guards, check-ilxgen-name-path.sh, main-fsi drift checks + allowlists - tests/projects/HotReloadDemo/: console demo app with hot reload session driver (hotreload-session.json + edited DemoTarget.fs); registered in FSharp.slnx under /Tests/HotReloadDemo/ (upstream migrated .sln -> .slnx since the prototype, so the old FSharp.sln entries were re-expressed) - eng/Build.ps1: Invoke-HotReloadDemoSmokeTest hook after testCoreClr and testDesktop runs, re-derived onto upstream's reworked test sections (TestSplit batching); runs the demo app with DOTNET_MODIFIABLE_ASSEMBLIES=debug and asserts the delta-emitted marker - hot-reload-verify.sh updated to build FSharp.slnx - tools/hot-reload/compare_roslyn.fsx: Roslyn delta comparison helper - docs: debug-emit.md hot reload heap tracing section, hot-reload-tgro-closure-matrix.md, release note entry in 11.0.100.md Deliberately not ported from the prototype branch: tmp.fsx (scratch), HOT_RELOAD_REVIEW_CHECKLIST.md and CLAUDE.md (workflow files, not product); xlf regeneration deferred (needs /p:UpdateXlfOnBuild=true build).
Mirror Roslyn's EditAndContinueCapabilities flags and parser semantics (exact-name matching, unknown capability words ignored, the AddDefinitionToExistingType aggregate) with an immutable typed model in FSharp.Compiler.EditAndContinue. Runtime capability strings are parsed once at the session boundary; a session always carries at least the Baseline capability. Includes parser tests and a design doc.
…bilities Thread the typed capability model through the hot reload session: FSharpChecker.StartHotReloadSession accepts an optional capabilities string sequence (Roslyn WatchHotReloadService parity), parsed once and stored on the session state; when omitted the session is conservative and assumes baseline-only support. Method additions now require the AddMethodToExistingType runtime capability. Without it the diff reports the new RudeEditKind.NotSupportedByRuntime (FSHRDL016) naming the missing capability, mirroring Roslyn's distinction between edits unsupported by hot reload and edits the connected runtime cannot apply. Classification consults a single capabilityForAddition seam so Phase B can flip field additions from always-rude to capability-gated by implementing emission. Rude-edit details now flow into the UnsupportedEdit error so hosts see the missing capability.
… additions Module-level values lower to a static field plus accessor methods with initialization in the startup class constructor, so adding one requires both AddStaticFieldToExistingType and AddMethodToExistingType; module-level functions lower to plain static methods and require only the latter. Both were previously unconditional DeclarationAdded rude edits. Instance fields remain rude until field-row emission lands (Phase B2); an emission-level test pins the cut line so a capability-enabled host gets a clean UnsupportedEdit rather than a half-emitted delta.
Adds Field-table support end-to-end in the delta writer pipeline: FieldDefinitionRowInfo row model, mirror table population, serializer table wiring, and SRM shadow-writer parity (including Field in the tracked parity tables). EncLog follows the Roslyn pattern recorded from a hotreload-delta-gen C# reference delta that adds a static field with an initializer: the parent TypeDef row is logged with the AddField operation immediately followed by the new Field row with the Default operation, and only the Field row enters EncMap. The pairs are spliced between the Module entry and Method entries so per-table EncLog sorting cannot separate them. Writer-level test asserts the exact EncLog sequence, EncMap content, and serialized table stream for an added static int32 field.
Added module-level values (let mutable x = ...) now emit complete, runtime-appliable deltas: the static backing fields on the startup-code class enter the Field table, accessors and the startup constructor are emitted as added methods, and the added value is readable/writable via its accessors after MetadataUpdater.ApplyUpdate. Making the deltas actually apply required aligning the EncLog with what CoreCLR's EnC applier (CMiniMdRW::ApplyDelta) and Roslyn emit for ALL added member kinds: an added member logs its PARENT row tagged with the Add* operation immediately followed by the member row with the Default operation (TypeDef for methods/fields, MethodDef for parameters, PropertyMap/EventMap for properties/events; map rows and MethodSemantics rows are plain Default entries). The previous shape - the Add* op on the member row itself - corrupted parent member lists at apply time and had never been runtime-applied by any test. Further fixes uncovered by the runtime-apply tests: - EditAndContinueOperation numeric values corrected to the CLR/SRM codes (AddParameter=3, AddProperty=4, AddEvent=5; previously 4/5/6). - Baseline #Strings size now uses SRM's trimmed size (StringHeap.TrimEnd semantics, Roslyn EmitBaseline parity); the padded stream size shifted every delta-heap string reference. - Added member names/signatures are written into the delta heaps instead of reusing fresh-compile heap offsets, with signature blobs remapped through remapSignatureBlobWith. - Return-parameter rows are no longer synthesized: the out-of-sequence seq-0 rows forced the CLR's indirect ParamPtr path and made ApplyUpdate reject the delta. ParamList of added methods stays monotone instead. DeltaBuilder resolves the startup-code class constructor as an updated method when the baseline already contains it (it is not a typed-tree binding, so the diff cannot pair it). Added field tokens chain into the next-generation baseline like added methods. Initialization semantics (validated at runtime and documented): the initializer runs lazily if the startup class was not yet type-initialized (the test observes 41); an already-initialized type reads default(T). New tests: checker-level delta validation for the added value, mdv validation of the Field rows and AddField pairing, and runtime ApplyUpdate tests for both added module functions and added module values. Instance field additions remain rejected (Phase B2).
…ule values Generation 2 adds a second mutable module value on top of the generation-1 field addition and applies it with MetadataUpdater.ApplyUpdate. This pins baseline chaining: generation-1 field/method/property tokens resolve in the chained baseline so only the new value is appended, and the startup-code constructor added in generation 1 is re-emitted as an updated method body. Also pins the already-initialized regime of the initialization semantics: the startup class was type-initialized in generation 1, so the second value reads default(int) rather than its initializer, while generation-1 state is preserved across the update.
… the typed-tree diff Replace the blanket lambda-shape digest equality check with a structured per-member lambda occurrence model: - Occurrence identity = traversal ordinal + parent-lambda chain + structural digest (curried arity, per-group parameter type identities, capture identity list, return type identity). Consecutive curried lambdas form one occurrence, matching IlxGen closure formation. Source ranges are diagnostics-only. - Old/new alignment = two LCS passes (full structural digest, then shape-only), so inserting/removing/reordering lambdas aligns the surviving occurrences instead of reporting spurious removed+added pairs. - Capture compatibility per matched pair with C#-parity classification: RenamingCapturedVariable, ChangingCapturedVariableType, ChangingCapturedVariableScope (cross-occurrence move post-pass), plus additions/removals (additions may become applicable in C4 via AddInstanceFieldToExistingType). - Classification: pure body edits within an unchanged lambda set remain plain MethodBody edits; lambda-set changes stay LambdaShapeChange rude edits, now with counts/ordinals in the message and a structured LambdaEdits payload on TypedTreeDiffResult for the C4 emitter. - Quotations, object expressions, local type functions, and types without a computable runtime identity keep the legacy whole-body digest path unchanged. Move the closure-mapping design doc into docs/ with C1 implementation notes.
Add EncMethodDebugInformation (CodeGen), replicating byte for byte the three Roslyn EnC CustomDebugInformation blob formats persisted per method in portable PDBs (EnC Local Slot Map, EnC Lambda and Closure Map, EnC State Machine State Map), including the syntax-offset-baseline optimization and the slot-map kind/ordinal byte packing. In F#, the syntax-offset slots carry C1 occurrence keys packed from the occurrence ordinal chain (16-bit segments, fail-closed past limits). Tests cover round-trips (temps, ordinal-flagged slots, negative baselines, static/this-only closure ordinals, negative state numbers), golden bytes, fail-closed limits, and cross-validation: a C# library built with the repo SDK has its Roslyn-emitted CDI rows decoded with the new decoder and re-encoded byte-identically. PDB writer/reader wiring is the next C2 commit.
… hotreloaddeltas When --enable:hotreloaddeltas is on, the fsc emit path computes per-method lambda occurrence data with the Phase-C1 extraction over the same optimized typed tree the baseline capture snapshots, serializes it with the C2 blob encoders, and carries it into the portable PDB writer through a new methodCustomDebugInfoRows side channel on the IL writer options: - TypedTreeDiff.collectMemberLambdaOccurrences: public C1 extraction over a CheckedImplFile, returning every member binding (empty occurrences for members the model cannot represent). - EncMethodDebugInformation.computeMethodCustomDebugInfoRows: occurrences -> EnC Lambda and Closure Map blobs keyed by IL method (compiled) name, with occurrence keys packed from the C1 ordinal chains; MethodOrdinal stays UndefinedMethodOrdinal; one closure scope per occurrence (IlxGen lowers every occurrence to its own closure class). The EnC Local Slot Map is omitted: the lowered slot layout is an IlxGen artifact, not derivable from the typed tree. - ilwritepdb attaches the rows as CustomDebugInformation on MethodDef parents, but only when the method name identifies exactly one method row; the producer likewise drops compiled names claimed by more than one member, so a map can never attach to the wrong method (overloads fail closed and simply carry no map). Flag-off builds pass the empty map everywhere and emit byte-identical output (EmittedIL gate: 1212 passed, 0 failed). New PdbCdiEmissionTests verify the flag-on PDB carries a decodable map with occurrence keys [0] and [0; 1] for a nested-lambda sample, and that the flag-off PDB contains no EnC CDI rows.
…sion When a baseline is captured, decode every method-level EnC CustomDebugInformation row of the baseline portable PDB (lambda/closure map plus the slot and state-machine maps) into FSharpEmitBaseline.EncMethodDebugInfos, keyed by MethodDef token - the CDI parent is a MethodDef handle, so token keying is unambiguous, unlike the name keying on the write side which exists only because the PDB writer lacks tokens. The fsc emit hook decodes the exact emitted PDB bytes; the checker path additionally reads the on-disk PDB as a sibling input because its in-memory baseline rewrite carries no CDI side channel. Fail safe/fail closed: flag-off and pre-C2 PDBs decode to the empty map (sessions start fine), and a method whose blobs do not decode is omitted rather than guessed. Generation chaining: EmitDeltaForCompilation recomputes per-method occurrence data from the fresh typed tree (computeRefreshedEncMethodDebugInfos, name-to-token resolution fail closed on non-unique names) and chainEncMethodDebugInfos replaces the updated methods' entries in the next-generation baseline, dropping entries the fresh compile did not produce so stale data can never be matched. The delta PDB does not yet re-emit EnC CDI rows; the in-memory chain is what C3 consumes, and PDB persistence across session restarts is the documented remaining gap.
Add ClosureNameAllocator, the F# analogue of Roslyn's
EncVariableSlotAllocator.TryGetPreviousLambda/TryGetPreviousClosure
expressed over the C1 lambda occurrence model: a matched occurrence
(same two-pass LCS as the C1 diff, equal capture sets, recorded
baseline name) reuses the baseline closure class name verbatim; an
unmatched, capture-incompatible, or unmappable occurrence gets a fresh
generation-suffixed name ({base}@hotreload#g{gen}_o{ord}, the
DebugId(ordinal, generation) analogue); removed occurrences fall out of
the chain-forward table so their names are never reused.
The index-pair core of the C1 alignment is extracted into
TypedTreeDiff.alignLambdaOccurrenceIndexPairs and shared by both
consumers, so diff classification and name allocation can never
disagree about pairing. Pure data transformation: no IO, no IlxGen
state; covered by ClosureNameAllocatorTests (match/add/remove/nested/
capture-incompatible/fail-closed plus three-generation chaining).
Gates: service HotReload 365/365 (356 + 9 new), component HotReload
134/134.
Two layers of coverage for the C3 naming machinery: - ClosureNameAllocatorTests now also drives the allocator over REAL C1 extraction (checker compiles via the shared DiffTestHarness, made internal for reuse): adding a filter lambda yields a generation-2 fresh name while the surviving map lambda reuses the baseline name, and generation 3 reuses both names from the chained table. - New component ClosureIdentityTests pins the metadata-level invariant the delta path relies on today (and C4 extends): flag-on recompiles of an unchanged lambda set produce closure classes with identical names across three body-edit generations, so deltas can update the existing closure method bodies in place. Gates: component HotReload 135/135, service HotReload 366/366, EmittedIL 1212 passed / 3 skipped / 0 failed.
Bridge the C1 lambda occurrence model to IlxGen's closure naming seam so the occurrence-keyed allocator can be wired into delta compiles: - LambdaOccurrence gains RootExprStamp, the unique stamp of the occurrence's outermost Expr.Lambda (extraction bookkeeping, never part of the structural digest or alignment). - GetIlxClosureFreeVars records stamp -> emitted closure type name into a new ConditionalWeakTable side channel (ClosureNameAllocationState, mirroring CompilerGeneratedNameMapState); recording is armed only by the emit hook for capture compiles, so flag-off output stays byte-identical. - The fsc emit path joins the recording with the same tree's occurrence extraction (ClosureNameAllocator.computeBaselineClosureNameRows, fail closed on ambiguous compiled names and on members with incompletely recorded occurrences) and threads the per-method chain -> name tables through TryEmitWithArtifacts/CompilerEmitArtifacts into the baseline capture, where they are re-keyed by MethodDef token and stored as FSharpEmitBaseline.EncClosureNames alongside EncMethodDebugInfos. Gates: compiler + both test projects build clean; component HotReload 136/136 (135 + new baseline-capture test), service HotReload 366/366, EmittedIL 1212/1212 (3 skipped).
Thread the C3 allocator into IlxGen lowering for session compiles:
- ICompilerEmitHook.PrepareForCodeGeneration now receives tcGlobals and the
optimized impl files about to be lowered. When a session with baseline
closure-name tables is active, the hook runs
HotReloadBaseline.computeOccurrenceKeyedClosureNames (previous-generation
occurrences vs fresh extraction, allocator per member with
generation = session.CurrentGeneration) and installs the resulting
stamp -> assigned-name table on the compiling CompilerGlobalState.
- The IlxGen closure call site consults the table FIRST and falls back to
sequence replay; the replay-map slot is still consumed unconditionally so
same-basename non-closure names keep their baseline replay positions.
Surviving closures therefore reuse baseline class names verbatim even when
the lambda set changes; added occurrences get {base}@hotreload#g{N}_o{i}.
- DeltaEmissionRequest.RefreshedClosureNameRows carries the allocator's
refreshed per-method tables (recomputed deterministically at delta emission)
and chainClosureNameRows chains them into the next-generation baseline with
the chainEncMethodDebugInfos replace-or-drop semantics.
- Fail closed everywhere: no session, empty baseline tables, ambiguous
compiled names, or unresolvable tokens leave lowering on pure sequence
replay; flag-off compiles see a single failed weak-table lookup.
Gates: compiler + both test projects build clean; component HotReload 138/138,
service HotReload 366/366, EmittedIL 1212/1212 (3 skipped); the
multi-generation closure-identity and body-edit runtime tests now exercise the
allocator path live.
… in delta compiles
Component coverage for the C3 lowering wiring, driven through the session/hook
plumbing (flag-on compiles against an active session):
- three body-edit generations of a 2-lambda method keep closure class names
byte-identical AND keep the chained session tables carrying the same names
(the compiles now take the allocator path, not bare sequence replay);
- a recompile with a lambda ADDED IN FRONT of the survivors — the case
sequence replay cannot handle — keeps every baseline closure name verbatim
and gives the added occurrence the generation-suffixed
{base}@hotreload#g1_o{i} name, which also chains forward for reuse.
Classification still rejects lambda-set changes at delta emission; emitting
the added member is Phase C4, documented in the design doc.
Gates: component HotReload 138/138, service HotReload 366/366.
Phase C4 sub-slice 1: writer + emitter support for ADDED TypeDef rows,
mirroring the Roslyn reference delta for "method gains its first capturing
lambda" (new display class). Reference recorded with a compiler-level
Compilation.EmitDifference harness (hot_reload_poc/src/csharp_enc_reference,
Roslyn 5.9.0) because the hotreload-utils workspace tool needs a net11-only
toolchain; the IDE pipeline ends in the same API, so the delta shape is
identical. Template captured verbatim in docs/hot-reload-member-additions.md.
Writer (FSharpDeltaMetadataWriter.emitWithTypeDefinitions):
- TypeDefinitionRowInfo / NestedClassRowInfo row models; TypeDef rows write
FieldList/MethodList as 0 (Roslyn EnC parity - members are linked through
the AddField/AddMethod EncLog pairs).
- EncLog: new TypeDef rows are plain Default entries placed before every
AddField/AddMethod pair that names them as the parent; NestedClass rows
are plain Default entries trailing the log. Both appear in EncMap.
- Serializer/SRM shadow writer/parity comparer extended for both tables.
Emitter (IlxDeltaEmitter):
- A fresh-compile type with the C3 allocator's generation-suffix marker
({base}@hotreload#g{N}_o{i}) and no baseline TypeDef token allocates the
next delta TypeDef row; its fields/methods (including instance capture
fields and the .ctor/Invoke pair) register as added members parented to
the NEW row. Extends is remapped (TypeRef/TypeDef); TypeSpec base types
rely on baseline passthrough and fail closed past the baseline table, as
do generic closure classes (GenericParam rows unsupported) - both gaps
documented.
- Added type tokens chain into the next-generation baseline TypeTokens and
are reported in UpdatedTypeTokens (Roslyn ChangedTypes parity).
Tests: exact-EncLog writer test mirroring the C# template (service), plus
emitter tests for a hand-constructed nested closure type incl. mdv
validation (component). Component HotReload 140/140, service HotReload
367/367.
Phase C4 sub-slice 2: the payoff slice. A member that gains a lambda now
produces an applicable delta carrying the new closure class.
Classification (TypedTreeDiff): Added-only lambda sets become method-body
edits when the runtime advertises NewTypeDefinition+AddMethodToExistingType,
otherwise NotSupportedByRuntime naming the missing capability (C# parity).
Removed-only sets are allowed at Baseline capabilities (deleted lambda
bodies just become unreachable; the baseline closure class stays unused).
Capture-set changes stay rude.
Emission: the delta compile's fresh rewrite contains the C3 allocator's
{base}@hotreload#g{N}_o{i} closure class; the emitter selects it as an added
TypeDef (sub-slice 1 machinery), and generation-suffixed names are excluded
from basic-name alias buckets (they never alias a baseline closure). The
type token chains into the next-generation baseline, so gen N+1 body-edits
the added lambda in place.
Wiring fixes the runtime test forced:
- checker.StartHotReloadSession rebuilds the baseline from disk, which
cannot carry EncClosureNames (CDI blobs have no name slots); it now
carries the tables over from the in-process capture session it replaces
when the MVID matches. Without this every watch-flow delta compile fell
back to sequence-replay naming.
- MemberRef/TypeSpec token passthrough is now content-validated: an added
lambda shifts the fresh compile's reference-row order, and positional
passthrough silently swapped ListModule.Filter/Map MethodSpec bindings
(BadImageFormatException at invoke). The baseline snapshots MemberRef row
contents and TypeSpec blobs (ILBaselineReader, which also gained the
missing InterfaceImpl row size - assemblies with interface impls had all
later table offsets misread); the remapper validates positional reuse,
falls back to content search, then appends (MemberRef) or fails closed
(TypeSpec). Appended MemberRef rows chain forward for later generations.
- remapTypeSpecBlobWith: TypeSpec blobs are a bare Type (II.23.2.14), not a
calling-convention-prefixed signature.
- The AsyncStateMachineAttribute heuristic now requires a MoveNext method,
so closure classes sharing the {method}@hotreload naming no longer pick
up a spurious attribute.
Runtime evidence (ApplyUpdate succeeds for added lambda creating a new
closure class): gen1 adds a third lambda in front of two survivors -> delta
EncLog carries the new TypeDef row + AddField/AddMethod pairs + NestedClass,
ApplyUpdate accepts, probe observes the new behavior; gen2 body-edits the
ADDED lambda -> method updates only, no new TypeDef rows.
Updated classification tests to the new semantics (added-without-capability
is NotSupportedByRuntime naming NewTypeDefinition; removed-only is a body
edit; async closure-chain shape changes follow the same gating).
Gates: component HotReload 141/141, service HotReload 367/367,
EmittedIL 1212/1212 (3 skipped). Both design docs updated.
Phase C4 sub-slice 3: - Negative runtime gate: a capability-less session rejects an added lambda with NotSupportedByRuntime (FSHRDL016) naming NewTypeDefinition, before any emission. - Removed-lambda runtime test: removing a lambda applies as a plain set of method-body updates (no TypeDef table entries; the baseline closure class stays, unused) and the new behavior takes effect - pinning the C#-parity allowance enabled in sub-slice 2. - mdv/EncLog pattern-parity assertions on the real generation-1 delta against the recorded C# new-display-class template: the new TypeDef row's plain Default entry precedes its Add* entries, every AddField/AddMethod parent entry is immediately followed by the member row with the Default operation, the NestedClass row trails, and EncMap carries the TypeDef and NestedClass rows but never the Add* parent entries. Gates: component HotReload 143/143, service HotReload 367/367.
… conduit The C2 PDB writer wiring added the methodCustomDebugInfoRows option to ilwritepdb.fsi; record the intentional drift and refresh the locked hashes.
Phase B2: instance fields added to existing CLASSES are emitted with the same (TypeDef, AddField) + (Field, Default) EncLog pairing as the B1b static path, validated against a recorded Roslyn EmitDifference reference (csharp_enc_reference field_add scenario) and applied at runtime. Classification: compareEntities now diffs the field segment structurally; a pure field addition on a TFSharpClass is gated on the negotiated AddInstanceFieldToExistingType/AddStaticFieldToExistingType capabilities (RudeEditKind.NotSupportedByRuntime names the missing one) instead of reporting TypeLayoutChange. Struct/record/union/enum layouts stay rude permanently (runtime restriction, C# identical), enforced fail-closed in the emitter via the fresh TypeDef's IsStructOrEnum. Constructor pairing: a mutable class let binding folds its initializer into the primary ctor, whose binding diffs as a plain MethodBody update; ctor return-type identity is now void (the IL truth) so .ctor edits resolve against baseline method tokens. [<DefaultValue>] val mutable needs no pairing: the entity surfaces as a TypeDefinition edit and the delta is a pure Field-row append - the metadata writer's empty-delta short-circuit now keys on row payload, not method updates alone. Runtime tests assert C# EnC semantics: existing instances read zeroed values for the added field, new instances run the updated ctor and see the initializer, and state survives multi-generation chains.
… identity The resumable-code/trait shape digest rendered builder-call type instantiations through TType.ToString() (tyToString), whose depth-limited LimitedToString(4) collapses a deeply-solved typar to the literal "True". That made the digest non-injective: a `task` whose return type is `int` both sides could render Bind<int,int,int> before an edit and Bind<int,True,True> after, producing a false StateMachineShapeChange (FSHRDL013) rude edit. Render the digest through tryTypeIdentityFromTType -- the same injective runtime-identity encoder the capture path already stores -- threading the method's typar->ordinal map into collectLoweredShapeInfo and traitConstraintShapeDigest, with a structured formatter and a display-string fallback only for types the encoder cannot represent. Update the architecture guard for the new traitConstraintShapeDigest signature. Addresses review feedback on dotnet#19941.
FSharpEditAndContinueLanguageService.UpdateActiveStatements had no callers: the live path is SetActiveStatements (FSharpHotReloadSession.SetActiveStatements -> SetSessionActiveStatements -> editAndContinueService.SetActiveStatements). Remove the dead member and its now-orphaned HotReloadSessionStore.UpdateActiveStatements backing, whose only caller was the dead forwarder. Addresses review feedback on dotnet#19941.
NatElkins
commented
Jun 23, 2026
| symbolChanges.RudeEdits | ||
| |> RudeEditDiagnostics.ofRudeEdits | ||
| |> List.map (fun diagnostic -> $"{diagnostic.Id}: {diagnostic.Message}") | ||
| |> String.concat Environment.NewLine |
Contributor
Author
There was a problem hiding this comment.
Agreed, and confirmed end-to-end: the structured {Id;Message;Kind} is flattened to UnsupportedEdit of string, the SDK bridge string-splits it back into a 4-state enum logged at Debug, so the reason never reaches the ENC diagnostic-codes channel C# uses (ReportRudeEdits/ReportCompilationDiagnostics). This is a cross-repo design change (FCS + SDK), not a one-liner. Proposed staging: (1) FCS-local — carry {Id;Severity;Message} through FSharpHotReloadError instead of a pre-joined blob, and have the bridge promote known FSHRDL* ids to a warning (surfacing the silent while/main/module-init cases); (2) full alignment — emit ENC-style codes through the same TransientDiagnostics channel, settling the id range with the SDK/Roslyn team (couples with the FSComp.txt thread). Leaving open to track.
NatElkins
commented
Jun 23, 2026
NatElkins
commented
Jun 23, 2026
NatElkins
commented
Jun 23, 2026
| 2025,fscAssemblyWildcardAndDeterminism,"An %s specified version '%s', but this value is a wildcard, and you have requested a deterministic build, these are in conflict." | ||
| 2026,fscHotReloadRequiresDebugInfo,"Hot reload delta capture (--enable:hotreloaddeltas) requires debug symbols (--debug+). Add '--debug+' or remove '--debug-' from the compilation options." | ||
| 2027,fscHotReloadIncompatibleWithOptimization,"Hot reload delta capture (--enable:hotreloaddeltas) is incompatible with optimizations (--optimize+). Add '--optimize-' or remove '--optimize+' from the compilation options." | ||
| hotReloadAdditionNotSupportedByRuntime,"Adding '%s' requires the runtime capability '%s', which the current runtime does not support." |
Contributor
Author
There was a problem hiding this comment.
Tracking with the ENC-channel thread above — these ENC-aligned hotReload* messages (UpdateMightNotHaveAnyEffect/ENC0118, InternalError, …) only become useful once the rude reason actually flows through a diagnostic channel, and the id range needs agreeing with the SDK/Roslyn team. Note the base-type/interface case is already caught as a rude TypeLayoutChange after the representation-hash fix, so a dedicated BaseTypeOrInterfaceUpdate message is optional polish. Leaving open as a follow-up.
NatElkins
commented
Jun 23, 2026
…error channel Rude-edit reasons were flattened to a single string at the point of failure (UnsupportedEdit of string), discarding the per-edit Id, Severity and symbol before they reached the public API and the dotnet-watch bridge. Carry them structurally instead: * RudeEditDiagnostic gains a Severity (FSharpDiagnosticSeverity). Every kind is Error today; severityOf is the single place to introduce a non-blocking Warning later (e.g. a "might not take effect" edit). * HotReloadError.UnsupportedEdit and the public FSharpHotReloadError.UnsupportedEdit now carry a list of structured diagnostics. A new public FSharpHotReloadRudeEdit record exposes Id, Severity, Message and SymbolName. * The active-statement, deleted-symbol, mapping-error and emit-exception paths wrap their ad-hoc reason via RudeEditDiagnostics.unsupported so every error still carries an id. The id namespace is unchanged and still owned solely by RudeEditDiagnostics.diagnosticId, so the channel itself is id-agnostic. Addresses review feedback on dotnet#19941.
…he swap point Document, at RudeEditDiagnostics.diagnosticId (the single place the rude-edit id namespace is decided), why F# keeps its own FSHRDL* codes rather than emitting Roslyn's ENC* codes, and note the closest ENC analogs. Kept as a separate commit from the channel work so aligning with the ENC codes later is an isolated change. Addresses review feedback on dotnet#19941.
NatElkins
added a commit
to NatElkins/sdk
that referenced
this pull request
Jun 23, 2026
…t-watch
The F# bridge collapsed the rude-edit reason to a record/DU ToString() dump and
logged it at Debug only, so an edit that forces a restart gave the user no reason.
Now that FSharpHotReloadError.UnsupportedEdit carries a structured rude-edit list
(Id + Severity + Message), read it via reflection (TryFormatRudeEdits) into a clean
"{Id}: {Message}" reason and report it as a warning instead of at Debug. The
extraction is defensive: any shape mismatch falls back to ToString(), so the bridge
stays correct against older FCS builds (where UnsupportedEdit still carries a string).
Pairs with the FCS-side structured diagnostics channel on dotnet/fsharp#19941.
Addresses review feedback on #1.
Make trying F# hot reload easier: - Add docs/hot-reload-setup.sh (one shot: build the compiler, clone + build the F#-aware SDK, sync them) and docs/hot-reload-sync-fcs.sh (the FCS -> redist sync). - Build only the SDK redist project rather than the whole repo, which is faster and side-steps unrelated test-only build breaks. - Source the SDK's eng/dogfood.sh to run the locally built CLI instead of hand-setting DOTNET_ROOT/PATH. - Add a "Fast path (one command)" section to the quickstart and fold the manual FCS copy and env setup into the scripts above.
The redist-only build (--projects redist.csproj) does a shallow restore that fails on a fresh clone (NETSDK1004 on a transitive project reference). Restore the whole repo first via a full build (which stops at a few unrelated broken test projects, expected), then build only the redist. Verified end to end by the prebuilt-SDK CI on linux.
T-Gro
reviewed
Jun 24, 2026
| sessionStore().UpdateImplementationFiles(updatedImplementation, ?key = projectKey) | ||
|
|
||
| Ok result | ||
| elif not (List.isEmpty delta.SequencePointUpdates) then |
Member
There was a problem hiding this comment.
This path resolves ijklam's "line-shift around a task needs restart" on head: removing/adding a non-captured line before a task applies as a sequence-point-only update, no rude. The earlier failure was the Windows build-lock, not the diff.
The captured variant (a let mutable hoisted across the await) is the one that's still rude in struct form (FSHRDL013, hoisted-layout change), and applies in class form (FSHARP_HOTRELOAD_CLASS_STATEMACHINES=1, 11 -> 24). Both confirmed on head with the runtime harness.
…oRange) dotnet/fsharp main turned Entity.Typars into a property and removed TyparsNoRange (SignatureHash.fs now uses tycon.Typars). The merge brought that in, so update the entity snapshot's IsGeneric check to tycon.Typars. Compiler builds clean locally.
… names `module A.B` is modelled as namespace A + module B, so the snapshot path was seeded with the file's qualified name "A.B" and then had "A" and "B" re-appended by the module traversal, rendering symbol names as "A.B.A.B.State" instead of "A.B.State". The doubling was display-only (consistent across baseline and delta, so symbol matching and the emitted deltas were unaffected) but wrong. canonicalQualifiedName drops the redundant dotted head; single-segment modules are unchanged. Adds an end-to-end regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This draft PR presents the complete F# hot reload implementation as a single branch against current main:
dotnet watch(with a companion dotnet-watch change, linked below) patches running F# processes in place via the standard EnC pipeline (MetadataUpdater.ApplyUpdate) — the same runtime contract C# uses. Unsupported edits degrade to the rebuild-and-restart flow watch uses today, so a limited scope still behaves as a complete feature.It is opened as a single draft per discussion with @T-Gro: one branch to review, build, and launch testing from. A decomposition into independently shippable PRs is laid out below and can begin whenever review reaches that stage. Earlier discussion: #11636.
Try it (30–45 min, two clones, copy-paste steps)
docs/hot-reload-quickstart.md walks from
git cloneto editing a running F# app — including adding aList.map (fun s -> s.ToUpper())to a live method and watching it apply in place, state preserved. Short version: build this branch, build NatElkins/sdkfsharp-hotreload-watch-v2(a complete .NET CLI with an F#-awaredotnet watch), copy the freshly builtFSharp.Compiler.Service.dllinto the SDK layout, add<OtherFlags>$(OtherFlags) --test:HotReloadDeltas</OtherFlags>to a console app, anddotnet watch run.What works
async, resume-point-stabletask, and generics--test:HotReloadClassStateMachines): adding or removing alet!/do!intaskandbackgroundTask, which emits class-form (reference-type) state machines so the change is anAddInstanceFieldToExistingTypeplus a method update, matching C#.taskSeqand other resumable CEs share the same lowering path. Off by default; flag-off codegen is byte-identical.[<CLIEvent>]eventsinline-annotation change) degrade to the standard rebuild-and-restart flow with a precise diagnostic. Adding or removing a mid-sequencelet!/do!is rude under the default struct state machines, but becomes a supported edit (anAddInstanceFieldToExistingTypeplus a method update, matching C#) under--test:HotReloadClassStateMachinesIsolation and disabling
The feature is designed so that a compile without the flag is indistinguishable from main, and so that the whole feature can be turned off at one point if something goes wrong:
--test:HotReloadDeltas(requires--debug+, incompatible with--optimize+), with class-form resumable state machines a further opt-in behind--test:HotReloadClassStateMachines. Both are off by default; no MSBuild property or SDK behavior changes in this repo.FSharpProjectSnapshot.fsis byte-identical to main (tracked-input staleness lives in the hot reload session layer instead).ICompilerEmitHookinterface with a no-op default. Guard scripts undertests/scripts/(run by the verification gate) enforce that only the intended files consume the hook and that theIlxGenname-generation path and fsi surface stay on their pinned shapes.DOTNET_WATCH_FSHARP_HOTRELOAD=0) that restores stock restart-on-edit behavior.Architecture
FSharpChecker.CreateHotReloadSessionreturns aFSharpHotReloadSessionholding per-project committed snapshots + emit baselines — theDebuggingSession/CommittedSolutionshape from Roslyn, built onFSharpProjectSnapshot(the snapshot contract, not the experimental workspace surface), so it composes with the FSharpWorkspace direction without depending on it. Solution-wide commit/discard semantics, runtime-capability updates, active-statement intake.AddMethodToExistingType,NewTypeDefinition,GenericUpdateMethod, …). Anything unclassifiable fails closed.EmitDifferencereference deltas, with mdv, and against CoreCLRApplyUpdatein runtime tests.Design documentation (rendered, on this branch)
FSharpHotReloadSession, per-project committed snapshots + baselines onFSharpProjectSnapshot, the FSharpWorkspace relationship, determinism pinsEmitDifferencereference templates (EncLog/EncMap shapes per edit kind) and the F# emission matrix incl. every intentional fail-closed caseEditAndContinueCapabilitiesparity) and per-capability gatingScale and review shape
~118 commits, 159 files, ~60k insertions, of which ~34k are tests and ~2.4k docs. The src/ changes are predominantly new self-contained modules (
IlxDeltaEmitter.fs,TypedTreeDiff.fs,HotReloadBaseline.fs, the AbstractIL EnC readers, the delta writer stack). Pre-existing files carry hook callouts plus one behavior-neutral refactor (ilwrite.fsMetadataTablerecord→class, exposing a baseline-row access seam for the delta writer); total deletions across the branch are 241 lines.Proposed path to merging in pieces
The fail-closed design means scope can grow capability by capability — each unsupported case is already a rude edit with a diagnostic, so every intermediate state is a complete, working feature. The natural sequence:
IlxGen/name-generation paths — reviewed on its own)Evidence
ApplyUpdate+ invocation, multi-generation chains, cross-process (disk-started) sessions#Stringsheap layout #19732, Determinism: closure type names differ between --parallelcompilation- and --parallelcompilation+ #19928 (open; flag-on compiles are insulated),$(ParallelCompilation)never reaches the Fsc task — targets pass the item@(ParallelCompilation)instead of the property #19935 (why the pin is compiler-level)./tests/scripts/hot-reload-verify.shKnown limitations / future work
Companion PRs