Tangling to disk

LoomTangler is the filesystem surface of the product. The compiler resolves a .loom file's {path} sinks to their content and target; the tangler writes them to disk. tangle takes a path, finds the looms under it, and writes every sink each one resolves — skipping the chapters a book already places, so every file is written once. It records each file it writes in the lock, and reads the lock back to find the files that lost their host: the orphans it can list or prune. watch keeps that output current while an app runs, re-tangling as the looms change, so a dev server already watching the tangled files reacts to a loom edit as it would to a hand-edit. It is the half of the pipeline the editor never runs — the editor projects to virtual code, the tangler emits real files.

The tangler service

LoomTangler holds the compiler, the filesystem, and the package configuration. writeFile first checks whether the target already holds the tangled content, and only touches the file when it does not. A rerun that changes nothing then leaves the file's modified time alone, so a bundler's dev server watching it sees a real change only when the tangled content actually changed — not on every tangle regardless. When the content does differ, writeFile creates the target's directory and writes it, returning the file either way. tangleFile asks the compiler to resolve one file's sinks — which fails with a TangleError if any module in the corpus carries an error — and writes each.

A path names either a folder or one loom. projectOf finds the project a path belongs to, through the package configuration. loomsOf resolves a path to the looms it covers. For a folder, those are the looms the project holds beneath it, listed by loomsIn under the same rule the compiler and the editor follow. For a file, it is the file itself, even a loom the project leaves out, because a person who names a loom means that loom.

tangle tangles each loom a path covers and records the result. It writes every sink, then folds the paths it produced into the lock at the workspace root, so a file enters the lock the first time it is tangled and every file is written once.

The lock records only the project's own files, and recordsFor decides which tangles produce them. A tangle of a folder does, since the walk lists only the looms the project holds. So does a tangle of a loom the project holds, or of a loom that belongs to no project, which no rule leaves out. A loom the project leaves out still tangles when it is named, but its files stay out of the lock, and the lock stays as it was. orphans then never lists those files, and a prune never deletes them.

The lock lives on disk beside the configuration. readLock reads loom.lock and decodes it, answering with the empty lock when the file is absent or malformed, so a first tangle starts from nothing rather than failing. writeLock writes it back as formatted JSON. workspaceOf finds the workspace root a path belongs to by walking up from a file's own directory, or from a directory itself, and the lock lives in that root's .loom folder. A loom that belongs to no workspace is its own root, so the lock of its last tangle lands in a .loom folder beside it. That folder holds no configuration, so it makes no workspace.

orphans names the files that lost their host and touches no disk. The lock covers the whole workspace, so orphan-finding does too: from a starting directory it finds the workspace root, tangles every loom the project holds beneath it, reads the lock, and returns the recorded paths the produced set no longer holds. A narrower scope would mislead — a file produced elsewhere in the workspace would look lost, and a pipe into rm would delete it. prune goes one step further: it removes each orphan and rewrites the lock to exactly what the build produces. Both resolve the produced set through the compiler, so a broken corpus fails them with its TangleError before either reads a lock or deletes a file — a prune never works from an incomplete build.

TypeScript

const isStorePath = (name: string): boolean =>
  name.split('/').some((segment) => storeDirs.has(segment))

export class LoomTangler extends Context.Service<LoomTangler>()('LoomTangler', {
  make: Effect.gen(function* () {
    const compiler = yield* LoomCompiler
    const fs = yield* FileSystem.FileSystem
    const config = yield* PackageConfig

    const matchesDisk = (file: TangledFile): Effect.Effect<boolean> =>
      fs.readFileString(file.path).pipe(Effect.option, Effect.map(Option.contains(file.content)))

    const writeFile = (file: TangledFile): Effect.Effect<TangledFile> =>
      matchesDisk(file).pipe(
        Effect.flatMap((same) =>
          same
            ? Effect.void
            : fs
                .makeDirectory(dirname(file.path), { recursive: true })
                .pipe(Effect.andThen(fs.writeFileString(file.path, file.content))),
        ),
        Effect.orDie,
        Effect.as(file),
      )

    const tangleFile = (entry: Path): Effect.Effect<ReadonlyArray<TangledFile>, TangleError> =>
      compiler.tangle(entry).pipe(Effect.flatMap((files) => Effect.forEach(files, writeFile)))

    const projectOf = (path: Path): Effect.Effect<Option.Option<LoomProject>> =>
      config.resolve(path).pipe(Effect.map(projectIn))

    const loomsOf = (path: Path): Effect.Effect<ReadonlyArray<Path>> =>
      fs.stat(path).pipe(
        Effect.orDie,
        Effect.flatMap((info) =>
          info.type === 'Directory'
            ? projectOf(path).pipe(Effect.map((project) => loomsIn(path, project)))
            : Effect.succeed<ReadonlyArray<Path>>([path]),
        ),
      )

    const recordsFor = (path: Path): Effect.Effect<boolean> =>
      fs.stat(path).pipe(
        Effect.orDie,
        Effect.flatMap((info) =>
          info.type === 'Directory'
            ? Effect.succeed(true)
            : projectOf(path).pipe(
                Effect.map(
                  Option.match({
                    onNone: () => true,
                    onSome: (project) => holds(project, path),
                  }),
                ),
              ),
        ),
      )

    const workspaceOf = (path: Path): Effect.Effect<Path> =>
      fs.stat(path).pipe(
        Effect.orDie,
        Effect.map((info) => workspaceRoot(info.type === 'Directory' ? path : dirname(path))),
      )

    const lockFile = (root: Path): Path => resolvePath(root, '.loom', 'loom.lock')

    const readLock = (root: Path): Effect.Effect<LoomLock> =>
      fs.readFileString(lockFile(root)).pipe(
        Effect.flatMap((text) => Effect.try(() => JSON.parse(text) as unknown)),
        Effect.flatMap(Schema.decodeUnknownEffect(LoomLockSchema)),
        Effect.orElseSucceed(() => emptyLock),
      )

    const writeLock = (root: Path, lock: LoomLock): Effect.Effect<void> =>
      fs
        .makeDirectory(dirname(lockFile(root)), { recursive: true })
        .pipe(
          Effect.andThen(fs.writeFileString(lockFile(root), JSON.stringify(lock, null, 2) + '\n')),
          Effect.orDie,
        )

    const tangle = (path: Path): Effect.Effect<ReadonlyArray<TangledFile>, TangleError> =>
      Effect.gen(function* () {
        const looms = yield* loomsOf(path)
        const written = (yield* Effect.forEach(looms, tangleFile)).flat()
        if (yield* recordsFor(path)) {
          const root = yield* workspaceOf(path)
          const lock = yield* readLock(root)
          yield* writeLock(
            root,
            recorded(
              lock,
              Array.map(written, (file) => relative(root, file.path)),
            ),
          )
        }
        return written
      })

    const orphanState = (
      from: Path,
    ): Effect.Effect<
      {
        readonly root: Path
        readonly produced: ReadonlyArray<string>
        readonly orphans: ReadonlyArray<string>
      },
      TangleError
    > =>
      Effect.gen(function* () {
        const root = workspaceRoot(from)
        const looms = loomsIn(root, yield* projectOf(root))
        const files = (yield* Effect.forEach(looms, (loom) => compiler.tangle(loom))).flat()
        const produced = Array.map(files, (file) => relative(root, file.path))
        const lock = yield* readLock(root)
        return { root, produced, orphans: orphansOf(lock, new Set(produced)) }
      })

    const orphans = (from: Path): Effect.Effect<ReadonlyArray<Path>, TangleError> =>
      orphanState(from).pipe(
        Effect.map(({ root, orphans }) => Array.map(orphans, (rel) => resolvePath(root, rel))),
      )

    const prune = (from: Path): Effect.Effect<ReadonlyArray<Path>, TangleError> =>
      Effect.gen(function* () {
        const { root, produced, orphans } = yield* orphanState(from)
        yield* Effect.forEach(orphans, (rel) =>
          fs.remove(resolvePath(root, rel), { force: true }).pipe(Effect.orDie),
        )
        yield* writeLock(root, pruned(produced))
        return Array.map(orphans, (rel) => resolvePath(root, rel))
      })

    ::[Keeping the tangle current]

    return { tangle, orphans, prune, watch }
  }),
}) {
  static readonly layer = Layer.effect(this, this.make).pipe(Layer.provide(LoomCompiler.layer))
}

Keeping the tangle current

watch keeps a path's tangled output current as the source looms change. A one-shot tangle emits the files once. After it, a .loom edit changes nothing on disk until tangle runs again, so a dev server watching the tangled TypeScript sees a loom edit only when someone re-runs the command by hand. watch runs it for them. It re-tangles on every save, so the dev server reacts to a loom edit exactly as it would to a hand-edit of the output.

watchRoot resolves the directory to watch — the path itself when the path is a directory, or the corpus directory the file sits in. Watching the whole corpus directory is deliberate. The entry composes sections from its sibling looms, so watch must re-tangle when any of those siblings changes, not only when the entry itself does. isLoomEvent keeps a change to a .loom file and drops the rest — an editor's swap file, a bare directory event, anything under a generated store.

On each batch watch evicts the changed files from the build cache through compiler.invalidate, then re-tangles the path. Composition re-inlines a changed section wherever another file draws it in, so evicting the one file that changed is enough. The matchesDisk guard writes only the outputs that truly changed, so the dev server's watcher fires on a real change, never on a tangle that produced identical bytes.

watch streams a Result for every pass, the first before any change. It tangles once immediately, so the output is current before watching begins, then once per batch of changes as they arrive. A pass that a broken corpus fails carries its TangleError as the Result's failure rather than the stream's, so a half-finished edit prints its error and the watch goes on. The one-shot command exits on that error. The watch cannot, or it would die on the first typo.

A single save can surface as several filesystem events, and a project-wide edit touches many looms at once. So watch collects the events that arrive within batchWindow, a short span, and re-tangles once for the whole batch rather than once per event. batchLimit caps a batch well above any real burst, so the window, not the count, is what closes it. It treats a create, a change, and a delete alike, each one a reason to re-tangle, because an editor that saves atomically — writing a temporary file and renaming it over the original — surfaces as a create, and watching only for a change would miss it.

TypeScript

const batchWindow = '50 millis'
const batchLimit = 1000

const isLoomEvent = (event: FileSystem.WatchEvent): boolean =>
  event.path.endsWith('.loom') && !isStorePath(event.path)

const watchRoot = (path: Path): Effect.Effect<Path> =>
  fs.stat(path).pipe(
    Effect.orDie,
    Effect.flatMap((info) =>
      info.type === 'Directory'
        ? Effect.succeed(path)
        : config
            .resolve(path)
            .pipe(
              Effect.map(({ corpusDir }) =>
                Option.getOrElse(Option.fromNullishOr(corpusDir), () => dirname(path)),
              ),
            ),
    ),
  )

const watch = (path: Path): Stream.Stream<Result.Result<ReadonlyArray<TangledFile>, TangleError>> =>
  Stream.unwrap(
    watchRoot(path).pipe(
      Effect.map((root) =>
        Stream.concat(
          Stream.fromEffect(Effect.result(tangle(path))),
          fs.watch(root).pipe(
            Stream.filter(isLoomEvent),
            Stream.groupedWithin(batchLimit, batchWindow),
            Stream.mapEffect((events) =>
              Effect.forEach(
                Array.dedupe(Array.map(events, (event) => resolvePath(root, event.path))),
                (changed) => compiler.invalidate(changed),
                { discard: true },
              ).pipe(Effect.andThen(Effect.result(tangle(path)))),
            ),
            Stream.orDie,
          ),
        ),
      ),
    ),
  )

What this module draws on

It draws on LoomCompiler to resolve each file's sinks to content and target, gating a broken corpus with its TangleError, and to evict a changed file from the build cache through compiler.invalidate. From LoomProject come projectIn, holds, and loomsIn, the rule that decides which looms a walk tangles and whose files the lock records, and storeDirs, the store folders a watch ignores a change in. Effect's FileSystem, with Bun's path module, tells a folder from a file, reads a target back to compare it against what tangle would write, writes each file, reads and writes the lock, and watches the corpus for changes. PackageConfig resolves a path's settings, which carry its project, and the corpus directory a file sits in, the scope watch watches. Stream carries the watch as a sequence of tangle passes, and Result carries each pass's outcome — a set of written files, or a TangleError. Option carries whether a target already exists, since a first tangle finds nothing there to compare against. Schema decodes the lock from disk. From LoomLock come the lock's shape and the set algebra over it — orphansOf, recorded, and pruned, with emptyLock for a workspace that has tangled nothing. workspaceRoot, from the language store, finds the workspace a path belongs to, whose .loom folder holds the lock. Effect supplies the array and effect primitives. TangledFile is the resolved sink the compiler hands back, and Path a file path.

TypeScript

import { Array, Context, Effect, FileSystem, Layer, Option, Result, Schema, Stream } from 'effect'
import { dirname, relative, resolve as resolvePath } from 'node:path'
import { type Path } from '@athrio/loom-ast/LoomCorpusAst'
import { workspaceRoot } from '@athrio/loom-lang-services/LoomStore'
import { LoomCompiler, type TangledFile, type TangleError } from './LoomCompiler'
import { PackageConfig } from './PackageConfig'
import { holds, loomsIn, projectIn, storeDirs, type LoomProject } from './LoomProject'
import { emptyLock, orphansOf, pruned, recorded, LoomLockSchema, type LoomLock } from './LoomLock'

The module

TypeScript

::[What this module draws on]

::[The tangler service]