The Loom language plugin

This module is Loom's adapter to Volar, the editor toolkit the Loom language server is built on. It hands Volar two things for every .loom file: a language plugin and a set of service plugins.

The language plugin projects a .loom file to a virtual-code tree — the source itself as the de dicto root, where Loom serves its own diagnostics and navigation, and the de re code sections beneath it, each embedded in its own language. Volar maps every diagnostic, hover, and definition back to where the author wrote it.

The service plugins are the editor features themselves: diagnostics, hover, completion. The server resolves the project's active languages, runs each one's plugins Effect, and flattens the results into the single list Volar registers.

The virtual-code plugin

loomLanguagePlugin is the LanguagePlugin Volar asks for. It claims every .loom path and projects the file to a virtual-code tree. The de dicto root is the source itself, which Loom's own service checks and navigates; the product sections are embedded codes, which each language service checks against a program of its own. It projects through LoomCompiler, whose compile verb runs the file's composition — discovering the corpus and inlining cross-file ::[…] transclusions — so the editor resolves a corpus the way the tangler does. projectIsolated runs that projection, then marks each composition root as its own module.

The file's text comes through a passive Source. editorSource reads the open file from the snapshot Volar hands over and each sibling from disk. It lists the corpus directory through loomsIn, the walk the compiler and the tangler share, so the editor discovers exactly the corpus that tangle writes. It never calls back into Volar, so running the composition cannot re-enter the projector.

After projecting, the plugin tells Volar what the file depends on, and Volar re-projects the file when any dependency changes. associate reads the corpus the projection reached and calls ctx.getAssociatedScript for every module the file transitively imports. The registration runs on the warm corpus, after the build, so the projection it triggers is a cache hit rather than a re-entrant build. It registers the whole transitive reach, not the direct imports alone, because Volar marks a dependent dirty one hop at a time on a snapshot change — so a change deep in a chain reaches every file that transcludes it only when each has declared the whole chain.

Volar's hooks are synchronous, so the projection runs through Runtime.runSync on the warm runtime built at startup. When a file's own text changes, updateVirtualCode sees the change through changed, evicts the file's build with the compiler's invalidate, then reprojects and re-registers. A reprojection Volar triggered because a dependency changed leaves the build cache alone — the file's own text is unchanged — and the changed dependency, evicted by its own update, rebuilds when the composition runs over it.

A file's build lasts only as long as Volar's script for it. When Volar deletes a script, disposeVirtualCode evicts the file's build too, so the next script at that path builds from the text it is created with. Formatting depends on this. Volar formats a loom through a temporary copy of it, which it deletes once the format is done, and creates again for the next format. A build that outlived the copy would hand the next format the text the copy held last, already formatted, and every format after the first would change nothing.

TypeScript

const editorSource = (uri: URI, snapshot: ts.IScriptSnapshot): Source => ({
  read: (path) =>
    Effect.try({
      try: () =>
        path === uri.fsPath
          ? snapshot.getText(0, snapshot.getLength())
          : readFileSync(path, 'utf8'),
      catch: (cause) => new ReadError({ path, cause }),
    }),
  list: Option.some((dir, project) => Effect.sync(() => loomsIn(dir, project))),
})

const associate = (
  compiler: LoomCompiler['Service'],
  ctx: CodegenContext<URI>,
  entry: string,
): Effect.Effect<void> =>
  compiler.reach(entry).pipe(
    Effect.flatMap((deps) =>
      Effect.forEach(deps, (dep) => Effect.sync(() => ctx.getAssociatedScript(URI.file(dep))), {
        discard: true,
      }),
    ),
  )

const changed = (previous: VirtualCode, next: ts.IScriptSnapshot): boolean =>
  previous.snapshot.getText(0, previous.snapshot.getLength()) !== next.getText(0, next.getLength())

const projectIsolated = (
  compiler: LoomCompiler['Service'],
  uri: URI,
  snapshot: ts.IScriptSnapshot,
  ctx: CodegenContext<URI>,
): Effect.Effect<VirtualCode> =>
  compiler.compile(editorSource(uri, snapshot), uri.fsPath).pipe(
    Effect.tap(() => associate(compiler, ctx, uri.fsPath)),
    Effect.map((tree) => isolateRoots(tree)),
  )

export const loomLanguagePlugin = (
  runtime: ManagedRuntime.ManagedRuntime<LoomCompiler | LoomConfig, never>,
): LanguagePlugin<URI> => ({
  getLanguageId: (uri) => (uri.path.endsWith('.loom') ? 'loom' : undefined),
  createVirtualCode: (uri, languageId, snapshot, ctx) =>
    languageId === 'loom'
      ? runtime.runSync(
          LoomCompiler.pipe(
            Effect.flatMap((compiler) => projectIsolated(compiler, uri, snapshot, ctx)),
          ),
        )
      : undefined,
  updateVirtualCode: (uri, previous, snapshot, ctx) =>
    runtime.runSync(
      LoomCompiler.pipe(
        Effect.flatMap((compiler) =>
          (changed(previous, snapshot)
            ? Effect.asVoid(compiler.invalidate(uri.fsPath))
            : Effect.void
          ).pipe(Effect.andThen(projectIsolated(compiler, uri, snapshot, ctx))),
        ),
      ),
    ),
  disposeVirtualCode: (uri) =>
    runtime.runSync(
      LoomCompiler.pipe(
        Effect.flatMap((compiler) => Effect.asVoid(compiler.invalidate(uri.fsPath))),
      ),
    ),
})

Isolating the roots

A product language service holds each composition root in a program of its own. A root with no import or export of its own is a global script, and two such roots would clash on a shared top-level name once a program holds them both. isolateRoots walks the projected tree and appends moduleMarker, an empty export {}, to every TypeScript-family root, turning each into a module with its own scope. tsLanguages names the TypeScript-family languages the marker applies to — every other language isolates itself or has no notion of a module. The marker carries no source mapping, so it stays invisible, and the plugin appends it once at projection, so a root's snapshot stays stable until its own source changes and Volar never re-parses it on a keystroke elsewhere.

TypeScript

const tsLanguages: ReadonlySet<string> = new Set(['typescript', 'tsx', 'javascript', 'jsx'])

const moduleMarker = '\nexport {}\n'

const isolateRoots = (code: VirtualCode): VirtualCode => ({
  ...code,
  snapshot: tsLanguages.has(code.languageId)
    ? stringSnapshot(code.snapshot.getText(0, code.snapshot.getLength()) + moduleMarker)
    : code.snapshot,
  embeddedCodes: Array.map(code.embeddedCodes ?? [], (child) => isolateRoots(child)),
})

Collecting the service plugins

loomServicePlugins is the collect-and-register pass for one workspace, the one whose root it is handed. The server runs it once for every workspace it meets, so every loom gets the languages its own workspace activates. Before it loads anything, it installs the host runtime on the global: Effect, the service contract, and the editor's TypeScript compiler. A loaded service then reads the host's instances rather than bundling its own and missing the capabilities the host provides.

The pass reads the workspace's active languages and hands them to loadActive. The loader always adds LoomLanguage and ProseLanguage, and loads every other id as a @athrio/loom-service-<id> package from the workspace's own store. The pass indexes the loaded services in the LanguageRegistry and runs each service's plugins Effect with the host's capabilities. A language activated but not installed in the store drops with a warning, and the rest load on.

The pass provides three capabilities. The TypescriptSdk is the editor's own compiler, so analysis matches the user's project. A FrameQuery, built over the warm corpus by frameQueryOver, lets the frame service read a file's Loom diagnostics. A Composition, built by compositionOver, hands a product service the corpus as a filesystem — every root as the code it composes, at the path it composes to — so its analyzer resolves a cross-file import against live composition rather than the output on disk. The editor still hands each service its Products directly, a resolved root with its members folded in; Composition adds only the rest of the corpus. Each service also gets its slice of the workspace's settings, keyed by language id, so it tunes itself from the configuration. A service that fails to build — an external binary that will not start — drops to no plugins rather than taking the others down. The pass dedupes by plugin name, so a plugin two services happen to share registers with Volar once.

frameQueryOver builds the one capability from the runtime. Each reader composes a file's corpus and answers from it: diagnostics returns the open module's Loom health, definition follows an anchor to the section it names, references finds every anchor that names the section under the cursor, rename and renameRange give the spans a rename rewrites and the one the rename box covers, and semanticTokens gives each of the file's tokens its colour. They read synchronously, on the same warm runtime the projection uses, and close over it, so the frame service can call them long after this pass, when the editor asks on a keystroke. format is the one reader that answers with a promise. It runs LoomFormatter over the text the editor sent, with the workspace's active languages, so each section's code reaches the formatter its own workspace installed. compositionOver builds the Composition reader the same way: rootsFor composes a file's corpus and returns every root as a ComposedFile, on that same warm runtime.

TypeScript

const dedupeByName = (
  plugins: ReadonlyArray<LanguageServicePlugin>,
): ReadonlyArray<LanguageServicePlugin> =>
  Array.fromIterable(new Map(plugins.map((plugin) => [plugin.name ?? plugin, plugin])).values())

const collect = (
  active: ReadonlyArray<LanguageService>,
  tsdk: typeof import('typescript'),
  settings: Record<string, Record<string, unknown>>,
  frameQuery: FrameQueryApi,
  composition: CompositionApi,
): Effect.Effect<ReadonlyArray<LanguageServicePlugin>> =>
  Effect.gen(function* () {
    const registry = yield* LanguageRegistry
    const collected = yield* Effect.forEach(registry.all, (service) =>
      service
        .plugins({ settings: settings[service.id] ?? {} })
        .pipe(Effect.catchCause(() => Effect.succeed([] as ReadonlyArray<LanguageServicePlugin>))),
    )
    return dedupeByName(collected.flat())
  }).pipe(
    Effect.provide(LanguageRegistry.layer),
    Effect.provide(Layer.succeed(ActiveLanguages, { all: active })),
    Effect.provideService(TypescriptSdk, tsdk),
    Effect.provideService(FrameQuery, frameQuery),
    Effect.provideService(Composition, composition),
  )

const frameQueryOver = (
  runtime: ManagedRuntime.ManagedRuntime<LoomCompiler | LoomFormatter, never>,
  languages: ReadonlyArray<LanguageService>,
): FrameQueryApi => ({
  diagnostics: (path) =>
    runtime.runSync(LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.diagnose(path)))),
  definition: (path, offset) =>
    runtime.runSync(
      LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.definition(path, offset))),
    ),
  references: (path, offset) =>
    runtime.runSync(
      LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.references(path, offset))),
    ),
  rename: (path, offset) =>
    runtime.runSync(LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.rename(path, offset)))),
  renameRange: (path, offset) =>
    runtime.runSync(
      LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.renameRange(path, offset))),
    ),
  navigationRange: (path, offset) =>
    runtime.runSync(
      LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.navigationRange(path, offset))),
    ),
  semanticTokens: (path) =>
    runtime.runSync(LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.semanticTokens(path)))),
  format: (path, text) =>
    runtime.runPromise(
      LoomFormatter.pipe(Effect.flatMap((formatter) => formatter.format(path, text, languages))),
    ),
})

const compositionOver = (
  runtime: ManagedRuntime.ManagedRuntime<LoomCompiler, never>,
): CompositionApi => ({
  rootsFor: (path) =>
    runtime.runSync(LoomCompiler.pipe(Effect.flatMap((compiler) => compiler.composition(path)))),
})

export const loomServicePlugins = (
  runtime: ManagedRuntime.ManagedRuntime<LoomCompiler | LoomFormatter, never>,
  tsdk: typeof import('typescript'),
  workspace: string,
): Effect.Effect<ReadonlyArray<LanguageServicePlugin>, never, LoomConfig> =>
  Effect.gen(function* () {
    const config = yield* LoomConfig
    const manifest = yield* config.manifest(workspace)
    const languages = Object.keys(manifest?.languages ?? {})
    const settings = manifest?.settings ?? {}
    yield* Effect.sync(() => installHostRuntime(tsdk))
    const active = yield* loadActive(languages, workspace)
    return yield* collect(
      active,
      tsdk,
      settings,
      frameQueryOver(runtime, active),
      compositionOver(runtime),
    )
  })

What the plugin draws on

The plugin draws on Volar for its contracts — LanguagePlugin and LanguageServicePlugin — and its CodegenContext and VirtualCode, which the per-call source and the cache check read. Effect captures the runtime and runs the collection pass. From the pipeline it takes LoomCompiler, which projects and caches the corpus, LoomFormatter, which formats a file, the Source seam with its ReadError, and loomsIn, which lists the looms a project holds under a folder. From @athrio/loom-lang-services it takes the LanguageService contract, the FrameQuery capability the collect pass provides, the LanguageRegistry that indexes the active set, and loadActive, which resolves a workspace's active services — LoomLanguage and each installed @athrio/loom-service-<id> package. installHostRuntime lends those loaded services the host's runtime. From @athrio/loom-config it reads the workspace's active languages out of the compiled manifest. URI keys a script for getAssociatedScript, and Bun's fs reads an unopened import.

TypeScript

import type { CodegenContext, LanguagePlugin, VirtualCode } from '@volar/language-core'
import type { LanguageServicePlugin } from '@volar/language-service'
import type {} from '@volar/typescript'
import { Array, Effect, Layer, ManagedRuntime, Option } from 'effect'
import { readFileSync } from 'node:fs'
import type * as ts from 'typescript'
import { URI } from 'vscode-uri'
import { ReadError, type Source } from '#ast/LoomCorpusAstBuilder'
import { LoomCompiler, stringSnapshot } from './LoomCompiler'
import { LoomFormatter } from './LoomFormatter'
import { loomsIn } from './LoomProject'
import { LoomConfig } from '@athrio/loom-config/LoomConfig'
import {
  Composition,
  type CompositionApi,
  FrameQuery,
  type FrameQueryApi,
  type LanguageService,
  TypescriptSdk,
} from '@athrio/loom-lang-services/LanguageService'
import { ActiveLanguages, LanguageRegistry } from '@athrio/loom-lang-services/LanguageRegistry'
import { loadActive } from '@athrio/loom-lang-services/LanguageLoader'
import { installHostRuntime } from '@athrio/loom-lang-services/Runtime'

The module

TypeScript

::[What the plugin draws on]

::[The virtual-code plugin]

::[Isolating the roots]

::[Collecting the service plugins]

Testing the plugin

Volar formats a loom through a temporary copy of it, a script at the loom's path with .tmp appended. It creates the copy with the loom's text, updates it with each pass's formatted text, and deletes it once the format is done. The next format creates the copy again, from the text the editor holds then.

The case replays that order through Volar's own createLanguage, over a runtime built the way the server builds its own. It creates the copy with a loom as written, updates it to the formatted loom, deletes it, and creates it again with the loom as written. The new script's root must hold the text it was created with, not the formatted text the deleted copy held last. The copy sits in a folder of its own, so its build reads no other loom.

TypeScript

import { describe, expect, it } from 'bun:test'
import { createLanguage, type VirtualCode } from '@volar/language-core'
import { Layer, ManagedRuntime } from 'effect'
import { mkdtempSync, realpathSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { URI } from 'vscode-uri'
import { LoomConfig } from '@athrio/loom-config/LoomConfig'
import { DocumentSource, LoomCompiler, stringSnapshot } from '../src/LoomCompiler'
import { loomLanguagePlugin } from '../src/LoomLanguagePlugin'
import { PackageConfig } from '../src/PackageConfig'

const runtime = ManagedRuntime.make(
  Layer.mergeAll(LoomCompiler.layer, LoomConfig.layer).pipe(
    Layer.provide(DocumentSource.layer),
    Layer.provide(PackageConfig.layer),
    Layer.provide(LoomConfig.layer),
  ),
)

const lines = (...rows: ReadonlyArray<string>): string => `${rows.join('\n')}\n`

const loomOf = (code: string): string =>
  lines('---', 'Language: TypeScript', '---', '', '# Answer', '', '=>', '', code)

const textOf = (code: VirtualCode | undefined): string =>
  code === undefined ? '' : code.snapshot.getText(0, code.snapshot.getLength())

describe('LoomLanguagePlugin — a script Volar deletes', () => {
  it('builds a new script at the same path from the text it is given', () => {
    const language = createLanguage<URI>([loomLanguagePlugin(runtime)], new Map(), () => {})
    const folder = realpathSync(mkdtempSync(join(tmpdir(), 'loom-plugin-')))
    const copy = URI.file(join(folder, 'answer.loom.tmp'))
    const written = loomOf('const  answer  =  42')
    language.scripts.set(copy, stringSnapshot(written), 'loom')
    language.scripts.set(copy, stringSnapshot(loomOf('const answer = 42')), 'loom')
    language.scripts.delete(copy)
    const again = language.scripts.set(copy, stringSnapshot(written), 'loom')
    expect(textOf(again?.generated?.root)).toBe(written)
  })
})

The test file

TypeScript

::[Testing the plugin]