The standalone executable

Loom composes any language, but the npm package asks for a JavaScript runtime before it runs at all. A person who writes Python or Rust should not need Node or Bun to tangle a loom. So Loom also ships as a standalone executable: one file that runs on its own, with no runtime and no package manager installed beside it.

Bun builds that file. bun build --compile takes a JavaScript program and writes an executable that holds both the program and Bun's own runtime. Loom compiles the bundle Vite writes to dist/main.js, the very bundle the npm package ships, rather than the CLI's TypeScript source. Compiling from the source fails, because a library under the Markdown service, vscode-markdown-languageservice, imports a default export from the ES-module build of vscode-uri, and that build has none. Vite's bundle has already settled that import, as the chapter on bundling the CLI explains.

The executable is large, between seventy and a hundred and twenty megabytes depending on the platform, since it carries a whole runtime. The release builds one for each platform Loom supports: macOS on Apple silicon and on Intel, Linux on x64 and on ARM, and Windows on x64. Each is built and tested on a machine of its own platform, so every executable the release ships has run its smoke test where it will run.

What changes inside an executable

The executable runs the same program as the npm package, but four things that program expects are missing from around it. A person who downloads the executable should need nothing else, no compiler and no further download, so each gap is closed inside the executable itself.

Node's resolver cannot find a package by its bare name inside an executable, though it finds a file at a known path. So the loader reads a language service's own manifest to find the file it imports, rather than asking the resolver.

There is no npm to install a language service with. The executable carries Bun, and Bun answers as itself when the BUN_BE_BUN variable is set, so the store runs the executable a second time as Bun to install a package.

loom helix and loom nvim give an editor Loom's grammar as a compiled library, which the npm package builds with the system's C compiler. The executable carries that library already built for its platform, so the commands write it out and need no compiler.

Nothing beside the executable holds TypeScript either, which the language server and loom format both need. The executable carries TypeScript 6, packed into one gzipped file, and unpacks it into the workspace's store the first time either finds none there.

Compiling the executable

The package's compile script runs this program, which builds the executable for the machine it runs on. The release runs it once on each platform's own machine. It works in three steps, and a fourth on macOS, and each writes into build, a folder Git ignores.

First it compiles the grammar into a library, under the name loom helix writes it by, loom.so or loom.dll on Windows. The compiler call is the one loom helix makes, and CC names another compiler, as it does there.

Second it packs TypeScript into typescript.json.gz, the shape the store unpacks: the package's package.json, the compiler in lib/typescript.js, and the declaration files of the standard library, lib.*.d.ts. That is about thirteen megabytes of text, and gzip brings it to about two. The TypeScript it packs is the workspace's own, the release of TypeScript 6 the package's devDependencies name.

Third it compiles dist/main.js into build/loom, with the library and the packed TypeScript as two more files beside the program. Bun embeds a file it cannot run as code under its name, and --asset-naming keeps that name exactly, so the executable finds each by the name it was given.

run spawns each command and fails with CompileError, naming the command, when one exits non-zero or cannot start. The program reports the error and exits non-zero, so a release never ships an executable built from a step that failed.

TypeScript

import { Array, Console, Data, Effect, pipe } from 'effect'
import { NodeRuntime } from '@effect/platform-node'
import { mkdirSync, readdirSync, readFileSync, writeFileSync } from 'node:fs'
import { createRequire } from 'node:module'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { gzipSync } from 'node:zlib'

const here = dirname(fileURLToPath(import.meta.url))
const build = join(here, 'build')

class CompileError extends Data.TaggedError('CompileError')<{
  readonly command: string
}> {}

const run = (command: ReadonlyArray<string>): Effect.Effect<void, CompileError> =>
  Effect.tryPromise({
    try: () => Bun.spawn([...command], { cwd: here, stdout: 'inherit', stderr: 'inherit' }).exited,
    catch: () => new CompileError({ command: command.join(' ') }),
  }).pipe(
    Effect.flatMap((code) =>
      code === 0 ? Effect.void : Effect.fail(new CompileError({ command: command.join(' ') })),
    ),
  )

The grammar's sources sit in the tree-sitter-loom package beside this one.

TypeScript

const grammarLibrary = process.platform === 'win32' ? 'loom.dll' : 'loom.so'

const grammarSource = join(here, '..', 'tree-sitter-loom', 'src')

const compiledGrammar = run([
  process.env.CC ?? 'cc',
  '-shared',
  '-fPIC',
  '-O2',
  '-I',
  grammarSource,
  join(grammarSource, 'parser.c'),
  join(grammarSource, 'scanner.c'),
  '-o',
  join(build, grammarLibrary),
])

typeScriptFiles names the files to pack, each by its path in the package.

TypeScript

const typeScriptDir = dirname(createRequire(import.meta.url).resolve('typescript/package.json'))

const typeScriptFiles = (): ReadonlyArray<string> => [
  'package.json',
  'lib/typescript.js',
  ...pipe(
    readdirSync(join(typeScriptDir, 'lib')),
    Array.filter((name) => name.startsWith('lib.') && name.endsWith('.d.ts')),
    Array.map((name) => `lib/${name}`),
  ),
]

const packedTypeScript = Effect.sync(() =>
  writeFileSync(
    join(build, 'typescript.json.gz'),
    gzipSync(
      JSON.stringify({
        files: Object.fromEntries(
          Array.map(typeScriptFiles(), (file) => [
            file,
            readFileSync(join(typeScriptDir, file), 'utf8'),
          ]),
        ),
      }),
      { level: 9 },
    ),
  ),
)

The compile runs on the Bun that runs this program, so it needs no bun on PATH.

TypeScript

const compiledExecutable = run([
  process.execPath,
  'build',
  join(here, 'dist', 'main.js'),
  join(build, grammarLibrary),
  join(build, 'typescript.json.gz'),
  '--compile',
  '--asset-naming=[name].[ext]',
  '--outfile',
  join(build, 'loom'),
])

On macOS a fourth step signs the executable again. Bun builds it from its own runtime, which comes signed, and then writes Loom's program into the file after that signature was made. The signature no longer matches the file, so codesign --verify rejects it. macOS still runs the file, since it checks only the pages of code it loads, but a stricter check by a security tool or a managed Mac would refuse it. signedExecutable signs the finished file ad hoc, with no certificate, as codesign --sign - does, and names it loom. Signing with a Developer ID, which lets Gatekeeper open a file downloaded through a browser, would replace that - with the identity.

TypeScript

const signedExecutable =
  process.platform === 'darwin'
    ? run(['codesign', '--force', '--sign', '-', '--identifier', 'loom', join(build, 'loom')])
    : Effect.void

const compile = Effect.gen(function* () {
  yield* Effect.sync(() => mkdirSync(build, { recursive: true }))
  yield* compiledGrammar
  yield* packedTypeScript
  yield* compiledExecutable
  yield* signedExecutable
  yield* Console.log(`compile: wrote ${join(build, 'loom')} with ${grammarLibrary} and TypeScript`)
})

NodeRuntime.runMain(
  compile.pipe(
    Effect.catchTag('CompileError', (error) =>
      Console.error(`compile: command failed — ${error.command}`).pipe(
        Effect.andThen(
          Effect.sync(() => {
            process.exitCode = 1
          }),
        ),
      ),
    ),
  ),
)

A smoke test for the executable

The tests of the CLI run its source under Bun, and none of these four differences shows there. So the executable has a test of its own, a smoke test, which runs the compiled file the way a person runs it and checks that each of its commands still answers.

The test runs build/loom, which the package's compile script writes. Each case runs the executable in a folder of its own, in an environment bareEnv strips of two things. Its PATH is an empty folder, so the executable finds no npm, bun, node or C compiler to lean on. Its package registry, which NPM_CONFIG_REGISTRY names, is an address where nothing answers, so a download it did not mean to make fails the case. A case that serves a registry or a release of its own names that address instead. Every other variable passes through, since Windows cannot open a network connection without the ones that name its system folders.

run spawns the executable and collects its exit status and both its output streams. It spawns without blocking, because a case may serve the executable a registry or a release from inside the test process, and a blocked process answers no request. A case checks the exit status together with the standard error, so a command that fails shows its own reason in the test's report. The registries and releases a case serves listen on 127.0.0.1 by number, since localhost may resolve to an address they do not listen on.

TypeScript

const cli = resolve(dirname(fileURLToPath(import.meta.url)), '..')

const binary = resolve(cli, 'build', process.platform === 'win32' ? 'loom.exe' : 'loom')

const scratch = (name: string): string =>
  realpathSync(mkdtempSync(join(tmpdir(), `loom-smoke-${name}-`)))

const emptyPath = scratch('path')

const nowhere = 'http://127.0.0.1:9/'

const bareEnv = (extra: Record<string, string> = {}): Record<string, string> => ({
  ...Object.fromEntries(
    Object.entries(process.env).filter(
      (entry): entry is [string, string] =>
        entry[1] !== undefined && entry[0].toUpperCase() !== 'PATH',
    ),
  ),
  PATH: emptyPath,
  NPM_CONFIG_REGISTRY: nowhere,
  ...extra,
})

const run = async (
  args: ReadonlyArray<string>,
  options: {
    readonly cwd: string
    readonly env: Record<string, string>
    readonly executable?: string
  },
) => {
  const child = Bun.spawn([options.executable ?? binary, ...args], {
    cwd: options.cwd,
    env: options.env,
    stdout: 'pipe',
    stderr: 'pipe',
  })
  const [stdout, stderr, status] = await Promise.all([
    new Response(child.stdout).text(),
    new Response(child.stderr).text(),
    child.exited,
  ])
  return { status, stdout, stderr }
}

const SMOKE_TIMEOUT = 60_000

A workspace to run it in

A case needs a workspace, a folder with a .loom/config.yaml, and often a loom inside it. A loom quoted inside this loom would be read as part of it, so lines builds each fixture from quoted lines and anchor spells an anchor in two pieces, as the tests of the CLI do.

TypeScript

const lines = (...rows: ReadonlyArray<string>): string => `${rows.join('\n')}\n`

const anchor = (name: string): string => `::${'['}${name}]`

const workspace = (...config: ReadonlyArray<string>): string => {
  const dir = scratch('workspace')
  mkdirSync(join(dir, '.loom'), { recursive: true })
  writeFileSync(join(dir, '.loom', 'config.yaml'), lines(...config))
  return dir
}

Tangling with nothing installed

The first cases are the ones every user meets. The executable reports this Loom's version. It tangles a loom whose code composes two sections into hello.ts, in a folder with no node_modules at all.

TypeScript

const helloLoom = lines(
  '---',
  'Language: TypeScript',
  '---',
  '',
  '# Hello [hello.ts]',
  '',
  '=>',
  '',
  anchor('Greeting'),
  '',
  "console.log(greet('world'))",
  '',
  '# Greeting',
  '',
  '=>',
  '',
  "const greet = (name: string): string => 'Hello, ' + name",
)

describe('the executable — tangling', () => {
  it(
    'reports its version',
    async () => {
      const { status, stdout, stderr } = await run(['--version'], { cwd: scratch('version'), env: bareEnv() })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain(`loom v${serviceVersion}`)
    },
    SMOKE_TIMEOUT,
  )

  it(
    'tangles a loom in a folder with no node_modules',
    async () => {
      const dir = workspace('corpus: .', 'primary: typescript')
      writeFileSync(join(dir, 'hello.loom'), helloLoom)
      const { status, stderr } = await run(['tangle', 'hello.loom'], { cwd: dir, env: bareEnv() })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      const tangled = readFileSync(join(dir, 'hello.ts'), 'utf8')
      expect(tangled).toContain("const greet = (name: string): string => 'Hello, ' + name")
      expect(tangled).toContain("console.log(greet('world'))")
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )
})

A signature macOS accepts

On macOS the compile signs the executable again, so that its signature covers the whole file. The case asks codesign to verify that signature strictly, the check a security tool or a managed Mac would make. Windows and Linux carry no such signature, so the case runs on macOS alone.

TypeScript

describe.skipIf(process.platform !== 'darwin')('the executable — signature', () => {
  it('carries a signature that codesign accepts', () => {
    const check = Bun.spawnSync(['codesign', '--verify', '--strict', binary], { stderr: 'pipe' })
    expect({ status: check.exitCode, stderr: check.stderr.toString() }).toMatchObject({
      status: 0,
    })
  })
})

A language service from the store

A case that needs a language service stands one in the store by hand, so it needs no release of the real service. The stand-in answers for TypeScript and formats by squeezing every double space to one. It reads Effect from the runtime the host lends, as a published service does, and declares this Loom's version, so nothing warns that it is stale.

writeStandIn writes the stand-in's files into a folder, and installStandIn writes them where the store keeps the TypeScript service. The manifest names the entry only in exports, under the key ".", the way the published service names it, so the case proves the loader finds an entry from a manifest.

TypeScript

const standIn = lines(
  `const { Effect } = globalThis.${runtimeKey}.modules.effect`,
  'export default {',
  "  id: 'typescript',",
  "  displayName: 'TypeScript',",
  "  extensions: ['.ts'],",
  '  plugins: () => Effect.succeed([]),',
  "  format: (request) => Effect.succeed(request.code.replaceAll('  ', ' ')),",
  '}',
)

const writeStandIn = (folder: string): void => {
  mkdirSync(join(folder, 'dist'), { recursive: true })
  writeFileSync(
    join(folder, 'package.json'),
    JSON.stringify({
      name: servicePackage('typescript'),
      version: serviceVersion,
      type: 'module',
      exports: { '.': './dist/index.js' },
    }),
  )
  writeFileSync(join(folder, 'dist', 'index.js'), standIn)
}

const installStandIn = (root: string): void =>
  writeStandIn(join(root, '.loom', 'services', 'node_modules', servicePackage('typescript')))

loom update on a service the store already holds at this Loom's version leaves it alone and says so, and it never runs a package manager to find that out.

A service the store lacks must arrive through the executable acting as Bun, with no package manager on PATH. registryServer stands in for npm's registry. It describes one package, the stand-in at this Loom's version, and serves it as the tarball a registry serves, with the checksum Bun checks the download against. standInTarball packs that tarball with tar, the stand-in's files under a package folder as npm packs them. It names both by relative paths from inside the folder, since GNU tar reads a Windows path such as C:\… as the name of a remote host.

TypeScript

const standInTarball = (): Uint8Array => {
  const dir = scratch('tarball')
  writeStandIn(join(dir, 'package'))
  Bun.spawnSync(['tar', '-czf', 'service.tgz', 'package'], { cwd: dir })
  const packed = new Uint8Array(readFileSync(join(dir, 'service.tgz')))
  rmSync(dir, { recursive: true, force: true })
  return packed
}

const registryServer = (tarball: Uint8Array) =>
  Bun.serve({
    hostname: '127.0.0.1',
    port: 0,
    fetch: (request) => {
      const name = servicePackage('typescript')
      const path = decodeURIComponent(new URL(request.url).pathname)
      if (path === `/${name}`)
        return Response.json({
          name,
          'dist-tags': { latest: serviceVersion },
          versions: {
            [serviceVersion]: {
              name,
              version: serviceVersion,
              dist: {
                tarball: new URL('/service.tgz', request.url).href,
                integrity: `sha512-${createHash('sha512').update(tarball).digest('base64')}`,
              },
            },
          },
        })
      if (path === '/service.tgz') return new Response(tarball)
      return new Response('not found', { status: 404 })
    },
  })


describe('the executable — languages', () => {
  it(
    'finds a current service in the store and leaves it alone',
    async () => {
      const dir = workspace('corpus: .', 'languages:', '  typescript: {}', 'primary: typescript')
      installStandIn(dir)
      const { status, stdout, stderr } = await run(['update', 'typescript'], { cwd: dir, env: bareEnv() })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain('is already in the store')
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )

  it(
    'installs a service the store lacks, acting as its own package manager',
    async () => {
      const dir = workspace('corpus: .', 'languages:', '  typescript: {}', 'primary: typescript')
      const registry = registryServer(standInTarball())
      const { status, stdout, stderr } = await run(['update', 'typescript'], {
        cwd: dir,
        env: bareEnv({ NPM_CONFIG_REGISTRY: registry.url.href }),
      })
      registry.stop()
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain(`installed ${servicePackage('typescript')} ${serviceVersion}`)
      const manifest = join(
        dir,
        '.loom',
        'services',
        'node_modules',
        servicePackage('typescript'),
        'package.json',
      )
      expect(JSON.parse(readFileSync(manifest, 'utf8')).version).toBe(serviceVersion)
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )
})

A language server with no TypeScript

This case drives loom lsp the way an editor does, over standard input and output. The workspace activates TypeScript and holds the stand-in, but no TypeScript compiler lies anywhere the server looks: not in the project, not beside the executable, not in the store. The server must unpack the TypeScript the executable carries into the store before it answers initialize. The registry points nowhere, so the case fails if the server tries to download TypeScript instead.

The case then opens a loom and asks the server to format it. The answer can come only from the stand-in, so a squeezed loom proves the executable loaded a service from the store.

TypeScript

const loomOf = (code: string): string =>
  lines('---', 'Language: TypeScript', '---', '', '# Answer', '', '=>', '', code)

const editorAt = async (folder: string) => {
  const child = spawn(binary, ['lsp', '--stdio'], {
    cwd: folder,
    env: bareEnv(),
    stdio: ['pipe', 'pipe', 'ignore'],
  })
  const connection = createMessageConnection(
    new StreamMessageReader(child.stdout),
    new StreamMessageWriter(child.stdin),
  )
  connection.onRequest(() => null)
  connection.listen()
  const close = (): void => {
    connection.dispose()
    child.kill()
  }
  return { connection, close }
}

describe('the executable — language server', () => {
  it(
    'unpacks the TypeScript it carries and formats with a service from the store',
    async () => {
      const dir = workspace('corpus: .', 'languages:', '  typescript: {}', 'primary: typescript')
      installStandIn(dir)
      const loom = join(dir, 'answer.loom')
      writeFileSync(loom, loomOf('const  answer  =  42'))
      const { connection, close } = await editorAt(dir)
      try {
        await connection.sendRequest('initialize', {
          processId: process.pid,
          rootUri: pathToFileURL(dir).href,
          workspaceFolders: [{ uri: pathToFileURL(dir).href, name: 'smoke' }],
          capabilities: {},
        })
        await connection.sendNotification('initialized', {})
        expect(
          existsSync(
            join(dir, '.loom', 'services', 'node_modules', 'typescript', 'lib', 'lib.es5.d.ts'),
          ),
        ).toBe(true)
        await connection.sendNotification('textDocument/didOpen', {
          textDocument: {
            uri: pathToFileURL(loom).href,
            languageId: 'loom',
            version: 1,
            text: readFileSync(loom, 'utf8'),
          },
        })
        const edits =
          (await connection.sendRequest<TextEdit[] | null>('textDocument/formatting', {
            textDocument: { uri: pathToFileURL(loom).href },
            options: { tabSize: 2, insertSpaces: true },
          })) ?? []
        expect(edits.map((edit) => edit.newText)).toEqual([loomOf('const answer = 42')])
      } finally {
        close()
        rmSync(dir, { recursive: true, force: true })
      }
    },
    SMOKE_TIMEOUT,
  )
})

Setting up Helix and Neovim

loom helix and loom nvim must write the grammar library the executable carries, with no C compiler to build one. The cases run them with the empty PATH, so a command that tried to compile would find no compiler and fail. Each points the editor's folders at a temporary one, as the tests of the CLI do, and checks that the library and the queries arrived.

TypeScript

describe('the executable — editors', () => {
  it(
    'gives Helix the grammar it carries, with no compiler',
    async () => {
      const config = scratch('helix')
      const { status, stderr } = await run(['helix'], {
        cwd: config,
        env: bareEnv({ XDG_CONFIG_HOME: config, APPDATA: config }),
      })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      const library = process.platform === 'win32' ? 'loom.dll' : 'loom.so'
      expect(existsSync(join(config, 'helix', 'runtime', 'grammars', library))).toBe(true)
      expect(existsSync(join(config, 'helix', 'runtime', 'queries', 'loom', 'highlights.scm'))).toBe(
        true,
      )
      rmSync(config, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )

  it(
    'gives Neovim the grammar it carries, with no compiler',
    async () => {
      const data = scratch('nvim')
      const { status, stderr } = await run(['nvim'], {
        cwd: data,
        env: bareEnv({ XDG_DATA_HOME: data, LOCALAPPDATA: data, NVIM_APPNAME: 'nvim' }),
      })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      const site = join(data, process.platform === 'win32' ? 'nvim-data' : 'nvim', 'site')
      expect(existsSync(join(site, 'parser', 'loom.so'))).toBe(true)
      expect(existsSync(join(site, 'queries', 'loom', 'highlights.scm'))).toBe(true)
      rmSync(data, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )
})

Upgrading in place

loom upgrade replaces the executable it runs from, so each case runs a copy, never build/loom itself. releaseServer stands in for GitHub. It answers the request for the latest release with a tag and two files, a gzipped replacement executable and the list of checksums, and serves both. The replacement is a short shell script that prints upgraded, small enough to check by eye. assetName is the name the release gives this platform's file, the same name loom upgrade looks for.

TypeScript

const assetName =
  process.platform === 'win32'
    ? `loom-windows-${process.arch}.exe.gz`
    : `loom-${process.platform}-${process.arch}.gz`

const replacement = new TextEncoder().encode(lines('#!/bin/sh', 'echo upgraded'))

const archive = Bun.gzipSync(replacement)

const sha256 = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex')

const releaseServer = (tag: string, listed: string) =>
  Bun.serve({
    hostname: '127.0.0.1',
    port: 0,
    fetch: (request) => {
      const base = new URL('/', request.url).href
      const path = new URL(request.url).pathname
      if (path === '/releases/latest')
        return Response.json({
          tag_name: tag,
          assets: [
            { name: assetName, browser_download_url: `${base}executable` },
            { name: 'SHA256SUMS', browser_download_url: `${base}sums` },
          ],
        })
      if (path === '/executable') return new Response(archive)
      if (path === '/sums') return new Response(`${listed}  ${assetName}\n`)
      return new Response('not found', { status: 404 })
    },
  })

const upgradeCopy = async (tag: string, listed: string) => {
  const server = releaseServer(tag, listed)
  const dir = scratch('upgrade')
  const copy = join(dir, process.platform === 'win32' ? 'loom.exe' : 'loom')
  copyFileSync(binary, copy)
  chmodSync(copy, 0o755)
  const result = await run(['upgrade'], {
    cwd: dir,
    env: bareEnv({ LOOM_RELEASES: new URL('/releases', server.url).href }),
    executable: copy,
  })
  server.stop()
  return { ...result, dir, copy }
}

A newer release whose checksum matches replaces the copy, and the case compares the copy's checksum with the replacement's rather than their bytes, so a failure reports two short lines instead of an executable's worth of numbers. On macOS and Linux the copy then runs as the replacement. Windows cannot run a shell script, so there the case checks that the running copy moved aside to a name ending in .old, the step that lets Windows replace it. A release whose listed checksum differs from the file's leaves the copy as it was, still Loom, and the command exits non-zero. A release no newer than this Loom changes nothing.

An executable that Homebrew installed is Homebrew's to replace, so loom upgrade names brew upgrade loom and leaves the file alone. The last case copies the executable under a Cellar folder, where Homebrew keeps its packages, and points the command at an address where nothing answers. The command must succeed without a request to that address, and the copy must keep its checksum.

TypeScript

describe('the executable — upgrade', () => {
  it(
    'replaces itself with a newer release whose checksum matches',
    async () => {
      const { status, stderr, dir, copy } = await upgradeCopy('v99.0.0', sha256(archive))
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(sha256(new Uint8Array(readFileSync(copy)))).toBe(sha256(replacement))
      if (process.platform === 'win32') expect(existsSync(`${copy}.old`)).toBe(true)
      else expect((await run([], { cwd: dir, env: bareEnv(), executable: copy })).stdout).toContain('upgraded')
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )

  it(
    'refuses a download whose checksum does not match the list',
    async () => {
      const { status, stderr, dir, copy } = await upgradeCopy('v99.0.0', sha256(new Uint8Array()))
      expect({ status, stderr }).toMatchObject({ status: 1 })
      expect(stderr).toContain('does not match')
      const after = await run(['--version'], { cwd: dir, env: bareEnv(), executable: copy })
      expect(after.stdout).toContain(`loom v${serviceVersion}`)
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )

  it(
    'leaves itself alone when the latest release is no newer',
    async () => {
      const { status, stdout, stderr, dir, copy } = await upgradeCopy('v0.0.1', sha256(archive))
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain('is up to date')
      const after = await run(['--version'], { cwd: dir, env: bareEnv(), executable: copy })
      expect(after.stdout).toContain(`loom v${serviceVersion}`)
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )

  it(
    'leaves an executable Homebrew installed to Homebrew',
    async () => {
      const dir = scratch('homebrew')
      const copy = join(dir, 'Cellar', 'loom', '9.9.9', 'bin', basename(binary))
      mkdirSync(dirname(copy), { recursive: true })
      copyFileSync(binary, copy)
      chmodSync(copy, 0o755)
      const { status, stdout, stderr } = await run(['upgrade'], {
        cwd: dir,
        env: bareEnv({ LOOM_RELEASES: nowhere }),
        executable: copy,
      })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain('brew upgrade loom')
      expect(sha256(new Uint8Array(readFileSync(copy)))).toBe(
        sha256(new Uint8Array(readFileSync(binary))),
      )
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )
})

A check-up for the executable

loom doctor must know the executable for what it is. The case puts the folder that holds the executable on PATH, and nothing else, and runs the doctor in a folder no workspace holds. The doctor reports a standalone executable, finds that a terminal runs this very file, and exits zero with no problem to report.

TypeScript

describe('the executable — doctor', () => {
  it(
    'knows itself as the standalone executable a terminal runs',
    async () => {
      const dir = scratch('doctor')
      const { status, stdout, stderr } = await run(['doctor'], {
        cwd: dir,
        env: bareEnv({ PATH: dirname(binary) }),
      })
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain(`runs from ${binary}, a standalone executable`)
      expect(stdout).toContain('a terminal runs this loom')
      expect(stdout).toContain('no problems found')
      rmSync(dir, { recursive: true, force: true })
    },
    SMOKE_TIMEOUT,
  )
})

Installing it with the script

Most people get the executable through the install script that the chapter on installing it sets out, so the smoke test runs that script too. It runs the script the way the one-line install does, with sh, against a release of its own. releaseServer stands in for GitHub's downloads. It serves one gzipped file for this platform and the list of checksums, and the script reads both from the address LOOM_DOWNLOAD names. The file inside is one line of shell that prints a version, so the script can run whatever the install put in place. It opens with no #! line, which a script would, so the install script takes it for a compiled executable that a later run may replace. A shell still runs it as a script. listed is a list of checksums that gives that file's true checksum.

The script names the file it needs from uname, which reports the same system and architecture that Node's process reports on every machine the release builds for, so assetName here names the same file.

TypeScript

const script = resolve(import.meta.dir, '..', '..', '..', 'scripts', 'install.sh')

const assetName = `loom-${process.platform}-${process.arch}.gz`

const standIn = Bun.gzipSync(new TextEncoder().encode('echo "loom v9.9.9"\n'))

const sha256 = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex')

const listed = `${sha256(standIn)}  ${assetName}\n`

const releaseServer = (sums: string | undefined) =>
  Bun.serve({
    hostname: '127.0.0.1',
    port: 0,
    fetch: (request) => {
      const path = new URL(request.url).pathname
      if (path === `/download/${assetName}`) return new Response(standIn)
      if (path === '/download/SHA256SUMS' && sums !== undefined) return new Response(sums)
      return new Response('not found', { status: 404 })
    },
  })

The script writes into the home folder, to ~/.loom/bin and to the shell's profile, so no case may run it against the real home folder. Each case makes a home of its own with freshHome, and install runs the script with an environment built from nothing. HOME names the case's home. PATH lists only the system's own folders, unless the case adds folders of its own, so no loom installed on the machine running the test shows up. SHELL says Zsh, unless the case names another shell. Nothing from the test's own environment passes through, so a ZDOTDIR or an XDG_CONFIG_HOME that would point the script at a real profile never reaches it.

installed names the folder the script installs into within a home.

TypeScript

const systemPath = '/usr/bin:/bin:/usr/sbin:/sbin'

const freshHome = (): string => realpathSync(mkdtempSync(join(tmpdir(), 'loom-home-')))

const installed = (home: string): string => join(home, '.loom', 'bin')

const install = async (
  home: string,
  sums: string | undefined,
  options: { readonly args?: ReadonlyArray<string>; readonly env?: Record<string, string> } = {},
) => {
  const server = releaseServer(sums)
  const child = Bun.spawn(['/bin/sh', script, ...(options.args ?? [])], {
    env: {
      HOME: home,
      PATH: systemPath,
      SHELL: '/bin/zsh',
      LOOM_DOWNLOAD: new URL('/download', server.url).href,
      ...options.env,
    },
    stdout: 'pipe',
    stderr: 'pipe',
  })
  const [stdout, stderr, status] = await Promise.all([
    new Response(child.stdout).text(),
    new Response(child.stderr).text(),
    child.exited,
  ])
  server.stop()
  return { status, stdout, stderr }
}

A checksum that matches

A file that matches the list lands in the folder as loom, executable, with no temporary file left beside it, and it runs. A file whose checksum differs from the one listed stops the install, and so does a release that carries no list. Either way no loom reaches the folder. Windows has no sh, and the script does not serve Windows, so every case of the script skips it there.

TypeScript

describe.skipIf(process.platform === 'win32')('the install script', () => {
  it('installs an executable whose checksum matches the list', async () => {
    const home = freshHome()
    const { status, stdout, stderr } = await install(home, listed)
    expect({ status, stderr }).toMatchObject({ status: 0 })
    expect(stdout).toContain('Installed loom v9.9.9')
    expect(statSync(join(installed(home), 'loom')).mode & 0o111).not.toBe(0)
    expect(readdirSync(installed(home))).toEqual(['loom'])
    rmSync(home, { recursive: true, force: true })
  })

  it('refuses an executable whose checksum does not match', async () => {
    const home = freshHome()
    const { status, stderr } = await install(home, `${sha256(new Uint8Array())}  ${assetName}\n`)
    expect({ status, stderr }).toMatchObject({ status: 1 })
    expect(stderr).toContain('does not match')
    expect(existsSync(join(installed(home), 'loom'))).toBe(false)
    rmSync(home, { recursive: true, force: true })
  })

  it('refuses a release that carries no list of checksums', async () => {
    const home = freshHome()
    const { status, stderr } = await install(home, undefined)
    expect({ status, stderr }).toMatchObject({ status: 1 })
    expect(stderr).toContain('SHA256SUMS')
    expect(existsSync(join(installed(home), 'loom'))).toBe(false)
    rmSync(home, { recursive: true, force: true })
  })
})

A folder on PATH

The script puts its folder on PATH through the shell's profile, and these cases read the profile it leaves behind. Zsh's profile, .zshrc, starts with a line of the person's own. The script runs twice. The first run adds the line that puts the folder on PATH and names the profile. The second finds the folder already named there and adds nothing. So the line appears once, after the person's own line, which stays as it was.

A home whose folder is already on PATH gets no profile at all. Bash gets the line in the profile Bash reads on this system, .bash_profile on macOS and .bashrc on Linux. Fish gets a fish_add_path line in its own config.fish. With --no-modify-path, or with LOOM_NO_MODIFY_PATH set, the script writes no profile and prints the line for the person to add.

TypeScript

const pathLine = 'export PATH="$HOME/.loom/bin:$PATH"'

const occurrences = (text: string, line: string): number => text.split(line).length - 1

describe.skipIf(process.platform === 'win32')('the install script — PATH', () => {
  it('adds the folder to PATH in the profile once, however often it runs', async () => {
    const home = freshHome()
    const zshrc = join(home, '.zshrc')
    writeFileSync(zshrc, "alias ll='ls -l'\n")
    const first = await install(home, listed)
    const second = await install(home, listed)
    expect({ status: first.status, stderr: first.stderr }).toMatchObject({ status: 0 })
    expect({ status: second.status, stderr: second.stderr }).toMatchObject({ status: 0 })
    expect(first.stdout).toContain(`Added ${installed(home)} to PATH in ${zshrc}`)
    expect(second.stdout).toContain(`${zshrc} already adds ${installed(home)} to PATH`)
    const profile = readFileSync(zshrc, 'utf8')
    expect(profile).toStartWith("alias ll='ls -l'\n")
    expect(occurrences(profile, pathLine)).toBe(1)
    rmSync(home, { recursive: true, force: true })
  })

  it('changes no profile when the folder is already on PATH', async () => {
    const home = freshHome()
    const { status, stderr } = await install(home, listed, {
      env: { PATH: `${installed(home)}:${systemPath}` },
    })
    expect({ status, stderr }).toMatchObject({ status: 0 })
    expect(existsSync(join(home, '.zshrc'))).toBe(false)
    rmSync(home, { recursive: true, force: true })
  })

  it('writes to the profile Bash reads on this system', async () => {
    const home = freshHome()
    const { status, stderr } = await install(home, listed, { env: { SHELL: '/bin/bash' } })
    expect({ status, stderr }).toMatchObject({ status: 0 })
    const profile = process.platform === 'darwin' ? '.bash_profile' : '.bashrc'
    expect(occurrences(readFileSync(join(home, profile), 'utf8'), pathLine)).toBe(1)
    rmSync(home, { recursive: true, force: true })
  })

  it('gives Fish a fish_add_path line in its own profile', async () => {
    const home = freshHome()
    const { status, stderr } = await install(home, listed, { env: { SHELL: '/usr/bin/fish' } })
    expect({ status, stderr }).toMatchObject({ status: 0 })
    expect(readFileSync(join(home, '.config', 'fish', 'config.fish'), 'utf8')).toContain(
      'fish_add_path "$HOME/.loom/bin"',
    )
    rmSync(home, { recursive: true, force: true })
  })

  it('leaves every profile alone with --no-modify-path or LOOM_NO_MODIFY_PATH', async () => {
    const flagged = freshHome()
    const unset = freshHome()
    const runs = [
      await install(flagged, listed, { args: ['--no-modify-path'] }),
      await install(unset, listed, { env: { LOOM_NO_MODIFY_PATH: '1' } }),
    ]
    runs.forEach(({ status, stdout, stderr }) => {
      expect({ status, stderr }).toMatchObject({ status: 0 })
      expect(stdout).toContain(pathLine)
    })
    expect(existsSync(join(flagged, '.zshrc'))).toBe(false)
    expect(existsSync(join(unset, '.zshrc'))).toBe(false)
    rmSync(flagged, { recursive: true, force: true })
    rmSync(unset, { recursive: true, force: true })
  })
})

Another loom already there

A person who installed Loom with npm or Bun before still has that loom, and the script must name it. The case lays out two such installs inside its home, the way each package manager lays them out, and puts both on PATH. npm's loom is a link, written relative to its folder, to the package under npm's lib/node_modules. Bun's is a link to the package in its .bun folder. The script names each loom, says who installed it, and gives the command that removes it. It removes neither.

TypeScript

const packageCopy = (copy: string): void => {
  mkdirSync(dirname(copy), { recursive: true })
  writeFileSync(copy, '#!/bin/sh\n', { mode: 0o755 })
}

describe.skipIf(process.platform === 'win32')('the install script — another loom', () => {
  it('names each older loom on PATH and the command that removes it', async () => {
    const home = freshHome()
    const npm = join(home, 'npm')
    const npmCopy = join(npm, 'lib/node_modules/@athrio/loom/dist/main.js')
    packageCopy(npmCopy)
    mkdirSync(join(npm, 'bin'))
    symlinkSync('../lib/node_modules/@athrio/loom/dist/main.js', join(npm, 'bin', 'loom'))
    const bun = join(home, '.bun')
    const bunCopy = join(bun, 'install/global/node_modules/@athrio/loom/dist/main.js')
    packageCopy(bunCopy)
    mkdirSync(join(bun, 'bin'))
    symlinkSync(bunCopy, join(bun, 'bin', 'loom'))
    const { status, stdout, stderr } = await install(home, listed, {
      env: { PATH: `${join(npm, 'bin')}:${join(bun, 'bin')}:${systemPath}` },
    })
    expect({ status, stderr }).toMatchObject({ status: 0 })
    expect(stdout).toContain(
      `Another loom is on your PATH, at ${join(npm, 'bin', 'loom')}, installed by npm. Remove it with:\n  npm uninstall -g @athrio/loom`,
    )
    expect(stdout).toContain(
      `Another loom is on your PATH, at ${join(bun, 'bin', 'loom')}, installed by Bun. Remove it with:\n  bun remove -g @athrio/loom`,
    )
    expect(existsSync(npmCopy)).toBe(true)
    expect(existsSync(bunCopy)).toBe(true)
    rmSync(home, { recursive: true, force: true })
  })
})

A loom that is not Loom's

A person may keep a loom of their own in the install folder, such as a script that switches between versions of Loom. The case puts one there, a script that even answers --version the way Loom does, and runs the install. The script refuses and names the file. The file keeps its text, and nothing else lands beside it.

TypeScript

describe.skipIf(process.platform === 'win32')('the install script — a loom kept by hand', () => {
  it('refuses to replace a loom that is not a Loom executable', async () => {
    const home = freshHome()
    const own = join(installed(home), 'loom')
    const text = '#!/bin/sh\necho "loom v0.0.10"\n'
    mkdirSync(installed(home), { recursive: true })
    writeFileSync(own, text, { mode: 0o755 })
    const { status, stderr } = await install(home, listed)
    expect({ status, stderr }).toMatchObject({ status: 1 })
    expect(stderr).toContain(`${own} is not a Loom executable`)
    expect(readFileSync(own, 'utf8')).toBe(text)
    expect(readdirSync(installed(home))).toEqual(['loom'])
    rmSync(home, { recursive: true, force: true })
  })
})

What the install test draws on

The install test runs on Bun's test runner, and Bun spawns the script, gzips the stand-in executable and serves the stand-in release. Node's modules make the homes, the profiles and the fake installs, read back what the script left, and compute a checksum.

TypeScript

import { describe, expect, it } from 'bun:test'
import { createHash } from 'node:crypto'
import {
  existsSync,
  mkdirSync,
  mkdtempSync,
  readdirSync,
  readFileSync,
  realpathSync,
  rmSync,
  statSync,
  symlinkSync,
  writeFileSync,
} from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join, resolve } from 'node:path'

The install test

TypeScript

::[What the install test draws on]

::[Installing it with the script]

::[A checksum that matches]

::[A folder on PATH]

::[Another loom already there]

::[A loom that is not Loom's]

What the smoke test draws on

The cases run on Bun's test runner, and Bun spawns the executable and tar, gzips the stand-in executables, and serves the stand-in registry and release. Node's modules make the folders and files a case needs, copy the executable to upgrade, compute a checksum, and turn paths into the file: addresses the language server names documents by. Volar's language-server package carries the client half of the protocol: createMessageConnection, the stream reader and writer it speaks through, and the TextEdit a format answers with. The language store names the TypeScript service's package and this Loom's version, and the host runtime names the key under which the stand-in finds Effect.

TypeScript

import { describe, expect, it } from 'bun:test'
import {
  createMessageConnection,
  StreamMessageReader,
  StreamMessageWriter,
  type TextEdit,
} from '@volar/language-server/node'
import { spawn } from 'node:child_process'
import { createHash } from 'node:crypto'
import {
  chmodSync,
  copyFileSync,
  existsSync,
  mkdirSync,
  mkdtempSync,
  readFileSync,
  realpathSync,
  rmSync,
  writeFileSync,
} from 'node:fs'
import { tmpdir } from 'node:os'
import { basename, dirname, join, resolve } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { servicePackage, serviceVersion } from '@athrio/loom-lang-services/LoomStore'
import { runtimeKey } from '@athrio/loom-lang-services/Runtime'

The smoke test

TypeScript

::[What the smoke test draws on]

::[A smoke test for the executable]

::[A workspace to run it in]

::[Tangling with nothing installed]

::[A signature macOS accepts]

::[A language service from the store]

::[A language server with no TypeScript]

::[Setting up Helix and Neovim]

::[Upgrading in place]

::[A check-up for the executable]