Continuous integration

Every push and every pull request runs the same gate: the code type-checks, the tests pass, the committed source matches the corpus it tangles from, the command-line tool carries none of Loom Design, and the standalone executable still runs. The release workflow waits on this one. It runs only after the gate passes on the default branch, so nothing publishes on a red build.

A push runs only the checks its change can affect. The repository holds a book, eight packages and a website, and a change to one rarely touches the rest. So the checks run under moon, a task runner made for repositories that hold several projects. Each check names the files it reads, and moon runs a check only when one of those files changed. A fix to the website's stylesheet type-checks and tests the website and leaves the language server's four hundred tests alone. A change to the AST, which every package builds on, runs nearly everything.

The tangle check is what a literate framework owes its readers. Loom commits its tangled output so a checkout builds without re-tangling, and that output is only trustworthy if it never drifts from the looms that produce it. The check re-tangles the corpus and fails on any difference, so a loom edited without a re-tangle cannot reach the default branch.

The Design check guards a licence rather than a build. Loom belongs to the Quadrivium Academy foundation under the MIT licence, while Loom Design is a commercial product of Athrio Media OÜ, so a bundle that quietly carries Design publishes the paid product under the free product's terms. Version 0.0.10 did exactly that, and the permissive grant it made on Design's source cannot be withdrawn. Design now lives in a repository of its own, which makes the mistake harder to repeat but not impossible, since a dependency added for convenience would be enough. So the check stays.

The tools, pinned

mise, a tool manager, pins the two tools every check runs on: Bun, at the version the workspace's manifest declares, and moon. A developer's machine and the CI runner read the same file, so a check that passes on one runs on the same tools on the other. Changing either version changes this file, and every check reads it, so a new tool asks every question again.

TOML

[tools]
bun = "1.2.20"
"npm:@moonrepo/cli" = "2.5.6"

The projects

A moon project is a directory whose moon.yml names its tasks. The workspace file lists the projects: the book, the eight packages, and the website. The book's project is the repository's root, since its one check reads the corpus and every directory the corpus writes into. The corpus folder itself holds looms and nothing else.

The projects are listed by name. Left to itself, moon names a project after its folder, and the root's folder is called loom, the same as the CLI's. Moon measures a change against the default branch, main, unless it is told which two commits to compare, as the workflow below tells it.

YAML

$schema: "https://moonrepo.dev/schemas/workspace.json"
vcs:
  defaultBranch: "main"
projects:
  book: "."
  loom-ast: "packages/loom-ast"
  loom-config: "packages/loom-config"
  loom-lang-services: "packages/loom-lang-services"
  loom-service-typescript: "packages/loom-service-typescript"
  loom-lang: "packages/loom-lang"
  loom: "packages/loom"
  loom-vscode: "packages/loom-vscode"
  tree-sitter-loom: "packages/tree-sitter-loom"
  site: "site"

The book

Every project's tasks need the workspace's packages, so each project has an install task the others depend on. It installs against the committed lockfile, from the repository's root, where the workspace is. A dependency runs before the task that needs it and never on its own, so a push that runs no task installs nothing. The install is never cached, because whether the packages are in place is a question about the machine, not about any file.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

The book's own task is the tangle check. It builds the CLI from the committed source and tangles the corpus with it. Self-tangling is deliberate here, since the check asks only whether the source and the corpus agree, not whether the tangler is the released one. git diff --exit-code then fails the task if the re-tangle changed anything, naming the files that drifted.

Its inputs are as wide as its question. They are the corpus, every directory the tangle writes into, and the lock that lists what it wrote, so a hand edit to a tangled file runs the check as surely as an edit to a loom does. The workspace's manifest, its lockfile and the pinned tools count too, since they decide which tangler runs. The project is the repository's root, so each input is named outright: the root's own **/* would be every file in the repository, and an edit to the README would tangle the book again.

YAML

  tangle:
    script: "bun run --filter @athrio/loom build && bun packages/loom/dist/main.js tangle corpus && git diff --exit-code"
    deps: ["install"]
    inputs:
      - "corpus/**/*"
      - "packages/**/*"
      - "site/**/*"
      - ".github/**/*"
      - "scripts/**/*"
      - ".loom/**/*"
      - "package.json"
      - "bun.lock"
      - "mise.toml"
      - "moon.yml"

The packages

Each package type-checks and tests itself. The type check earns its place because the bundler does not do it: Vite strips types rather than reading them, so a package without a checker could ship an identifier that resolves to nothing. Both tasks run in the package's own directory, so bun test finds that package's tests and no other's.

A package reads more than its own files. Its code imports the packages it depends on, straight from their source, so a change to a dependency must check the package again. Each package's sources therefore name its own source and tests and those of every package it builds on, along with the configuration they all share: the base TypeScript config, the workspace's manifest and lockfile, and Bun's settings. A path that begins with a slash is measured from the repository's root, and every other path from the package. Every task also reads its own moon.yml and the pinned tools, so a change to the task, or to what runs it, runs it again.

The tests of most packages share one helper, @athrio/effect-test, which runs an Effect as a test. It has no tests or checks of its own, so it is no project; the packages whose tests use it read it as a source.

The AST's tasks

The AST is the leaf. It builds on no other package, so its sources are its own and the shared configuration.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "test/**/*"
    - "package.json"
    - "tsconfig.json"
    - "/packages/effect-test/**/*"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The configuration's tasks

The configuration package also builds on no other package, so its tasks read what the AST's read.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "test/**/*"
    - "package.json"
    - "tsconfig.json"
    - "/packages/effect-test/**/*"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The language services' tasks

The contract every language service satisfies builds on the AST. It has no tests, so its one task is the type check.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "package.json"
    - "tsconfig.json"
    - "/packages/loom-ast/src/**/*"
    - "/packages/loom-ast/package.json"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The TypeScript service's tasks

The TypeScript service builds on the language services' contract, and through it on the AST.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "test/**/*"
    - "package.json"
    - "tsconfig.json"
    - "/packages/loom-lang-services/src/**/*"
    - "/packages/loom-lang-services/package.json"
    - "/packages/loom-ast/src/**/*"
    - "/packages/loom-ast/package.json"
    - "/packages/effect-test/**/*"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The language's tasks

The Loom language, with the compiler, the tangler and the language server, builds on every package before it: the AST, the configuration, the language services' contract and the TypeScript service.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "test/**/*"
    - "package.json"
    - "tsconfig.json"
    - "/packages/loom-ast/src/**/*"
    - "/packages/loom-ast/package.json"
    - "/packages/loom-config/src/**/*"
    - "/packages/loom-config/package.json"
    - "/packages/loom-lang-services/src/**/*"
    - "/packages/loom-lang-services/package.json"
    - "/packages/loom-service-typescript/src/**/*"
    - "/packages/loom-service-typescript/package.json"
    - "/packages/effect-test/**/*"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The CLI's tasks

The CLI builds on the language and so on everything the language builds on. It also reads the grammar of tree-sitter-loom: its build copies the grammar's sources into the bundle, and its tests run loom helix and loom nvim, which compile them. Besides its type check and its tests it carries the Design check. That check builds the tool and packs it the way npm would, then reads what came out. A file named for Design in the tarball fails the task: one with design, overlay or shell in its name, or one named ui. The ui must start the file's name, because a bundled chunk's name ends in a hash, and a hash can spell ui by chance. A source map fails the task as well when it lists one of Design's files among its sources, the path by which Design's TypeScript reached the registry the first time. Design's name inside a string is allowed, since the design command has to say which package to install. The tarball is removed once it has been read, so the check leaves no file behind on a developer's machine.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "test/**/*"
    - "package.json"
    - "tsconfig.json"
    - "vite.config.ts"
    - "/packages/loom-ast/src/**/*"
    - "/packages/loom-ast/package.json"
    - "/packages/loom-config/src/**/*"
    - "/packages/loom-config/package.json"
    - "/packages/loom-lang-services/src/**/*"
    - "/packages/loom-lang-services/package.json"
    - "/packages/loom-service-typescript/src/**/*"
    - "/packages/loom-service-typescript/package.json"
    - "/packages/loom-lang/src/**/*"
    - "/packages/loom-lang/package.json"
    - "/packages/tree-sitter-loom/src/**/*"
    - "/packages/tree-sitter-loom/queries/**/*"
    - "/packages/tree-sitter-loom/editors/**/*"
    - "/packages/effect-test/**/*"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  nodesign:
    script: |
      set -e
      bun run build
      npm pack --ignore-scripts --silent > /dev/null
      contents=$(tar tzf athrio-loom-*.tgz)
      rm -f athrio-loom-*.tgz
      if echo "$contents" | grep -Eiq 'design|overlay|shell|(^|/)ui\.'; then
        echo "the tarball carries Loom Design:" >&2
        echo "$contents" | grep -Ei 'design|overlay|shell|(^|/)ui\.' >&2
        exit 1
      fi
      if grep -oh '"sources":\[[^]]*\]' dist/*.js.map | grep -q 'loom-design'; then
        echo "a source map carries Loom Design's source" >&2
        exit 1
      fi
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The smoke check runs the standalone executable. It builds the bundle, compiles it into build/loom for the runner's own platform with the grammar library and TypeScript inside, and runs the smoke test from the chapter on the standalone executable against that file. The release runs the same test on every platform before it ships an executable; this check runs it on Linux for every change. The executable differs from the source the tests run, in ways those tests cannot see. It finds a service in the store by another route, and it installs packages without npm. Only a check of the compiled file catches a change that breaks either. The check also runs the install script against a release it serves itself. It reaches the network once, to install TypeScript into a store the way the language server does for a user. Its inputs are the package's sources, the program that compiles the executable, the smoke test, and the install script.

YAML

  smoke:
    script: "bun run build && bun run compile && bun test ./smoke/binary.smoke.ts ./smoke/install.smoke.ts"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "smoke/**/*"
      - "compile.ts"
      - "/scripts/install.sh"
      - "moon.yml"
      - "/mise.toml"

The editor extension's tasks

The VS Code extension has no tests yet, so its one task is the type check of its client.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "package.json"
    - "tsconfig.json"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The grammar's tasks

The tree-sitter grammar's test reads every loom in the repository with both the grammar and Loom's own classifier, and compares the two, line by line. So its sources are wider than a package's usual. They are the grammar itself, its scanner, its queries and its generated parser, the language and the AST that supply the classifier, and every .loom file in the book, the docs and the examples. A loom edited anywhere runs the test again, since the edit may reach a corner of the grammar no test has met.

The test needs two tools beyond Bun. The tree-sitter command arrives as a package, and a C compiler builds the parser, which every runner and every machine that builds Loom already has.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "grammar.js"
    - "src/**/*"
    - "queries/**/*"
    - "editors/**/*"
    - "test/**/*"
    - "package.json"
    - "tree-sitter.json"
    - "tsconfig.json"
    - "/packages/loom-ast/src/**/*"
    - "/packages/loom-ast/package.json"
    - "/packages/loom-lang/src/**/*"
    - "/packages/loom-lang/package.json"
    - "/packages/effect-test/**/*"
    - "/corpus/**/*.loom"
    - "/docs/**/*.loom"
    - "/examples/**/*.loom"
    - "/tsconfig.base.json"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    command: "bunx tsc --noEmit -p tsconfig.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

The website's tasks

The website builds on no package of the workspace. Its sources are its own code, its assets, the two pages and configs that build it, and the licence it shows on a page of its own. It type-checks twice, once as the client the browser runs and once as the server Bun runs, since the two compile under different configs.

The website also ships, as a container image, and the image task builds it. It builds site/Dockerfile with the whole repository as its context and tags the image four ways: latest, sha- with the commit's short hash, v with the version the CLI's manifest carries, and main- with the build's time and the short hash. It then pushes every tag to the registry. The main- tag is the one the cluster follows. Flux watches the registry, picks the newest build by the time in that tag, and rolls the website forward to it. The other tags cannot order builds: a hash says nothing about time, and several builds share one version. The image's inputs are the site's own sources and Loom's book, since the source page draws the book and the build weaves it into the image. A change that touches none of them builds no image, so a fix to a package's tests leaves the website that is already running alone. The task runs in CI alone, so no developer's moon pushes an image, and moon never caches it, because what it makes lives in the registry rather than in any file.

YAML

$schema: "https://moonrepo.dev/schemas/project.json"

fileGroups:
  sources:
    - "src/**/*"
    - "public/**/*"
    - "index.html"
    - "package.json"
    - "tsconfig.json"
    - "tsconfig.server.json"
    - "vite.config.ts"
    - "/LICENSE"
    - "/package.json"
    - "/bun.lock"
    - "/bunfig.toml"

tasks:
  install:
    command: "bun install --frozen-lockfile"
    options:
      cache: false
      internal: true
      runFromWorkspaceRoot: true

  typecheck:
    script: "bunx tsc --noEmit -p tsconfig.json && bunx tsc --noEmit -p tsconfig.server.json"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "moon.yml"
      - "/mise.toml"

  test:
    command: "bun test"
    deps: ["install"]
    inputs:
      - "@group(sources)"
      - "test/**/*"
      - "moon.yml"
      - "/mise.toml"

  image:
    script: >-
      docker build . --file site/Dockerfile
      --tag ghcr.io/athrio-com/loom-website:latest
      --tag ghcr.io/athrio-com/loom-website:sha-${GITHUB_SHA::7}
      --tag ghcr.io/athrio-com/loom-website:v$(node -p "require('./packages/loom/package.json').version")
      --tag ghcr.io/athrio-com/loom-website:main-$(date -u +%Y%m%d%H%M%S)-${GITHUB_SHA::7}
      && docker push --all-tags ghcr.io/athrio-com/loom-website
    inputs:
      - "@group(sources)"
      - "Dockerfile"
      - "moon.yml"
      - "/corpus/**/*"
    options:
      cache: false
      runInCI: "only"
      runFromWorkspaceRoot: true

What the workflow answers to

A push to main checks what it changed and ships the website if the website changed. A pull request runs the checks alone, since an image from an unmerged branch has nowhere to go. A newer run on the same branch cancels an older one still going, because only the latest commit can be the one to ship. A cancelled run loses nothing: the next run measures its change from the last build that passed, so it takes the cancelled run's change on as well.

YAML

name: CI

on:
  push:
    branches: [main]
  pull_request:

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

The last build that passed

The commit a change is measured from depends on what carries the change. A pull request is measured from the commit it asks to be merged into. A push to main is measured from the last build that passed: the head of the latest run of this workflow on main that succeeded. Every change since that commit arrived either with this push or with a run that failed or was cancelled, and this run takes all of them on. Until a build has passed, the commit the push began from stands in.

Left to itself, moon would measure a push from the commit before its head. That misses changes: when main was merged into a branch before the branch was pushed, the merge's first parent is the branch's own last commit, and the branch's own work would build nothing. The last build that passed is the base that misses nothing. The first job asks GitHub for it before anything else and hands it to the second job, and it reads the repository and the workflow's own runs, nothing more.

YAML

jobs:
  check:
    name: The book, typed, tested and tangled
    runs-on: ubuntu-latest
    permissions:
      contents: read
      actions: read
    outputs:
      base: ${{ steps.base.outputs.sha }}
    steps:
      - name: The commit the change is measured from
        id: base
        env:
          GH_TOKEN: ${{ github.token }}
          PULL_BASE: ${{ github.event.pull_request.base.sha }}
          PUSH_BEFORE: ${{ github.event.before }}
        run: |
          base=$PULL_BASE
          if [ -z "$base" ]; then
            base=$(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/ci.yaml/runs?branch=main&event=push&status=success&per_page=1" --jq '.workflow_runs[0].head_sha // empty')
          fi
          echo "sha=${base:-$PUSH_BEFORE}" >> "$GITHUB_OUTPUT"

The gate, asked of the change

The first job checks the repository out with its whole history, which moon needs to find what changed between the two commits, and fetches a file's contents only when something reads the file. It gets Bun and moon from mise, at the versions mise.toml pins. It then asks moon ci for the five checks, measured from the base it found, one at a time: run side by side on the runner, the type checks and the tests slow each other down until a test misses its time limit.

YAML

      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
          filter: blob:none
      - uses: jdx/mise-action@v5
        with:
          install_args: bun npm:@moonrepo/cli
      - name: The book's checks, asked of the change
        run: moon ci --concurrency 1 :typecheck :test :tangle :nodesign :smoke
        env:
          MOON_BASE: ${{ steps.base.outputs.sha }}
          MOON_HEAD: ${{ github.sha }}

Shipping the website

The second job runs for a push to main alone, once the first has passed, and measures its change from the same base. It reads the repository and writes packages, nothing else. It checks the repository out the same way, gets moon from mise, logs in to GitHub's registry with the token the run is given, and asks moon ci for the image. When the website did not change, moon builds nothing and the job passes.

YAML

  ship:
    name: The website's image, pushed to the registry
    needs: check
    if: github.event_name == 'push'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    env:
      MOON_BASE: ${{ needs.check.outputs.base }}
      MOON_HEAD: ${{ github.sha }}
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
          filter: blob:none
      - uses: jdx/mise-action@v5
        with:
          install_args: npm:@moonrepo/cli
      - name: Log in to the registry
        run: echo '${{ secrets.GITHUB_TOKEN }}' | docker login ghcr.io -u '${{ github.actor }}' --password-stdin
      - name: The website's image, pushed to the registry
        run: moon ci :image

The workflow

YAML

::[What the workflow answers to]

::[The last build that passed]

::[The gate, asked of the change]

::[Shipping the website]