The looms in a project

A project is a workspace together with the patterns it ignores. LoomProject holds the two. Its root is the workspace, the directory that holds .loom/config.yaml. Its ignore holds the patterns .loom/config.yaml lists under ignore:.

Each pattern is written like a line of a .gitignore file and matched against paths relative to the root. So drafts/ leaves out every folder named drafts, wherever it sits, while /notes.loom leaves out only the notes.loom at the root.

projectIn reads a file's project from its build settings. A file in no workspace belongs to no project, and projectIn answers Option.none for it.

export interface LoomProject {
  readonly root: Path
  readonly ignore: ReadonlyArray<string>
}

export const projectIn = (settings: BuildSettings): Option.Option<LoomProject> =>
  Option.map(Option.fromNullishOr(settings.workspaceRoot), (root) => ({
    root,
    ignore: settings.ignore ?? [],
  }))

A set of patterns speaks for the folder it belongs to. A .gitignore speaks for its own folder, and the project's patterns speak for the root. A Layer pairs a folder with the patterns that speak for it. Its base is the folder. Its rules are the patterns, compiled by the ignore package, which matches paths the way Git does. layerOf builds a layer from a list of patterns or from the text of an ignore file.

relativeIn places a path within a folder. It gives the path relative to the folder, with forward slashes as Git writes them, and gives nothing for a path outside the folder or for the folder itself.

verdictIn asks one layer about one path. A layer can speak only for a path inside its folder. A folder is asked about with a trailing slash, because a pattern like drafts/ matches folders and nothing else. The layer answers in one of three ways. Option.some(true) means a pattern ignores the path. Option.some(false) means a ! pattern brings it back. Option.none means no pattern spoke to the path, and the question passes to the next layer.

interface Layer {
  readonly base: Path
  readonly rules: Ignore
}

const layerOf = (base: Path, patterns: string | ReadonlyArray<string>): Layer => ({
  base,
  rules: ignore().add(patterns),
})

const relativeIn = (base: Path, path: Path): Option.Option<string> => {
  const rel = relative(base, path)
  return rel === '' || rel.split(sep)[0] === '..' || isAbsolute(rel)
    ? Option.none()
    : Option.some(rel.split(sep).join('/'))
}

const verdictIn = (layer: Layer, path: Path, isDir: boolean): Option.Option<boolean> =>
  Option.flatMap(relativeIn(layer.base, path), (rel) =>
    Match.value(layer.rules.test(isDir ? `${rel}/` : rel)).pipe(
      Match.when({ ignored: true }, () => Option.some(true)),
      Match.when({ unignored: true }, () => Option.some(false)),
      Match.orElse(() => Option.none<boolean>()),
    ),
  )

Loom leaves out what Git ignores, so a scratch folder, a build's output, or a second checkout of the repository never joins the project. To learn what Git ignores, Loom reads the pattern files the repository keeps, as Git does.

Each .gitignore speaks for the folder it sits in and everything below it. The repository's info/exclude speaks for the whole repository. It holds the patterns a clone keeps without committing them. Extra checkouts made with git worktree are often excluded there, so reading the .gitignore files alone would let those checkouts in.

Git also reads an excludes file in each person's home folder, and Loom does not. That file follows a person into every repository they open, so a pattern there would quietly change which looms a project holds. Loom reads Git's patterns but not Git's index of committed files, so a loom that matches a pattern stays out even if someone committed it.

gitDirOf finds a repository's Git folder from the repository's top folder. In an ordinary clone, .git is the Git folder itself. In a linked worktree or a submodule, .git is a file whose gitdir: line points at the real Git folder. Reading a folder as text fails, so the text of .git tells the two cases apart. commonDirOf takes one step more. A worktree's Git folder names, in a commondir file, the folder it shares with the main checkout, and info/exclude lives in that shared folder.

textOf reads a file's text, and gives nothing for a file that is missing or is a folder. fileLayer turns an ignore file into a layer for the folder it speaks for, when the file exists. repositoryAt gathers the layers a repository starts with: its info/exclude, then the .gitignore in its top folder.

const textOf = (file: Path): Option.Option<string> => {
  try {
    return Option.some(readFileSync(file, 'utf8'))
  } catch {
    return Option.none()
  }
}

const fileLayer = (base: Path, file: Path): Option.Option<Layer> =>
  Option.map(textOf(file), (text) => layerOf(base, text))

const gitDirOf = (top: Path): Path =>
  pipe(
    textOf(join(top, '.git')),
    Option.map((text) => resolvePath(top, text.replace(/^gitdir:/, '').trim())),
    Option.getOrElse(() => join(top, '.git')),
  )

const commonDirOf = (gitDir: Path): Path =>
  pipe(
    textOf(join(gitDir, 'commondir')),
    Option.map((text) => resolvePath(gitDir, text.trim())),
    Option.getOrElse(() => gitDir),
  )

const repositoryAt = (top: Path): ReadonlyArray<Layer> =>
  Array.getSomes([
    fileLayer(top, join(commonDirOf(gitDirOf(top)), 'info', 'exclude')),
    fileLayer(top, join(top, '.gitignore')),
  ])

The Git rules in force inside a folder form a stack of layers. The repository's layers come first and the deepest folder's .gitignore comes last. GitRules is that stack, or nothing for a folder in no repository.

entering gives the stack inside a folder from the stack outside it. A folder with a .git of its own begins a repository, so its layers replace the stack. Any other folder puts its .gitignore, if it has one, on top. gitRulesAt builds the stack for any folder by entering each folder from the filesystem's root down to it. A walk that starts deep inside a repository therefore honours every .gitignore above its starting point.

type GitRules = Option.Option<ReadonlyArray<Layer>>

const entering = (dir: Path, outside: GitRules): GitRules =>
  existsSync(join(dir, '.git'))
    ? Option.some(repositoryAt(dir))
    : Option.map(outside, (layers) => [
        ...layers,
        ...Option.toArray(fileLayer(dir, join(dir, '.gitignore'))),
      ])

const gitRulesAt = (dir: Path): GitRules => {
  const parent = dirname(dir)
  return entering(dir, parent === dir ? Option.none() : gitRulesAt(parent))
}

Git settles a path by asking its layers from the deepest up. The first layer with a pattern for the path decides. Within one file, a later pattern overrides an earlier one. ignores asks in the same order, with one layer placed above all of Git's: the project's own.

The project's layer lists defaultIgnore first and the configured patterns after it. So a ! pattern under ignore: can bring back what the default leaves out: !*.fixture.loom makes fixtures tangle again. Such a pattern can bring back what Git ignores, too, provided the folder that holds the loom comes back with it. Git keeps the same limit, since a walk never looks inside a folder it does not open. A path that no layer speaks to belongs to the project.

The default leaves out test fixtures. A fixture is a loom a test reads. It is often broken on purpose, to prove that Loom refuses it. A project that tangled its own fixtures would refuse to build. Naming a fixture *.fixture.loom keeps it out of every walk. The default stops short of *.test.loom. A loom that tangles cli.test.ts is naturally called cli.test.loom, and that loom must tangle.

const defaultIgnore: ReadonlyArray<string> = ['*.fixture.loom']

const ignores = (own: Layer, git: GitRules, path: Path, isDir: boolean): boolean =>
  pipe(
    [own, ...Array.reverse(Option.getOrElse(git, (): ReadonlyArray<Layer> => []))],
    Array.findFirst((layer) => verdictIn(layer, path, isDir)),
    Option.getOrElse(() => false),
  )

A walk descends a folder one level at a time and decides at each entry. It never enters a folder it would only throw away, such as a node_modules with thousands of files. The walk carries a Scope, which answers three questions. opens says whether to enter a folder. keeps says whether a loom belongs. into gives the scope to carry inside a folder.

storeDirs names the four folders that hold no one's looms: node_modules holds installed packages, dist holds built output, .git holds Git's history, and .loom holds Loom's own store. Every walk skips them, and the tangler's watch skips a change inside them.

A walk inside a project carries projectScope. It opens a folder unless the folder is one of the store folders, is a workspace of its own, or is ignored. A folder that holds its own configuration is a separate workspace, and its looms belong to that project and its lock. isWorkspace makes that call by the rule the language store uses to find a workspace, so a .loom folder with no configuration in it, such as one that only keeps a loose loom's lock, closes nothing off. The scope keeps a loom unless the loom is ignored. Entering a folder brings that folder's Git rules into the scope. scopeAt builds the scope a walk starts with in a given folder, from the project's own layer and the Git rules in force in that folder.

A walk outside any project carries looseScope, which skips only the store folders. Loom walks this way through the folder of a loom no project holds.

export const storeDirs: ReadonlySet<string> = new Set(['node_modules', '.loom', 'dist', '.git'])

interface Scope {
  readonly opens: (dir: Path) => boolean
  readonly keeps: (loom: Path) => boolean
  readonly into: (dir: Path) => Scope
}

const projectScope = (own: Layer, git: GitRules): Scope => ({
  opens: (dir) =>
    !storeDirs.has(basename(dir)) && !isWorkspace(dir) && !ignores(own, git, dir, true),
  keeps: (loom) => !ignores(own, git, loom, false),
  into: (dir) => projectScope(own, entering(dir, git)),
})

const looseScope: Scope = {
  opens: (dir) => !storeDirs.has(basename(dir)),
  keeps: () => true,
  into: () => looseScope,
}

const scopeAt = (project: LoomProject, dir: Path): Scope =>
  projectScope(layerOf(project.root, [...defaultIgnore, ...project.ignore]), gitRulesAt(dir))

walk lists the looms under a folder in the scope it is given. It enters each folder the scope opens and keeps each loom the scope keeps. A loom is any entry whose name ends in .loom and that is not a folder, so the .loom store itself never counts as one. entriesIn reads one folder's entries, and lists nothing for a folder it cannot read. The compiler then builds a corpus of one, the file alone.

loomsIn is the walk every caller uses. Given a folder and its project, it walks the folder in that project's scope, or in the loose scope when there is no project.

const entriesIn = (dir: Path): ReadonlyArray<Dirent> => {
  try {
    return readdirSync(dir, { withFileTypes: true })
  } catch {
    return []
  }
}

const walk = (dir: Path, scope: Scope): ReadonlyArray<Path> =>
  Array.flatMap(entriesIn(dir), (entry) => {
    const full = resolvePath(dir, entry.name)
    return Match.value(entry).pipe(
      Match.when(
        (e) => e.isDirectory() && scope.opens(full),
        () => walk(full, scope.into(full)),
      ),
      Match.when(
        (e) => !e.isDirectory() && e.name.endsWith('.loom') && scope.keeps(full),
        (): ReadonlyArray<Path> => [full],
      ),
      Match.orElse((): ReadonlyArray<Path> => []),
    )
  })

export const loomsIn = (dir: Path, project: Option.Option<LoomProject>): ReadonlyArray<Path> =>
  walk(
    dir,
    Option.match(project, {
      onNone: () => looseScope,
      onSome: (p) => scopeAt(p, dir),
    }),
  )

A walk settles membership for every loom under a folder. Sometimes the question concerns a single loom. The compiler asks it of the file whose corpus it builds, and the tangler asks it of a loom it was pointed at directly.

holds answers the question the way a walk from the root would, without listing the root. It steps down from the root through each folder on the way to the loom, carrying the scope the walk would carry. It stops at the first folder the walk would not open. Past the last folder, it asks whether the walk would keep the loom. A loom outside the root is not the project's at all. foldersBetween lists the folders on the way, from just below the root down to the folder that holds the loom.

const foldersBetween = (root: Path, dir: Path): ReadonlyArray<Path> =>
  dir === root || dirname(dir) === dir ? [] : [...foldersBetween(root, dirname(dir)), dir]

export const holds = (project: LoomProject, loom: Path): boolean =>
  Option.isSome(relativeIn(project.root, loom)) &&
  pipe(
    foldersBetween(project.root, dirname(loom)),
    Array.reduce(Option.some(scopeAt(project, project.root)), (scope, dir) =>
      scope.pipe(
        Option.filter((s) => s.opens(dir)),
        Option.map((s) => s.into(dir)),
      ),
    ),
    Option.exists((scope) => scope.keeps(loom)),
  )

The ignore package compiles a pattern list the way Git matches it, and answers for a path with test. Node's fs reads folders, ignore files, and Git's pointer files synchronously, because the editor discovers a corpus inside Volar's synchronous hook, which cannot wait. Node's path module places one path relative to another. Effect supplies Option for an answer that may be absent, the Array combinators, Match, and pipe. BuildSettings carries a file's workspace and its configured patterns, and Path types a file path. The language store supplies isWorkspace, which tells a nested workspace by its configuration.

import { Array, Match, Option, pipe } from 'effect'
import ignore, { type Ignore } from 'ignore'
import { existsSync, readdirSync, readFileSync, type Dirent } from 'node:fs'
import {
  basename,
  dirname,
  isAbsolute,
  join,
  relative,
  resolve as resolvePath,
  sep,
} from 'node:path'
import { type Path } from '@athrio/loom-ast/LoomCorpusAst'
import { type BuildSettings } from './PackageConfig'
import { isWorkspace } from '@athrio/loom-lang-services/LoomStore'