Command Resolution

Background

A C++ language server needs to know how to compile every file in a project. This information comes from the compilation database (CDB), typically a compile_commands.json file generated by the build system. Each entry in the CDB contains a source file path and a compilation command, for example:

g++ -std=c++20 -O2 -fPIC -I../include -DNDEBUG -c src/foo.cpp

This command records how the build system invokes the compiler during an actual build. However, the language server cannot use it directly, for three reasons.

First, this is a driver-level command, not a frontend command. The g++ above is a compiler driver -- it selects the correct frontend, linker, and standard library. The language server actually uses Clang's frontend (cc1), and converting from g++ to cc1 involves a large amount of implicit work: determining the target triple, injecting system header search paths, setting default language standards, and more. None of this information appears explicitly in the CDB -- it is implicitly provided by the compiler.

Second, the command contains semantically irrelevant options. -O2 and -fPIC only affect code generation, not semantic analysis -- the language server does not do code generation and has no use for these options. Similarly, -c (compile mode) and -o (output file) are build-artifact directives that are meaningless to a language server. Passing them unfiltered to the frontend adds unnecessary complexity and can even cause errors.

Third, commands in large projects are highly redundant. In a project with tens of thousands of source files, the vast majority use the same compiler and semantic options, differing only in include paths and macro definitions. Without deduplication, each file requires an independent toolchain query, wasting memory and startup time.

Of these three problems, the first has the greatest impact on user experience: missing implicit information. When the language server cannot correctly obtain system header paths, users see errors on standard library headers -- #include <vector> reports "file not found," or GCC built-in type traits are flagged as undeclared identifiers. In clangd's issue tracker, these problems are consistently among the most frequently reported (clangd#1262, clangd#1691).

clangd's solution is the --query-driver flag: users manually specify which compilers need probing, and clangd then queries those compilers for system paths. This approach has two problems. First, it is manual -- users must know which compiler their project uses and correctly configure a glob pattern. Second, the flag is not independently useful -- it relies on existing CDB commands to trigger probing (clangd#1219). For cross-compilation and embedded development scenarios, users frequently need to debug their configuration repeatedly before clangd correctly identifies the toolchain.

clice automates the entire command processing pipeline: after reading commands from the CDB, it automatically identifies the compiler family (GCC, Clang, MSVC, etc.), automatically probes the toolchain, and minimizes startup overhead through multi-level deduplication. Users do not need to manually configure any toolchain-related parameters.

Design

The core task of command processing is to convert the raw driver commands in the CDB into cc1 arguments consumable by the Clang frontend. This conversion involves four conceptual layers: argument classification, command separation, toolchain probing, and search path extraction.

Argument Classification

When loading the CDB, each compilation option is classified into one of seven categories:

• Discarded: Options related to build artifacts that the language server does not need. These include output file (-o), compile mode (-c), dependency scanning (-M family), PCH building (-emit-pch), and options the driver itself would ignore for this input.

• Codegen: Options that only affect the code generation backend and do not affect semantic analysis. These include position-independent code (-fPIC), stack protection (-fstack-protector), frame pointer (-fomit-frame-pointer), debug info (-g family), LTO, etc. They do not change the AST or diagnostic output.

  Note: -O and -fsanitize= are not in this category despite appearing code-generation-related. -O defines the __OPTIMIZE__ macro, and -fsanitize=address affects __has_feature(address_sanitizer). They alter preprocessor state and are therefore semantic options.

• Diagnostics: Warning-control options (-Wall, -Wno-*, -Werror). They change which diagnostics are emitted but neither the AST nor toolchain probing results.

• User-content: Options that may differ per file but do not affect toolchain probing results. These include include paths (-I, -isystem, -iquote, -idirafter), macro definitions (-D, -U), and forced includes (-include). Their meaning is "what this particular file additionally needs."

• Semantic: The remaining recognized options -- they affect compilation semantics and play a role in toolchain probing. Examples include -std=c++20, -target, -march=, etc.

• Input: The source file itself, which occupies an explicit slot in the parsed command rather than being a plain string among others.

• Unknown: Options the parser does not recognize (for example, flags from a compiler newer than clice's embedded LLVM). They participate in command identity, but unknown tokens coming from the CDB are not rendered into the compile commands clice executes; unknown tokens the user wrote in configuration rules are kept.

Classification is based on Clang's own option table (OptTable), using option IDs rather than string matching.

Structured Commands

After classification, each command is parsed once into a structured configuration (CompileConfig): an interned sequence of classified arguments in which the input file occupies an explicit slot. Identical configurations deduplicate to a single instance identified by a stable ConfigID; strings are interned for pointer-stable comparison. Everything downstream — toolchain probing, rendering, rules, identity — consumes the structured form; nothing re-parses command strings.

Two derived views matter:

• Rendering: the same structured configuration can be rendered on demand as a driver command line (for probing or for handing agents a runnable command) or combined with probe results into frontend arguments. Rendering normalizes alias spellings, so identity is not disturbed by cosmetic variation in the CDB.
• Entry identity: the frontend-relevant view of the configuration, together with the input's slot position and working directory, is hashed into the entry's identity hash — the stable identity used for CDB diffing, index snapshot validation, and pinned context choices.

The classification also buys toolchain probing cache efficiency. Probing requires actually invoking the compiler driver (e.g., running g++ -dumpmachine or clang++ -###), typically taking 100ms or more. The probing result depends only on the driver and probe-relevant options -- user-content options (-I, -D) do not affect the system paths or target triple output by the driver. Therefore, regardless of what different -I paths files may have, files sharing the probe-relevant view share one probing result. In real projects, tens of thousands of files typically collapse to a few dozen distinct probe keys.

Compilation Database

CompilationDatabase holds the entries of every loaded source — a compile_commands.json registered by the configuration or discovered under the workspace. Each source loads, reloads and unloads independently; its entries are parsed and deduplicated into ConfigIDs and kept in file order, because generators emit a file's several entries in a fixed configuration order. The database stores facts only: it does not decide which of a file's entries wins.

Commands written by hand — a rule's default_command, the builtin fallback — go through the same normalization as an entry (intern_command), with the input slot synthesized at the end, so a declared command is probed, edited and rendered exactly like a database entry.

Build

Build is the one reader of the configuration's [[rules]] and the one place that knows which command a file compiles with — a pure function of the configuration and the database, shared by the server and every CLI entry point (clice index, clice lint, clice inspect load a workspace the same way). Every consumer — the context resolver, the dependency scan, the indexer, the context protocol — asks it rather than the database:

• Entries. A file's database entries in build order: the sources of the rules matching the file first, then those of the other active rules, each in declaration order, discovered sources last; within one source, file order. The first entry is the default selection; a user's pin (clice/switchContext) can choose another. Entries never disappear because a pattern does not name their file — the rules only decide priority.
• Edits. The remove and append lists of every matching active rule, applied in declaration order — a rule's removes before its appends, a later remove reaching what an earlier rule appended. A header borrowing a host's command carries the edits of both files, each rule once, so it sees the same macros the host compiles with.
• Commands. What a file compiles under: its entries, or — when it has none — the default_command of the first matching rule that declares one. A file with neither gets the builtin command, whose driver follows the language clang assigns to the extension (clang++ -std=c++20 for every C++ spelling and for the ambiguous .h, -x cuda --cuda-device-only for CUDA, plain clang for the rest).
• Members. The translation units of the build: files with entries, plus the sources on disk that a default_command rule's patterns claim (headers never count), enumerated from the directories the patterns name. The dependency scan runs every command of every member, so a header reachable through only one of a file's entries still finds that host. The background index admits a member unless a matching rule says index = false; that check sits at the index queue, so every path that enqueues work — startup, a database reload, a save — honours it.

Hosting

A header without a command of its own compiles as part of a translation unit that includes it. The hosting layer ranks the includers the build compiles in a language the header can be part of — a .h in any, a .hpp in C++ and the languages built on it (Objective-C++, CUDA), a .cuh only in CUDA, so a C++ header is never compiled as C: units whose entries come from the databases the header's own rules name first, then the unit sharing the header's stem, then one in its directory, then path proximity. The first with an include chain to the header is its default host — the same answer for the editor, the background index, a save-time rescan and clice inspect. A file the build does not compile (one on the builtin command) never hosts.

Discovery

When no rule declares a source, the build's databases are the compile_commands.json files found under the workspace: at startup the root's and those of its direct subdirectories, and whenever a file without an entry is opened, the ones in the directories from the file's up to the root — a project deeper in the tree is loaded the first time one of its files is opened, and never scanned for otherwise. Discovered databases rank after declared ones, shallower before deeper, then by path; a file several of them list compiles under the first by default and offers the others as candidates, while a file only a later database lists takes its command from it. Finding more than one is reported once at startup, with the tagged rules that would turn them into switchable configurations instead. The file tracker keeps discovering on its poll, so a database generated after startup — at the root or in a new subdirectory — loads when it appears. A discovered database that vanishes keeps serving its entries, but yields to the present ones: a build/ directory regenerated as out/ hands the files both list to out/ at once, and takes them back when it returns.

Inference

A file with neither an entry nor a host, and no matching rule with a default_command, borrows the command of a nearby translation unit of its own language — a .h matches any, a .c never borrows a C++ command and a .cpp never a C one. The lender is a unit in the file's directory, the one sharing its stem first and then the first by name; else, for a header, the unit whose header search directories (-I, -isystem, -iquote) contain it, the nearest directory first — the unit's own code finds the header by that path, so its command is the one the header is written for; else the unit closest by path. The borrowed command carries the rule edits of both files and is labeled Inferred: the decision log names the lender, diagnostics about missing files get a guidance note, and the file stays outside the build — it is not indexed in the background, and clice inspect resolves it the same way.

Resolution

The final command of a file is composed from those layers in a fixed order: the user's selection for the file in the editor (a pinned entry or a pinned host and include occurrence), else the file's first entry, else — for a header — its default host's command, else the first matching rule's default_command, else a nearby unit's command (inference), else the builtin; then the rule edits of the file (and of the host or lender it borrows from), then a run's extras (a lint plan's clang-tidy arguments). Selections apply to editor compiles only; background compiles resolve without them and cache nothing, so the index and the CLI see the build's own answer.

Patterns are compiled against absolute paths: a relative pattern is anchored at the configuration file's directory (.. segments included), or at the workspace root for rules passed through initializationOptions; an absolute or **-led one matches as written. Rules carrying a configuration tag apply only while that tag is active; the distinct tags form the configuration menu, and the active one is resolved at startup from --configuration, the persisted selection, or default_configuration (see Compilation Context). When any rule declares a source (a database or a default_command), discovery is off: the declaration is the whole intent, and a database sitting at the workspace root is not consulted. Only a configuration declaring no source at all falls back to discovery.

Configuration granularity is preserved throughout the pipeline: search-path extraction for dependency scanning is per effective command (different -I sets produce different search configs), while toolchain probing deduplicates further still, since user-content options do not affect probe results.

Toolchain

Toolchain converts driver-level commands into cc1 arguments. Its design centers on two core capabilities:

Compiler family identification. Toolchain identifies the compiler family from the executable name -- the CompilerFamily enum includes GCC, Clang, MSVC, ClangCL, NVCC, Intel, and Zig. The identification logic handles various naming variants: version suffixes (clang++-17), architecture prefixes (arm-none-eabi-g++), and Windows .exe suffixes. The family determines which probing strategy is used.

Caching strategy. The cache has two layers. The probe layer caches raw driver invocation results, keyed by the probe-relevant shape of the command (driver, probe-relevant flags, input kind — .c and .cpp may trigger different driver rules; the working directory joins the key when the command is sensitive to it). The synthesis layer turns probe results into the per-input-kind flag sets that command rendering consumes. Failed probes are cached too (negative caching), but transient failures are retried after a cooldown rather than being remembered forever.

Search Paths

SearchConfig extracts header search paths from cc1 arguments, organizing them into a four-tier structure:

1. Quoted (-iquote): Search paths for #include "foo.h"
2. Angled (-I): Search paths for #include <foo.h>
3. System (-isystem, -internal-isystem, etc.): System header paths
4. After (-idirafter): Paths searched after system directories

This four-tier model corresponds to Clang's internal search layout. Paths within tiers are deduplicated (starting from the Angled tier), with the deduplication algorithm replicating Clang's behavior: if the same path appears in both Angled and System tiers, the one in Angled is kept. This ensures #include_next correctness.

SearchConfig is the foundational input for include path completion, include path resolution, dependency graph construction, and other features.

Implementation

Loading and Parsing

CDB loading uses simdjson for streaming JSON parsing, processing entries one by one:

1. Read each entry's directory, file, and arguments (or command) fields
2. Expand response files (@file), driver-mode aware, tolerating UTF-16 encoded files and nesting
3. Translate NVCC commands into an equivalent clang CUDA invocation
4. Filter out non-C/C++ files (e.g., .rc, .asm, .def)
5. Resolve relative file paths to absolute paths
6. Parse and classify each option once into the structured configuration, absolutizing relative include paths against directory
7. Deduplicate identical configurations into shared ConfigIDs

The parsing process also handles a special case: CMake-generated CDBs sometimes contain -Xclang -include-pch -Xclang <pchfile> sequences (CMake's PCH workaround), which are detected and discarded during loading.

Toolchain Probing

Different compiler families use different probing strategies:

GCC: A two-step process. First, call the GCC driver to obtain two key pieces of information -- the target triple (-dumpmachine) and the install path (-print-search-dirs). Second, inject this information into Clang's driver (--target= and --gcc-install-dir=), letting Clang's driver emulate GCC's behavior and produce cc1 arguments. This allows the Clang frontend to correctly locate GCC's standard library and system headers.

Clang / Zig: Invoke the driver with the -### option, which prints the complete cc1 command line without actually executing compilation. Parse the first cc1 line from the output. For Zig, the driver path consists of two parts (zig cc or zig c++), requiring special handling during probing.

The external driver's version may be newer than clice's embedded LLVM version, and its cc1 output may contain options that clice does not recognize. During parsing, unknown options are silently dropped to ensure compatibility.

MSVC / ClangCL: Switch Clang's driver to MSVC-compatible mode via the --driver-mode=cl directive, then use Clang's driver to obtain cc1 arguments.

After probing completes, the temporary probe file's path and module-output-related flags (which reference the deleted temp file) are stripped from the results. If clice's resource dir differs from what the probe returned, all related paths are replaced to ensure the frontend uses builtin headers from a matching version.

Startup warm-up. During the dependency scanning phase at server startup, all unique toolchain cache keys are collected and probed in parallel. Pipe draining and process waiting for probe subprocesses also run concurrently to avoid deadlocks from full pipes. Once warm-up completes, all subsequent toolchain queries hit the cache.

Search Path Extraction

After toolchain probing yields cc1 arguments, search path extraction iterates through these arguments, routing include path options by type into the four tiers. All paths are resolved to absolute paths and normalized (removing . and ..).

The -iprefix / -iwithprefix / -iwithprefixbefore options must be processed cooperatively in order of appearance: -iprefix sets a prefix, subsequent -iwithprefix prepends the prefix to its path and places it in the After tier, and -iwithprefixbefore places it in the Angled tier.

After the four tiers are concatenated, deduplication begins from the Angled tier. The Quoted tier does not participate in deduplication -- the same path appearing in both Quoted and Angled is legitimate, and both are preserved. This replicates Clang's internal behavior.

FAQ

• Why classify by option ID rather than string matching?

  clice's argument parsing is based on Clang's own option table (generated from Options.inc), using option IDs for classification. String matching is prone to missing edge cases: Clang's option syntax has many forms -- -std=c++20 is joined form, -I /path is separate form, -Wall is flag form, and some options have / prefixes (MSVC style). Classifying by ID means all these syntax variants are correctly handled by Clang's parser, and the classification logic only needs to care about "what is this option," not "how is it spelled."

• Why don't user-content options participate in the toolchain cache key?

  This is the core benefit of the classification design. -I and -D do not change the system paths, target triple, or language defaults output by the compiler driver. Excluding them from the cache key reduces the number of keys from "one per file" to "one per configuration." A project with tens of thousands of files typically has only a few dozen distinct cache keys, requiring only a few dozen subprocess calls at startup.

• Why must search path deduplication precisely match Clang's behavior?

  If the language server's header search order differs from the actual compiler's, #include may resolve to different files -- when identically named headers exist in different directories, search order determines which one is used. The semantics of #include_next depend even more directly on the deduplication result of search directories. Strict matching ensures that the language server sees exactly the same code as the compiler.

• Why does GCC probing use a two-step process instead of directly parsing GCC's -v output?

  The Clang frontend needs GCC's target triple and install path to correctly locate GCC's standard library headers. Directly parsing GCC's -v output is feasible but would introduce additional text parsing logic that is fragile across different GCC versions with varying output formats. By injecting the information into Clang's driver via --target= and --gcc-install-dir=, Clang itself handles the search path assembly, which is more reliable.

• Why is compiler family identification based on executable filename rather than path resolution?

  Compiler driver behavior is affected by the name used to invoke it. For example, /usr/bin/clang++ is typically a symlink to /usr/lib/llvm-20/bin/clang, but invoking it as clang++ automatically enables C++ mode and links C++ libraries. Using realpath to resolve to the actual path before identification would lose the semantic information carried by the invocation name. Similarly, if arm-none-eabi-g++ were resolved to some generic GCC binary path, the cross-compilation context would be lost.

• How are files without a CDB entry handled?

  A header first looks for a source file that includes it through the dependency graph and borrows that file's command (see Compilation Context). Otherwise the first matching rule with a default_command supplies the command — the way to describe a project whose files all share one set of flags, or a scratch directory. A file with none of those borrows a nearby unit's command (see Inference); only one with no compatible unit either gets the builtin command: clang or clang++ -std=c++20 based on the file's language, with the resource dir injected, so basic semantic analysis remains available and a guidance note explains that the command was guessed.

Known Limitations

• Discovery never scans the tree. A nested project's database loads when one of its files is opened; until then its units are not indexed. A rule naming the database (or default_command) covers it from the start.

• A borrowed command serves the editor only. A file compiling under an inferred command is not a member of the build: it is not indexed in the background and cannot host a header. Listing it in a database or under a default_command rule makes it one.

• Incomplete support for some compiler families. Intel compilers (icc, icx, dpcpp) are recognized but currently fall through to the generic Clang driver path without dedicated probing logic, so their special system paths may not be correctly discovered. NVCC has dedicated probing (parsing nvcc --dryrun output into a clang CUDA invocation), but only with a GCC or Clang host compiler -- nvcc driving MSVC cl is not supported yet, and multi-architecture commands parse the newest architecture only.

• SearchConfig does not support all search path options. -cxx-isystem (system directories effective only in C++ mode), -iwithsysroot (prepends sysroot to path), and HeaderMap support are not yet implemented. These options are uncommon in practice but may appear in specific Apple or cross-compilation toolchains.

• Global impact of configuration rules. [[rules]] in clice.toml can append or remove options from compilation commands. If the user modifies a rule that affects all files (e.g., appending a global -I), all files' compilation configurations change, potentially triggering a full re-index. There is currently no mechanism to detect which rule changes actually affect which files.

• MSVC-style option parsing. On non-Windows systems, MSVC-style option prefixes (/U, /D, /I) must be handled specially to prevent Unix absolute paths (such as /Users/...) from being misparsed as MSVC options. This is currently resolved by dynamically adjusting option visibility based on the driver name, but edge cases may still exist.

• Compiler launchers are not recognized. CDB commands wrapped in a launcher (ccache g++ ..., sccache clang++ ...) are parsed as if the launcher were the compiler, so family identification and toolchain probing target the wrong binary. Strip the launcher from the compilation database, or override the flags with configuration rules.