Command Resolution

Background

A C++ language server needs to know how to compile every file in a project. This information comes from the compilation database (CDB), typically a compile_commands.json file generated by the build system. Each entry in the CDB contains a source file path and a compilation command, for example:

g++ -std=c++20 -O2 -fPIC -I../include -DNDEBUG -c src/foo.cpp

This command records how the build system invokes the compiler during an actual build. However, the language server cannot use it directly, for three reasons.

First, this is a driver-level command, not a frontend command. The g++ above is a compiler driver -- it selects the correct frontend, linker, and standard library. The language server actually uses Clang's frontend (cc1), and converting from g++ to cc1 involves a large amount of implicit work: determining the target triple, injecting system header search paths, setting default language standards, and more. None of this information appears explicitly in the CDB -- it is implicitly provided by the compiler.

Second, the command contains semantically irrelevant options. -O2 and -fPIC only affect code generation, not semantic analysis -- the language server does not do code generation and has no use for these options. Similarly, -c (compile mode) and -o (output file) are build-artifact directives that are meaningless to a language server. Passing them unfiltered to the frontend adds unnecessary complexity and can even cause errors.

Third, commands in large projects are highly redundant. In a project with tens of thousands of source files, the vast majority use the same compiler and semantic options, differing only in include paths and macro definitions. Without deduplication, each file requires an independent toolchain query, wasting memory and startup time.

Of these three problems, the first has the greatest impact on user experience: missing implicit information. When the language server cannot correctly obtain system header paths, users see errors on standard library headers -- #include <vector> reports "file not found," or GCC built-in type traits are flagged as undeclared identifiers. In clangd's issue tracker, these problems are consistently among the most frequently reported (clangd#1262, clangd#1691).

clangd's solution is the --query-driver flag: users manually specify which compilers need probing, and clangd then queries those compilers for system paths. This approach has two problems. First, it is manual -- users must know which compiler their project uses and correctly configure a glob pattern. Second, the flag is not independently useful -- it relies on existing CDB commands to trigger probing (clangd#1219). For cross-compilation and embedded development scenarios, users frequently need to debug their configuration repeatedly before clangd correctly identifies the toolchain.

clice automates the entire command processing pipeline: after reading commands from the CDB, it automatically identifies the compiler family (GCC, Clang, MSVC, etc.), automatically probes the toolchain, and minimizes startup overhead through multi-level deduplication. Users do not need to manually configure any toolchain-related parameters.

Design

The core task of command processing is to convert the raw driver commands in the CDB into cc1 arguments consumable by the Clang frontend. This conversion involves four conceptual layers: argument classification, command separation, toolchain probing, and search path extraction.

Argument Classification

When loading the CDB, each compilation option is classified into one of four categories:

• Discarded: Options related to build artifacts that the language server does not need. These include output file (-o), compile mode (-c), dependency scanning (-M family), PCH building (-emit-pch), and C++20 modules (-fmodule-file, etc., managed by the language server itself).

• Codegen-only: Options that only affect the code generation backend and do not affect semantic analysis. These include position-independent code (-fPIC), stack protection (-fstack-protector), frame pointer (-fomit-frame-pointer), debug info (-g family), LTO, etc. They do not change the AST or diagnostic output.

  Note: -O and -fsanitize= are not in this category despite appearing code-generation-related. -O defines the __OPTIMIZE__ macro, and -fsanitize=address affects __has_feature(address_sanitizer). They alter preprocessor state and are therefore semantic options.

• User-content: Options that may differ per file but do not affect toolchain probing results. These include include paths (-I, -isystem, -iquote, -idirafter), macro definitions (-D, -U), and forced includes (-include). Their meaning is "what this particular file additionally needs."

• Semantic: All remaining options -- they affect compilation semantics and play a role in toolchain probing. Examples include -std=c++20, -Wall, -target, -march=, etc.

Classification is based on Clang's own option table (OptTable), using option IDs rather than string matching.

Command Separation

After classification, each compilation command is split into two parts:

• CanonicalCommand: Driver path + all semantic options. Represents "the identity and semantic configuration of the compiler."
• Patch: All user-content options. Represents "the include paths and macro definitions this file additionally needs."

Together with the working directory, these form CompilationInfo -- an abstract representation of a file's complete compilation configuration.

The core purpose of this separation is toolchain probing cache efficiency. Toolchain probing requires actually invoking the compiler driver (e.g., running g++ -dumpmachine or clang++ -###), typically taking 100ms or more. The probing result depends only on the driver and semantic options -- user-content options (-I, -D) do not affect the system paths or target triple output by the driver. Therefore, regardless of what different -I paths files may have, as long as the semantic options are the same, they can share the same probing result.

In real projects, tens of thousands of files may have only a few dozen distinct CanonicalCommand instances, meaning the toolchain needs to be probed only a few dozen times rather than tens of thousands.

Both CanonicalCommand and CompilationInfo are deduplicated via ObjectSet -- instances with identical content exist only once in memory and are shared by pointer. String arguments are interned through StringSet, ensuring pointer stability and enabling direct comparison.

Compilation Database

CompilationDatabase loads compile_commands.json, parsing, classifying, and deduplicating each entry into CompilationEntry (file path ID -> CompilationInfo). All entries are sorted by file path ID for binary search lookups.

On lookup, CompilationDatabase assembles a CompilationInfo into a CompileCommand -- the final output of the command processing pipeline, containing the complete compilation flags and source file path, ready to be submitted to toolchain probing or the Clang frontend.

For files without a CDB entry (e.g., a file the user opens that is not part of the project), CompilationDatabase synthesizes a default command -- selecting clang or clang++ -std=c++20 based on the file extension.

CompilationDatabase also provides the ability to group by configuration: ConfigGroup aggregates files that share the same CompilationInfo. This is the right granularity for extracting search path configurations during dependency scanning -- different -I paths produce different groups. For toolchain probing, the granularity is coarser (user-content options don't affect probing results), so Toolchain further deduplicates on top of ConfigGroup.

Configuration Rules

Beyond the compilation commands in the CDB itself, users can append or remove compilation options via [[rules]] in clice.toml. Each rule contains file matching patterns (globs) and lists of options to append or remove.

When looking up a file's compilation command, matching rules are applied on top of the CDB command -- specified options are first removed from the base command, then new options are appended. This allows users to fine-tune compilation flags at the project level without modifying the build system's output.

Toolchain

Toolchain converts driver-level commands into cc1 arguments. Its design centers on two core capabilities:

Compiler family identification. Toolchain identifies the compiler family from the executable name -- the CompilerFamily enum includes GCC, Clang, MSVC, ClangCL, NVCC, Intel, and Zig. The identification logic handles various naming variants: version suffixes (clang++-17), architecture prefixes (arm-none-eabi-g++), and Windows .exe suffixes. The family determines which probing strategy is used.

Caching strategy. Probing results are cached by (driver path, file extension, non-user-content flags). File extension is part of the cache key because .c and .cpp may trigger different driver rules. Failed probes are also cached (negative caching) to avoid retrying the same nonexistent compiler repeatedly.

Search Paths

SearchConfig extracts header search paths from cc1 arguments, organizing them into a four-tier structure:

1. Quoted (-iquote): Search paths for #include "foo.h"
2. Angled (-I): Search paths for #include <foo.h>
3. System (-isystem, -internal-isystem, etc.): System header paths
4. After (-idirafter): Paths searched after system directories

This four-tier model corresponds to Clang's internal search layout. Paths within tiers are deduplicated (starting from the Angled tier), with the deduplication algorithm replicating Clang's behavior: if the same path appears in both Angled and System tiers, the one in Angled is kept. This ensures #include_next correctness.

SearchConfig is the foundational input for include path completion, include path resolution, dependency graph construction, and other features.

Implementation

Loading and Parsing

CDB loading uses simdjson for streaming JSON parsing, processing entries one by one:

1. Read each entry's directory, file, and arguments (or command) fields
2. Filter out non-C/C++ files (e.g., .rc, .asm, .def)
3. Resolve relative file paths to absolute paths
4. Classify each option in the arguments field, routing them into canonical or patch
5. Absolutize relative paths in include path options (resolved against directory)
6. Deduplicate CanonicalCommand and CompilationInfo via ObjectSet
7. Sort all entries by file path ID

The parsing process also handles a special case: CMake-generated CDBs sometimes contain -Xclang -include-pch -Xclang <pchfile> sequences (CMake's PCH workaround), which are detected and discarded during loading.

Toolchain Probing

Different compiler families use different probing strategies:

GCC: A two-step process. First, call the GCC driver to obtain two key pieces of information -- the target triple (-dumpmachine) and the install path (-print-search-dirs). Second, inject this information into Clang's driver (--target= and --gcc-install-dir=), letting Clang's driver emulate GCC's behavior and produce cc1 arguments. This allows the Clang frontend to correctly locate GCC's standard library and system headers.

Clang / Zig: Invoke the driver with the -### option, which prints the complete cc1 command line without actually executing compilation. Parse the first cc1 line from the output. For Zig, the driver path consists of two parts (zig cc or zig c++), requiring special handling during probing.

The external driver's version may be newer than clice's embedded LLVM version, and its cc1 output may contain options that clice does not recognize. During parsing, unknown options are silently dropped to ensure compatibility.

MSVC / ClangCL: Switch Clang's driver to MSVC-compatible mode via the --driver-mode=cl directive, then use Clang's driver to obtain cc1 arguments.

After probing completes, the temporary probe file's path and module-output-related flags (which reference the deleted temp file) are stripped from the results. If clice's resource dir differs from what the probe returned, all related paths are replaced to ensure the frontend uses builtin headers from a matching version.

Startup warm-up. During the dependency scanning phase at server startup, all unique toolchain cache keys are collected and probed in parallel. Pipe draining and process waiting for probe subprocesses also run concurrently to avoid deadlocks from full pipes. Once warm-up completes, all subsequent toolchain queries hit the cache.

Search Path Extraction

After toolchain probing yields cc1 arguments, search path extraction iterates through these arguments, routing include path options by type into the four tiers. All paths are resolved to absolute paths and normalized (removing . and ..).

The -iprefix / -iwithprefix / -iwithprefixbefore options must be processed cooperatively in order of appearance: -iprefix sets a prefix, subsequent -iwithprefix prepends the prefix to its path and places it in the After tier, and -iwithprefixbefore places it in the Angled tier.

After the four tiers are concatenated, deduplication begins from the Angled tier. The Quoted tier does not participate in deduplication -- the same path appearing in both Quoted and Angled is legitimate, and both are preserved. This replicates Clang's internal behavior.

FAQ

• Why classify by option ID rather than string matching?

  clice's argument parsing is based on Clang's own option table (generated from Options.inc), using option IDs for classification. String matching is prone to missing edge cases: Clang's option syntax has many forms -- -std=c++20 is joined form, -I /path is separate form, -Wall is flag form, and some options have / prefixes (MSVC style). Classifying by ID means all these syntax variants are correctly handled by Clang's parser, and the classification logic only needs to care about "what is this option," not "how is it spelled."

• Why don't user-content options participate in the toolchain cache key?

  This is the core benefit of the two-level separation design. -I and -D do not change the system paths, target triple, or language defaults output by the compiler driver. Excluding them from the cache key reduces the number of keys from "one per file" to "one per configuration." A project with tens of thousands of files typically has only a few dozen distinct cache keys, requiring only a few dozen subprocess calls at startup.

• Why must search path deduplication precisely match Clang's behavior?

  If the language server's header search order differs from the actual compiler's, #include may resolve to different files -- when identically named headers exist in different directories, search order determines which one is used. The semantics of #include_next depend even more directly on the deduplication result of search directories. Strict matching ensures that the language server sees exactly the same code as the compiler.

• Why does GCC probing use a two-step process instead of directly parsing GCC's -v output?

  The Clang frontend needs GCC's target triple and install path to correctly locate GCC's standard library headers. Directly parsing GCC's -v output is feasible but would introduce additional text parsing logic that is fragile across different GCC versions with varying output formats. By injecting the information into Clang's driver via --target= and --gcc-install-dir=, Clang itself handles the search path assembly, which is more reliable.

• Why is compiler family identification based on executable filename rather than path resolution?

  Compiler driver behavior is affected by the name used to invoke it. For example, /usr/bin/clang++ is typically a symlink to /usr/lib/llvm-20/bin/clang, but invoking it as clang++ automatically enables C++ mode and links C++ libraries. Using realpath to resolve to the actual path before identification would lose the semantic information carried by the invocation name. Similarly, if arm-none-eabi-g++ were resolved to some generic GCC binary path, the cross-compilation context would be lost.

• How are files without a CDB entry handled?

  A default command is synthesized. Based on the file extension, either clang or clang++ -std=c++20 is selected, and the resource dir is injected. This ensures basic semantic analysis remains available even when a file is not in the CDB. For header files, the system also attempts to find a source file that includes it via the dependency graph and uses that source file's compilation command as context (see Compilation Context).

Known Limitations

• Incomplete support for some compiler families. NVCC and Intel compilers (icc, icx, dpcpp) are recognized but currently fall through to the generic Clang driver path without dedicated probing logic. This means these compilers' special system paths may not be correctly discovered.

• SearchConfig does not support all search path options. -cxx-isystem (system directories effective only in C++ mode), -iwithsysroot (prepends sysroot to path), and HeaderMap support are not yet implemented. These options are uncommon in practice but may appear in specific Apple or cross-compilation toolchains.

• Global impact of configuration rules. [[rules]] in clice.toml can append or remove options from compilation commands. If the user modifies a rule that affects all files (e.g., appending a global -I), all files' compilation configurations change, potentially triggering a full re-index. There is currently no mechanism to detect which rule changes actually affect which files.

• MSVC-style option parsing. On non-Windows systems, MSVC-style option prefixes (/U, /D, /I) must be handled specially to prevent Unix absolute paths (such as /Users/...) from being misparsed as MSVC options. This is currently resolved by dynamically adjusting option visibility based on the driver name, but edge cases may still exist.