Compilation Context

Background

C++'s #include is a very primitive code reuse mechanism — it is essentially textual substitution. Unlike the module systems universally adopted by modern languages, it has no isolation: what a header file sees when included depends entirely on the preprocessor state before the inclusion point. This means the same file can produce entirely different compilation results under different conditions, and makes compiler caching optimizations much harder.

Source files produce different results due to different compilation commands (Debug/Release, different platforms); header files produce entirely different expansions depending on which source file includes them and what #define and #include directives precede the inclusion.

When a user opens a file in their editor, the language server must choose an appropriate compilation context to serve it — diagnostics, completion, and go-to-definition all depend on this context. Existing language servers (not just those for C++) have no formal concept of compilation context — they implicitly pick a context but give users no way to perceive or control this choice. When the context is wrong, compilation produces errors and the user sees a flood of false-positive diagnostics. clice is the first language server to introduce compilation context as a formal concept.

Taking clangd as an example, the lack of a compilation context concept has led to long-standing issues in the community:

• Non-self-contained headers are not supported. This is one of the earliest reported and still unresolved issues in the clangd community (clangd #45). Since clangd has no concept of header context, it cannot restore the correct preprocessor state for non-self-contained headers, and users see a flood of meaningless errors when opening such files.

• Guessing compilation commands for headers. When clangd selects a compilation command for a header file, it mainly relies on path heuristics — guessing the "most relevant" source file from the compilation database (CDB) based on filename and directory structure, then borrowing that source file's compilation command. This guessing is unreliable and frequently picks the wrong file, leading to spurious diagnostics (clangd #1505).

• Cannot select context in multi-configuration projects. When a file appears multiple times in compile_commands.json (Debug/Release, different platforms), clangd picks only one entry, and the user has no control over which one is chosen (clangd #681).

clice's design incorporates the compilation context framework from the start — this concept is naturally woven into the entire system. Users can not only see the currently active compilation context, but also query available contexts and switch between them.

Design

A compilation context is the complete set of conditions needed to fully restore the compilation state of a file. Its meaning varies by file type.

For source files, the compilation context is their compilation command. A source file may correspond to multiple compilation commands in the CDB — common in multi-configuration projects. Each command represents a different compilation context, and which one is selected determines the file's behavior in the language server. For example, the same file can produce different code under Debug and Release:

void initialize() {
    auto* conn = create_connection();
#ifndef NDEBUG
    conn->debug_label = "main";       // Only present in Debug builds
    log("connection created");
#endif
    conn->start();
}

Under Debug builds (-O0, without NDEBUG defined) and Release builds (-O2 -DNDEBUG), the visible symbols, diagnostics, and completion suggestions can all differ.

For header files, the compilation context falls into two cases. First, it helps to understand what "self-contained" means: a header file is self-contained if it does not depend on #define or #include directives provided by the includer and can compile independently on its own. The vast majority of modern C++ headers are self-contained — they #include all their dependencies and make no assumptions about the includer's state.

Self-contained headers can be treated like source files. However, headers are typically not in the CDB (the CDB only records compilation commands for source files), so even self-contained headers need to find a source file that includes them via the dependency graph and borrow that source file's compilation command (search paths, language standard, etc.). The key distinction is that self-contained headers do not need synthesized prefix code — borrowing the compilation command is sufficient.

Strictly speaking, "self-contained" is a tendency rather than an absolute property. Even if a header can compile independently, the includer can still alter its semantics through -D flags or preceding #define / #undef directives. In practice, however, this is rarely done — a header designed to be self-contained should not depend on external preprocessor state.

Non-self-contained headers have compilation results that depend on the source file that includes them. A classic example is X macros:

// errors.def — not self-contained, requires the includer to define the X macro
X(Ok,           0, "success")
X(NotFound,     1, "not found")
X(InvalidArg,   2, "invalid argument")
X(Timeout,      3, "operation timed out")

Different source files include the same errors.def, each defining a different X macro, expanding into entirely different code:

// error_enum.cpp — generates an enum
#define X(name, code, msg) name = code,
enum ErrorCode {
#include "errors.def"
};
#undef X

// error_message.cpp — generates a string mapping
#define X(name, code, msg) case code: return msg;
const char* error_message(int code) {
    switch(code) {
#include "errors.def"
    }
}
#undef X

errors.def expands into completely different ASTs in the two source files — one is an enum definition, the other a switch statement. For such non-self-contained headers, the compilation context is called a Header Context, determined by two elements:

1. Host source file: A source file that directly or indirectly includes the header, providing the base compilation command
2. Include position: The specific position in the host source file's include tree where the header is included

The include position is important because the same header may be included multiple times within the same source file, with different preprocessor states each time (e.g., in the X macro pattern, a different macro is defined before each inclusion). All include positions in a source file can be flattened into an ordered sequence, with each position assigned a unique number. The host source file plus this number uniquely identifies a header context.

How the Compilation Context is Determined

When a file needs to be compiled, the compilation context is automatically determined by the following priority:

1. User selection takes priority. If the user has actively selected a compilation context via clice/switchContext, that selection is used. Choices are persisted in the workspace cache and restored automatically when the file is opened in a later session.

2. Direct CDB lookup. If the file has an entry in the compilation database, that entry's compilation command is used. In multi-configuration projects where one file has several commands, the first one is used by default; the user can switch to another entry by command hash.

3. Dependency graph lookup. If the file is not in the CDB (the common case for headers), the dependency graph's include relationships are used to find a source file that includes it, and that source file's compilation command is borrowed. Host candidates are ranked by relevance: a source sharing the header's stem (utils.h -> utils.cpp) wins, then sources in the same directory, then path proximity, with a lexicographic tie-break for determinism. For self-contained headers, borrowing the compilation command is sufficient; for non-self-contained headers, prefix code must also be synthesized to restore the preprocessor state — the distinction is made automatically, see the next section.

4. Rule default command. A file with neither an entry nor a host takes the default_command of the first matching rule that declares one — the way to describe a project whose files all share one set of flags.

5. Borrowed command. For a file with neither a CDB entry nor any host, and no matching rule with a default_command — a freshly created source or a header nothing includes yet — the command of a nearby translation unit of the same language: a unit in the file's directory (same stem first), else the unit whose header search directories (-I etc.) contain the header, else the unit closest by path (see Inference). The command source is labeled Inferred, the decision log names the lender, and diagnostics about missing files come with a guidance note. A borrowed command serves the editor only; the file is not a member of the build and does not join the background index.

6. Builtin fallback. When everything above falls through, an isolated file compiles with a synthesized builtin command, so basic language services keep working.

Beyond automatic resolution, clice also provides three LSP extension requests that let users explicitly query and switch compilation contexts:

clice/queryContext lists all available compilation contexts for a file. For headers, it returns the source files that can serve as hosts, ranked by relevance; for source files, it returns all CDB entries, using key flags from the compilation command (-D, -std, -O, etc.) as human-readable descriptions, each carrying a command hash identifying the entry. Results support pagination. For self-contained headers and source files, the context list is deduplicated by the entry's identity hash, computed over the frontend-relevant view of the effective command plus the input's slot position and working directory: two contexts with the same identity produce identical compilation results, so only the best-ranked representative is kept — otherwise a widely-included header like <vector> could list thousands of contexts. Non-self-contained headers are not merged this way: different hosts synthesize different prefixes, so every host is a distinct context. When a guard-less header is included several times by one host, each include position is returned as a separate entry distinguished by an occurrence number.

clice/currentContext returns the currently active user selection (host + occurrence, or command hash). If the user has not actively switched (i.e., the automatically selected default context is in use), it returns empty.

clice/switchContext switches to the user's chosen new context. The choice is validated: the host must actually (transitively) include the header, the occurrence must be in range, and the command hash must correspond to a real CDB entry of the file, otherwise the request fails.

Besides the requests, the file's preprocessor-inactive regions (bodies of untaken #if branches) surface through semantic tokens: under the current compilation context, every token inside them carries the inactive modifier and editors render the regions dimmed. A context switch recompiles and the server requests a semantic-tokens refresh, so the client re-pulls and the dimming flips — the most immediate visual feedback of switching.

Switching Build Configurations

Context selection is per file. The workspace-wide counterpart is the build configuration: rules in clice.toml can carry a configuration tag (see Command Resolution), the distinct tags form a menu, and exactly one of them is active in a server process — its tagged rules apply, untagged rules always apply, and everything a file compiles with follows from that: the databases consulted, the default commands, the edits. The active configuration is resolved once at startup, from three layers in priority order:

1. --configuration <tag> on the command line, which pins the session; clice serve, clice index, clice lint and clice inspect all take it.
2. The persisted selection: state.json under the cache directory, next to the store. It survives cache layout upgrades, which discard the store but not this file.
3. default_configuration from the configuration, else the first declared tag.

A name in either of the first two layers that no rule declares is reported as guidance and skipped, and the selection file is left as it is — a renamed tag falls back to the default until the user picks again. That is the server's rule, since an editor must always start; clice index, clice lint and clice inspect run scripted and fail on an undeclared --configuration instead of reporting success for another configuration.

Two extension requests drive the menu. clice/listConfigurations returns the declared tags with the active, selected and default names. clice/switchConfiguration validates the name and persists it as the selection; the running server keeps its configuration, and the request tells the client that a restart applies the choice. Switching is a restart on purpose: a server process holds one configuration's world — its loaded databases, dependency graph, contexts, artifacts and index — and a fresh start into that world is the cold start every session already performs, warm from the configuration's own cache. In VS Code the status bar shows the active configuration whenever the rules declare any; picking another one persists it and restarts the server, after which every open document goes through the ordinary open path and its diagnostics, inactive regions and semantic tokens flip. A restart in socket mode drops every connected client.

Each configuration owns an index library of its own under the cache store, index/<tag>~<hash> (index/default for a configuration without tags): the persisted symbol index, the compile-command snapshot it was built against, the context choices and the artifact records. Switching therefore never mixes rows of two configurations and never reindexes a configuration on the way back: the library it left is reopened as is, and the usual reconciliation picks up only what changed on disk meanwhile. Only the active configuration is indexed. PCH and PCM blobs stay in the shared content-addressed namespaces but carry the configuration in their keys, since the dependency stamps that vouch for a blob live in the configuration's library; header-context files are pure content and shared. Libraries of configurations no longer in use are not reclaimed automatically.

Automatic Self-Containedness Detection

The vast majority of headers are self-contained, and synthesizing a prefix for them is pure waste — it requires computing the include chain, reading every file on it, and pulling in a large number of preceding dependencies. clice therefore uses a try-then-fall-back strategy for headers without a CDB entry:

1. Trial compile: treat the header as self-contained and compile it directly with the borrowed host command (no prefix).
2. Diagnostic scoring: if the trial's diagnostics hit a strictly curated set of "missing context" signals — unknown type name, undeclared identifier, unterminated conditional directive — the header is judged non-self-contained. The set is deliberately narrow: a false positive merely costs one pointless prefix synthesis, while misclassifying ordinary in-progress typing errors would cause meaningless recompiles.
3. Fallback recompile: after the verdict, the header is recompiled with a synthesized prefix. The trial's error diagnostics are never published to the editor; the user only sees the final result.

Files with .def or .inc extensions (the X macro convention) skip the trial and are treated as non-self-contained directly. Only the "needs context" verdict is persisted in the workspace cache — it is the expensive side, and restarts should not repeat the trial; "self-contained" remains a session-local impression whose re-validation is free (the trial is the regular compile). Re-evaluation timing is deliberately split: dependency changes (a chain file saved, a file the header includes changed, external on-disk modifications) re-run the trial, while user typing never does — otherwise a half-typed unknown type name would keep triggering pointless prefix synthesis. Saving the header itself also resets the persisted verdict.

Header Context in Practice

The header context is conceptually clear (host source file + include position), but given a header context, how to actually make Clang compile the header under the correct preprocessor state is an engineering problem that requires choosing an approach.

The prefix synthesis discussed here only applies to headers the user has opened. Headers on disk that are not open do not need special handling — they are included and processed normally when each source file is compiled. The indexing system collects symbol information from headers as part of each source file's indexing pass, and the header's per-file index shard merges the variants produced under different source files (see Index Design for the merging mechanism).

clice uses prefix synthesis + -include injection: based on the host source file and include position from the header context, it extracts all content before the target header along the include chain, synthesizes it into prefix code, writes it to a prefix file on disk, and then injects that file into the compilation command via Clang's -include flag. The core advantage of this approach is its natural compatibility with PCH optimization — -include'd files are processed through Clang's predefines buffer before the main file, so when the preamble PCH is built, the prefix content is baked into the same PCH as the header's own preamble region; even a header with no directives of its own (an X macro style .def file) gets a PCH built for its prefix. On PCH reuse Clang validates and subsumes matching -includes, so the prefix is never processed twice. PCHs are cached by content key (preamble text + canonical flags), so files with identical prefixes automatically share one. When the user subsequently edits the header body, the large volume of header inclusions in the prefix does not need to be reprocessed each time. Detailed rationale for this design choice is in the FAQ section below. PCH construction, caching, and invalidation mechanisms are described in Incremental Compilation Design.

The synthesis process has four stages. The following example illustrates the process. Suppose the project has these files:

// main.cpp
#include <vector>
#define DEBUG 1
#include "utils.h"
int main() { ... }

// utils.h
#pragma once
#include <string>
#include "math.h"
void util_func();

The user opens math.h, which is not in the CDB, so prefix code must be synthesized for it.

Step 1: Find the host source file. Using the DependencyGraph's reverse include graph, perform a BFS upward from math.h to find source files that transitively include it. In this example, math.h is included by utils.h, which is included by main.cpp, and main.cpp has a CDB entry, so it is selected as the host. If the user has specified a preferred host via clice/switchContext, that preference takes priority.

Step 2: Compute the include chain. From host main.cpp to target math.h, BFS on the forward include graph finds the shortest path: [main.cpp, utils.h, math.h].

Step 3: Synthesize the prefix code. Along the include chain, for each file (except the final math.h), read its content, find the #include line that includes the next file, and extract everything before that line. Each fragment is prefixed with a #line directive to ensure diagnostics point to the correct original positions. The synthesized prefix code for this example is:

#line 1 "main.cpp"
#include <vector>
#define DEBUG 1
#line 1 "utils.h"
#pragma once
#include <string>

The content after #include "utils.h" in main.cpp (int main() { ... }) is truncated, and the content after #include "math.h" in utils.h (void util_func();) is also truncated. This prefix code restores the preprocessor state that math.h would see in the original compilation: <vector> and <string> have been included, and DEBUG is defined as 1.

The prefix code is written to a disk cache, named by the xxh3 hash of its content, so identical content is stored only once.

Step 4: Inject into the compilation command. Using the host main.cpp's CDB compilation command, replace the source file path with math.h and inject the prefix file via the -include flag:

clang -std=c++17 -Iinclude -include cache/header_context/a1b2c3d4.h math.h

Clang processes the -include prefix file first, then compiles math.h. The effect is that math.h is compiled as if at the #include "math.h" position in main.cpp, with the correct preprocessor state.

Several engineering details matter here. Include directives are resolved against the host command's real search paths and matched by absolute path, so same-named headers in different directories cannot be confused; since the prefix file lives in the cache directory, quoted includes with relative paths are rewritten to their resolved absolute paths, otherwise they would be looked up against the wrong base directory. Matching prefers includes outside #if blocks (an occurrence in an untaken branch must not shadow the real one); when the cut lands inside #if blocks (most commonly an include guard on an intermediate header), balancing #endifs are appended so the prefix stays well-formed — the guard condition is still evaluated by the compiler, preserving the semantics.

Beyond the prefix there is also a suffix: the content after the include position (mirrored along the chain — the direct includer's remainder first, the host's last) is synthesized into a suffix file and injected by appending a single #include line to the header's buffer at compile time. X macro fragments embedded in enums or function bodies thus see their surrounding braces close — the token stream runs continuously through prefix, main file and suffix. When the cut lands inside #if blocks, the prefix closes them with #endifs and the suffix reopens the same depth with #if 1s, keeping both sides balanced. Includes of the header itself along the chain (other occurrences) are redirected to a disk snapshot of it — the header's own path is remapped to the buffer with the trailing suffix include, so keeping them verbatim would recurse forever. The appended line sits past the editor's visible content, and diagnostics inside the suffix file are never attributed to the header itself.

The synthesis result (host, prefix file path, suffix file path, content hash, and the include chain with a content snapshot) is owned by the context resolver and outlives the editing session — closing and reopening the header reuses it. The chain files' content is embedded in the prefix and the compiler never opens them, so regular dependency tracking is blind to them — staleness is handled by the chain snapshot (two-layer mtime + content hash detection): any change to a chain file re-synthesizes the prefix. Saving a chain file forces content re-validation even when its mtime is unchanged.

Multi-Context in Indexing

Compilation context affects not only live compilation but also index construction. The same header file may produce different symbols under different compilation contexts. For example:

// config.h
#ifdef USE_V2
    using Handler = AsyncHandler;    // Visible in context A
#else
    using Handler = SyncHandler;     // Visible in context B
#endif

If the index only records the result from one context, go-to-definition or find-references would miss information from the other context. Each file's index shard therefore stores the rows produced under different compilation contexts as separate variants, and the project index records which translation units contributed them; queries return the union of the live variants — for config.h, find-references can find references to both AsyncHandler and SyncHandler. See Index Design for the specific merging and deduplication mechanisms.

FAQ

• Can the same source file provide multiple contexts for the same header?

  Yes. If the header has a header guard or #pragma once, multiple #include directives for it are effective only the first time, so a source file provides at most one context. But if the header has no include protection (e.g., X macro-style .def files), every #include introduces a new context. As described above, all effective include positions within a source file are numbered in order, so even when the same header is included multiple times, each inclusion can be uniquely distinguished by its number.

• Is injecting the prefix file via -include fully equivalent to the original compilation?

  For mechanisms defined by the C++ standard, yes. The only ways user code can observe file names and line numbers are __FILE__, __LINE__, and std::source_location. According to the C++ standard, #line directives change the values of __LINE__ and __FILE__, and std::source_location behavior is also determined by them. Therefore, the #line directives in the prefix file ensure that all three correctly reflect the target header's position information.

  However, two GCC/Clang compiler extensions behave differently: __INCLUDE_LEVEL__ is 0 under -include mode (the target file is the main file), whereas in the original compilation it reflects the nested include depth; __BASE_FILE__ returns the target header's path under -include mode, whereas in the original compilation it returns the host source file's path. These two extensions are rarely used in practice and have no impact on language server functionality.

• Why write the prefix file to disk instead of using a virtual file?

  Clang supports a virtual file system, so in theory the synthesized prefix code could exist only in memory without being written to disk. The choice to write disk files is mainly for debuggability: when users encounter problems, they can directly inspect the prefix file contents on disk, making it easier to diagnose issues and file bug reports. There is no particularly deep technical reason.

• Why not use an approach that "compiles the host source file as the main unit and stops at the target position"?

  Another approach that was considered: instead of synthesizing a prefix file, compile the host source file as the main compilation unit directly. Clang provides a declaration-level callback mechanism that allows checking the current source location after each top-level declaration is parsed. Using this mechanism, compilation can be stopped when parsing reaches the end of the target header based on the include position from the header context. At that point, the compiler has naturally accumulated the correct preprocessor state and can provide language services for the target header directly in that state.

  This approach has clear advantages: no files need to be synthesized, no include chains need to be tracked, and specifying the host source file alone solves the problem. Moreover, for non-self-contained headers like X macros that are embedded inside function bodies, since the compilation subject is the complete source file, the surrounding context (braces, statements, etc.) before and after the include position is intact, so there are no mismatched syntax issues.

  However, this approach has a critical flaw: it cannot effectively leverage PCH optimization. PCH works by precompiling and caching a contiguous preamble segment (typically a sequence of #include directives) at the beginning of a file. In the prefix synthesis approach, the target header is the compilation subject, and the synthesized prefix code is its preamble, which can be fully cached by PCH. But in the host compilation approach, the compilation subject is the host source file, and the target header is embedded somewhere in the include tree — typically not in the host source file's preamble region but somewhere in the middle of the file. Clang's PCH mechanism cannot split at such a position, so none of the content before the target header can be cached by PCH.

  For a header the user is actively editing, the file body is the most frequently updated part, requiring recompilation on each edit. Without PCH caching for the prefix content, every compilation would need to reprocess the large volume of preceding header inclusions in the host source file — an unacceptable cost in large projects. The prefix synthesis approach makes the target header the compilation subject and the prefix code a separately cacheable preamble, fully preserving PCH generation and usage.

Known Limitations

• Occurrence numbering only covers the direct includer. By design, a header context is uniquely determined by the host source file plus a position number in its include tree. The current implementation's occurrence only distinguishes multiple includes of the target in its direct includer (which covers the typical X macro usage), not a flattened numbering over the host's entire include tree — when the same header is transitively included by one host via different intermediate paths, only the context corresponding to the shortest chain is presented.

• The shortest chain may not match the real first inclusion. The include chain from host to target is the shortest path in the include graph. In a real compilation the header may first be reached via a different, longer path, and the preprocessor state accumulated before it may differ. In practice this rarely causes observable differences, but strictly speaking the synthesized state may not correspond exactly to any real compilation.

• Cross-configuration conditional includes within one TU. Dependency scanning records include edges per compilation configuration, but host lookup uses the union across configurations. In extreme cases, an include edge that only holds under configuration A could lend configuration B's command to a header.

• Switching a build configuration restarts the server. The selection is persisted and applied by the next server start rather than swapped in-process; open documents are re-opened by the client afterwards. Without any declared database, discovery loads every compile_commands.json at the workspace root and in its immediate subdirectories, and the ones above a file when it is opened; a nested project's database is not scanned for before then.