Semantic Tokens
clice classifies every token of a document with its own token-kind vocabulary, which is richer than the standard LSP token types and consistent across all clice replies. Clients that prefer standard LSP kinds can map them through configuration.
Lexical Tokens
Kinds derived from the token stream itself, independent of the AST.
Comments
Line, block and documentation comments receive comment tokens
Literals
Numbers, characters and strings receive literal tokens
Keywords
Alternative operator spellings and contextual specifiers retain keyword tokens
Preprocessor directives
#if chains keep directive kinds; disabled branches keep lexical kinds; pragma arguments stay plain
Inactive regions
Tokens in untaken branches keep their lexical kinds and carry the inactive modifier; unclassified tokens become plain identifier carriers, so even a lone } line dims
Header names
Quoted and angled include filenames receive string tokens
Inactive preamble regions
Untaken branches among the leading directives dim the same way
Literal prefixes and suffixes
Literal prefixes, suffixes and separators do not receive distinct tokens yet
Escape sequences
Escape sequences are not highlighted distinctly inside literals yet
Declarator vs operator disambiguation
Declarator and expression operators do not receive distinct token kinds yet
Primitive token type
Built-in types use a distinct token kind instead of plain keyword
Bracket token types
Matching brackets do not receive pair-specific token kinds yet
Declarations
Names classified by the declaration they define or reference.
Namespaces
Namespace definitions, references, nesting and aliases receive namespace tokens
Types
Type definitions and references keep their respective type kinds
Functions and methods
Function declarations, definitions and calls receive function tokens
Variables
Variable declarations and references keep their respective variable kinds
Templates
Template parameters receive type or variable kinds, and template names carry templated
Concepts
Concept definitions and constraint uses receive concept tokens
Labels
Labels and their goto references receive label tokens
Structured bindings
Structured binding names receive variable tokens at definition and use
The opening [ deliberately carries no token; only the binding names themselves are highlighted.
Lambda init-captures
Lambda init-captures receive variable tokens
Deduction guides
Deduction guides and their guided templates receive type tokens
Explicit instantiation
Explicit class instantiations highlight template names and written arguments
Variable templates
Variable template declarations, definitions and specializations receive variable tokens
Out-of-line member definitions
Qualified names keep method kinds and modifiers
Alias templates
The alias name carries the type kind and the templated modifier
Template template parameters
Template-template parameters receive type tokens at declaration and use
Friend declarations
Befriended names resolve to their targets; inline friends define
Function explicit instantiation directives
Identifiers in a function explicit-instantiation directive are painted
Variable explicit instantiation directives
Identifiers in a variable explicit-instantiation directive are painted
Explicit instantiation member bodies
A dependent name paints as its actual resolution: agreeing kinds keep the modifiers all instantiations share, disagreeing kinds paint a conflict
References
Reference sites retain the semantic kind of the declaration they resolve to, including names reached through language-specific lookup rules.
Member initializer lists
Member initializer lists highlight initialized names as fields
Using declarations
The introduced name keeps its target's kind
sizeof...
The pack parameter keeps its type-parameter token
using enum
Using declarations highlight enum names at the using site
Dependent names
Dependent names resolve through known primary templates
Dependent members of a known template (Box<T>) resolve to the primary template's declarations and keep their kinds. Members of a bare template parameter have no candidate declaration and currently get no token; heuristic coloring for such names remains open.
Lambda captures
By-copy and by-reference captures reference the captured variable; this stays a keyword
Range-based for
Range-for variables keep variable tokens at definitions and uses
Enum underlying types
The enum-base reference keeps its type kind
Dependent using declarations
Dependent using declarations remain unpainted
The introduced name and its uses currently get no token; the reserved dependent-name modifier is not emitted yet.
Modules
Module declarations
Module declarations tokenize contextual keywords, dotted names and private fragments
Module partitions
Module declarations tokenize partition names
module and import as identifiers
Contextual keywords keep their semantic kinds outside module declarations
Token Modifiers
Declaration vs definition
Declaration and definition modifiers distinguish the two sites
Static
Static members and locals carry the static modifier
Readonly
Const values and methods, plus enum members, carry the readonly modifier
Readonly is currently value-based: a pointer to const counts as readonly even though the pointer itself can change.
Virtual and abstract
Virtual methods and abstract classes carry virtual or abstract modifiers
Deprecated
Deprecated declarations and uses carry the deprecated modifier
Default library
Symbols from system headers carry the default-library modifier
Scope modifiers
Symbols do not carry function, class, file or global scope modifiers yet
Mutable reference and pointer
Mutable reference and pointer arguments do not carry a modifier yet
Deduced
Deduced types do not carry a dedicated modifier yet
User-defined operators
Overloaded operators do not differ from built-in operators yet
Conflict & Ambiguity
C++ allows structurally different entities to share one name. When a single written name refers to entities of different kinds at once, no single token type is correct; such names receive the dedicated conflict token type, which clients typically display in a neutral color.
Type vs function
A name naming both renders as conflict
Type vs variable
A name naming both renders as conflict
Same-kind overload sets
A name naming only functions is no conflict
Injected class name
An injected class name keeps its class token when used as a constructor
The written name renders as the class; the constructor reference it implies paints nothing extra — the ( stays token-free.
Token Correctness
Shapes clice pins deliberately, including issues clangd got wrong.
Constructors and destructors
Constructors and destructors use method tokens with dedicated modifiers
A destructor name renders as two tokens: the ~ carries the method kind and the declaration/definition modifiers, the class name after it stays a reference to the class.
Anonymous parameters
Unnamed parameters produce no tokens
The punctuation after an unnamed parameter's type stays token-free.
Operator names
The operator keyword and call-site punctuation stay plain
An operator's written name is keyword plus punctuation, so no name token is painted: operator keeps its keyword classification and call sites emit nothing on the operator symbol.
Destructors of class templates
The ~ shape holds under templates
Conversion operators
Written as keywords, converting uses paint nothing extra
Pseudo-destructor on a template parameter
The ~ paints nothing; the type name keeps its kind
Defaulted and deleted members
Special-member names keep their definition tokens
Attributes
Attribute names
Attribute names and their expressions do not receive semantic tokens yet
Macros
Tokens inside macro definition bodies keep their lexical kinds; highlighting them from their expansions belongs to a future expansion-preview feature.
Macro definition and expansion
Macro definitions and expansions receive semantic tokens
Expansion sites and arguments
Expansion names are macros, written arguments keep their semantics, definition bodies stay lexical
Object-like vs function-like macros
Object-like and function-like macros do not receive distinct token kinds yet
Other Known Gaps
Curated issues without a fixture yet:
-
autoparameters must not be highlighted as template type parameters (clangd#1390) - Nested name specifier in a pointer-to-member should get a token (clangd#2235)
-
::newshould keep thenewkeyword highlighted (clangd#1627) -
co_yield/co_awaitlose highlighting when the coroutine return type is a template (clangd#2437) - Token modifiers should apply to operands of overloaded operators (clangd#2547)
- Dependent template names (
obj.template get<int>()), members imported from a dependent base viausing, and dependent names with mixed-kind overload sets (clangd#484, clangd#686, clangd#1057)
Inactive Code Regions
Every token inside an untaken preprocessor branch carries the inactive modifier while keeping its lexical kind, so editors dim the region by styling the modifier without losing the syntax colors underneath. Tokens without a classification in dead code — bare identifiers and plain punctuation — are emitted as the unstyled identifier type, giving the whole region token coverage. The clice VS Code extension renders the regions dimmed out of the box; other editors style the modifier directly (e.g. @lsp.mod.inactive in Neovim).
- Dim inactive preprocessor branches (clangd#132)
- Correct inactive boundaries with
#elifchains (clangd#602) - Preserve syntax highlighting within inactive regions (clangd#1664)
- Keep inactive regions distinct from comments (clangd#1545)
- Unreachable code dimming (clangd#1828)
Format String Highlighting
-
std::format/std::printplaceholder highlighting (clangd#1709) - Highlight invalid format specifiers as errors
Protocol Support
- Full document semantic tokens (
textDocument/semanticTokens/full) - UTF-16 delta-encoded token positions
- Range-based semantic tokens (
textDocument/semanticTokens/range) — only compute tokens for the visible viewport, critical for large files - Delta updates (
textDocument/semanticTokens/full/delta) — send only changes since the previous response