Repository navigation
Tags
Tags
fix(server): no project without its cache; lone files get one (#1088) ## What changed **A project never runs without its cache.** Before, a project whose cache could not be used ran without one: no PCH, no module builds, the index kept in memory and lost at exit, with a line in the log the user rarely sees. Now it serves nothing, and the user is told which directory, why, and what to do, then to restart clice: | When | Told | | --- | --- | | the cache directory cannot be created or written | give clice write access to it, or set `cache_dir` in clice.toml | | its index writer lock cannot be opened | same | | another project owns the directory | set `cache_dir` to a directory of this project's own | | it is on a remote filesystem (NFS, SMB, a Windows drive under WSL) | set `cache_dir` to a directory on a local disk | | its index database cannot be read | delete its `index` directory, or set `cache_dir` elsewhere | | every directory the project could use is taken by the server's other projects | set `cache_dir` to a directory of its own | | clice's temporary directory, for the files outside every folder, cannot be used | point TMPDIR at a directory of your own | `clice lint` without `--index` refuses an unusable cache as `clice index` already did. The reading commands (`clice query`, `clice inspect`) are unchanged. With the degraded mode gone go its code paths: the PCH, PCM and index builds' branches for a project without a store, the worker's "index too large to send" for a reply with nowhere to write, the memory-only session over an unreadable database header. Users on WSL who open a project under `/mnt/c` now get that error until they set `cache_dir` to a local directory; before, it ran with an index that was never saved. A file opened with no workspace folder and no project above it (`code test.cpp`, Neovim's single-file mode) is a quick try: it now works as a folder's file does, and leaves nothing behind. - **A cache of its own.** Such files had no cache directory, so no PCH and no module builds: go to definition into a header they include answered nothing while hover worked. Their project now keeps a cache in clice's temporary directory, `<temp>/clice-<uid>/<pid>/` (`<temp>\clice\<pid>\` on Windows), whatever `cache_dir` the editor's settings give (that is the folders'), holding only PCHs and module files. It is removed when the project goes, once nothing maps its files; a server killed before leaves it to the next one, which removes the directories of exited servers. - **One temporary directory.** clice's other temporary files go there too: the compiler query's input, the control socket when the cache directory's path is too long for one, `clice inspect`'s PCH. The directory is the user's alone (0700); one of that name another user owns is an error naming it, never written into. - **Rename.** A rename from such a file was refused outright (`a rename edits the sources of a workspace folder; open one`). One whose edits all stay in the file is now made; one that would change another file is refused, naming it. The configuration page says so now, in English and Chinese: the `cache_dir` option's description (and the schema), the paragraphs on cache fallback and network filesystems, and a sentence on the files outside every folder. ## Tests - `compile/cache/13_unusable_cache_dir` now pins the error: the server serves nothing and says why; `clice lint` exits 2 with the same message. The cases of the degraded mode go (`contexts/self_containment/08_unusable_cache`, `index/transfer/01_large_unit`'s "oversized index without store"), as does the worker's pathless unit test. - New case `project/folders/21_cache_dirs_taken`: a project whose every cache directory is taken serves nothing and says so. - New case `project/folders/20_lone_file`: go to definition into an included header (answers nothing on main), the cache removed with the server under a preamble index large enough to be mapped, a new lone file waiting for the previous one's project to go, and an unusable temporary directory reported. - `rename/refusals/04_no_workspace_folder`: the lone-file rename now succeeds; a rename of a symbol declared in a header beside the file is refused naming it. - The unit tests' served-project and indexer fixtures carry a cache store, as every project that builds now does. - Unit tests for the temporary directory's mode and the removal of exited servers' directories; the control socket's long-path test follows the socket's new place. - The case framework gives each server a temporary directory of its own under a non-ASCII root, as it does the workspace and cache.
fix(server): no project without its cache; lone files get one (#1088) ## What changed **A project never runs without its cache.** Before, a project whose cache could not be used ran without one: no PCH, no module builds, the index kept in memory and lost at exit, with a line in the log the user rarely sees. Now it serves nothing, and the user is told which directory, why, and what to do, then to restart clice: | When | Told | | --- | --- | | the cache directory cannot be created or written | give clice write access to it, or set `cache_dir` in clice.toml | | its index writer lock cannot be opened | same | | another project owns the directory | set `cache_dir` to a directory of this project's own | | it is on a remote filesystem (NFS, SMB, a Windows drive under WSL) | set `cache_dir` to a directory on a local disk | | its index database cannot be read | delete its `index` directory, or set `cache_dir` elsewhere | | every directory the project could use is taken by the server's other projects | set `cache_dir` to a directory of its own | | clice's temporary directory, for the files outside every folder, cannot be used | point TMPDIR at a directory of your own | `clice lint` without `--index` refuses an unusable cache as `clice index` already did. The reading commands (`clice query`, `clice inspect`) are unchanged. With the degraded mode gone go its code paths: the PCH, PCM and index builds' branches for a project without a store, the worker's "index too large to send" for a reply with nowhere to write, the memory-only session over an unreadable database header. Users on WSL who open a project under `/mnt/c` now get that error until they set `cache_dir` to a local directory; before, it ran with an index that was never saved. A file opened with no workspace folder and no project above it (`code test.cpp`, Neovim's single-file mode) is a quick try: it now works as a folder's file does, and leaves nothing behind. - **A cache of its own.** Such files had no cache directory, so no PCH and no module builds: go to definition into a header they include answered nothing while hover worked. Their project now keeps a cache in clice's temporary directory, `<temp>/clice-<uid>/<pid>/` (`<temp>\clice\<pid>\` on Windows), whatever `cache_dir` the editor's settings give (that is the folders'), holding only PCHs and module files. It is removed when the project goes, once nothing maps its files; a server killed before leaves it to the next one, which removes the directories of exited servers. - **One temporary directory.** clice's other temporary files go there too: the compiler query's input, the control socket when the cache directory's path is too long for one, `clice inspect`'s PCH. The directory is the user's alone (0700); one of that name another user owns is an error naming it, never written into. - **Rename.** A rename from such a file was refused outright (`a rename edits the sources of a workspace folder; open one`). One whose edits all stay in the file is now made; one that would change another file is refused, naming it. The configuration page says so now, in English and Chinese: the `cache_dir` option's description (and the schema), the paragraphs on cache fallback and network filesystems, and a sentence on the files outside every folder. ## Tests - `compile/cache/13_unusable_cache_dir` now pins the error: the server serves nothing and says why; `clice lint` exits 2 with the same message. The cases of the degraded mode go (`contexts/self_containment/08_unusable_cache`, `index/transfer/01_large_unit`'s "oversized index without store"), as does the worker's pathless unit test. - New case `project/folders/21_cache_dirs_taken`: a project whose every cache directory is taken serves nothing and says so. - New case `project/folders/20_lone_file`: go to definition into an included header (answers nothing on main), the cache removed with the server under a preamble index large enough to be mapped, a new lone file waiting for the previous one's project to go, and an unusable temporary directory reported. - `rename/refusals/04_no_workspace_folder`: the lone-file rename now succeeds; a rename of a symbol declared in a header beside the file is refused naming it. - The unit tests' served-project and indexer fixtures carry a cache store, as every project that builds now does. - Unit tests for the temporary directory's mode and the removal of exited servers' directories; the control socket's long-path test follows the socket's new place. - The case framework gives each server a temporary directory of its own under a non-ASCII root, as it does the workspace and cache.
fix(editor): completion gaps, clearer messages, slow tidy checks (#1082) ## Related issue Fixes #1017. Fixes #1080. ## What changed Editor papercuts, completion gaps and clice's own messages, each saying who reports, what is wrong, where and what to do. ### Completion - In a function template or generic lambda the code instantiates with several types, completion offers the members every instantiation has, each name's overloads across them in one item (at most eight compared; inside an `if constexpr` branch only the instantiations taking it). Before, it offered nothing. - A name a header the file does not include yet declares completes after a written namespace (`std::vec`, `::ns::inner::Na`, or `inner::Na` inside `namespace ns`), with its `#include`, also when the file does not know the namespace yet. A list after a bare qualifier (`std::`) is marked incomplete, so the first letter typed asks again. - `#` at a line start lists the directives. - Completion inside assertion macros (`EXPECT_EQ(u.|...)`) no longer reads every name of the PCH for the completion clang throws away while it parses the macro's condition tentatively: fmt's `EXPECT_EQ(..., u.⟦⟧str())` takes about half as long (A/B on one box, both orders). - `clice inspect code-completion` answers through the editor's entry, so `#include` paths, directives and module names match the server. ### Messages - A module that does not build: `module 'B' does not build: <first error> (and N more errors); imported through A → B`, source clice, code `module-not-built`, a note leading to the error; a cycle says so (`module-cycle`). Header units say clice does not build them (`header-unit`). - A worker crash note says it is a bug of clice, where the crash reports are, and links the issue tracker (`worker-crash`); the pool's "keeps crashing" popups too. - The inferred-command warning names the file whose command was borrowed, and says when a file lies outside the project. - A compiler that cannot be queried is named in one sentence, with what to do. - A `compile_commands.json` that is no valid JSON is not loaded, and the message gives the line and column. - A command whose directory does not exist warns at the top of the file (`missing-directory`); files the database names that do not exist are one log line per load. - A `.clang-tidy` that does not parse points at its lines; an extension written with its dot is refused instead of crashing the worker. - Project files that are not UTF-8 are named once at load, with the fix. - A client that takes only pushed diagnostics gets them anew when it saves an edited file, when a save retries a crashed compile, and when a change on disk (a saved header) leaves an open file stale: the open files recompile, as a pulling client's re-pull makes them. Saves in a row compile an open file twice at most. ### Slow clang-tidy checks (#1017) The checks too slow for an editor run on open files when `slow_clang_tidy_checks = true` under `[diagnostics]`; unset, clice names the slow checks the `.clang-tidy` enables once; `false` stays silent. `clice lint` runs them all. ### Other fixes - Hover keeps a written `constexpr` on a defaulted function and none clang adds. - Hover of an alias read from the preamble's PCH names the parameters of what it stands for (`__make_integer_seq<sequence, unsigned long, Size>`, not `value-parameter-0-0`): clang reads a dependent specialization of a builtin template back as an alias of its canonical form. - A header compiling in its host's context keeps the verdict of the self-containment check it just ran, and takes its host's include tree again when the host changed while it was read: a host saved meanwhile no longer leaves the header compiled alone, with false errors, as current. - Formatting a whole selected line leaves the next line alone. - Add #include is offered with the cursor right after the name (`printf|(`). - Define in host goes after the host's own definitions of the class's members. - Member and designated initializers highlight as writes (index format bumped). ### VS Code - clice's commands sit together in the editor's context menu below Go to Definition: Switch Header and Source (Module Interface and Implementation), Switch Compilation Context (when the file has more than one), Switch Build Configuration (when the rules declare any), Show Compile Command. The three `{}` language status items are gone. - Standard library headers without an extension open as C++ by their first line. - A header name typed after a dead end asks again (Enter on the line does not); extensionless headers get the header icon. Accepted limitations in this area are now `// LIMITATION:` comments where a fix would go. ## Tests - Unit and case suites on a Linux box (RelWithDebInfo), the full case suite five times in a row; the Windows-only cases and the VS Code e2e on a Windows box. - The header-context races, under 40 parallel copies of `project/unlisted_files/08_member_hosts_headers`: about 80 failures in 400 runs before, 1 in about 1200 after (a host read while it is being written, tracked separately). - The Zed e2e's download test reports the extension's own reason (GitHub's answer) when it starts no download. - New cases: several instantiations, written scopes, tentative parses, defaulted constexpr hover, broken JSON database, missing directory, outside the project, dotted tidy extension, other encodings named, selection through a newline, cursor after a name, host member defined twice, saves in a row, a save retrying a crashed compile; snapshots updated for the new texts and codes. - fmt A/B (`node tools/ecosystem/run.ts perf fmt/editing`, both orders): member completion in `EXPECT_EQ` 55–63 ms on main, 25–31 ms here; answers unchanged.
ci: trustworthy CI and ecosystem round, tests to one standard (#1076) ## What changed CI and the nightly ecosystem round become signals worth trusting, and the tests something to read with confidence. ### The tests say exactly what clice does - The feature unit tests that compiled a source and checked an answer (completion, semantic tokens, folding, selection, outline, navigation, document links, diagnostics, code actions, formatting, hover, inlay hints, rename, counterpart ranking) are now cases, merged by behavior into the existing ones instead of one case per old test. A review of every deleted test restored the checks the move had lost. The unit tests that stay check what no product boundary shows cheaply. They have exact expectations, no duplicates, names of at most four words, and they wait on events instead of polling. Each anomaly trap asserts the anomalies its test provokes. Fixtures skip the toolchain query and run under a watchdog. - Cases show blocks and exact checks instead of partial matches, and anchors instead of literal positions. Known-wrong reasons are written so a docs reader can follow them. Crash spacing and slot revival use test hooks instead of waiting it out (a crash case went from 33 s to 6 s). The modularize cases compile the build their plan describes, every configuration of a merged plan included, through one framework helper. - The teardown gate reads every process a test started: servers, commands, `inspect` runs. Each process's session is found by pid, and stderr is read for one that wrote none. One classifier of clice's log lines serves the cases, the editors and the ecosystem round: a worker death counts once, anomalies by id, any other error-level line. A declared error that no line matches fails, and so does a command killed by a signal the test did not send. - Coverage of the product surface (every method, command, flag and config key) is recorded on the linux-x64 shards, and a CI job fails on what nothing exercised (`pixi run coverage-test`). - The editor clients run end-to-end on fixtures of their own, `tests/editors/NN_*` with one `scenario.json` read by both Neovim and VS Code: contexts and paging, source/header switching, refactoring, build configurations, restarts, VS Code's check for conflicting C++ extensions. The servers' logs are gated afterwards. Zed's test installs this build as the extension's newest version. - Recorded LSP sessions and their replay are gone. `CLICE_LSP_TRACE=1` writes the LSP messages of a server an editor starts to `trace.log` beside the other logs instead. ### CI - Every target builds on x64 Linux, macOS included (cross-compiled with xclang), and runs its suites on its own OS. - `main.yml` runs on `merge_group` with every filter on, the docs-site check included, and the commit-message check reads the queued commit, ready for a merge queue on main. Main pushes keep their full run: releases promote the packages a green main run tested. - Every job and the slow install steps have a time bound. Windows runners turn Defender's real-time scan off before installing. On main pushes, old cache entries are pruned to the newest one per key. - The unit runner takes zest's own command line and runs one test at a time. ### Product fixes the stricter tests and the round found - **Shutdown**: the background index stops before the workers. A stopping pool used to fail every dispatch, so a pass walked its whole queue through it before the stop could save. The include-tree runs are cancelled with it and awaited once the pool has stopped. Shutdown is also bounded: stuck for 60 s, the server logs the step it is stuck at and exits. - **Toolchain queries**: a compiler query that has not answered within 5 s is stopped, along with the programs it started, and clang reads the command itself. On Windows, what a program of the Cygwin or MSYS2 runtime starts leaves the query's job and outlives the stop; native drivers' programs do not. - **Self-containment**: a header is judged by which compile has fewer errors, alone or in its includer's context, counting the compiler's errors in every file and in the preamble. A tie keeps the header self-contained. C's warning about a struct first named in a prototype counts as an error, clang-tidy's findings do not. Without a synthesized context there is no comparison, and a comparison an edit cuts short is made again. This replaces the list of "missing context" diagnostics. - **Hover**: macro expansions that clang-format cannot lay out quickly (blocks inside expressions, deep nesting) keep their own line breaks instead of stalling the hover. The index card of an Objective-C class shows its `@interface`, attributes included. - **Template parameters**: lookups of a substituted parameter across merged patterns are bounds-checked (index, signature help, hover, unification, bindings). The crash needs clice's own module layering to reproduce; the ecosystem round's clice regression job covers it. - **Rename**: a declaration in a file outside the project refuses the rename as "not a source of the project" whether or not the background index has reached that file yet. Before, a rename right after opening said the file "changed since it was indexed". - **Logging**: `CLICE_LOG_LEVEL` sets the stderr log level, replacing `--log-level` on every subcommand (Neovim's config sets it through `cmd_env`). All logging switches are now environment variables. ### Toolchain LLVM 23.1.5 (xclang 23.1.5.1), rules_cc 0.2.28, index format 63. ## Tests - Box runs on the final tree: Linux RelWithDebInfo and Debug (ASan), unit and every case, green. The Windows-specific toolchain test also ran on a Windows box. macOS arm64 RelWithDebInfo and Debug, cross-built on Linux, were green on the tree before the review's fixes. - Negative controls for the new tests: the compiler-query test fails without the process group (Linux and Windows); the rename case fails without its fix, with exactly the message an earlier CI run hit; the log classifier's new test fails on the old classifier; the known-issue matcher's new test fails on the old pattern. - Ecosystem round on this branch: [run 43289149904](https://github.com/clice-io/clice/actions/runs/43289149904), 76 jobs. - Two red jobs (abseil/clang on Windows, abseil/gcc on Linux), both the known highlights disagreement in template code: the server's answer holds more than the unit's own index. This branch had narrowed that known entry to diffs that only add lines, which missed the shape where `inspect` answers nothing. The entry is fixed here, with a test, so the next round reads them as known. - Perf: one slower row confirmed, a cold `initialize` on macOS at 1.21×, which the night before (on main) measured too. `llvm/sessions` edit-after-open is 1.01×; the 1.25× of the round before was noise. - The regression numbers that rose 1.5× or more (clice on Linux, flang on Windows) rose in every phase, the checkout's preparation included, while the perf jobs' A/B on one runner finds indexing unchanged: different runners.
refactor(index): derive symbol records from shards, fix known defects (… …#1075) ## Related issue Fixes #1066. ## What changed The index is rethought around one rule: **what the index says about a symbol is read off what it stores, never reported beside it.** That removes a class of defects instead of patching them, and several mechanisms with it. `src/index/` gets the subdirectories the other layers have. The persisted format moves to 62: an existing index is rebuilt once. ### Design - **Symbol records follow the shards.** A record's files — naming, declaring, defining the symbol, instantiating code that names it — are derived by diffing a file's old and new shard when it is replaced, instead of being a union of what units reported that never shrank. A file whose entries leave takes its symbols along; a symbol is defined exactly while some file defines it (`HasDefinition` is no longer stored anywhere); every file declaring a symbol is known, not one. - **Symbols below external scope that several files name are records of the same kind.** The per-unit sets in the manifests, their own record type, the walk over them at query time and the check at merge time go. A named namespace enclosed by an anonymous one now takes its first declaration's file into its identity, as every other internal entity does, so two units' namespaces of one name are two symbols. - **Shards, instantiation shards and manifests are kept like every other record** (changed, written, cached), with one writer each; eleven fields of hand-kept state go. - **A preamble's index is a container of the worker layer** around a plain TU index. The TU index loses its preamble fields and accessors, the query layer no longer judges whether a preamble matches a buffer, and `DocumentLink` moves to the feature layer. ### `src/index/` `shard/`, `indexing/`, `project_index/`, `database/`, `search/`, `query/`, each with its persisted layout beside its reader; `symbol`, `format` and `columns` are what they share. The first commit is the move alone. ### Defects fixed - The index of a project including library templates grew by half: an overload set a library template names without a resolver was recorded from what the unit happened to declare before the header, a variant per unit. - A header moved or copied unchanged — every update of the VS Code extension moves clang's own headers — made each unit's first index result be rejected and run again: the variants and instantiation keys a unit may leave out are now told per file. - A symbol several places define (a weak and a strong function, a `main` per program) showed once in `workspace/symbol` and `clice query symbol-search`; `clice query definition` gave one. Each definition is listed now. - `path:` found a symbol only by the file of its definition, not by the headers declaring it. - A macro of the compile command (`-D`) or of the compiler answered "symbol not found" to `clice query`: references, definition and symbol search answer, naming `<command line>` or `<built-in>`. - A save the database refused was retried forever without a word: the user is told once, and past 256 MB of unsaved index a server stops indexing until a save commits. - References and the hierarchies asked from one file silently lacked the uses in another open file edited since its last compile: they wait for that compile. - A compile with errors that read a file whose open document has unsaved text says so at the top of the file. - An evicted document answered document symbols, folding ranges and semantic tokens with null when none was asked before the eviction. - A file turned to compiling after its index attempt failed kept the empty tokens and folds it was answered meanwhile until the next edit. - Semantic tokens of a file with code its context skips classified every name of the file from the index on each request; only the skipped code is, now. - Opening a file again read and hashed its preamble's index whole each time; once per process is enough. - An open file's references missed what its preamble's headers instantiate until the background index reached them: the preamble's index holds those records. - With `skip_header_bodies = false`, an open source kept its headers out of project-wide answers until it closed: open documents are indexed in the background like closed ones. ### New option `project.index_library_bodies` (default `false`): the index leaves out the bodies of the functions system headers define. The compile is unchanged — diagnostics and instantiations are what they were; only the names used inside those bodies are not walked, in every unit that includes the library. A header opened in the editor is indexed whole. On abseil (425 units) the default stores 20.2 MB of file shards in 209 s where the option stores 22.4 MB in 218 s. ### Known limitations Written where a fix would go, as `// LIMITATION:` comments: a symbol flag only some units report, the cost of merging a header that differs in every unit, the stall of preparing a save, instantiation keys that resolve to nothing, a database whose lock file the reader may not write, hover from the index without type facts, and the two prices of `skip_header_bodies`. ## Tests Unit tests and cases pass on Linux in RelWithDebInfo and Debug. New cases: `query/names/15_declaring_header_path`, `16_several_definitions`, `17_predefined_macros`; `workspace_symbols/results/07_several_definitions`; `navigation/projects/13_unsaved_edit_references`, `14_preamble_instantiations`; `index/read_only/12_failed_attempt_refresh`; `index/batch/10_library_bodies`. New and rewritten unit tests cover the derived records (a definition leaving its file, definitions of several files, a symbol moving between two files of one unit, a local symbol across files and behind a precompiled header, whichever unit merges first), a copied header, a library template's overload set, the preamble index container, library bodies, a refused save, and the rerun after a variant left. Not covered by a test: the wait of the background index past 256 MB of unsaved index, which needs that much index and a database refusing it.
fix(command): search what the compiler searches, frameworks included (#… …1073) ## What changed clice asks the user's compiler for its job and search list, then compiles with its own LLVM 23 frontend. Whatever an older or foreign frontend set up by itself was silently missing; with Apple clang (LLVM ≤ 20), CMake's default on macOS, every framework header was "file not found". **Header search follows the queried compiler** - An older clang's frontend sets up part of the search list itself (Darwin's framework directories before LLVM 21, FreeBSD's before 16, ...). clice now adds the directories the compiler's `-v` list prints that its job does not already search, matched by directory and by kind of search (plain or framework), with sysroot- and prefix-composed paths counted as the job's own. GCC's printed list goes through the same rule, so a directory the command names (`-iwithprefixbefore`, `-iwithsysroot`) is not searched a second time as a system one. - GCC on macOS (Homebrew) lists the SDK's framework directory as a plain one: it is searched as a framework directory, as clang's driver for the target searches it. - A MinGW clang of another version uses clice's builtin headers like any other clang, its own right behind them (clang 20's beside clice's frontend broke `<windows.h>` in C++). - The language-specific system directories are searched for their language, as clang does: `-c-isystem` (clang's driver makes it of `C_INCLUDE_PATH`) for C, `-cxx-isystem` for C++ and Objective-C++, `-objc-isystem` for Objective-C, `-objcxx-isystem` for Objective-C++. **clice's own header model** (dependency scan, include resolution, include completion, include insertion, header contexts) reads framework directories (`-F`, `-iframework`, `-internal-iframework`: `<Foo/Bar.h>` in `Foo.framework/Headers`, then `PrivateHeaders`; the first directory holding the framework wins; a versioned framework's header is spelled `<Foo/Bar.h>`), the language-specific system directories, `-iwithsysroot` and `-iframeworkwithsysroot`, and `-imacros`: a file it names is a forced file of the unit, entered before every `-include` as clang does, so the scan sees it and a header opened alone that the compile enters through it (Zephyr's `autoconf.h`) gets its context. What it still does not mirror is marked `LIMITATION:` where a fix would go: header maps, clang's subframework lookup (the SDK's umbrella frameworks) and MSVC's lookup along the include stack. **Include completion** - A typed name matches a whole name first, then a prefix, then the words of a name (`view` finds `string_view`). - Files that do not look like headers are ranked after the headers instead of being dropped; hidden ones come last, and so do clang's builtin headers that must not be included directly (`<avxintrin.h>`). That list is generated from the resource directory at build time. - A framework directory offers its frameworks as `Foo/`, then the headers of `Headers/`. - A name several directories hold is listed once, at its best rank. **Module name completion** matches the same way, the part up to the last `.` as typed (`import my.` lists `my.*` only); names that match alike keep the unit's own partitions first, then the nearest modules. **Code completion layout**: the completion code moved from `src/syntax` into `src/feature/code_completion`, one file per kind of completion. The server's entry is the overload `code_complete(const CompletionInputs&)`, which takes the document's `SourceText` and the LSP request, and gathers the index's names for the compile itself — only when `all_scopes` is on, the one setting that uses them. ## Tests - Unit tests for the queried search lists (older frontends, composed and framework-kind directories, GCC framework directories, GCC listing the command's own directories, MinGW builtin headers, the language-specific system directories), the search configuration (forced files in clang's order), framework lookup in the include resolver, clice's builtin directory as include completion reads it, and internal builtin headers ranking last. - Cases for include completion (word matching, frameworks, other files and hidden ones last, a name two directories hold), module name completion (three match tiers, dotted scope, declarations, partition units, no partitions outside a named module, buffers declaring or exporting a module before the file is saved), a framework header whose host only the scan finds, and the headers of an `-imacros` file. The unit tests that called the completion helpers directly became cases. - Compat: the project includes `<CoreFoundation/CoreFoundation.h>` on macOS (Apple clang, its `cc` alias, Homebrew GCC 13/14/15, Homebrew LLVM 18) and `<windows.h>` under MinGW, with a new scenario for the runner's LLVM clang targeting MinGW. Each of these fails on main. New `import std` scenarios build the standard library module through CMake 4.5 (downloaded by CI) with MSYS2's clang and libc++, and with GCC 15 and libstdc++. The scenarios are split into files by kind of toolchain or build. - Linux unit tests and cases pass on a box; Windows and macOS through CI. The accepted limitations of header search and hosting now live in the code as `// LIMITATION:` comments (the convention is added to the C++ style guide) instead of the issue board.
perf(template-resolver): one cache rule, no queries in system headers (… …#1061) ## Related issue Follow-up to #1056: the index walk was still slower than before #1032, most on Windows. ## What changed The background index stops resolving dependent names inside system headers, and the template resolver keeps its work for the unit by one rule. - **No resolving in system headers.** Where the indexer's place is a system header other than the focus file, it asks the resolver nothing: dependent references inside a library's template bodies stay unresolved. Every unit including the library resolved them again (92-94% of the resolver's steps on abseil), for names only a reader of the library's code asks about. Opening such a header makes it the focus file, which resolves as before; what the project instantiates still comes from the instantiation records. - **One cache rule (`kept_step`).** A step that read none of its caller's bindings is kept for the unit by its input, with the parameters it found free, the visibility it judged and the defaults it asked about; a later query takes it where all of these hold again. A binding that only renames a parameter to another template's free one (`variant`'s `_Types` as a function template's `_Types...`) is kept too, for the same renaming. Rewrites are kept only for the types that look a name up. - **Standard traits by rule** (`template_resolver/std.cpp`): `conditional`, `enable_if`, `type_identity`, `remove_*`, `add_*`, `decay` by the standard's answer; `conjunction`, `disjunction` and `negation` by their arguments' values. A condition not known leaves its alternatives unrewritten until a lookup looks through them, layer by layer, each layer kept by its canonical form. - **Fewer wasted steps.** A step budget of 2048 per query; substitution, which walks a type as deep as it is written, counts no depth; a pack bound to a caller's expansion expands over it; a pack bound to itself is no binding read; folding stops after an unknown operand; a qualifier rooted at an unbound parameter fails at once. - **`clice inspect template-resolver <file> --audit`** lists, for each place in the file, the resolver's queries and steps and where it gave up or left a choice open. Same-box abseil index walk (`clice index --jobs 4`, both orders): Linux 618 s before #1032, 683 s on the first rewrite, about 525 s now; Windows 825, 1002 and 713 s. Walls are at or under the pre-#1032 level on both. The audit of MSVC's `<variant>` drops from 207k to 81k steps. ## Tests - `Indexer.LibraryTemplatesUnresolved`: a dependent name resolves in the user's header and in a library's text the focus file holds, not in a library header. - `TemplateResolver.*`: the standard type and value rules, conditional base chains twelve layers deep from the class and from a function template, an expansion of a caller's pack, scoped bindings in either order, per-view keeping across includes. - Case `tests/inspect/template_resolver/19_audit`: a cycle, an ambiguous base and an undecided partial specialization, each at its place. - Both suites pass in RelWithDebInfo and Debug on Linux boxes, and in CI.
refactor: clean up support and server, and move to kotatsu 0.1.0.275 (#… …1059) ## What changed The last cleanup before v1.0: `src/support` and `src/server`, kotatsu 0.1.0.275 with what it now provides deleted here, and `CompilationUnit` as the only way code asks clang where a token is. `src/`: +1309 −1877 lines (net −568). ### support - Small modules merged into the one that uses them: `bitmap` and `format` into the prelude, `bytes` into `text`, `environment` into `process`, `anomaly` into `logging`, `timer` beside logging; `shell`'s splitting moved to `command/argument`; `object_pool` folded into the command table, its only user. - One set of logging macros: every `LOG_*` formats through `logging::log`, `LOG_FATAL` through `logging::fatal`, and `LOG_ANOMALY` / `LOG_GUIDANCE` gate and report through the same module. - `failure_record` is `failure`. ### kotatsu 0.1.0.275 With simdjson 5.1.1.clice.1, spdlog 1.17.0, croaring 5.5.5 and lmdb 0.9.36; Bazel 9.3.2, as kotatsu builds with. Deleted here because kotatsu provides it: the crash stack walker (kotatsu writes the trace from the signal handler), the host's command quoting and the splitting that keeps views into the command (`split_plain`), the per-OS process code (exiting with the parent, power throttling, SIGPIPE, thread stacks, local time, core counts, returning freed heap, address-space limits), the database's platform `#ifdef`s (record locks, sparse files, sizes), the directory case-sensitivity probe, the UTF-8 helpers, and the old URI and meta APIs (`lsp::URI::from_path` / `from_string`, `meta::enum_values` / `enum_names`). `modules/` is regenerated; its macOS configurations were generated on a macOS runner with Xcode's SDK 26.2, as the SDK xclang 23.1.3.6 downloads is no longer served. `scripts/symbolize.py` reads the new frame format and still reads old logs. ### CompilationUnit Code that holds a unit no longer reaches clang's `SourceManager`: the indexer, semantics, identity, hover, semantic tokens, folding, inlay hints, signature help, completion, code actions and the stateless worker ask the unit. New members: `written_location`, `seen_location`, `produced_bounds`, `before`, `seen_name`, `concatenated`, `expansion_level`, `written_range(CharSourceRange)`, `positions`, `in_library`, `imported`, `next_token`, `previous_token`, and on `FocusFile` `parsed_by_pch`, `written_offset`, `macro_invocations`. Every member is documented and a unit test holds it to its comment. Helpers handed only a `Decl` or an `ASTContext` (the declaration printer, `decls`, `scan`, the code actions' type printing) keep clang's. ### worker On Linux each worker process is named after the program and the worker (`clice SF-0`, `clice SL-1`), which `top` and `pgrep` show; before, every worker was `clice`. macOS and Windows have no such name. ### npm workspace The lockfile moves past the open Dependabot alerts (follow-redirects, qs, fast-uri, `@vitest/mocker`, `@azure/msal-node`, serialize-javascript), minimatch through an override to 11. ### server `ServedProject::invalidate` is inlined into its callers; `reload_databases` returns whether anything changed. ### Renames | Before | After | | --- | --- | | `ScopedTimer`, `ms_f()` | `Timer`, `precise_ms()` | | `logging::critical` | `logging::fatal` | | `anomaly_should_report` | `logging::should_report` | | `support/failure_record` | `support/failure` | | `print_argv`, `host_quoting` | `join_command`, `sys::cmd::native_quoting` | | workers' `test_env` | `env_string` | | `token_class(macro, SourceManager)`, `written_name(name, SourceManager)` | `token_class(macro, CompilationUnit&)`, `written_name(name, ASTContext)` | ### Behavior changes - Crash logs: the stack trace is kotatsu's format, each frame with its module and load base; the `main executable base` line is gone (`symbolize.py` reads both). - Log lines that printed a command (`print_argv`) quote it by the host's rules: Windows' on Windows. - Directory case sensitivity is read from the directory itself: case-sensitive APFS volumes, WSL directories and ext4 casefold directories are judged as they are. - Windows: closing the console stops the server and `clice index` cleanly; clang-tidy's `User` comes from the account name. - macOS: the index database's address-space check before growing counts what is mapped (it read 0). - The simdjson error text in `--config-overlay` errors changed with the library. - `CompilationUnit::file_id(path)` no longer aborts an assertion build when no file is there. - `CompilationUnit::loaded_file_content` read a header a PCH served from the disk on demand, so the include tree hashed the disk's text for a file the index was built from the PCH's copy of; it now answers none for such a file, as its comment and the include tree's say. ### Known limit, not fixed here A unit that reads two hardlinked spellings of one file names both by the first: clang reads them as one file. Its dependencies then miss the second, and its index files the second's entries under the first. Now documented and pinned by `CompilationUnit.FilePathOfHardLinks`. ## Tests - Unit and cases on Linux (RelWithDebInfo and Debug), Windows (cross-built, run on a Windows runner) and macOS. - A real worker crash log, symbolized with the updated `scripts/symbolize.py`, resolves every clice frame to its source line. - New unit tests for every `CompilationUnit` / `FocusFile` member (`unittests/compile/`); `split_command` tests moved to `unittests/command`. - `pixi run format`, `pixi run check-layers`, `npm run check`, `npm test -w tools`; the VS Code extension builds and packages with the updated lockfile.
perf(worker): return freed memory on macOS and Windows (#1053) ## What changed On macOS and Windows clice kept most of the memory it had freed. With every file closed, it used about twice as much memory as clangd. There were two causes. - **mimalloc collects only the calling thread's heap.** A block freed on another thread waits in its owner's page until the owner allocates or collects. A stateful worker builds an AST on a pool thread and frees it on the loop thread, and the pool thread then sits idle. So the AST's pages stayed committed. - **An idle stateless worker kept 58–104 MiB** after a few requests, in pages that no collect returns. A fresh one holds 3–8 MiB. The fix: - **`release_free_memory_everywhere`** runs `release_free_memory` on every thread of libuv's pool, then on the calling thread. - The pool's collects wait up to 50 ms for each other, so that each runs on its own thread. A thread still busy after that is left to its next collect. - A call made while one is running returns at once and makes the running one do one more pass. - Under glibc, `malloc_trim` already covers every arena. It runs once, on a pool thread, because on a large heap it takes long enough to stall the loop. - **Callers.** The stateful worker calls it after a compile, after an evict, and when the last hold on a document goes. It no longer collects inside its compile task. The master's poll tick, the background index pass and a project's load also call it. - **mimalloc's `page_reclaim_on_free` is on.** A page left by a thread that has ended is taken over by the thread freeing into it, whose collects return it. - **`WorkerScheduler` replaces a heavy idle worker.** - At the `min_workers` floor, an idle stateless worker that has run a request and holds more than 32 MiB gets a fresh process started beside it. The next scale-down removes the heavy one, which keeps serving until then. - Every scale-down, the idle one and the one under memory pressure, now takes the heaviest idle worker, the newest of equals. Before, a scale-down under memory pressure would have removed the fresh replacement, and the cycle would have repeated every idle period. Memory, whole process tree, abseil, in MB: | Platform | State | Before | After | clangd | |---|---|---|---|---| | macOS | one file open | 272–285 | 249 | 258 | | macOS | all closed, 90 s later | 295–329 | 90–91 | 134 | | Windows | one file open | 548–564 | 471 | 510 | | Windows | all closed, 90 s later | 635–656 | 266–267 | 355 | ## Tests - New `WorkerScheduler` unit cases: - a heavy idle worker is replaced, spawning first and scaling down after; - a light worker is kept; - an unused worker is kept; - a scale-down takes the heaviest; - under memory pressure, the heavy worker goes rather than its replacement. - No automated test pins the collect on every pool thread. A pool case that evicted a large unit's AST passed with that collect removed too: the AST's large blocks go back to the system whichever thread frees them, and what mimalloc keeps is many small ones. The memory table above is what shows it. - Unit tests and cases pass on Linux in RelWithDebInfo and Debug. - The memory numbers above come from runs on macOS and Windows runners, each repeated at least twice.
feat(command): option tables for nvcc and Intel, nvcc with a cl host (#… …1049) ## What changed clice reads the commands of compilers whose options clang's table does not know by tables of their own, instead of hand-written string matching, and reads nvcc commands whose host compiler is cl — nvcc's default on Windows. **Option tables** - `src/command/toolchains/nvcc_options.inc`: every option `nvcc --help` lists, in its long (`--name`) and short (`-name`) spellings, values separate or after `=`, joined only where nvcc takes them joined (`-I -D -U -L -l -O -t`). The translation switches on the parsed option; what has no effect on the parse is dropped with its value (`-Xptxas -O3`, `--dependency-output k.d`, `-lineinfo`), and `--no-exceptions`, `--ftemplate-depth`, `--stdlib`, `--disable-warnings` become clang's options. - `src/command/toolchains/own_options.inc`: per family, spellings clang's table reads otherwise or not at all, each read as the clang option it means or dropped when only the build reads it — Intel's `-qopenmp`/`-fiopenmp`/`/Qiopenmp`/`/Qopenmp[-]`, `/Qstd:` (the standard was lost before), icx-cl's `/fsycl`; GCC's module mapper and dependency outputs (`-fmodule-mapper=`, `-fdeps-*`, `-aux-info`, …). `parse_arguments` tries this table where clang's sees an unknown option or takes a `/` spelling for an input. - Everything `-Xclang` passes on is read as one cc1 command line, as the driver passes it: an option the compile never takes goes as its driver spelling does (a prebuilt `-fmodule-file=name=path`), and CMake's `-Xclang -include-pch -Xclang <pch>` is recognized wherever its two halves stand. - The toolchain query, the context labels and the nvcc query ask the command by option id rather than grepping its rendered strings: whether it names a standard (`-std=`, also through `/clang:` and `-Xclang`), cl's `/std:` and version, a universal build's second `-arch`, an asked-for `-fdelayed-template-parsing`, the nvcc host view and architecture selection. Context labels now show cl's `/D`, `/O2`, `/std:` (they showed none of them before). **nvcc with a cl host** - An nvcc command keeps its host compiler (`Command::host`): the one `-ccbin` names, else cl on Windows and GCC elsewhere. `-Xcompiler` pieces are split as that host splits its command line (CMake writes `-Xcompiler=" -Zi -Ob0 -Od /RTC1"`), and with a cl host they are read as cl's options — `-Fd…` no longer reads as a framework directory, `/EHsc`, `/MDd`, `/Zc:…` take effect. A GCC host's warnings clang does not know go, as in a GCC command. - The query runs nvcc's dryrun with cl's directory on `PATH`, the only place nvcc looks for cl: the directory `-ccbin` names, else, outside a developer prompt, the newest toolset Visual Studio installed (`llvm::findVCToolChainViaSetupConfig`, newly in `modules/`; `LLVMWindowsDriver` linked explicitly). It reads the dryrun from stdout, quoted for cmd, as Windows' nvcc prints it; cudafe++'s `--ms_c++14 --microsoft_version=1951` give the dialect and `-fms-compatibility-version`; clang's driver in cl mode builds the job with nvcc's `/EHsc`, and a `.cuh` is selected as CUDA by `-x cuda`. - Before, the query failed with "Unsupported nvcc host compiler" on Windows, and clang's own reading lacked the CUDA setup. **Also** - In clang's unqueried reading of an nvcc command, the tokens only nvcc's dryrun reads (`-ccbin=…`) no longer reach clang's driver as unknown arguments. - Compat: a Windows scenario, CMake + Ninja + conda's nvcc 13 with MSVC (`cmake ninja nvcc msvc`); the CUDA toolkit is in the `toolchains` environment on win-64, so the Windows unit and case jobs do not download it. - Docs: the command-resolve design page and the compat tool list. ## Tests - Unit: `OwnOptions.*` (new), nvcc's spellings, values and clang counterparts, `-Xcompiler` split per host, host from `-ccbin`, cl-host spellings and rule edits, the cl dryrun (`DryrunCLHost`) and the whole cl-host query on a fake nvcc (`NVCCHostCL`), `-Xclang` read by cc1 incl. a split PCH, `NamesStandard` by id, deferral through `-Xclang`, context labels for cl and clang commands. - Cases: `command/reading/06_intel_drivers` (Intel spellings), new `command/reading/07_nvcc_options`; two nvcc cases pin `-ccbin g++` so their output stays the same on Windows, where the default host is now cl. - Run on boxes: Linux RelWithDebInfo and Debug unit tests and cases; Windows unit tests and cases; Windows compat (all MSVC, clang-cl, MinGW, icx and the new nvcc scenarios); Linux compat for the nvcc, gcc, clang and icpx scenarios.
PreviousNext