fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) (#1600)
Fixes #1581. ## What was wrong `codegraph init` / `codegraph index` died with `Segmentation fault` — the whole CLI process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's `clang/test/Parser/parser_overflow.c`, 16,384 nested `{`). The reporter's diagnosis is exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the native kernel's **recursive walker** (`visit_node` → `visit_for_calls_and_structure` → …, one frame per AST level) overflowed the thread's stack. A native overflow can't be caught the way a wasm abort can, and a parse worker is a thread of the `codegraph` process, so the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial index. Two things made "just give the worker a bigger stack" the wrong fix: - it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the 8 MiB **main** thread too; - the walkers are shared by every kernel-routed language (20 of them), and each has several recursion points with different frame sizes, so no single stack size is a provable bound. Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its own `RangeError` per file and stores a partial result with a `parse_error`. The kernel just needed a way to get there instead of dying. ## What this does **The kernel guards its own recursion against the calling thread's real stack bounds and defers a too-deep file to wasm** — the same `defer:` routing signal it already uses for files with parse errors, which `src/extraction/kernel/index.ts` treats as "take the wasm path for this file", silently. - `codegraph-kernel/src/stack.rs`: per-thread stack bounds from the OS, computed once per thread and cached — glibc/musl `pthread_getattr_np` + `pthread_attr_getstack`, macOS `pthread_get_stackaddr_np` + `pthread_get_stacksize_np`, Win32 `GetCurrentThreadStackLimits` (a hand-declared `kernel32` extern; no `windows-sys`). `exhausted()` is one thread-local load and one compare: true once the stack pointer is within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on the 4 MiB worker, the 8 MiB main thread, and any `resourceLimits.stackSizeMb` alike. - `stack_guard!()` (defined in `lib.rs`) is the first statement of every recursive walker function — all **150** self-recursive or on-cycle functions across the 15 walker modules, found by script (every cycle in the call graph, not just direct self-calls). It returns `Default::default()` (`()`, `false`, `None`, `""`) so an exhausted walk simply stops descending; a hook returning `false` sends its caller down the generic child walk, whose own guard returns at once. - `extract_file` runs the whole walk under `stack::run_guarded`: if the flag is set afterwards the (truncated) result is discarded and replaced by `defer: nesting too deep for the native walker — wasm recovery handles it`. - `parse-pool.ts`: a comment at `new Worker(scriptPath)` records why there is deliberately no `resourceLimits.stackSizeMb` bump. - No new crates beyond `libc` as a direct unix dependency (already in `Cargo.lock` transitively). No wire/ABI change. Net effect for the reporter's repo: `deep.c` goes to the wasm path, lands as `function foo` plus a recorded parse warning, and the other 31,607 files index normally. `CODEGRAPH_KERNEL=0` and the `exclude` workaround are no longer needed. ## Tests **Rust unit tests** (`cargo test`, 21 passed — 7 new in `stack.rs`): the walkers for C, C++, Rust, TypeScript and Python are driven on a **1 MiB** thread (a quarter of Node's worker default) with 30k-deep nesting and must return `defer:` instead of crashing; shallow files are untouched; the latch resets between runs; the OS bounds are sane on the main thread and describe a small thread's own stack. **`__tests__/kernel-deep-nesting.test.ts`** (new, 8 tests — skips without a staged `.node`, fails under `CODEGRAPH_KERNEL_EXPECT=1` if the kernel is missing, like the other kernel suites): - every default-routed language (all 20) survives a 60k-deep expression on the main thread — clean result or the wasm fallback's partial result, never a crash; - the reporter's exact 16,384-brace C file is indexed (partial) on the main thread; - 200-deep expressions in every language still take the kernel path clean (the guard never trips on normal code); - inside a **default-sized 4 MiB `worker_threads` Worker** through `dist/`: the reporter's `deep.c` and a 60k-deep expression in every language come back `deferred` with exit 0, and a normal file still extracts natively; - end-to-end through the built CLI: `codegraph init` on a repo holding `deep.c` + `ok.c` exits 0 and records both files, with both functions. **Existing kernel suites**: all 15 (`kernel-*-parity`, `kernel-scaffold`, `kernel-retry-materialize`, `kernel-grammar-parity`) pass unchanged, 147 tests — the guard never fires on the parity fixtures. **Reporter's probes** (`one.js` from the issue, default 4 MiB worker, this build): `deep.c` → `deferred`, exitCode=0 (was rc=132/139); `deep100k.c` → `deferred`, exitCode=0. Main thread: `deep.c` / `deep100k.c` → wasm partial with `Parse error: Maximum call stack size exceeded`; a 6,000-term binary expression and a 3,000-branch `else if` chain stay on the kernel path with clean results. **Perf** (same `dist/`, only the `.node` swapped via `CODEGRAPH_KERNEL_PATH`; interleaved main/new ×3, `codegraph init`, macOS arm64): | repo | main (median) | guarded (median) | nodes / edges | |---|---|---|---| | express (141 files) | 0.60 s (0.58–0.65) | 0.61 s (0.58–0.61) | 1,084 / identical | | redis (786 C/H files) | 4.44 s (4.39–4.66) | 4.49 s (4.41–4.70) | 19,942 / 76,446 identical | Within run-to-run noise, as expected for one TLS load + compare per recursion entry. **Linux (Docker, `node:22-bookworm`, kernel built in-container, `docker run --rm --init`)** — the reporter's platform and the glibc `pthread_getattr_np` bounds path: ``` === platform === Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov 4 21:21:47 UTC 2025 aarch64 GNU/Linux v22.22.3 -rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node === reporter repro (issue #1581): 16,384-brace deep.c, codegraph init === │ └ Done init exit code: 0 file: deep.c file: deep100k.c file: ok.c function: add function: bar function: foo === worker probe: kernel raw extract in a default 4 MiB worker === deep.c: deferred deep.c: worker exitCode=0 deep100k.c: deferred deep100k.c: worker exitCode=0 ok.c: kernel nodes=2 ok.c: worker exitCode=0 === cargo test stack:: (glibc pthread_getattr_np bounds path) === test stack::tests::os_bounds_are_sane_on_this_platform ... ok test stack::tests::small_stack_reports_its_own_bounds ... ok test stack::tests::normal_files_are_untouched_by_the_guard ... ok test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok test stack::tests::latch_resets_between_runs ... ok test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s === vitest: kernel-deep-nesting + kernel-scaffold === ✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms ✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms Test Files 2 passed (2) Tests 18 passed (18) ``` (The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.) **Windows (Parallels ARM64 VM, MSVC 14.44, `cargo 1.97`, kernel built on the VM, `GetCurrentThreadStackLimits` path)**: ``` head: cbf8485 fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) === cargo build --release (win32-arm64) === Finished `release` profile [optimized] target(s) in 2m 04s staged: 35086848 bytes === cargo test (stack guard unit tests) === test stack::tests::normal_files_are_untouched_by_the_guard ... ok test stack::tests::os_bounds_are_sane_on_this_platform ... ok test stack::tests::small_stack_reports_its_own_bounds ... ok test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok test stack::tests::latch_resets_between_runs ... ok test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.49s === reporter repro: codegraph init on a 16,384-brace deep.c === └ Done init exit code: 0 === vitest: deep-nesting + scaffold (CODEGRAPH_KERNEL_EXPECT=1) === ✓ __tests__/kernel-scaffold.test.ts (10 tests) 55ms ✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 67239ms ✓ every default-routed language survives a 60k-deep expression on the main thread 52801ms ✓ inside a default-sized (4 MiB) parse worker, through dist/ > defers a 60k-deep expression in every default-routed language 13050ms ✓ end-to-end: codegraph init on a repo holding the deep file > exits 0 and records deep.c alongside the normal files 936ms Test Files 2 passed (2) Tests 18 passed (18) ``` (The end-to-end test is what reads the Windows index back through `node:sqlite` — `files` = `deep.c`, `ok.c`; functions `add`, `foo`.) Full `npm test` on this branch (macOS arm64, kernel staged): **190 files passed, 3,185 tests passed, 10 skipped, 0 failed.** Clippy note: `cargo clippy` on the current toolchain (1.92) reports 18 pre-existing lints (`manual_contains`, `unnecessary_to_owned`, …) in walker code this PR only touched by inserting guard lines; none are in `stack.rs`/`lib.rs`. Left alone to keep the diff reviewable. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
This commit is contained in:
Generated
+1
@@ -47,6 +47,7 @@ name = "codegraph-kernel"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"libc",
|
||||
"napi",
|
||||
"napi-build",
|
||||
"napi-derive",
|
||||
|
||||
@@ -62,6 +62,12 @@ tree-sitter-luau = "=1.2.0"
|
||||
# kotlin grammar C (see build.rs — no kotlin crate dep is possible).
|
||||
tree-sitter-language = "0.1"
|
||||
|
||||
# Stack-bounds queries for the walker stack guard (src/stack.rs, #1581):
|
||||
# pthread_getattr_np / pthread_get_stackaddr_np. Already in the lock file
|
||||
# transitively; Windows uses a hand-declared kernel32 extern instead.
|
||||
[target.'cfg(unix)'.dependencies]
|
||||
libc = "0.2"
|
||||
|
||||
[build-dependencies]
|
||||
napi-build = "2"
|
||||
cc = "1"
|
||||
|
||||
@@ -776,6 +776,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitNode -----------------------------------------------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -852,6 +853,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors ----------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// Receiver present (out-of-line `Cls::method` def) → method instead.
|
||||
if self.variant == Variant::Cpp && self.receiver_type_of(node).is_some() {
|
||||
self.extract_method(node);
|
||||
@@ -892,6 +894,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let receiver_type = if self.variant == Variant::Cpp { self.receiver_type_of(node) } else { None };
|
||||
|
||||
if !self.inside_class_like() && receiver_type.is_none() {
|
||||
@@ -956,6 +959,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// extractClass for cpp class_specifier (skipBodilessClass, #1093).
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -976,6 +980,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// Extract a struct-like declaration while preserving its semantic kind.
|
||||
fn extract_aggregate(&mut self, node: Node<'t>, kind: &'static str) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -995,6 +1000,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -1028,6 +1034,7 @@ impl<'t> Walker<'t> {
|
||||
/// extractTypeAlias for type_definition / alias_declaration. Returns true
|
||||
/// when children were consumed (typedef struct/enum bodies).
|
||||
fn extract_type_alias(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
return false;
|
||||
@@ -1502,10 +1509,12 @@ impl<'t> Walker<'t> {
|
||||
// --- function bodies -----------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -1591,6 +1600,7 @@ impl<'t> Walker<'t> {
|
||||
/// grammars: base_class_clause (#1043), the field_declaration Go-embedding
|
||||
/// shape, and the field_declaration_list recursion that reaches it.
|
||||
fn extract_inheritance(&mut self, node: Node<'t>, class_row: u32) {
|
||||
stack_guard!();
|
||||
let extends_kind = edge_kind_index("extends").unwrap();
|
||||
for i in 0..node.named_child_count() {
|
||||
let Some(child) = node.named_child(i) else { continue };
|
||||
@@ -1703,6 +1713,7 @@ impl<'t> Walker<'t> {
|
||||
/// normalizeValue for cFamilySpec: bare identifiers, and the
|
||||
/// pointer_expression unwrap (`&fn`; `&Cls::m` keeps the qualified name).
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, mode: Mode, explicit_ref: bool, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1749,6 +1760,7 @@ impl<'t> Walker<'t> {
|
||||
/// scanFnRefSubtree: capture-only walk of subtrees the main walkers skip
|
||||
/// (variable-declaration initializers). Halts at nested functions/lambdas.
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -500,6 +500,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode, C#-relevant branches) -----------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -571,10 +572,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody ------------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -626,6 +629,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors --------------------------------------------------------------
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// skipBodilessClass unset: a bodiless `record Empty;` still mints a node.
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -656,6 +660,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_struct(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// Body gate — EXCEPT C# positional records (`record struct M(…);`,
|
||||
// node type record_declaration), complete definitions with no body.
|
||||
// A bodiless `struct Fwd;` mints NO node. (#831)
|
||||
@@ -685,6 +690,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_interface(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -703,6 +709,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -890,6 +897,7 @@ impl<'t> Walker<'t> {
|
||||
/// extractMethod (1737) — method_declaration + constructor_declaration.
|
||||
/// Signature is ALWAYS undefined (no getSignature hook); isAsync is real.
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
if !self.inside_class_like() {
|
||||
// Unreachable on non-erroring C# (top-level `void M(){}` parses as
|
||||
// local_function_statement; erroring files defer) — mirror the TS
|
||||
@@ -924,6 +932,7 @@ impl<'t> Walker<'t> {
|
||||
/// extractFunction — only reachable for a method outside any class
|
||||
/// (unreachable on non-erroring C#; kept faithful to the generic tail).
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
if let Some(body) = node.child_by_field_name("body") {
|
||||
@@ -1087,6 +1096,7 @@ impl<'t> Walker<'t> {
|
||||
/// (object initializers are initializer_expression), so this is
|
||||
/// unreachable — mirrored from the shared TS path like java.rs.
|
||||
fn extract_anonymous_class(&mut self, node: Node<'t>, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
let type_node = node
|
||||
.child_by_field_name("constructor")
|
||||
.or_else(|| node.child_by_field_name("type"))
|
||||
@@ -1243,6 +1253,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// walkCsharpTypePosition (5955).
|
||||
fn walk_type_position(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
match node.kind() {
|
||||
"predefined_type" => {}
|
||||
"identifier" => {
|
||||
@@ -1362,6 +1373,7 @@ impl<'t> Walker<'t> {
|
||||
/// normalizeValue (function-ref.ts:525) for CSHARP_SPEC: bare identifiers,
|
||||
/// the transparent `argument` layer, and the `this.Member` special.
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1412,6 +1424,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -596,6 +596,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the main walk (visitNode, tree-sitter.ts:936-1303) ---------------
|
||||
|
||||
fn visit(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// The visitNode hook (dart.ts:144-157) — the constants branch.
|
||||
if node.kind() == "static_final_declaration" {
|
||||
let mut cursor = node.walk();
|
||||
@@ -669,6 +670,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractFunction / extractMethod (:1517 / :1737) ------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// No receiver hook. Name first (resolveName inside extract_name).
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
@@ -770,6 +772,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractClass (:1679) — classes, mixins, extensions ---------------
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let resolved_body = self.resolve_body(node);
|
||||
// No skipBodilessClass. Anonymous `extension on String` → the name
|
||||
// fallback finds the ON type's type_identifier — a class named after
|
||||
@@ -800,6 +803,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractEnum (:1914) ----------------------------------------------
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let body = match self.resolve_body(node) {
|
||||
Some(b) => b,
|
||||
None => return,
|
||||
@@ -1221,6 +1225,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let name = self.text(node);
|
||||
if !name.is_empty() && !is_builtin_type(name) {
|
||||
@@ -1239,6 +1244,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody (:5129-5286) — dart rows -----------------------
|
||||
|
||||
fn visit_body(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
let kind = node.kind();
|
||||
@@ -1348,6 +1354,7 @@ impl<'t> Walker<'t> {
|
||||
/// normalizeValue with DART_SPEC's one layer (`argument` → fan out).
|
||||
/// Named arguments are NOT captured (named_argument is not a layer).
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1378,6 +1385,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -380,6 +380,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitNode ------------------------------------------------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -417,10 +418,12 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -448,6 +451,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors --------------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// (getReceiverType only matches method_declaration's receiver field —
|
||||
// function_declaration has none, so no reroute happens here)
|
||||
let name = self.extract_name(node);
|
||||
@@ -524,6 +528,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// extractTypeAlias for Go: type_spec → struct / interface / plain alias.
|
||||
fn extract_type_alias(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
return false;
|
||||
@@ -842,6 +847,7 @@ impl<'t> Walker<'t> {
|
||||
/// (constraint_elem) and struct embedding (field_declaration without a
|
||||
/// field_identifier), plus the field_declaration_list recursion.
|
||||
fn extract_inheritance(&mut self, node: Node<'t>, class_row: u32) {
|
||||
stack_guard!();
|
||||
let extends_kind = edge_kind_index("extends").unwrap();
|
||||
for i in 0..node.named_child_count() {
|
||||
let Some(child) = node.named_child(i) else { continue };
|
||||
@@ -894,6 +900,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let type_name = self.text(node).to_string();
|
||||
if !type_name.is_empty() && !is_builtin_type(&type_name) {
|
||||
@@ -980,6 +987,7 @@ impl<'t> Walker<'t> {
|
||||
/// normalizeValue with GO_SPEC's transparent layers (literal_element,
|
||||
/// expression_list — both fan out to named children).
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1010,6 +1018,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -485,6 +485,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode, Java-relevant branches) -----------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -534,10 +535,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody ----------------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -577,6 +580,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors --------------------------------------------------------------
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -600,6 +604,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
if !self.inside_class_like() {
|
||||
// (object-literal parents don't exist in Java; a stray top-level
|
||||
// method extracts as a function, mirroring extractMethod's tail)
|
||||
@@ -627,6 +632,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// extractFunction — only reachable for a method outside any class.
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
if let Some(body) = node.child_by_field_name("body") {
|
||||
@@ -653,6 +659,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_interface(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -671,6 +678,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -902,6 +910,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// extractAnonymousClass — `new T() { ... }`.
|
||||
fn extract_anonymous_class(&mut self, node: Node<'t>, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
let type_node = node
|
||||
.child_by_field_name("constructor")
|
||||
.or_else(|| node.child_by_field_name("type"))
|
||||
@@ -1101,6 +1110,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let type_name = self.text(node).to_string();
|
||||
if !type_name.is_empty() && !is_builtin_type(&type_name) {
|
||||
@@ -1193,6 +1203,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -648,6 +648,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode, Kotlin-relevant branches) -----------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
if self.try_visit_hook(node) {
|
||||
self.scan_fn_ref_subtree(node, 0);
|
||||
return;
|
||||
@@ -719,10 +720,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody ----------------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -777,6 +780,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors ------------------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// getReceiverType short-circuit (1522) — extension fns at any scope.
|
||||
if self.receiver_type_of(node).is_some() {
|
||||
self.extract_method(node);
|
||||
@@ -810,6 +814,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let receiver = self.receiver_type_of(node);
|
||||
let name = self.extract_name(node);
|
||||
let qualified_override = receiver.as_ref().map(|r| format!("{r}::{name}"));
|
||||
@@ -861,6 +866,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let resolved_body = self.resolve_body(node);
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -887,6 +893,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_interface(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -905,6 +912,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = self.resolve_body(node) else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -1283,6 +1291,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1362,6 +1371,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
+37
-17
@@ -16,6 +16,20 @@
|
||||
|
||||
#![deny(clippy::all)]
|
||||
|
||||
/// First statement of every recursive walker function (see stack.rs, #1581):
|
||||
/// once the stack pointer is inside the red zone, stop descending — the
|
||||
/// latched flag makes `stack::run_guarded` discard the walk and defer the
|
||||
/// file to wasm. `Default::default()` covers every walker return type in use
|
||||
/// (`()`, `bool`, `Option<_>`, `String`); a hook returning `false` just sends
|
||||
/// its caller down the generic child walk, whose own guard returns at once.
|
||||
macro_rules! stack_guard {
|
||||
() => {
|
||||
if $crate::stack::exhausted() {
|
||||
return ::core::default::Default::default();
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
mod buffers;
|
||||
mod ccpp;
|
||||
mod cfnptr;
|
||||
@@ -33,6 +47,7 @@ mod rlang;
|
||||
mod ruby;
|
||||
mod rustlang;
|
||||
mod scala;
|
||||
mod stack;
|
||||
mod swift;
|
||||
mod textutil;
|
||||
mod python;
|
||||
@@ -216,23 +231,28 @@ pub fn cfnptr_strip_c(text: String) -> String {
|
||||
|
||||
#[napi]
|
||||
pub fn extract_file(file_path: String, content: String, language: String) -> Result<ExtractBuffers> {
|
||||
let out = match language.as_str() {
|
||||
"java" => java::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"python" => python::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"go" => go::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"c" | "cpp" => ccpp::extract(&file_path, &content, &language).map_err(Error::from_reason)?,
|
||||
"rust" => rustlang::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"csharp" => csharp::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"ruby" => ruby::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"php" => php::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"swift" => swift::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"kotlin" => kotlin::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"r" => rlang::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"lua" | "luau" => lua::extract(&file_path, &content, &language).map_err(Error::from_reason)?,
|
||||
"scala" => scala::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
"dart" => dart::extract(&file_path, &content).map_err(Error::from_reason)?,
|
||||
_ => tsjs::extract(&file_path, &content, &language).map_err(Error::from_reason)?,
|
||||
};
|
||||
// The whole walk runs under the stack guard (stack.rs, #1581): a file
|
||||
// nested deeply enough to overflow this thread's stack comes back as a
|
||||
// `defer:` error — the TS side's routine "take the wasm path" signal —
|
||||
// instead of a SIGSEGV that kills the entire indexer process.
|
||||
let out = stack::run_guarded(|| match language.as_str() {
|
||||
"java" => java::extract(&file_path, &content),
|
||||
"python" => python::extract(&file_path, &content),
|
||||
"go" => go::extract(&file_path, &content),
|
||||
"c" | "cpp" => ccpp::extract(&file_path, &content, &language),
|
||||
"rust" => rustlang::extract(&file_path, &content),
|
||||
"csharp" => csharp::extract(&file_path, &content),
|
||||
"ruby" => ruby::extract(&file_path, &content),
|
||||
"php" => php::extract(&file_path, &content),
|
||||
"swift" => swift::extract(&file_path, &content),
|
||||
"kotlin" => kotlin::extract(&file_path, &content),
|
||||
"r" => rlang::extract(&file_path, &content),
|
||||
"lua" | "luau" => lua::extract(&file_path, &content, &language),
|
||||
"scala" => scala::extract(&file_path, &content),
|
||||
"dart" => dart::extract(&file_path, &content),
|
||||
_ => tsjs::extract(&file_path, &content, &language),
|
||||
})
|
||||
.map_err(Error::from_reason)?;
|
||||
Ok(ExtractBuffers {
|
||||
meta: out.meta.into(),
|
||||
nodes: out.nodes.into(),
|
||||
|
||||
@@ -418,6 +418,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the main walk (visitNode, tree-sitter.ts:936-1303) ---------------
|
||||
|
||||
fn visit(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
|
||||
// The visitNode hook (lua.ts:105-151) runs FIRST.
|
||||
@@ -487,6 +488,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractFunction / extractMethod (1517 / 1737) --------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// :1522 receiver short-circuit IS the method routing.
|
||||
if let Some(receiver) = self.receiver_type(node) {
|
||||
let receiver = receiver.to_string();
|
||||
@@ -522,6 +524,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>, receiver: String) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let docstring = preceding_docstring(node, self.src);
|
||||
let signature = self.signature_of(node);
|
||||
@@ -654,6 +657,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody (5129-5286) — the hook-free body walk ----------
|
||||
|
||||
fn visit_body(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// maybeCaptureFnRefs (5137) fires in the body walker too.
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -750,6 +754,7 @@ impl<'t> Walker<'t> {
|
||||
/// normalizeValue with LUA_SPEC's one transparent layer (expression_list
|
||||
/// fans out to named children).
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -780,6 +785,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -530,6 +530,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode, PHP-relevant branches) ------------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
if self.try_visit_hook(node) {
|
||||
self.scan_fn_ref_subtree(node, 0);
|
||||
return;
|
||||
@@ -609,10 +610,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody --------------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -671,6 +674,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors ----------------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
if let Some(body) = node.child_by_field_name("body") {
|
||||
@@ -696,6 +700,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -715,6 +720,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>, kind: &'static str) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -736,6 +742,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_interface(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -754,6 +761,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -1080,6 +1088,7 @@ impl<'t> Walker<'t> {
|
||||
/// nests inside `anonymous_class`, so findAnonymousClassBody finds no
|
||||
/// DIRECT child) — mirrored from the shared TS path for shape.
|
||||
fn extract_anonymous_class(&mut self, node: Node<'t>, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
let type_node = node
|
||||
.child_by_field_name("constructor")
|
||||
.or_else(|| node.child_by_field_name("type"))
|
||||
@@ -1206,6 +1215,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn walk_php_type_position(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
match node.kind() {
|
||||
"primitive_type" => {}
|
||||
"name" => {
|
||||
@@ -1249,6 +1259,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1337,6 +1348,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -320,6 +320,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitNode ------------------------------------------------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -356,10 +357,12 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -390,6 +393,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors --------------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
if let Some(body) = node.child_by_field_name("body") {
|
||||
@@ -414,6 +418,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -431,6 +436,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -789,6 +795,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -298,6 +298,7 @@ impl<'t> Walker<'t> {
|
||||
// else plain recursion over namedChildren in order.
|
||||
|
||||
fn visit(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
if self.hook(node) {
|
||||
return;
|
||||
}
|
||||
@@ -313,6 +314,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// The visitNode hook (r.ts:180-309). Returns true when consumed.
|
||||
fn hook(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
match node.kind() {
|
||||
"call" => self.hook_call(node),
|
||||
"binary_operator" => self.hook_binary_operator(node),
|
||||
@@ -321,6 +323,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn hook_call(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
let fname = match self.callee_name(node) {
|
||||
Some(f) => f,
|
||||
None => return false,
|
||||
@@ -405,6 +408,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn hook_binary_operator(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
let op = match node.child_by_field_name("operator") {
|
||||
Some(o) => self.text(o),
|
||||
None => return false,
|
||||
@@ -477,6 +481,7 @@ impl<'t> Walker<'t> {
|
||||
/// `list(…)` entries become methods. Non-method argument subtrees are
|
||||
/// NEVER visited (`representation(…)`, `signature(…)` invisible).
|
||||
fn extract_class_members(&mut self, class_call: Node<'t>, class_row: u32) {
|
||||
stack_guard!();
|
||||
let args = match class_call.child_by_field_name("arguments") {
|
||||
Some(a) => a,
|
||||
None => return,
|
||||
@@ -543,6 +548,7 @@ impl<'t> Walker<'t> {
|
||||
/// `method` node positioned at the ARGUMENT node, signature from the raw
|
||||
/// parameters text, body walked hook-aware inside the method scope.
|
||||
fn emit_method_arg(&mut self, entry: Node<'t>) {
|
||||
stack_guard!();
|
||||
let entry_name = match entry.child_by_field_name("name") {
|
||||
Some(n) => n,
|
||||
None => return,
|
||||
|
||||
@@ -383,6 +383,7 @@ impl<'t> Walker<'t> {
|
||||
/// the source of the module multiply-capture quirk (scan runs with the
|
||||
/// module already POPPED, so candidates re-attribute to the outer scope).
|
||||
fn try_visit_hook(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
if kind == "call" && node.child_by_field_name("receiver").is_none() {
|
||||
if let Some(method) = node.child_by_field_name("method") {
|
||||
@@ -451,6 +452,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode, Ruby-relevant branches) ----------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// Language hook FIRST (tree-sitter.ts:943) — a handled subtree is
|
||||
// scanned for fn-ref candidates and never reaches the ladder (or the
|
||||
// maybeCaptureFnRefs call below).
|
||||
@@ -522,10 +524,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody ------------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -596,6 +600,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors --------------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
if let Some(body) = node.child_by_field_name("body") {
|
||||
@@ -619,6 +624,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -634,6 +640,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -872,6 +879,7 @@ impl<'t> Walker<'t> {
|
||||
/// qualify); `block_argument` is a transparent layer; specials are the
|
||||
/// `method(:sym)` call form and hook-DSL `simple_symbol`s.
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -938,6 +946,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -440,6 +440,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitNode ------------------------------------------------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -503,6 +504,7 @@ impl<'t> Walker<'t> {
|
||||
/// impl method's body, whose parent walk passes through the outer fn) or
|
||||
/// the stack top is class-like (trait members).
|
||||
fn extract_fn_or_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let receiver = self.receiver_type_of(node);
|
||||
let as_method = receiver.is_some() || self.inside_class_like();
|
||||
|
||||
@@ -569,6 +571,7 @@ impl<'t> Walker<'t> {
|
||||
/// extractInterface — kind `trait` (interfaceKind), inheritance from
|
||||
/// trait_bounds, body children visited with the trait pushed.
|
||||
fn extract_interface(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -589,6 +592,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// Extract a Rust struct or union with a body; unit structs remain skipped.
|
||||
fn extract_aggregate(&mut self, node: Node<'t>, kind: &'static str) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -611,6 +615,7 @@ impl<'t> Walker<'t> {
|
||||
/// extractEnum — body required; enum_variant children → enum_member nodes
|
||||
/// (name field only, payloads never walked); other children re-dispatched.
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -705,6 +710,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// getRootModule (languages/rust.ts:124).
|
||||
fn root_module(&self, n: Node) -> String {
|
||||
stack_guard!();
|
||||
let Some(first) = n.named_child(0) else {
|
||||
return self.text(n).to_string();
|
||||
};
|
||||
@@ -724,6 +730,7 @@ impl<'t> Walker<'t> {
|
||||
if prefix.is_empty() { seg.to_string() } else { format!("{prefix}::{seg}") }
|
||||
}
|
||||
fn collect<'t>(w: &Walker<'t>, n: Node<'t>, prefix: &str, paths: &mut Vec<(String, Node<'t>)>) {
|
||||
stack_guard!();
|
||||
match n.kind() {
|
||||
"identifier" => paths.push((join(prefix, w.text(n)), n)),
|
||||
"scoped_identifier" => {
|
||||
@@ -991,6 +998,7 @@ impl<'t> Walker<'t> {
|
||||
/// every field has a field_identifier), and the field_declaration_list
|
||||
/// recursion that reaches it.
|
||||
fn extract_inheritance(&mut self, node: Node<'t>, class_row: u32) {
|
||||
stack_guard!();
|
||||
let extends_kind = edge_kind_index("extends").unwrap();
|
||||
for i in 0..node.named_child_count() {
|
||||
let Some(child) = node.named_child(i) else { continue };
|
||||
@@ -1094,6 +1102,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let type_name = self.text(node).to_string();
|
||||
if !type_name.is_empty() && !is_builtin_type(&type_name) {
|
||||
@@ -1111,10 +1120,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody -----------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -1270,6 +1281,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -471,6 +471,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// scalaBaseTypeName (tree-sitter.ts:201-224).
|
||||
fn scala_base_type_name(&self, node: Option<Node<'t>>) -> Option<String> {
|
||||
stack_guard!();
|
||||
let node = node?;
|
||||
match node.kind() {
|
||||
"type_identifier" | "identifier" => Some(self.text(node).to_string()),
|
||||
@@ -495,6 +496,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// emitScalaTypeRefs (scala.ts:27-45) — the hook's own builtin set.
|
||||
fn emit_scala_type_refs(&mut self, type_node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if type_node.kind() == "type_identifier" {
|
||||
let name = self.text(type_node);
|
||||
if !name.is_empty() && !is_scala_builtin(name) {
|
||||
@@ -529,6 +531,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the main walk (visitNode, tree-sitter.ts:936-1303) ---------------
|
||||
|
||||
fn visit(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// The visitNode hook (scala.ts:131-198) runs FIRST.
|
||||
if self.hook(node) {
|
||||
self.scan_fn_ref_subtree(node, 0);
|
||||
@@ -591,6 +594,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// The visitNode hook (scala.ts:131-198). Returns true when consumed.
|
||||
fn hook(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
match node.kind() {
|
||||
"val_definition" | "var_definition" => {
|
||||
let is_val = node.kind() == "val_definition";
|
||||
@@ -691,6 +695,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractMethod → extractFunction routing (:1737 / :1517) ----------
|
||||
|
||||
fn extract_method_or_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// No receiver hook, no methodsAreTopLevel: inside class-like → method,
|
||||
// else → function (the object/object_expression parent check never
|
||||
// matches scala node kinds).
|
||||
@@ -735,6 +740,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractClass (:1679) — classes, objects, traits ------------------
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>, kind: &'static str) {
|
||||
stack_guard!();
|
||||
let resolved_body = node.child_by_field_name("body"); // template_body
|
||||
// No skipBodilessClass — bodiless mints (scala-complete).
|
||||
let name = self.extract_name(node);
|
||||
@@ -765,6 +771,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractEnum (:1914) ----------------------------------------------
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let body = match node.child_by_field_name("body") {
|
||||
Some(b) => b,
|
||||
None => return, // bodiless enum mints nothing
|
||||
@@ -812,6 +819,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractImport (:3170-3236) ---------------------------------------
|
||||
|
||||
fn extract_import(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let import_text = self.text(node).trim();
|
||||
// extractImport hook (scala.ts:200-211): `path` field is FIRST-MATCH-
|
||||
// WINS → the FIRST dotted segment names the import.
|
||||
@@ -1133,6 +1141,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let name = self.text(node);
|
||||
if !name.is_empty() && !is_builtin_type(name) {
|
||||
@@ -1151,6 +1160,7 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody (:5129-5286) — scala rows ----------------------
|
||||
|
||||
fn visit_body(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
let kind = node.kind();
|
||||
@@ -1266,6 +1276,7 @@ impl<'t> Walker<'t> {
|
||||
/// normalizeValue with SCALA_SPEC's unwrap (postfix_expression → first
|
||||
/// named child — eta-expansion `handler _`). No layers.
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1294,6 +1305,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
//! Stack-budget guard for the recursive walkers (#1581).
|
||||
//!
|
||||
//! Every language walker recurses per AST level (`visit_node` →
|
||||
//! `visit_for_calls_and_structure` → …). tree-sitter's own parser is
|
||||
//! iterative, so a pathologically nested file — clang's
|
||||
//! `parser_overflow.c` nests 16,384 `{`, fuzzer corpora go deeper — parses
|
||||
//! fine and then overflows the WALKER's native stack. A native overflow is
|
||||
//! uncatchable: the parse worker is a thread of the `codegraph` process, so
|
||||
//! the SIGSEGV takes the whole indexer down with no message, no partial
|
||||
//! index and no per-file fallback. Worker threads get Node's 4 MiB default
|
||||
//! stack; the main thread's 8 MiB only moves the cliff (100k levels still
|
||||
//! kill it).
|
||||
//!
|
||||
//! The guard turns "about to overflow" into the kernel's existing `defer:`
|
||||
//! routing signal: `exhausted()` is checked at the top of every recursive
|
||||
//! walker function (the `stack_guard!` macro in lib.rs), returns `true` once
|
||||
//! the stack pointer is within `RED_ZONE` of the thread's stack limit, and
|
||||
//! latches a per-thread flag. `run_guarded` wraps a whole extraction: when
|
||||
//! the flag is set afterwards the result is discarded and replaced by a
|
||||
//! `defer:` error, which the TS side (`src/extraction/kernel/index.ts`)
|
||||
//! already treats as "this file takes the wasm path" — and the wasm walker
|
||||
//! catches its own JS `RangeError` per file, so the file lands as a partial
|
||||
//! result with a recorded parse error instead of a dead process.
|
||||
//!
|
||||
//! The per-thread stack bounds come from the OS (glibc/musl
|
||||
//! `pthread_getattr_np`, macOS `pthread_get_stackaddr_np`, Win32
|
||||
//! `GetCurrentThreadStackLimits`), computed once per thread and cached, so
|
||||
//! the guard is exact on the 4 MiB worker, the 8 MiB main thread and any
|
||||
//! `resourceLimits.stackSizeMb` alike. Where the bounds are unavailable the
|
||||
//! guard falls back to a fixed descent budget measured from the entry stack
|
||||
//! pointer. Hot path: one thread-local load and one compare.
|
||||
|
||||
use std::cell::Cell;
|
||||
|
||||
/// Headroom kept free below the deepest walker frame: the napi return path,
|
||||
/// tree-sitter's node accessors and the error formatting all still need to
|
||||
/// run after the guard trips, and the frames BETWEEN two guard checks (an
|
||||
/// `extract_class` between two `visit_node`s) are never more than a few KiB.
|
||||
const RED_ZONE: usize = 256 * 1024;
|
||||
|
||||
/// Descent budget when the OS can't report the thread's stack bounds — safe
|
||||
/// on anything from Node's 4 MiB worker default upwards.
|
||||
const FALLBACK_BUDGET: usize = 1024 * 1024;
|
||||
|
||||
thread_local! {
|
||||
/// Lowest stack-pointer value the walker may reach before the guard
|
||||
/// trips. `0` = not computed yet for this thread.
|
||||
static THRESHOLD: Cell<usize> = const { Cell::new(0) };
|
||||
/// `true` when the thread's threshold came from real OS bounds (fixed
|
||||
/// for the thread's lifetime) rather than the per-call fallback budget.
|
||||
static THRESHOLD_IS_OS: Cell<bool> = const { Cell::new(false) };
|
||||
/// Latched by `exhausted()`; read by `run_guarded` after the walk.
|
||||
static OVERFLOWED: Cell<bool> = const { Cell::new(false) };
|
||||
}
|
||||
|
||||
/// Approximate current stack pointer: the address of a local. Stacks grow
|
||||
/// downward on every target the kernel ships for (x86_64 / aarch64).
|
||||
#[inline(always)]
|
||||
fn current_sp() -> usize {
|
||||
let marker = 0u8;
|
||||
std::hint::black_box(&marker) as *const u8 as usize
|
||||
}
|
||||
|
||||
/// Low (deepest) address of the calling thread's stack, from the OS.
|
||||
#[cfg(target_os = "linux")]
|
||||
fn os_stack_low() -> Option<usize> {
|
||||
// SAFETY: plain pthread queries on the calling thread; `attr` is
|
||||
// initialised by pthread_getattr_np and destroyed before returning.
|
||||
unsafe {
|
||||
let mut attr: libc::pthread_attr_t = std::mem::zeroed();
|
||||
if libc::pthread_getattr_np(libc::pthread_self(), &mut attr) != 0 {
|
||||
return None;
|
||||
}
|
||||
let mut addr: *mut libc::c_void = std::ptr::null_mut();
|
||||
let mut size: libc::size_t = 0;
|
||||
let rc = libc::pthread_attr_getstack(&attr, &mut addr, &mut size);
|
||||
libc::pthread_attr_destroy(&mut attr);
|
||||
if rc != 0 || addr.is_null() || size == 0 {
|
||||
return None;
|
||||
}
|
||||
Some(addr as usize)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "macos")]
|
||||
fn os_stack_low() -> Option<usize> {
|
||||
// SAFETY: plain pthread queries on the calling thread.
|
||||
unsafe {
|
||||
let me = libc::pthread_self();
|
||||
// pthread_get_stackaddr_np returns the HIGH end (the stack base).
|
||||
let high = libc::pthread_get_stackaddr_np(me) as usize;
|
||||
let size = libc::pthread_get_stacksize_np(me);
|
||||
if high == 0 || size == 0 || size > high {
|
||||
return None;
|
||||
}
|
||||
Some(high - size)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
fn os_stack_low() -> Option<usize> {
|
||||
#[link(name = "kernel32")]
|
||||
extern "system" {
|
||||
// Win8+ (the bundled Node runtime needs Win10 anyway). Reports the
|
||||
// full RESERVED range; Windows commits pages on demand down to it.
|
||||
fn GetCurrentThreadStackLimits(low_limit: *mut usize, high_limit: *mut usize);
|
||||
}
|
||||
let mut low: usize = 0;
|
||||
let mut high: usize = 0;
|
||||
// SAFETY: both out-pointers are valid for the duration of the call.
|
||||
unsafe { GetCurrentThreadStackLimits(&mut low, &mut high) };
|
||||
if low == 0 || high <= low {
|
||||
return None;
|
||||
}
|
||||
Some(low)
|
||||
}
|
||||
|
||||
#[cfg(not(any(target_os = "linux", target_os = "macos", windows)))]
|
||||
fn os_stack_low() -> Option<usize> {
|
||||
None
|
||||
}
|
||||
|
||||
/// Arm the guard for one extraction on the calling thread: clear the latch
|
||||
/// and (re)compute the threshold. OS bounds are computed once per thread;
|
||||
/// the fallback budget is re-anchored at every call's entry stack pointer.
|
||||
pub fn begin() {
|
||||
OVERFLOWED.with(|o| o.set(false));
|
||||
let cached = THRESHOLD.with(|t| t.get());
|
||||
if cached != 0 && THRESHOLD_IS_OS.with(|f| f.get()) {
|
||||
return;
|
||||
}
|
||||
match os_stack_low() {
|
||||
Some(low) => {
|
||||
THRESHOLD.with(|t| t.set(low.saturating_add(RED_ZONE)));
|
||||
THRESHOLD_IS_OS.with(|f| f.set(true));
|
||||
}
|
||||
None => {
|
||||
THRESHOLD.with(|t| t.set(current_sp().saturating_sub(FALLBACK_BUDGET).max(1)));
|
||||
THRESHOLD_IS_OS.with(|f| f.set(false));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `true` once the walker has descended to within `RED_ZONE` of the stack
|
||||
/// limit. Latches `OVERFLOWED` so `run_guarded` can discard the result. A
|
||||
/// thread that never called `begin()` (a direct unit-test call) arms itself
|
||||
/// lazily from the current position.
|
||||
#[inline(always)]
|
||||
pub fn exhausted() -> bool {
|
||||
let threshold = THRESHOLD.with(|t| t.get());
|
||||
if threshold == 0 {
|
||||
begin();
|
||||
return exhausted();
|
||||
}
|
||||
if current_sp() < threshold {
|
||||
OVERFLOWED.with(|o| o.set(true));
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the guard tripped since the last `begin()`.
|
||||
pub fn overflowed() -> bool {
|
||||
OVERFLOWED.with(|o| o.get())
|
||||
}
|
||||
|
||||
/// Run one extraction under the guard. A walk that tripped the guard returns
|
||||
/// a `defer:` error — the TS side's routine "take the wasm path" signal —
|
||||
/// regardless of what the truncated walk produced.
|
||||
pub fn run_guarded<T>(f: impl FnOnce() -> Result<T, String>) -> Result<T, String> {
|
||||
begin();
|
||||
let out = f();
|
||||
if overflowed() {
|
||||
return Err(
|
||||
"defer: nesting too deep for the native walker — wasm recovery handles it".to_string(),
|
||||
);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// 1 MiB is a quarter of Node's worker default; a guard that holds here
|
||||
/// holds on every real thread. Without the guard these walks SIGSEGV the
|
||||
/// test process instead of failing an assertion.
|
||||
const SMALL_STACK: usize = 1 << 20;
|
||||
const DEPTH: usize = 30_000;
|
||||
|
||||
fn on_small_stack<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static) -> T {
|
||||
std::thread::Builder::new()
|
||||
.stack_size(SMALL_STACK)
|
||||
.spawn(f)
|
||||
.expect("spawn")
|
||||
.join()
|
||||
.expect("walker thread panicked")
|
||||
}
|
||||
|
||||
fn nested_parens(prefix: &str, suffix: &str) -> String {
|
||||
format!("{prefix}{}1{}{suffix}", "(".repeat(DEPTH), ")".repeat(DEPTH))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn os_bounds_are_sane_on_this_platform() {
|
||||
// Every shipped target has an OS implementation; the fallback budget
|
||||
// is only for platforms the kernel is not built for.
|
||||
let low = os_stack_low().expect("OS stack bounds available");
|
||||
let sp = current_sp();
|
||||
assert!(low < sp, "stack low {low:#x} must be below the current sp {sp:#x}");
|
||||
assert!(sp - low < 1 << 31, "implausible stack size {}", sp - low);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn small_stack_reports_its_own_bounds() {
|
||||
on_small_stack(|| {
|
||||
let low = os_stack_low().expect("OS stack bounds available");
|
||||
let used = current_sp() - low;
|
||||
// std/the OS round the requested size up a little (macOS reports
|
||||
// 1,060,864 for a 1 MiB request); the point is that the bounds
|
||||
// describe THIS thread's small stack, not the main thread's.
|
||||
assert!(
|
||||
used <= SMALL_STACK + 128 * 1024,
|
||||
"used {used} is not within the {SMALL_STACK}-byte stack"
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deep_braces_c_defer_instead_of_crashing() {
|
||||
let src = format!("void foo(void) {{\n{}{}\n}}\n", "{".repeat(DEPTH), "}".repeat(DEPTH));
|
||||
let r = on_small_stack(move || run_guarded(|| crate::ccpp::extract("deep.c", &src, "c")));
|
||||
let err = r.err().expect("deep nesting must defer");
|
||||
assert!(err.starts_with("defer:"), "unexpected error: {err}");
|
||||
}
|
||||
|
||||
type Extract = fn(&str) -> Result<crate::buffers::EmitOut, String>;
|
||||
|
||||
#[test]
|
||||
fn deep_parens_cpp_rust_ts_python_defer_instead_of_crashing() {
|
||||
let cases: [(&str, Extract, String); 4] = [
|
||||
("deep.cpp", |s| crate::ccpp::extract("deep.cpp", s, "cpp"), nested_parens("int f() { return ", "; }\n")),
|
||||
("deep.rs", |s| crate::rustlang::extract("deep.rs", s), nested_parens("fn f() -> i32 { ", " }\n")),
|
||||
("deep.ts", |s| crate::tsjs::extract("deep.ts", s, "typescript"), nested_parens("function f() { return ", "; }\n")),
|
||||
("deep.py", |s| crate::python::extract("deep.py", s), nested_parens("def f():\n return ", "\n")),
|
||||
];
|
||||
for (name, extract, src) in cases {
|
||||
let r = on_small_stack(move || run_guarded(|| extract(&src)));
|
||||
let err = r.err().unwrap_or_else(|| panic!("{name}: deep nesting must defer"));
|
||||
assert!(err.starts_with("defer:"), "{name}: unexpected error: {err}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normal_files_are_untouched_by_the_guard() {
|
||||
let src = "int add(int a, int b) { return a + b; }\nint main(void) { return add(1, 2); }\n";
|
||||
let r = on_small_stack(move || run_guarded(|| crate::ccpp::extract("ok.c", src, "c")));
|
||||
assert!(r.is_ok(), "a shallow file must not defer: {:?}", r.err());
|
||||
assert!(!overflowed());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn latch_resets_between_runs() {
|
||||
let deep = format!("void foo(void) {{\n{}{}\n}}\n", "{".repeat(DEPTH), "}".repeat(DEPTH));
|
||||
on_small_stack(move || {
|
||||
assert!(run_guarded(|| crate::ccpp::extract("deep.c", &deep, "c")).is_err());
|
||||
// The latch from the deep file must not poison the next, shallow one.
|
||||
let ok = run_guarded(|| crate::ccpp::extract("ok.c", "int x;\n", "c"));
|
||||
assert!(ok.is_ok(), "latch leaked into the next run: {:?}", ok.err());
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -264,6 +264,7 @@ fn first_simple_identifier<'t>(node: Option<Node<'t>>) -> Option<Node<'t>> {
|
||||
/// lastNamedOfType (function-ref.ts:600): rightmost matching DESCENDANT in
|
||||
/// document order (deeper matches override).
|
||||
fn last_simple_identifier<'t>(node: Node<'t>) -> Option<Node<'t>> {
|
||||
stack_guard!();
|
||||
let mut found: Option<Node<'t>> = None;
|
||||
for i in 0..node.named_child_count() {
|
||||
let Some(child) = node.named_child(i) else { continue };
|
||||
@@ -560,6 +561,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode, Swift-relevant branches) -----------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -632,6 +634,7 @@ impl<'t> Walker<'t> {
|
||||
/// THE DEDICATED PROPERTY BRANCH (tree-sitter.ts:1113-1193, #1020).
|
||||
/// Returns skipChildren.
|
||||
fn dedicated_property_branch(&mut self, node: Node<'t>) -> bool {
|
||||
stack_guard!();
|
||||
let owner_row = self.top_row();
|
||||
let info = self.swift_property_info(node);
|
||||
let mut computed_prop: Option<(u32, String)> = None;
|
||||
@@ -707,6 +710,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn walk_attr_args(&mut self, n: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.extract_static_member_ref(n);
|
||||
for i in 0..n.named_child_count() {
|
||||
if let Some(c) = n.named_child(i) {
|
||||
@@ -718,10 +722,12 @@ impl<'t> Walker<'t> {
|
||||
// --- visitFunctionBody ---------------------------------------------------------
|
||||
|
||||
fn visit_function_body(&mut self, body: Node<'t>) {
|
||||
stack_guard!();
|
||||
self.visit_for_calls_and_structure(body);
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
@@ -775,6 +781,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractors -----------------------------------------------------------------
|
||||
|
||||
fn extract_function(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
if name == "<anonymous>" {
|
||||
if let Some(body) = node.child_by_field_name("body") {
|
||||
@@ -802,6 +809,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_method(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -823,6 +831,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_class(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -845,6 +854,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_struct(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
// Body gate (:1876) — bodiless mints nothing (record exemption is C#).
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
@@ -866,6 +876,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_enum(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let Some(body) = node.child_by_field_name("body") else { return };
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
@@ -900,6 +911,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_interface(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let name = self.extract_name(node);
|
||||
let extra = Extra {
|
||||
docstring: preceding_docstring(node, self.src),
|
||||
@@ -1151,6 +1163,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let type_name = self.text(node).to_string();
|
||||
if !type_name.is_empty() && !is_builtin_type(&type_name) {
|
||||
@@ -1317,6 +1330,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
}
|
||||
@@ -1381,6 +1395,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -478,6 +478,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn find_initializer_returned_object(&self, call: Node<'t>, depth: u32) -> Option<Node<'t>> {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return None;
|
||||
}
|
||||
@@ -499,6 +500,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
fn function_returned_object(&self, fn_node: Node<'t>) -> Option<Node<'t>> {
|
||||
fn as_object<'t>(n: Node<'t>) -> Option<Node<'t>> {
|
||||
stack_guard!();
|
||||
match n.kind() {
|
||||
"object" | "object_expression" => Some(n),
|
||||
"parenthesized_expression" => {
|
||||
@@ -873,6 +875,7 @@ impl<'t> Walker<'t> {
|
||||
fn extract_ts_tuple_contract_names(&mut self, value: Node<'t>, alias_row: u32, alias_name: &str) {
|
||||
let mut tuples: Vec<Node> = Vec::new();
|
||||
fn collect<'t>(n: Node<'t>, depth: u32, out: &mut Vec<Node<'t>>) {
|
||||
stack_guard!();
|
||||
if depth > 6 {
|
||||
return;
|
||||
}
|
||||
@@ -1230,6 +1233,7 @@ impl<'t> Walker<'t> {
|
||||
// --- extractInheritance (TS/JS clauses) ---------------------------------------------------
|
||||
|
||||
pub(super) fn extract_inheritance(&mut self, node: Node<'t>, class_row: u32) {
|
||||
stack_guard!();
|
||||
let extends_kind = edge_kind_index("extends").unwrap();
|
||||
let implements_kind = edge_kind_index("implements").unwrap();
|
||||
for i in 0..node.named_child_count() {
|
||||
@@ -1298,6 +1302,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn extract_type_refs_from_subtree(&mut self, node: Node<'t>, from_row: u32) {
|
||||
stack_guard!();
|
||||
if node.kind() == "type_identifier" {
|
||||
let type_name = self.text(node).to_string();
|
||||
if !type_name.is_empty() && !is_builtin_type(&type_name) {
|
||||
|
||||
@@ -550,6 +550,7 @@ impl<'t> Walker<'t> {
|
||||
|
||||
/// scanFnRefSubtree: capture-only walk of subtrees the main walkers skip.
|
||||
fn scan_fn_ref_subtree(&mut self, node: Node<'t>, depth: u32) {
|
||||
stack_guard!();
|
||||
if depth > 12 {
|
||||
return;
|
||||
}
|
||||
@@ -607,6 +608,7 @@ impl<'t> Walker<'t> {
|
||||
// --- the dispatcher (visitNode) --------------------------------------------
|
||||
|
||||
fn visit_node(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
let mut skip_children = false;
|
||||
|
||||
@@ -686,6 +688,7 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
fn visit_for_calls_and_structure(&mut self, node: Node<'t>) {
|
||||
stack_guard!();
|
||||
let kind = node.kind();
|
||||
self.maybe_capture_fn_refs(node);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user