在昨天的實測中,Ollama 成功自主呼叫了 find_importers,並找出 4 個直接 import rag_common 的檔案與行號。
但如果你仔細看過真實世界的 Python 程式碼,就會發現單純的比對很快會陣亡:
import rag_common as rc
from rag_common import COLLECTION_NAME as C_NAME
from . import rag_common
如果只靠文字搜尋或正則表達式(Regex),遇到 import ... as ... 的別名,或是巢狀類別裡面的 method,Regex 根本無法得知當前變數代表誰,更無法區分這行是「函式定義」還是「剛好字串叫這個名字」。
今天我們要進入靜態分析的重頭戲:用 Python 原生 ast.NodeVisitor 走訪語法樹,精準解析 Symbol 的階層(Qualified Name)與 Import 別名映射(Alias Mapping)。
Python 原生提供了 ast.walk(),但它會把所有語法節點無差別攤平:
它分不出 run() 到底是頂層的一般函式,還是 class TaskRunner 裡面的成員方法。
它遺失了作用域(Scope)上下文。
為了保留完整的階層資訊,我們需要繼承標準函式庫的 ast.NodeVisitor,利用呼叫堆疊(Call Stack)在走訪過程中動態記錄當前的所屬類別。
SymbolVisitor 與語法提取我們在 app/codebase.py 中實作完整的 AST 萃取器:
# app/codebase.py (擴充 AST 萃取能力)
import ast
from dataclasses import dataclass
from typing import List, Optional
@dataclass
class ExtractedSymbol:
name: str # 原始名稱,如 "run"
qualified_name: str # 完整限定名稱,如 "TaskRunner.run"
kind: str # "class" | "function" | "method"
start_line: int
end_line: int
docstring: Optional[str] = None
@dataclass
class ExtractedImport:
module_name: str # 匯入來源,如 "rag_common"
imported_name: Optional[str] # 匯入的具體對象,如 "COLLECTION_NAME"
alias: Optional[str] # 本地別名,如 "rc" 或 "C_NAME"
line: int
class CodebaseASTVisitor(ast.NodeVisitor):
def __init__(self):
self.scope_stack: List[str] = []
self.symbols: List[ExtractedSymbol] = []
self.imports: List[ExtractedImport] = []
def visit_ClassDef(self, node: ast.ClassDef):
# 1. 記錄 Class 本體
q_name = ".".join(self.scope_stack + [node.name])
doc = ast.get_docstring(node)
self.symbols.append(ExtractedSymbol(
name=node.name,
qualified_name=q_name,
kind="class",
start_line=node.lineno,
end_line=node.end_lineno or node.lineno,
docstring=doc
))
# 2. 將 Class 推入 Scope Stack,繼續走訪內部方法
self.scope_stack.append(node.name)
self.generic_visit(node)
self.scope_stack.pop()
def visit_FunctionDef(self, node: ast.FunctionDef):
self._handle_function(node)
def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef):
self._handle_function(node)
def _handle_function(self, node):
kind = "method" if self.scope_stack else "function"
q_name = ".".join(self.scope_stack + [node.name])
doc = ast.get_docstring(node)
self.symbols.append(ExtractedSymbol(
name=node.name,
qualified_name=q_name,
kind=kind,
start_line=node.lineno,
end_line=node.end_lineno or node.lineno,
docstring=doc
))
# 函式內部可能有內部函式(closure),推入 stack
self.scope_stack.append(node.name)
self.generic_visit(node)
self.scope_stack.pop()
def visit_Import(self, node: ast.Import):
# 處理 import a as b
for alias in node.names:
self.imports.append(ExtractedImport(
module_name=alias.name,
imported_name=None,
alias=alias.asname,
line=node.lineno
))
def visit_ImportFrom(self, node: ast.ImportFrom):
# 處理 from a import b as c
module = node.module or ""
for alias in node.names:
self.imports.append(ExtractedImport(
module_name=module,
imported_name=alias.name,
alias=alias.asname,
line=node.lineno
))
在 tests/unit/test_ast_parse.py 驗證解析器能否正確辨識 as 別名與方法階層:
# tests/unit/test_ast_parse.py
import ast
from app.codebase import CodebaseASTVisitor
def test_ast_visitor_aliases_and_methods():
source = (
"import rag_common as rc\n"
"from config import SETTINGS as CFG\n"
"\n"
"class Pipeline:\n"
" def run(self):\n"
" pass\n"
"\n"
"def standalone_func():\n"
" pass\n"
)
tree = ast.parse(source)
visitor = CodebaseASTVisitor()
visitor.visit(tree)
# 1. 驗證 Import 別名捕捉
assert len(visitor.imports) == 2
assert visitor.imports[0].module_name == "rag_common"
assert visitor.imports[0].alias == "rc"
assert visitor.imports[1].imported_name == "SETTINGS"
assert visitor.imports[1].alias == "CFG"
# 2. 驗證 Class 與 Method 階層辨識
symbols_by_name = {s.name: s for s in visitor.symbols}
assert "Pipeline" in symbols_by_name
assert symbols_by_name["Pipeline"].kind == "class"
assert "run" in symbols_by_name
assert symbols_by_name["run"].kind == "method"
assert symbols_by_name["run"].qualified_name == "Pipeline.run"
assert "standalone_func" in symbols_by_name
assert symbols_by_name["standalone_func"].kind == "function"
assert symbols_by_name["standalone_func"].qualified_name == "standalone_func"
執行測試確認綠燈:
uv run pytest tests/unit/test_ast_parse.py -v
tests/unit/test_ast_parse.py::test_ast_visitor_aliases_and_methods PASSED [100%]
============================== 1 passed in 0.04s ==============================
mobileai-local-rag 上的解析成果當我們將這套 Visitor 套用到範例 repository 的 7 支檔案上時,解析報告有了質的飛躍:
uv run python -m app.cli parse-report
輸出摘要:
=== AST Parsing Report: mobileai-local-rag ===
Files Parsed: 7
Total Symbols Found: 23
- Classes: 2
- Methods: 9
- Functions: 12
Total Imports Mapped: 28 (including 5 aliased imports)
Parse Failures: 0
Status: CLEAN
現在,程式不只知道某一行出現了 rag_common,還能明確辨別:
import rag_common as rc。build() 屬於全域函式,哪一個屬於 Class 的專屬成員方法。今天我們完成了「把文字轉成結構化語法樹」的核心技術升級:
as 改名後的實際來源。但如果每次使用者提問、或 Ollama 調用工具時,我們都要重新打開原始碼跑一次 ast.parse,專案一大速度就會大幅拖慢。
明天在 Day 10 中,我們將正式把今天抓出來的 23 個 Symbol 與 28 條 Import 關聯,寫入結構化的 SQLite 索引資料表!