Repository 指南 · 概覽 · v1
karpathy / nanochat
“The best ChatGPT that $100 can buy.” — 作者自述
- Model Training
- Python
- MIT · 開源授權
- 原始 repository
- karpathy/nanochat
- 來源平台
- GitHub
- 擁有者 / 組織
- karpathy
- 授權
- MIT · 開源授權
- 分析的版本
- 92d63d4e8bb4df75c3b71618f31ddde2378b2bcd
- 最後驗證
2026-11-15 檢查過原始專案:預設分支仍在分析的版本。
本頁中文由負責的 AI 編輯依英文正式版本 v1 翻譯;程式碼名稱、路徑、行號與數字都經確定性檢查,與英文版一致。陳述與依據表保留英文原文,因為那是獨立驗證者核對過的紀錄。看英文原文
karpathy/nanochat repository 概覽
本頁用語說明
- Entrypoint record(進入點記錄)
- 分析器標記為「程式可能從這裡開始執行」的檔案。附有 __main__ guard 摘錄時,表示檔案裡有 if __name__ == "__main__": 區塊,摘錄顯示它呼叫了什麼;沒有摘錄時,只是依檔名判斷。
- Bounded static execution path(有界靜態執行路徑)
- 不執行程式、直接從原始碼重建出的呼叫鏈,走到固定步數就停。它說明在紙面上能追到多遠,不代表實際執行時會發生什麼。
- Unresolved boundary(未解析邊界)
- 靜態路徑停下的地方:下一個呼叫進入外部函式庫,或不執行就無法確定。這是本次分析的邊界,不是 repository 的缺陷。
- Static relations(靜態關係)
- 在原始碼中找到的呼叫與 import。「external or unresolved」表示指向分析範圍以外。
- Module role(模組角色)
- 分析器依呼叫進出次數推論的檔案標籤(例如 core、entry/orchestration、leaf)。它描述的是在呼叫圖中的位置,不是作者的設計意圖。
- Test-like files(類測試檔案)
- 名稱或位置看起來像測試的檔案。本次分析只計數,不執行。
- Observed · inferred · author-claimed · unresolved
- 每句話的依據:直接讀自分析的檔案;由分析器從檔案推論;repository 作者自述(metadata、README);或本次分析無法確定。
- Verified(已驗證)
- 本頁有兩種用法。在分析器的記錄裡(「verified provenance」「verified entrypoint」),它表示這筆記錄是直接從分析的檔案讀到的,也就是本指南所說的 observed;以進入點來說,就是檔案裡確實有 __main__ guard 的文字。分析器在 2026-10-04 修正後,只有 cli.py、main.py 這類像進入點的檔名、沒有 guard 的檔案,會記為 inferred(推測);在那之前分析的指南仍把這類檔案標成 verified,它依然只是對檔案用途的推測。它不代表程式被執行或測試過。「最後驗證」是這份指南最後一次通過本實驗室檢查的日期,檢查對象是所標示版本的分析記錄,不是對 repository 本身的審查。
一個以 Python 撰寫、採 MIT 授權的 repository,作者描述它是一個花費約 $100 就能取得的類 ChatGPT 系統。靜態分析找到七個 `__main__` 進入點、一個被大量共用的輔助模組 `nanochat/common.py`,以及位於 `scripts/` 下的訓練與評估腳本。在 2803 個靜態關係中,有 2223 個仍是外部或未解析。
它是什麼
平台 metadata 列出 Python 為主要語言,另有少量 Jupyter Notebook 和 Shell 程式碼,授權為 MIT。沒有發布任何 release。
這個 repository 的 GitHub 描述把它定位為一個花費約 $100 就能取得的類 ChatGPT 系統。這是作者的說法,這裡沒有查核。
從結構來看,nanochat/engine.py 定義了一個負責生成的 Engine 類別、一個 KVCache 類別和一個 sample_next_token 函式。nanochat/checkpoint_manager.py 定義了 build_model、load_model 和 save_checkpoint 等輔助函式。scripts/ 下的訓練類腳本有許多對外的跨模組呼叫。綜合來看,這暗示它是一個用來訓練和執行 GPT 風格聊天模型的程式碼庫。這個解讀是根據名稱和呼叫結構所做的詮釋。
它如何啟動
分析器找到七個帶有 __main__ 守衛的檔案:
nanochat/dataset.py、nanochat/engine.py、tasks/common.pyscripts/base_eval.py、scripts/chat_eval.py、scripts/infer_bench.pytests/test_attention_fallback.py
nanochat/dataset.py 建立一個 argparse CLI,用來下載預訓練資料集的分片。它有設定分片數量和平行 worker 數量的選項。scripts/chat_eval.py 需要指定一個模型來源(sft 或 rl),並接受一個選填的任務名稱。nanochat/engine.py 中的守衛在文件中被說明為一個快速的內嵌測試,用來比較單純的 model.generate 和 Engine.generate。
這份分析資料中僅有的有界靜態路徑從 nanochat/dataset.py 開始,進入 nanochat/common.py 中的 get_base_dir。它們結束於 os.environ.get、os.path.expanduser 和 os.makedirs 等標準函式庫呼叫。
依呼叫度數來看,scripts/base_train.py、scripts/chat_sft.py 和 scripts/chat_rl.py 看起來像是協調者(orchestrator)。它們如何被啟動,這裡沒有確定。
結構
依分析器的目錄計數,這 53 個檔案分布如下:
nanochat/(14)、scripts/(9)、dev/(7)tasks/(6)、tests/(6)、runs/(4)- 六個根目錄檔案,以及
.claude/下的一個檔案
分析器把 nanochat 和 tests 視為可能的子系統。
靜態呼叫度數暗示一種「中樞加腳本」的形態:
nanochat/common.py接收 143 個已解析的跨模組呼叫,本身沒有發出任何呼叫。它包含get_base_dir、print0、autodetect_device_type和compute_init等共用輔助函式。- 在分析器依靜態呼叫度數排名的十個模組中,繼
nanochat/common.py之後被呼叫最多的是nanochat/engine.py(17 個呼入)、tasks/common.py(16)、nanochat/flash_attention.py(16)和nanochat/checkpoint_manager.py(12)。 scripts/base_train.py(58 個呼出)、scripts/chat_sft.py(42)和scripts/base_eval.py(37)主要是呼叫套件內的程式碼。
tasks/common.py 定義了 Task、TaskMixture、TaskSequence、HubDataset 和 render_mc。nanochat/flash_attention.py 對外提供 flash_attn_func 和 flash_attn_with_kvcache,另外還有 _sdpa_attention 和 _load_flash_attention_3。這讀起來像是一個帶有後備路徑的 attention 層。
整體而言,分析器記錄了 2803 個關係:2527 個呼叫和 276 個 import。
依賴與測試
只有一個依賴清單,pyproject.toml。
- 執行期記錄:
filelock、kernels、numpy、psutil、pyarrow、rustbpe、tiktoken、torch(固定為==2.9.1)和wandb。 - 開發記錄:
ipykernel、matplotlib、pytest和python-dotenv。
解析器還以 optional 範圍記錄了兩次 torch 和一次 setuptools。這些 optional 記錄對應的是什麼,沒有確定。依賴解析只涵蓋 requirements 檔案、package.json 和 pyproject.toml。lock 檔案和 setup.py 之類的清單不會被解析,版本限制則照原樣保留。
分析器算出六個類似測試的檔案。tests/test_attention_fallback.py 定義了 TestFA3VsSDPA、TestSDPAOnly 和 TestOverrideMechanism。從名稱來看,這些是在比較不同的 attention 實作。這次分析沒有執行任何測試,所以測試是否通過沒有確定。
先讀哪些
建議的閱讀順序:
README.md和pyproject.toml,分析器把它們標記為專案層級的重要檔案。nanochat/common.py,因為它接收的已解析跨模組呼叫比其他任何模組都多(143)。- 被標記的進入點:
nanochat/dataset.py、nanochat/engine.py、scripts/base_eval.py、scripts/chat_eval.py和scripts/infer_bench.py。 nanochat/checkpoint_manager.py中的build_model和load_model,了解已儲存的模型如何被載入。scripts/base_train.py,傳出呼叫最多的模組,了解各個部分如何組合起來進行訓練。
這次分析的限制
在 2803 個靜態關係中,有 2223 個仍是外部或未解析。
這次分析只有靜態分析:動態 import、monkey-patching、reflection 和動態分派都不會被解析。這一點在 nanochat/flash_attention.py 中很重要,其中 _load_flash_attention_3 和 _resolve_use_fa3 看起來會在不同的 attention 實作之間做選擇。在某台機器上實際執行哪條路徑,沒有確定。
這份分析資料中的有界執行路徑只涵蓋 nanochat/dataset.py 到達 get_base_dir 的部分。訓練、評估和聊天流程都沒有從頭到尾追蹤。
安裝、建置和執行方式的推論並不完整,也沒有任何指令經過驗證,所以這裡不提供指令。
解析範圍也有上限:非常大的檔案、壓縮過的 bundle 和 vendored 目錄只會列入清單,不會被解析。關係擷取在每個檔案和每個 repository 都有數量上限。
這次分析無法確定的事
這些是撰寫模型對分析記錄自己的說明。lim_3、exec_* 或 claim_… 這類識別碼指的是那次分析裡的記錄;陳述與依據表引用的也是同一批記錄。
- scripts/base_train.py、scripts/chat_sft.py 和 scripts/chat_rl.py 如何被呼叫沒有確定;它們不在偵測到的 __main__ 進入點之列。
- pyproject.toml 中兩筆 optional torch 記錄和一筆 optional setuptools 記錄的意義沒有確定。
- 分析資料沒有說明 dev/、runs/ 和 .claude/ 目錄的內容和用途;存在 Shell 程式碼位元組,但位置不明。
- 執行期使用哪個 attention 後端(FA3 或 SDPA)沒有確定。
- 六個測試檔案是否通過沒有確定;沒有執行任何測試。
陳述與依據 — 40 條陳述,40 條經獨立驗證者確認(英文原文)
上面每一句都是一條陳述,各自引用分析裡的依據編號。編號是這次分析內部的;表格列出每條陳述靠的是什麼、依據有多強。
Every substantive statement above is a claim bound to grounding IDs of the analyzed revision. Global grounding IDs are namespaced by the analysis run.
| Claim | Epistemic status | Verifier | Grounding (global IDs) |
|---|---|---|---|
| c1 GitHub metadata lists Python as the primary language, with Jupyter Notebook and Shell bytes also present, and the license is MIT. | observed | supported | analysis_50dfee49d6903e32:meta_primary_language, analysis_50dfee49d6903e32:meta_language_bytes, analysis_50dfee49d6903e32:meta_license |
| c2 No release is published according to platform metadata. | observed | supported | analysis_50dfee49d6903e32:meta_latest_release |
| c3 The repository description frames the project as a ChatGPT-like system obtainable for about $100. | author_claimed | supported | analysis_50dfee49d6903e32:meta_description |
| c4 nanochat/engine.py defines an Engine class, a KVCache class and a sample_next_token function. | observed | supported | analysis_50dfee49d6903e32:py_class_2648a0b9102d, analysis_50dfee49d6903e32:py_class_99dd2946c374, analysis_50dfee49d6903e32:py_func_81063d17fa08 |
| c5 nanochat/checkpoint_manager.py defines build_model, load_model and save_checkpoint functions. | observed | supported | analysis_50dfee49d6903e32:py_func_cdb79cea93b5, analysis_50dfee49d6903e32:py_func_085478f8a510, analysis_50dfee49d6903e32:py_func_e512e99d6d24 |
| c6 Symbol names and training-style scripts under scripts/ that make many outgoing cross-module calls suggest a codebase for training and running a GPT-style chat model; this is an interpretation. | inferred | supported | analysis_50dfee49d6903e32:py_class_2648a0b9102d, analysis_50dfee49d6903e32:py_class_99dd2946c374, analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18 |
| c7 Seven files contain a __main__ guard: nanochat/dataset.py, nanochat/engine.py, tasks/common.py, scripts/base_eval.py, scripts/chat_eval.py, scripts/infer_bench.py and tests/test_attention_fallback.py. | observed | supported | analysis_50dfee49d6903e32:py_entry_6f78b0b1883f, analysis_50dfee49d6903e32:py_entry_d1f565fa3410, analysis_50dfee49d6903e32:py_entry_21a180b9528c, analysis_50dfee49d6903e32:py_entry_4e39b6126807, analysis_50dfee49d6903e32:py_entry_269c07384799, analysis_50dfee49d6903e32:py_entry_405999f52cfe, analysis_50dfee49d6903e32:py_entry_9395ac6dfb21 |
| c8 The __main__ block of nanochat/dataset.py builds an argparse CLI to download pretraining dataset shards, with options for number of files and number of parallel workers. | observed | supported | analysis_50dfee49d6903e32:py_entry_6f78b0b1883f |
| c9 scripts/chat_eval.py's __main__ block requires a --source argument (sft or rl) and accepts an optional task name. | observed | supported | analysis_50dfee49d6903e32:py_entry_269c07384799 |
| c10 The __main__ block in nanochat/engine.py is documented as an inline test comparing model.generate with Engine.generate. | observed | supported | analysis_50dfee49d6903e32:py_entry_d1f565fa3410 |
| c11 The bounded static paths start at nanochat/dataset.py __main__, enter get_base_dir in nanochat/common.py, and end at os.environ.get, os.path.expanduser, os.path.join and os.makedirs. | inferred | supported | analysis_50dfee49d6903e32:exec_de60eb0f815c, analysis_50dfee49d6903e32:exec_bebf565646e1, analysis_50dfee49d6903e32:exec_9b057297bbd5, analysis_50dfee49d6903e32:exec_d13940480c8e, analysis_50dfee49d6903e32:py_func_8dd5e10657c8 |
| c12 scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py have no incoming and 58, 42 and 16 outgoing resolved cross-module calls, and are classified as orchestration candidates. | inferred | supported | analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18, analysis_50dfee49d6903e32:role_17 |
| c13 How the training scripts are launched is not established; they are not among the detected __main__ entrypoints and run inference is partial. | unresolved | supported | analysis_50dfee49d6903e32:lim_2, analysis_50dfee49d6903e32:role_14 |
| c14 The 53 files split into nanochat (14), scripts (9), dev (7), tasks (6), tests (6), runs (4), root (6) and .claude (1). | inferred | supported | analysis_50dfee49d6903e32:summary_repository |
| c15 The analyzer marks the top-level directories nanochat and tests as likely subsystems. | observed | supported | analysis_50dfee49d6903e32:subsys_1, analysis_50dfee49d6903e32:subsys_2 |
| c16 nanochat/common.py has 143 incoming and 0 outgoing resolved cross-module calls and defines helpers including get_base_dir, print0, autodetect_device_type and compute_init, acting as a shared hub. | inferred | supported | analysis_50dfee49d6903e32:role_2, analysis_50dfee49d6903e32:py_func_8dd5e10657c8, analysis_50dfee49d6903e32:py_func_08a788aac199, analysis_50dfee49d6903e32:py_func_d0824d300379, analysis_50dfee49d6903e32:py_func_1b28a2b96f1d |
| c17 Among the ten modules the analyzer ranks by static call degree, the next most-called after nanochat/common.py are nanochat/engine.py (17), tasks/common.py (16), nanochat/flash_attention.py (16) and nanochat/checkpoint_manager.py (12). | inferred | supported | analysis_50dfee49d6903e32:role_6, analysis_50dfee49d6903e32:role_23, analysis_50dfee49d6903e32:role_8, analysis_50dfee49d6903e32:role_1 |
| c18 scripts/base_train.py (58 outgoing), scripts/chat_sft.py (42) and scripts/base_eval.py (37) mostly call into other modules, suggesting a hub-and-scripts shape. | inferred | supported | analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18, analysis_50dfee49d6903e32:role_13 |
| c19 tasks/common.py defines the classes Task, TaskMixture, TaskSequence and HubDataset, and a render_mc function. | observed | supported | analysis_50dfee49d6903e32:py_class_85f419f13bf1, analysis_50dfee49d6903e32:py_class_abf6926242ba, analysis_50dfee49d6903e32:py_class_092d56013510, analysis_50dfee49d6903e32:py_class_648e5421d94e, analysis_50dfee49d6903e32:py_func_d47df70bac2d |
| c20 nanochat/flash_attention.py exposes flash_attn_func and flash_attn_with_kvcache plus _sdpa_attention and _load_flash_attention_3, which reads like an attention layer with a fallback path. | inferred | supported | analysis_50dfee49d6903e32:py_func_830c5da3511c, analysis_50dfee49d6903e32:py_func_421d40c044d9, analysis_50dfee49d6903e32:py_func_e1cd46b6be90, analysis_50dfee49d6903e32:py_func_ad663fbc2b26 |
| c21 The analyzer recorded 2803 static relations: 2527 calls and 276 imports. | observed | supported | analysis_50dfee49d6903e32:relation_counts |
| c22 The repository contains one build/dependency manifest, pyproject.toml. | observed | supported | analysis_50dfee49d6903e32:ev_manifest_1 |
| c23 Runtime dependency records are filelock, kernels, numpy, psutil, pyarrow, rustbpe, tiktoken, torch (==2.9.1) and wandb. | observed | supported | analysis_50dfee49d6903e32:dep_1, analysis_50dfee49d6903e32:dep_2, analysis_50dfee49d6903e32:dep_3, analysis_50dfee49d6903e32:dep_4, analysis_50dfee49d6903e32:dep_5, analysis_50dfee49d6903e32:dep_6, analysis_50dfee49d6903e32:dep_7, analysis_50dfee49d6903e32:dep_8, analysis_50dfee49d6903e32:dep_9 |
| c24 Dev dependency records are ipykernel, matplotlib, pytest and python-dotenv. | observed | supported | analysis_50dfee49d6903e32:dep_13, analysis_50dfee49d6903e32:dep_14, analysis_50dfee49d6903e32:dep_15, analysis_50dfee49d6903e32:dep_16 |
| c25 The parser also recorded torch twice and setuptools once with scope optional. | observed | supported | analysis_50dfee49d6903e32:dep_10, analysis_50dfee49d6903e32:dep_11, analysis_50dfee49d6903e32:dep_12 |
| c26 What the optional torch and setuptools records correspond to in pyproject.toml is not established by the packet. | unresolved | supported | analysis_50dfee49d6903e32:dep_11, analysis_50dfee49d6903e32:dep_10, analysis_50dfee49d6903e32:lim_3a |
| c27 Dependency records come only from requirements files, package.json and pyproject.toml; setup.py, setup.cfg, Pipfile and lock files are not parsed, and constraints are recorded as written. | observed | supported | analysis_50dfee49d6903e32:lim_3a |
| c28 There are six test-like files; tests/test_attention_fallback.py defines TestFA3VsSDPA, TestSDPAOnly and TestOverrideMechanism, which by name compare attention implementations. | inferred | supported | analysis_50dfee49d6903e32:ev_tests_1, analysis_50dfee49d6903e32:py_class_dbf5ff79cf52, analysis_50dfee49d6903e32:py_class_38e524d2473b, analysis_50dfee49d6903e32:py_class_f742a2ec4b10 |
| c29 No tests were run as part of this analysis, so whether the tests pass is not established. | unresolved | supported | analysis_50dfee49d6903e32:lim_2, analysis_50dfee49d6903e32:lim_1 |
| c30 README.md and pyproject.toml are flagged as project-level important files and are a reasonable first read. | observed | supported | analysis_50dfee49d6903e32:important_1, analysis_50dfee49d6903e32:important_2 |
| c31 nanochat/common.py is worth reading early because it receives the most resolved cross-module calls. | inferred | supported | analysis_50dfee49d6903e32:role_2 |
| c32 nanochat/dataset.py, nanochat/engine.py, scripts/base_eval.py, scripts/chat_eval.py and scripts/infer_bench.py are flagged as important because they are likely entrypoints. | observed | supported | analysis_50dfee49d6903e32:important_4, analysis_50dfee49d6903e32:important_5, analysis_50dfee49d6903e32:important_6, analysis_50dfee49d6903e32:important_7, analysis_50dfee49d6903e32:important_8 |
| c33 build_model and load_model in nanochat/checkpoint_manager.py appear to be where saved models are loaded. | inferred | supported | analysis_50dfee49d6903e32:py_func_cdb79cea93b5, analysis_50dfee49d6903e32:py_func_085478f8a510 |
| c34 scripts/base_train.py has the most outgoing cross-module calls (58), so it likely shows how the package pieces are combined for training. | inferred | supported | analysis_50dfee49d6903e32:role_14 |
| c35 2223 of the 2803 static relations are external or unresolved; 580 are resolved locally. | observed | supported | analysis_50dfee49d6903e32:relation_counts |
| c36 The analysis is static only; reflection, dynamic imports, monkey-patching, runtime wiring and dynamic dispatch are not resolved. | observed | supported | analysis_50dfee49d6903e32:lim_1 |
| c37 Which attention implementation _load_flash_attention_3 and _resolve_use_fa3 select at runtime on a given machine is not established. | unresolved | supported | analysis_50dfee49d6903e32:py_func_ad663fbc2b26, analysis_50dfee49d6903e32:py_func_b589073abe13, analysis_50dfee49d6903e32:lim_1 |
| c38 The bounded execution paths in the packet only cover nanochat/dataset.py reaching get_base_dir; training, evaluation and chat flows are not traced end to end. | inferred | supported | analysis_50dfee49d6903e32:exec_de60eb0f815c, analysis_50dfee49d6903e32:exec_270399b88681 |
| c39 Install, run and build inference is partial and installation commands are not verified. | observed | supported | analysis_50dfee49d6903e32:lim_2 |
| c40 Analysis is bounded: files over 400 kB, minified bundles and vendored directories are inventoried but not parsed, and relation extraction is capped per file and per repository. | observed | supported | analysis_50dfee49d6903e32:lim_6 |
來源、權利與說明 · attribution-license-templates/v0.1
權利聲明。原始 repository 託管於 GitHub。其原始碼、文件、名稱、媒體與相關素材的權利,仍屬各自的作者、貢獻者與其他權利人所有,並受該 repository 的授權條款約束。
Original repository hosted on GitHub. Repository source code, documentation, names, media, and related project materials remain subject to the rights of their respective authors, contributors, and other rights holders and to applicable repository license terms.
平台聲明。GitHub 是所連結 repository 的來源託管平台。GitHub 與相關標誌為 GitHub, Inc. 的商標。除非另有明確說明,EVEMISS Technology 與 GitHub 之間沒有隸屬或背書關係。
GitHub is the source hosting platform for the linked repository. GitHub and related marks are trademarks of GitHub, Inc. EVEMISS Technology is not affiliated with or endorsed by GitHub unless explicitly stated otherwise.
這一頁怎麼來的。本頁由對應版本的 repository 分析與 AI 輔助的編輯工具產生。技術陳述綁定所分析的版本;原始 repository 更新後,可能重新驗證。
This page was produced using revision-aware repository analysis and AI-assisted editorial tooling. Technical claims are tied to the analyzed repository revision and may be revalidated when the source repository changes.
本頁包含 AI 輔助分析,並經 EVEMISS Technology 人工與 AI 協力審閱。 · 回報權利疑慮 · 所有 repository 指南