Repository guide · overview · v1
karpathy / nanochat
“The best ChatGPT that $100 can buy.” — as described by its authors
- Model Training
- Python
- MIT · open-source license
- Original repository
- karpathy/nanochat
- Source platform
- GitHub
- Repository owner / organization
- karpathy
- License
- MIT · open-source license
- Analyzed revision
- 92d63d4e8bb4df75c3b71618f31ddde2378b2bcd
- Last verified
Source checked 2026-11-15: default branch still at the analyzed revision.
karpathy/nanochat repository overview
Terms used on this page
- Entrypoint record
- A file the analyzer marks as a place where execution can start. When it carries a __main__ guard excerpt, the file contains an if __name__ == "__main__": block and the excerpt shows what that block calls; without an excerpt, the file was flagged by its name only.
- Bounded static execution path
- A call chain reconstructed from the source code without running it, stopped after a fixed number of steps. It shows how far the code can be followed on paper, not what happens at run time.
- Unresolved boundary
- Where a static path stops because the next call goes into an external library or cannot be resolved without running the code. It marks the edge of this analysis, not a defect in the repository.
- Static relations
- Calls and imports found in the source. "External or unresolved" relations point outside the analyzed files.
- Module role
- The analyzer's label for a file, inferred from how many calls go in and out (for example core, entry/orchestration, leaf). It describes a position in the call graph, not the authors' design.
- Test-like files
- Files whose names or locations look like tests. This analysis counts them; it does not run them.
- Observed · inferred · author-claimed · unresolved
- How each statement is supported: read directly from the analyzed files; derived by the analyzer from them; stated by the repository's authors (metadata, README); or not established by this analysis.
- Verified
- Two uses on these pages. In an analyzer record ("verified provenance", "a verified entrypoint") it means the record was read directly from the analyzed files, which these guides call observed; for an entrypoint, the __main__ guard text is present. Since the analyzer fix of 2026-10-04, a file that only has an entrypoint-like name such as cli.py or main.py is recorded as inferred; guides analyzed before that still call such a file verified, and it is still only a guess about how the file is used. It does not mean the code was run or tested. "Last verified" is the date the guide last passed the lab's checks against the analysis record of the stated revision; it is not a review of the repository itself.
A Python, MIT-licensed repository whose author describes it as a ChatGPT-like system obtainable for about $100. Static analysis finds seven `__main__` entrypoints, a heavily shared helper module `nanochat/common.py`, and training and evaluation scripts under `scripts/`. Of 2803 static relations, 2223 remain external or unresolved.
What it is
Platform metadata lists Python as the primary language, with smaller amounts of Jupyter Notebook and Shell code, and an MIT license. No release is published.
The repository's GitHub description frames it as a ChatGPT-like system that can be obtained for about $100. That is the author's claim and is not checked here.
Structurally, nanochat/engine.py defines a generation Engine class, a KVCache class and a sample_next_token function. nanochat/checkpoint_manager.py defines helpers such as build_model, load_model and save_checkpoint. Training-style scripts under scripts/ make many outgoing cross-module calls. Taken together, this suggests a codebase for training and running a GPT-style chat model. That reading is an interpretation of names and call structure.
How it starts
The analyzer found seven files with a __main__ guard:
nanochat/dataset.py,nanochat/engine.py,tasks/common.pyscripts/base_eval.py,scripts/chat_eval.py,scripts/infer_bench.pytests/test_attention_fallback.py
nanochat/dataset.py builds an argparse CLI for downloading pretraining dataset shards. It has options for the number of shards and the number of parallel workers. scripts/chat_eval.py requires a model source (sft or rl) and accepts an optional task name. The guard in nanochat/engine.py is documented as a quick inline test that compares the plain model.generate with Engine.generate.
The only bounded static paths in this packet start at nanochat/dataset.py and go into get_base_dir in nanochat/common.py. They end at standard-library calls such as os.environ.get, os.path.expanduser and os.makedirs.
By call degree, scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py look like orchestrators. How they are launched is not established here.
Structure
By the analyzer's directory counts, the 53 files split into:
nanochat/(14),scripts/(9),dev/(7)tasks/(6),tests/(6),runs/(4)- six root files and one file under
.claude/
The analyzer treats nanochat and tests as likely subsystems.
Static call degree suggests a hub-and-scripts shape:
nanochat/common.pyreceives 143 resolved cross-module calls and makes none. It holds shared helpers such asget_base_dir,print0,autodetect_device_typeandcompute_init.- Among the ten modules the analyzer ranks by static call degree, the next most-called after
nanochat/common.pyarenanochat/engine.py(17 incoming),tasks/common.py(16),nanochat/flash_attention.py(16) andnanochat/checkpoint_manager.py(12). scripts/base_train.py(58 outgoing),scripts/chat_sft.py(42) andscripts/base_eval.py(37) mostly call into the package.
tasks/common.py defines Task, TaskMixture, TaskSequence, HubDataset and render_mc. nanochat/flash_attention.py exposes flash_attn_func and flash_attn_with_kvcache, plus _sdpa_attention and _load_flash_attention_3. That reads like an attention layer with a fallback path.
Overall, the analyzer recorded 2803 relations: 2527 calls and 276 imports.
Dependencies and tests
There is one dependency manifest, pyproject.toml.
- Runtime records:
filelock,kernels,numpy,psutil,pyarrow,rustbpe,tiktoken,torch(pinned==2.9.1) andwandb. - Dev records:
ipykernel,matplotlib,pytestandpython-dotenv.
The parser also recorded torch twice and setuptools once with scope optional. What these optional records correspond to is not established. Dependency parsing covers requirements files, package.json and pyproject.toml only. Lock files and manifests such as setup.py are not parsed, and version constraints are kept as written.
The analyzer counts six test-like files. tests/test_attention_fallback.py defines TestFA3VsSDPA, TestSDPAOnly and TestOverrideMechanism. By name, these compare attention implementations. No test run is part of this analysis, so whether the tests pass is not established.
Read first
Suggested reading order:
README.mdandpyproject.toml, which the analyzer flags as project-level important files.nanochat/common.py, because it receives more resolved cross-module calls than any other module (143).- The flagged entrypoints:
nanochat/dataset.py,nanochat/engine.py,scripts/base_eval.py,scripts/chat_eval.pyandscripts/infer_bench.py. build_modelandload_modelinnanochat/checkpoint_manager.py, to see how saved models are loaded.scripts/base_train.py, the module with the most outgoing calls, to see how the pieces are combined for training.
Limits of this analysis
Of the 2803 static relations, 2223 remain external or unresolved.
The analysis is static only: dynamic imports, monkey-patching, reflection and dynamic dispatch are not resolved. This matters in nanochat/flash_attention.py, where _load_flash_attention_3 and _resolve_use_fa3 appear to choose between attention implementations. Which path runs on a given machine is not established.
The bounded execution paths in this packet only cover nanochat/dataset.py reaching get_base_dir. Training, evaluation and chat flows are not traced end to end.
Install, build and run inference is partial and no commands are verified, so none are given here.
Parsing is also bounded: very large files, minified bundles and vendored directories are inventoried but not parsed. Relation extraction is capped per file and per repository.
What this analysis could not establish
The writing model's own notes on the analysis record. Identifiers such as lim_3, exec_* or claim_… name records of that analysis; the claims table cites the same records.
- How scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py are invoked is not established; they are not listed among detected __main__ entrypoints.
- The meaning of the two optional torch records and the optional setuptools record in pyproject.toml is not established.
- Contents and purpose of the dev/, runs/ and .claude/ directories are not described in the packet; Shell bytes exist but their location is unknown.
- Which attention backend (FA3 or SDPA) is used at runtime is not established.
- Whether the six test files pass is not established; no tests were run.
Claims and evidence — 40 claims, 40 supported by an independent verifier
Every statement above is a claim that cites grounding IDs from the analysis. IDs are internal to the analysis run; the columns show what each claim rests on and how strong that ground is.
Every substantive statement above is a claim bound to grounding IDs of the analyzed revision. Global grounding IDs are namespaced by the analysis run.
| Claim | Epistemic status | Verifier | Grounding (global IDs) |
|---|---|---|---|
| c1 GitHub metadata lists Python as the primary language, with Jupyter Notebook and Shell bytes also present, and the license is MIT. | observed | supported | analysis_50dfee49d6903e32:meta_primary_language, analysis_50dfee49d6903e32:meta_language_bytes, analysis_50dfee49d6903e32:meta_license |
| c2 No release is published according to platform metadata. | observed | supported | analysis_50dfee49d6903e32:meta_latest_release |
| c3 The repository description frames the project as a ChatGPT-like system obtainable for about $100. | author_claimed | supported | analysis_50dfee49d6903e32:meta_description |
| c4 nanochat/engine.py defines an Engine class, a KVCache class and a sample_next_token function. | observed | supported | analysis_50dfee49d6903e32:py_class_2648a0b9102d, analysis_50dfee49d6903e32:py_class_99dd2946c374, analysis_50dfee49d6903e32:py_func_81063d17fa08 |
| c5 nanochat/checkpoint_manager.py defines build_model, load_model and save_checkpoint functions. | observed | supported | analysis_50dfee49d6903e32:py_func_cdb79cea93b5, analysis_50dfee49d6903e32:py_func_085478f8a510, analysis_50dfee49d6903e32:py_func_e512e99d6d24 |
| c6 Symbol names and training-style scripts under scripts/ that make many outgoing cross-module calls suggest a codebase for training and running a GPT-style chat model; this is an interpretation. | inferred | supported | analysis_50dfee49d6903e32:py_class_2648a0b9102d, analysis_50dfee49d6903e32:py_class_99dd2946c374, analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18 |
| c7 Seven files contain a __main__ guard: nanochat/dataset.py, nanochat/engine.py, tasks/common.py, scripts/base_eval.py, scripts/chat_eval.py, scripts/infer_bench.py and tests/test_attention_fallback.py. | observed | supported | analysis_50dfee49d6903e32:py_entry_6f78b0b1883f, analysis_50dfee49d6903e32:py_entry_d1f565fa3410, analysis_50dfee49d6903e32:py_entry_21a180b9528c, analysis_50dfee49d6903e32:py_entry_4e39b6126807, analysis_50dfee49d6903e32:py_entry_269c07384799, analysis_50dfee49d6903e32:py_entry_405999f52cfe, analysis_50dfee49d6903e32:py_entry_9395ac6dfb21 |
| c8 The __main__ block of nanochat/dataset.py builds an argparse CLI to download pretraining dataset shards, with options for number of files and number of parallel workers. | observed | supported | analysis_50dfee49d6903e32:py_entry_6f78b0b1883f |
| c9 scripts/chat_eval.py's __main__ block requires a --source argument (sft or rl) and accepts an optional task name. | observed | supported | analysis_50dfee49d6903e32:py_entry_269c07384799 |
| c10 The __main__ block in nanochat/engine.py is documented as an inline test comparing model.generate with Engine.generate. | observed | supported | analysis_50dfee49d6903e32:py_entry_d1f565fa3410 |
| c11 The bounded static paths start at nanochat/dataset.py __main__, enter get_base_dir in nanochat/common.py, and end at os.environ.get, os.path.expanduser, os.path.join and os.makedirs. | inferred | supported | analysis_50dfee49d6903e32:exec_de60eb0f815c, analysis_50dfee49d6903e32:exec_bebf565646e1, analysis_50dfee49d6903e32:exec_9b057297bbd5, analysis_50dfee49d6903e32:exec_d13940480c8e, analysis_50dfee49d6903e32:py_func_8dd5e10657c8 |
| c12 scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py have no incoming and 58, 42 and 16 outgoing resolved cross-module calls, and are classified as orchestration candidates. | inferred | supported | analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18, analysis_50dfee49d6903e32:role_17 |
| c13 How the training scripts are launched is not established; they are not among the detected __main__ entrypoints and run inference is partial. | unresolved | supported | analysis_50dfee49d6903e32:lim_2, analysis_50dfee49d6903e32:role_14 |
| c14 The 53 files split into nanochat (14), scripts (9), dev (7), tasks (6), tests (6), runs (4), root (6) and .claude (1). | inferred | supported | analysis_50dfee49d6903e32:summary_repository |
| c15 The analyzer marks the top-level directories nanochat and tests as likely subsystems. | observed | supported | analysis_50dfee49d6903e32:subsys_1, analysis_50dfee49d6903e32:subsys_2 |
| c16 nanochat/common.py has 143 incoming and 0 outgoing resolved cross-module calls and defines helpers including get_base_dir, print0, autodetect_device_type and compute_init, acting as a shared hub. | inferred | supported | analysis_50dfee49d6903e32:role_2, analysis_50dfee49d6903e32:py_func_8dd5e10657c8, analysis_50dfee49d6903e32:py_func_08a788aac199, analysis_50dfee49d6903e32:py_func_d0824d300379, analysis_50dfee49d6903e32:py_func_1b28a2b96f1d |
| c17 Among the ten modules the analyzer ranks by static call degree, the next most-called after nanochat/common.py are nanochat/engine.py (17), tasks/common.py (16), nanochat/flash_attention.py (16) and nanochat/checkpoint_manager.py (12). | inferred | supported | analysis_50dfee49d6903e32:role_6, analysis_50dfee49d6903e32:role_23, analysis_50dfee49d6903e32:role_8, analysis_50dfee49d6903e32:role_1 |
| c18 scripts/base_train.py (58 outgoing), scripts/chat_sft.py (42) and scripts/base_eval.py (37) mostly call into other modules, suggesting a hub-and-scripts shape. | inferred | supported | analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18, analysis_50dfee49d6903e32:role_13 |
| c19 tasks/common.py defines the classes Task, TaskMixture, TaskSequence and HubDataset, and a render_mc function. | observed | supported | analysis_50dfee49d6903e32:py_class_85f419f13bf1, analysis_50dfee49d6903e32:py_class_abf6926242ba, analysis_50dfee49d6903e32:py_class_092d56013510, analysis_50dfee49d6903e32:py_class_648e5421d94e, analysis_50dfee49d6903e32:py_func_d47df70bac2d |
| c20 nanochat/flash_attention.py exposes flash_attn_func and flash_attn_with_kvcache plus _sdpa_attention and _load_flash_attention_3, which reads like an attention layer with a fallback path. | inferred | supported | analysis_50dfee49d6903e32:py_func_830c5da3511c, analysis_50dfee49d6903e32:py_func_421d40c044d9, analysis_50dfee49d6903e32:py_func_e1cd46b6be90, analysis_50dfee49d6903e32:py_func_ad663fbc2b26 |
| c21 The analyzer recorded 2803 static relations: 2527 calls and 276 imports. | observed | supported | analysis_50dfee49d6903e32:relation_counts |
| c22 The repository contains one build/dependency manifest, pyproject.toml. | observed | supported | analysis_50dfee49d6903e32:ev_manifest_1 |
| c23 Runtime dependency records are filelock, kernels, numpy, psutil, pyarrow, rustbpe, tiktoken, torch (==2.9.1) and wandb. | observed | supported | analysis_50dfee49d6903e32:dep_1, analysis_50dfee49d6903e32:dep_2, analysis_50dfee49d6903e32:dep_3, analysis_50dfee49d6903e32:dep_4, analysis_50dfee49d6903e32:dep_5, analysis_50dfee49d6903e32:dep_6, analysis_50dfee49d6903e32:dep_7, analysis_50dfee49d6903e32:dep_8, analysis_50dfee49d6903e32:dep_9 |
| c24 Dev dependency records are ipykernel, matplotlib, pytest and python-dotenv. | observed | supported | analysis_50dfee49d6903e32:dep_13, analysis_50dfee49d6903e32:dep_14, analysis_50dfee49d6903e32:dep_15, analysis_50dfee49d6903e32:dep_16 |
| c25 The parser also recorded torch twice and setuptools once with scope optional. | observed | supported | analysis_50dfee49d6903e32:dep_10, analysis_50dfee49d6903e32:dep_11, analysis_50dfee49d6903e32:dep_12 |
| c26 What the optional torch and setuptools records correspond to in pyproject.toml is not established by the packet. | unresolved | supported | analysis_50dfee49d6903e32:dep_11, analysis_50dfee49d6903e32:dep_10, analysis_50dfee49d6903e32:lim_3a |
| c27 Dependency records come only from requirements files, package.json and pyproject.toml; setup.py, setup.cfg, Pipfile and lock files are not parsed, and constraints are recorded as written. | observed | supported | analysis_50dfee49d6903e32:lim_3a |
| c28 There are six test-like files; tests/test_attention_fallback.py defines TestFA3VsSDPA, TestSDPAOnly and TestOverrideMechanism, which by name compare attention implementations. | inferred | supported | analysis_50dfee49d6903e32:ev_tests_1, analysis_50dfee49d6903e32:py_class_dbf5ff79cf52, analysis_50dfee49d6903e32:py_class_38e524d2473b, analysis_50dfee49d6903e32:py_class_f742a2ec4b10 |
| c29 No tests were run as part of this analysis, so whether the tests pass is not established. | unresolved | supported | analysis_50dfee49d6903e32:lim_2, analysis_50dfee49d6903e32:lim_1 |
| c30 README.md and pyproject.toml are flagged as project-level important files and are a reasonable first read. | observed | supported | analysis_50dfee49d6903e32:important_1, analysis_50dfee49d6903e32:important_2 |
| c31 nanochat/common.py is worth reading early because it receives the most resolved cross-module calls. | inferred | supported | analysis_50dfee49d6903e32:role_2 |
| c32 nanochat/dataset.py, nanochat/engine.py, scripts/base_eval.py, scripts/chat_eval.py and scripts/infer_bench.py are flagged as important because they are likely entrypoints. | observed | supported | analysis_50dfee49d6903e32:important_4, analysis_50dfee49d6903e32:important_5, analysis_50dfee49d6903e32:important_6, analysis_50dfee49d6903e32:important_7, analysis_50dfee49d6903e32:important_8 |
| c33 build_model and load_model in nanochat/checkpoint_manager.py appear to be where saved models are loaded. | inferred | supported | analysis_50dfee49d6903e32:py_func_cdb79cea93b5, analysis_50dfee49d6903e32:py_func_085478f8a510 |
| c34 scripts/base_train.py has the most outgoing cross-module calls (58), so it likely shows how the package pieces are combined for training. | inferred | supported | analysis_50dfee49d6903e32:role_14 |
| c35 2223 of the 2803 static relations are external or unresolved; 580 are resolved locally. | observed | supported | analysis_50dfee49d6903e32:relation_counts |
| c36 The analysis is static only; reflection, dynamic imports, monkey-patching, runtime wiring and dynamic dispatch are not resolved. | observed | supported | analysis_50dfee49d6903e32:lim_1 |
| c37 Which attention implementation _load_flash_attention_3 and _resolve_use_fa3 select at runtime on a given machine is not established. | unresolved | supported | analysis_50dfee49d6903e32:py_func_ad663fbc2b26, analysis_50dfee49d6903e32:py_func_b589073abe13, analysis_50dfee49d6903e32:lim_1 |
| c38 The bounded execution paths in the packet only cover nanochat/dataset.py reaching get_base_dir; training, evaluation and chat flows are not traced end to end. | inferred | supported | analysis_50dfee49d6903e32:exec_de60eb0f815c, analysis_50dfee49d6903e32:exec_270399b88681 |
| c39 Install, run and build inference is partial and installation commands are not verified. | observed | supported | analysis_50dfee49d6903e32:lim_2 |
| c40 Analysis is bounded: files over 400 kB, minified bundles and vendored directories are inventoried but not parsed, and relation extraction is capped per file and per repository. | observed | supported | analysis_50dfee49d6903e32:lim_6 |
Sources, rights and disclosure · attribution-license-templates/v0.1
Rights notice. Original repository hosted on GitHub. Repository source code, documentation, names, media, and related project materials remain subject to the rights of their respective authors, contributors, and other rights holders and to applicable repository license terms.
Platform notice. GitHub is the source hosting platform for the linked repository. GitHub and related marks are trademarks of GitHub, Inc. EVEMISS Technology is not affiliated with or endorsed by GitHub unless explicitly stated otherwise.
How this page is produced. This page was produced using revision-aware repository analysis and AI-assisted editorial tooling. Technical claims are tied to the analyzed repository revision and may be revalidated when the source repository changes.
AI-assisted analysis. Reviewed by EVEMISS Technology through human–AI collaborative review. · Report a rights concern · All repository guides