EVEMISSTechnology

Repository guide · overview · v1

karpathy / nanochat

“The best ChatGPT that $100 can buy.” — as described by its authors

  • Model Training
  • Python
  • MIT · open-source license
Original repository
karpathy/nanochat
Source platform
GitHub
Repository owner / organization
karpathy
License
MIT · open-source license
Analyzed revision
92d63d4e8bb4df75c3b71618f31ddde2378b2bcd
Last verified

Source checked 2026-11-15: default branch still at the analyzed revision.

karpathy/nanochat repository overview

Terms used on this page
Entrypoint record
A file the analyzer marks as a place where execution can start. When it carries a __main__ guard excerpt, the file contains an if __name__ == "__main__": block and the excerpt shows what that block calls; without an excerpt, the file was flagged by its name only.
Bounded static execution path
A call chain reconstructed from the source code without running it, stopped after a fixed number of steps. It shows how far the code can be followed on paper, not what happens at run time.
Unresolved boundary
Where a static path stops because the next call goes into an external library or cannot be resolved without running the code. It marks the edge of this analysis, not a defect in the repository.
Static relations
Calls and imports found in the source. "External or unresolved" relations point outside the analyzed files.
Module role
The analyzer's label for a file, inferred from how many calls go in and out (for example core, entry/orchestration, leaf). It describes a position in the call graph, not the authors' design.
Test-like files
Files whose names or locations look like tests. This analysis counts them; it does not run them.
Observed · inferred · author-claimed · unresolved
How each statement is supported: read directly from the analyzed files; derived by the analyzer from them; stated by the repository's authors (metadata, README); or not established by this analysis.
Verified
Two uses on these pages. In an analyzer record ("verified provenance", "a verified entrypoint") it means the record was read directly from the analyzed files, which these guides call observed; for an entrypoint, the __main__ guard text is present. Since the analyzer fix of 2026-10-04, a file that only has an entrypoint-like name such as cli.py or main.py is recorded as inferred; guides analyzed before that still call such a file verified, and it is still only a guess about how the file is used. It does not mean the code was run or tested. "Last verified" is the date the guide last passed the lab's checks against the analysis record of the stated revision; it is not a review of the repository itself.

A Python, MIT-licensed repository whose author describes it as a ChatGPT-like system obtainable for about $100. Static analysis finds seven `__main__` entrypoints, a heavily shared helper module `nanochat/common.py`, and training and evaluation scripts under `scripts/`. Of 2803 static relations, 2223 remain external or unresolved.

What it is

Platform metadata lists Python as the primary language, with smaller amounts of Jupyter Notebook and Shell code, and an MIT license. No release is published.

The repository's GitHub description frames it as a ChatGPT-like system that can be obtained for about $100. That is the author's claim and is not checked here.

Structurally, nanochat/engine.py defines a generation Engine class, a KVCache class and a sample_next_token function. nanochat/checkpoint_manager.py defines helpers such as build_model, load_model and save_checkpoint. Training-style scripts under scripts/ make many outgoing cross-module calls. Taken together, this suggests a codebase for training and running a GPT-style chat model. That reading is an interpretation of names and call structure.

How it starts

The analyzer found seven files with a __main__ guard:

  • nanochat/dataset.py, nanochat/engine.py, tasks/common.py
  • scripts/base_eval.py, scripts/chat_eval.py, scripts/infer_bench.py
  • tests/test_attention_fallback.py

nanochat/dataset.py builds an argparse CLI for downloading pretraining dataset shards. It has options for the number of shards and the number of parallel workers. scripts/chat_eval.py requires a model source (sft or rl) and accepts an optional task name. The guard in nanochat/engine.py is documented as a quick inline test that compares the plain model.generate with Engine.generate.

The only bounded static paths in this packet start at nanochat/dataset.py and go into get_base_dir in nanochat/common.py. They end at standard-library calls such as os.environ.get, os.path.expanduser and os.makedirs.

By call degree, scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py look like orchestrators. How they are launched is not established here.

Structure

By the analyzer's directory counts, the 53 files split into:

  • nanochat/ (14), scripts/ (9), dev/ (7)
  • tasks/ (6), tests/ (6), runs/ (4)
  • six root files and one file under .claude/

The analyzer treats nanochat and tests as likely subsystems.

Static call degree suggests a hub-and-scripts shape:

  • nanochat/common.py receives 143 resolved cross-module calls and makes none. It holds shared helpers such as get_base_dir, print0, autodetect_device_type and compute_init.
  • Among the ten modules the analyzer ranks by static call degree, the next most-called after nanochat/common.py are nanochat/engine.py (17 incoming), tasks/common.py (16), nanochat/flash_attention.py (16) and nanochat/checkpoint_manager.py (12).
  • scripts/base_train.py (58 outgoing), scripts/chat_sft.py (42) and scripts/base_eval.py (37) mostly call into the package.

tasks/common.py defines Task, TaskMixture, TaskSequence, HubDataset and render_mc. nanochat/flash_attention.py exposes flash_attn_func and flash_attn_with_kvcache, plus _sdpa_attention and _load_flash_attention_3. That reads like an attention layer with a fallback path.

Overall, the analyzer recorded 2803 relations: 2527 calls and 276 imports.

Dependencies and tests

There is one dependency manifest, pyproject.toml.

  • Runtime records: filelock, kernels, numpy, psutil, pyarrow, rustbpe, tiktoken, torch (pinned ==2.9.1) and wandb.
  • Dev records: ipykernel, matplotlib, pytest and python-dotenv.

The parser also recorded torch twice and setuptools once with scope optional. What these optional records correspond to is not established. Dependency parsing covers requirements files, package.json and pyproject.toml only. Lock files and manifests such as setup.py are not parsed, and version constraints are kept as written.

The analyzer counts six test-like files. tests/test_attention_fallback.py defines TestFA3VsSDPA, TestSDPAOnly and TestOverrideMechanism. By name, these compare attention implementations. No test run is part of this analysis, so whether the tests pass is not established.

Read first

Suggested reading order:

  1. README.md and pyproject.toml, which the analyzer flags as project-level important files.
  2. nanochat/common.py, because it receives more resolved cross-module calls than any other module (143).
  3. The flagged entrypoints: nanochat/dataset.py, nanochat/engine.py, scripts/base_eval.py, scripts/chat_eval.py and scripts/infer_bench.py.
  4. build_model and load_model in nanochat/checkpoint_manager.py, to see how saved models are loaded.
  5. scripts/base_train.py, the module with the most outgoing calls, to see how the pieces are combined for training.

Limits of this analysis

Of the 2803 static relations, 2223 remain external or unresolved.

The analysis is static only: dynamic imports, monkey-patching, reflection and dynamic dispatch are not resolved. This matters in nanochat/flash_attention.py, where _load_flash_attention_3 and _resolve_use_fa3 appear to choose between attention implementations. Which path runs on a given machine is not established.

The bounded execution paths in this packet only cover nanochat/dataset.py reaching get_base_dir. Training, evaluation and chat flows are not traced end to end.

Install, build and run inference is partial and no commands are verified, so none are given here.

Parsing is also bounded: very large files, minified bundles and vendored directories are inventoried but not parsed. Relation extraction is capped per file and per repository.

What this analysis could not establish

The writing model's own notes on the analysis record. Identifiers such as lim_3, exec_* or claim_… name records of that analysis; the claims table cites the same records.

  • How scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py are invoked is not established; they are not listed among detected __main__ entrypoints.
  • The meaning of the two optional torch records and the optional setuptools record in pyproject.toml is not established.
  • Contents and purpose of the dev/, runs/ and .claude/ directories are not described in the packet; Shell bytes exist but their location is unknown.
  • Which attention backend (FA3 or SDPA) is used at runtime is not established.
  • Whether the six test files pass is not established; no tests were run.
Claims and evidence — 40 claims, 40 supported by an independent verifier

Every statement above is a claim that cites grounding IDs from the analysis. IDs are internal to the analysis run; the columns show what each claim rests on and how strong that ground is.

Every substantive statement above is a claim bound to grounding IDs of the analyzed revision. Global grounding IDs are namespaced by the analysis run.

Claim Epistemic status Verifier Grounding (global IDs)
c1 GitHub metadata lists Python as the primary language, with Jupyter Notebook and Shell bytes also present, and the license is MIT. observed supported analysis_50dfee49d6903e32:meta_primary_language, analysis_50dfee49d6903e32:meta_language_bytes, analysis_50dfee49d6903e32:meta_license
c2 No release is published according to platform metadata. observed supported analysis_50dfee49d6903e32:meta_latest_release
c3 The repository description frames the project as a ChatGPT-like system obtainable for about $100. author_claimed supported analysis_50dfee49d6903e32:meta_description
c4 nanochat/engine.py defines an Engine class, a KVCache class and a sample_next_token function. observed supported analysis_50dfee49d6903e32:py_class_2648a0b9102d, analysis_50dfee49d6903e32:py_class_99dd2946c374, analysis_50dfee49d6903e32:py_func_81063d17fa08
c5 nanochat/checkpoint_manager.py defines build_model, load_model and save_checkpoint functions. observed supported analysis_50dfee49d6903e32:py_func_cdb79cea93b5, analysis_50dfee49d6903e32:py_func_085478f8a510, analysis_50dfee49d6903e32:py_func_e512e99d6d24
c6 Symbol names and training-style scripts under scripts/ that make many outgoing cross-module calls suggest a codebase for training and running a GPT-style chat model; this is an interpretation. inferred supported analysis_50dfee49d6903e32:py_class_2648a0b9102d, analysis_50dfee49d6903e32:py_class_99dd2946c374, analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18
c7 Seven files contain a __main__ guard: nanochat/dataset.py, nanochat/engine.py, tasks/common.py, scripts/base_eval.py, scripts/chat_eval.py, scripts/infer_bench.py and tests/test_attention_fallback.py. observed supported analysis_50dfee49d6903e32:py_entry_6f78b0b1883f, analysis_50dfee49d6903e32:py_entry_d1f565fa3410, analysis_50dfee49d6903e32:py_entry_21a180b9528c, analysis_50dfee49d6903e32:py_entry_4e39b6126807, analysis_50dfee49d6903e32:py_entry_269c07384799, analysis_50dfee49d6903e32:py_entry_405999f52cfe, analysis_50dfee49d6903e32:py_entry_9395ac6dfb21
c8 The __main__ block of nanochat/dataset.py builds an argparse CLI to download pretraining dataset shards, with options for number of files and number of parallel workers. observed supported analysis_50dfee49d6903e32:py_entry_6f78b0b1883f
c9 scripts/chat_eval.py's __main__ block requires a --source argument (sft or rl) and accepts an optional task name. observed supported analysis_50dfee49d6903e32:py_entry_269c07384799
c10 The __main__ block in nanochat/engine.py is documented as an inline test comparing model.generate with Engine.generate. observed supported analysis_50dfee49d6903e32:py_entry_d1f565fa3410
c11 The bounded static paths start at nanochat/dataset.py __main__, enter get_base_dir in nanochat/common.py, and end at os.environ.get, os.path.expanduser, os.path.join and os.makedirs. inferred supported analysis_50dfee49d6903e32:exec_de60eb0f815c, analysis_50dfee49d6903e32:exec_bebf565646e1, analysis_50dfee49d6903e32:exec_9b057297bbd5, analysis_50dfee49d6903e32:exec_d13940480c8e, analysis_50dfee49d6903e32:py_func_8dd5e10657c8
c12 scripts/base_train.py, scripts/chat_sft.py and scripts/chat_rl.py have no incoming and 58, 42 and 16 outgoing resolved cross-module calls, and are classified as orchestration candidates. inferred supported analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18, analysis_50dfee49d6903e32:role_17
c13 How the training scripts are launched is not established; they are not among the detected __main__ entrypoints and run inference is partial. unresolved supported analysis_50dfee49d6903e32:lim_2, analysis_50dfee49d6903e32:role_14
c14 The 53 files split into nanochat (14), scripts (9), dev (7), tasks (6), tests (6), runs (4), root (6) and .claude (1). inferred supported analysis_50dfee49d6903e32:summary_repository
c15 The analyzer marks the top-level directories nanochat and tests as likely subsystems. observed supported analysis_50dfee49d6903e32:subsys_1, analysis_50dfee49d6903e32:subsys_2
c16 nanochat/common.py has 143 incoming and 0 outgoing resolved cross-module calls and defines helpers including get_base_dir, print0, autodetect_device_type and compute_init, acting as a shared hub. inferred supported analysis_50dfee49d6903e32:role_2, analysis_50dfee49d6903e32:py_func_8dd5e10657c8, analysis_50dfee49d6903e32:py_func_08a788aac199, analysis_50dfee49d6903e32:py_func_d0824d300379, analysis_50dfee49d6903e32:py_func_1b28a2b96f1d
c17 Among the ten modules the analyzer ranks by static call degree, the next most-called after nanochat/common.py are nanochat/engine.py (17), tasks/common.py (16), nanochat/flash_attention.py (16) and nanochat/checkpoint_manager.py (12). inferred supported analysis_50dfee49d6903e32:role_6, analysis_50dfee49d6903e32:role_23, analysis_50dfee49d6903e32:role_8, analysis_50dfee49d6903e32:role_1
c18 scripts/base_train.py (58 outgoing), scripts/chat_sft.py (42) and scripts/base_eval.py (37) mostly call into other modules, suggesting a hub-and-scripts shape. inferred supported analysis_50dfee49d6903e32:role_14, analysis_50dfee49d6903e32:role_18, analysis_50dfee49d6903e32:role_13
c19 tasks/common.py defines the classes Task, TaskMixture, TaskSequence and HubDataset, and a render_mc function. observed supported analysis_50dfee49d6903e32:py_class_85f419f13bf1, analysis_50dfee49d6903e32:py_class_abf6926242ba, analysis_50dfee49d6903e32:py_class_092d56013510, analysis_50dfee49d6903e32:py_class_648e5421d94e, analysis_50dfee49d6903e32:py_func_d47df70bac2d
c20 nanochat/flash_attention.py exposes flash_attn_func and flash_attn_with_kvcache plus _sdpa_attention and _load_flash_attention_3, which reads like an attention layer with a fallback path. inferred supported analysis_50dfee49d6903e32:py_func_830c5da3511c, analysis_50dfee49d6903e32:py_func_421d40c044d9, analysis_50dfee49d6903e32:py_func_e1cd46b6be90, analysis_50dfee49d6903e32:py_func_ad663fbc2b26
c21 The analyzer recorded 2803 static relations: 2527 calls and 276 imports. observed supported analysis_50dfee49d6903e32:relation_counts
c22 The repository contains one build/dependency manifest, pyproject.toml. observed supported analysis_50dfee49d6903e32:ev_manifest_1
c23 Runtime dependency records are filelock, kernels, numpy, psutil, pyarrow, rustbpe, tiktoken, torch (==2.9.1) and wandb. observed supported analysis_50dfee49d6903e32:dep_1, analysis_50dfee49d6903e32:dep_2, analysis_50dfee49d6903e32:dep_3, analysis_50dfee49d6903e32:dep_4, analysis_50dfee49d6903e32:dep_5, analysis_50dfee49d6903e32:dep_6, analysis_50dfee49d6903e32:dep_7, analysis_50dfee49d6903e32:dep_8, analysis_50dfee49d6903e32:dep_9
c24 Dev dependency records are ipykernel, matplotlib, pytest and python-dotenv. observed supported analysis_50dfee49d6903e32:dep_13, analysis_50dfee49d6903e32:dep_14, analysis_50dfee49d6903e32:dep_15, analysis_50dfee49d6903e32:dep_16
c25 The parser also recorded torch twice and setuptools once with scope optional. observed supported analysis_50dfee49d6903e32:dep_10, analysis_50dfee49d6903e32:dep_11, analysis_50dfee49d6903e32:dep_12
c26 What the optional torch and setuptools records correspond to in pyproject.toml is not established by the packet. unresolved supported analysis_50dfee49d6903e32:dep_11, analysis_50dfee49d6903e32:dep_10, analysis_50dfee49d6903e32:lim_3a
c27 Dependency records come only from requirements files, package.json and pyproject.toml; setup.py, setup.cfg, Pipfile and lock files are not parsed, and constraints are recorded as written. observed supported analysis_50dfee49d6903e32:lim_3a
c28 There are six test-like files; tests/test_attention_fallback.py defines TestFA3VsSDPA, TestSDPAOnly and TestOverrideMechanism, which by name compare attention implementations. inferred supported analysis_50dfee49d6903e32:ev_tests_1, analysis_50dfee49d6903e32:py_class_dbf5ff79cf52, analysis_50dfee49d6903e32:py_class_38e524d2473b, analysis_50dfee49d6903e32:py_class_f742a2ec4b10
c29 No tests were run as part of this analysis, so whether the tests pass is not established. unresolved supported analysis_50dfee49d6903e32:lim_2, analysis_50dfee49d6903e32:lim_1
c30 README.md and pyproject.toml are flagged as project-level important files and are a reasonable first read. observed supported analysis_50dfee49d6903e32:important_1, analysis_50dfee49d6903e32:important_2
c31 nanochat/common.py is worth reading early because it receives the most resolved cross-module calls. inferred supported analysis_50dfee49d6903e32:role_2
c32 nanochat/dataset.py, nanochat/engine.py, scripts/base_eval.py, scripts/chat_eval.py and scripts/infer_bench.py are flagged as important because they are likely entrypoints. observed supported analysis_50dfee49d6903e32:important_4, analysis_50dfee49d6903e32:important_5, analysis_50dfee49d6903e32:important_6, analysis_50dfee49d6903e32:important_7, analysis_50dfee49d6903e32:important_8
c33 build_model and load_model in nanochat/checkpoint_manager.py appear to be where saved models are loaded. inferred supported analysis_50dfee49d6903e32:py_func_cdb79cea93b5, analysis_50dfee49d6903e32:py_func_085478f8a510
c34 scripts/base_train.py has the most outgoing cross-module calls (58), so it likely shows how the package pieces are combined for training. inferred supported analysis_50dfee49d6903e32:role_14
c35 2223 of the 2803 static relations are external or unresolved; 580 are resolved locally. observed supported analysis_50dfee49d6903e32:relation_counts
c36 The analysis is static only; reflection, dynamic imports, monkey-patching, runtime wiring and dynamic dispatch are not resolved. observed supported analysis_50dfee49d6903e32:lim_1
c37 Which attention implementation _load_flash_attention_3 and _resolve_use_fa3 select at runtime on a given machine is not established. unresolved supported analysis_50dfee49d6903e32:py_func_ad663fbc2b26, analysis_50dfee49d6903e32:py_func_b589073abe13, analysis_50dfee49d6903e32:lim_1
c38 The bounded execution paths in the packet only cover nanochat/dataset.py reaching get_base_dir; training, evaluation and chat flows are not traced end to end. inferred supported analysis_50dfee49d6903e32:exec_de60eb0f815c, analysis_50dfee49d6903e32:exec_270399b88681
c39 Install, run and build inference is partial and installation commands are not verified. observed supported analysis_50dfee49d6903e32:lim_2
c40 Analysis is bounded: files over 400 kB, minified bundles and vendored directories are inventoried but not parsed, and relation extraction is capped per file and per repository. observed supported analysis_50dfee49d6903e32:lim_6

Sources, rights and disclosure · attribution-license-templates/v0.1

Rights notice. Original repository hosted on GitHub. Repository source code, documentation, names, media, and related project materials remain subject to the rights of their respective authors, contributors, and other rights holders and to applicable repository license terms.

Platform notice. GitHub is the source hosting platform for the linked repository. GitHub and related marks are trademarks of GitHub, Inc. EVEMISS Technology is not affiliated with or endorsed by GitHub unless explicitly stated otherwise.

How this page is produced. This page was produced using revision-aware repository analysis and AI-assisted editorial tooling. Technical claims are tied to the analyzed repository revision and may be revalidated when the source repository changes.

AI-assisted analysis. Reviewed by EVEMISS Technology through human–AI collaborative review. · Report a rights concern · All repository guides