955 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Andrej Karpathy	22d1607f6d	delete these files, they are now in scripts/ folder and better	2024-05-30 20:11:59 +00:00
Andrej Karpathy	fc40ffc4aa	adjust readme to new scripts/ dir	2024-05-30 20:10:04 +00:00
Andrej Karpathy	a0d5dbbda6	scripts folder	2024-05-30 20:00:07 +00:00
Andrej Karpathy	35aa110244	more refactor	2024-05-30 19:39:29 +00:00
Andrej Karpathy	f8f9e49ba3	Merge branch 'master' into feature/libdir	2024-05-30 19:07:51 +00:00
Andrej Karpathy	5f547601a0	further hide the c pytorch bridge logic	2024-05-30 18:20:29 +00:00
Andrej Karpathy	ca27eb8fa5	bugfix don't train on val batch	2024-05-30 18:18:05 +00:00
Andrej	0e2298247a	Merge pull request #496 from ChrisDryden/patch-7 ... Nit: duplicated line in header of project	2024-05-30 09:13:30 -07:00
Andrej Karpathy	862c957d3a	fix default number of gpus from 4 to 8 for consistency with run124M.sh	2024-05-30 14:50:09 +00:00
Andrej	ed805f60be	Merge pull request #494 from karpathy/feature/pytorch_parity ... pytorch parity v1	2024-05-30 07:47:58 -07:00
Andrej Karpathy	5959851d2d	small comments for pyrun124M.sh	2024-05-30 14:47:43 +00:00
Andrej Karpathy	9b935a8f04	fix bug with advancing position in dataloader	2024-05-30 13:16:07 +00:00
Chris Dryden	3bca31fde1	Nit: duplicated line in header of project	2024-05-29 20:03:13 -07:00
Andrej Karpathy	555d07502a	more feature parity work, i think it's quite close now	2024-05-30 02:29:40 +00:00
Andrej Karpathy	ea88a343b5	also updated pyrun124M.sh	2024-05-30 02:05:41 +00:00
Andrej Karpathy	03c2e85601	sharded dataloader for PyTorch	2024-05-30 02:05:06 +00:00
Andrej Karpathy	3fdf88e4d4	add logfile similar to C	2024-05-29 23:59:35 +00:00
Andrej Karpathy	083c50a52e	small stylistic touchups that match the c code	2024-05-29 23:50:13 +00:00
Andrej Karpathy	1f6fdfbd9a	add pyrun124M.sh	2024-05-29 23:30:27 +00:00
Andrej Karpathy	c413bfe0ca	pytorch parity v1	2024-05-29 23:25:55 +00:00
Andrej Karpathy	4cc8b1570b	hardcode grad_clip to 1.0, delete confusion around legacy code that used -c flag for it, but became inactive in the if else block	2024-05-29 19:01:12 +00:00
Andrej	55f3665b64	Merge pull request #493 from vyom1611/master-2 ... Modal benchmarking script updated to replace deprecated calls	2024-05-29 07:50:10 -07:00
vyom1611	062f096da1	Modal benchmarking script updated to replace deprecated calls	2024-05-29 20:01:41 +05:30
Andrej	2db7716d68	Merge pull request #476 from lancerts/layernorm-bk-fix ... Fix "Mismatch of dweight at layernorm_backward.cu #428"	2024-05-29 07:30:52 -07:00
Andrej	c3a65a556f	Merge pull request #487 from huoyushequ/patch-1 ... `softmax_autoregressive_backward_kernel` does not use share memory in the kernel	2024-05-28 18:52:38 -07:00
QIAOQIAN	27cf5223e0	softmax_autoregressive_backward_kernel does not use share memory in the kernel ... we do not need to launch the kernel with 256 bytes share memory, so remove it	2024-05-29 09:23:46 +08:00
Andrej Karpathy	a664aa3094	include datasets as req	2024-05-28 21:43:11 +00:00
Andrej	afe40be994	Merge pull request #479 from rosslwheeler/feature/ci-manual-trigger ... Adding manual trigger for CI	2024-05-28 12:46:27 -07:00
Andrej Karpathy	88dd670c42	link to recent post from readme	2024-05-28 17:31:31 +00:00
Andrej Karpathy	a6a2f042f8	keep -y 1 flag	2024-05-28 15:24:30 +00:00
Andrej Karpathy	a7f01088dc	repro scripts for 124M and 350M models	2024-05-28 15:08:04 +00:00
Andrej Karpathy	037ec67a5d	small tweak to launch	2024-05-28 15:07:07 +00:00
Andrej Karpathy	0c83a23188	tweak the logfile parser and visualizer	2024-05-28 01:38:15 +00:00
Ross Wheeler	c60d7de466	Adding manual trigger for CI	2024-05-27 16:50:06 -07:00
lancerts	47c670b885	amend	2024-05-27 15:13:11 -07:00
lancerts	c4c985cb87	spell explicitly uint to unsigned int	2024-05-27 14:44:47 -07:00
lancerts	f27ca4df41	fix the issue Mismatch of dweight at layernorm_backward.cu	2024-05-27 14:39:57 -07:00
Andrej Karpathy	888c0d4be3	also update the other files	2024-05-27 21:15:09 +00:00
Andrej Karpathy	eed242702a	experiment with adding the llmc lib directory	2024-05-27 21:10:28 +00:00
Andrej	b67880a879	Merge pull request #471 from karpathy/feature/resume ... resume training v1	2024-05-27 13:38:52 -07:00
Andrej Karpathy	24a8c03cb6	Merge branch 'master' into feature/resume	2024-05-27 20:30:30 +00:00
Andrej Karpathy	69d0583ac8	conditionally include dirent on not windows	2024-05-27 20:28:52 +00:00
Andrej	3b4d8bece6	Merge pull request #474 from rosslwheeler/Windows-dirent-fix ... Adding dirent.h support to unistd.h	2024-05-27 13:27:58 -07:00
Ross Wheeler	bb80f93618	Minor cleanup ... Added static inline to the functions	2024-05-27 13:16:34 -07:00
Ross Wheeler	758acdffbf	Adding dirent.h support to unistd.h	2024-05-27 13:08:07 -07:00
Andrej Karpathy	63f0e25f5a	make compiler happy	2024-05-27 19:32:31 +00:00
Andrej Karpathy	b75738c4eb	resume optimization, seems to be working	2024-05-27 18:02:51 +00:00
Andrej Karpathy	f93a30fbbf	more careful with conditional MPI use	2024-05-27 15:54:55 +00:00
Andrej Karpathy	d295cb8d81	part 1 of v1 of resume training functionality, writes the files but doesn't load them yet, coming up in a bit	2024-05-27 15:49:03 +00:00
Andrej	12999f7082	Merge pull request #470 from ngc92/bugfix ... bugfix for layernorm correctness (dev/cuda did not trip, but end to end test `make test_gpt2cu PRECISION=FP32 && ./test_gpt2cu ` did)	2024-05-27 06:39:56 -07:00