338 commits

Author	SHA1	Message	Date
Andrej Karpathy	2543b62b88	Merge branch 'streams-io' of https://github.com/ngc92/llm.c into ngc92-streams-io	2024-06-23 00:48:05 +00:00
Andrej Karpathy	2677d82dcb	add shell script that downloads the .bin files of FineWeb100B directly	2024-06-20 22:39:09 +00:00
Andrej Karpathy	0bc8753bc0	update vislog to be a bit more fancy, has GPT-2/3 perf for hellaswag	2024-06-19 01:51:09 +00:00
Andrej Karpathy	b29478e20b	add starter pack .sh script for faster quickstart	2024-06-19 01:44:36 +00:00
Andrej	4577d58cba	Merge pull request #611 from rosslwheeler/feature/dataloader_test ... Bug fixes and warning fixes for test_dataloader.c	2024-06-18 18:08:05 -07:00
Andrej	88f0543c38	Merge pull request #594 from karpathy/feature/hf_eval ... add scripts to export to HF and run Eleuther evals	2024-06-18 18:04:14 -07:00
Andrej Karpathy	1a66151a80	small fixes and readme	2024-06-19 01:03:36 +00:00
Ross Wheeler	24923829c4	Adding suggested changes ... Ternary change to test_dataloader and header cleanup	2024-06-18 10:38:33 -07:00
Erik Schultheis	3ebd5abe82	fixup tests	2024-06-18 19:58:03 +03:00
Erik Schultheis	188e7274e6	add unit test to CI	2024-06-18 13:23:56 +03:00
Ross Wheeler	30095bcb0b	Fixes on accessing past memory buffer	2024-06-17 21:37:35 -07:00
rosslwheeler	4e0221de0d	Merge branch 'karpathy:master' into feature/dataloader_test	2024-06-17 18:18:23 -07:00
Ross Wheeler	37023dd9a1	Cleanup for data loader	2024-06-17 18:15:29 -07:00
Andrej	f104009bc7	Merge pull request #600 from gordicaleksa/add_faster_layernorm_kernel ... Use faster kernel for LayerNorm forward	2024-06-17 14:19:22 -07:00
Andrej Karpathy	14d9cd671e	resolve merge conflict in staate file between use_master_weights and DataLoader should_shuffle	2024-06-17 19:16:42 +00:00
Andrej Karpathy	de53e71729	small touchups	2024-06-17 18:55:16 +00:00
Aleksa Gordic	9b5ec8f719	Set v memory to random floats >=0	2024-06-17 17:44:00 +02:00
Aleksa Gordic	a0a8795be1	Fix guarding logic, refactor	2024-06-17 14:12:05 +02:00
Aleksa Gordic	c6040eef5f	Even better kernel - modification of what we use for fused residual fwd	2024-06-17 09:53:58 +02:00
Andrej	13c893ffec	Merge pull request #561 from lancerts/fix-warning ... Fix the compiler warnings and errors	2024-06-16 09:25:51 -07:00
Andrej	1973631c6b	Merge pull request #583 from vyom1611/FLT_MAX-fix ... Replaced hard-coded max float with FLT_MAX	2024-06-16 09:25:03 -07:00
Aleksa Gordic	d5601d16da	Fix stuff in layernorm forward	2024-06-16 09:34:01 +02:00
Aleksa Gordic	d003274b7c	Remove warp size - moved to common.h	2024-06-15 20:23:59 +02:00
Aleksa Gordic	09910acc0b	Remove blockreduce from classifier fused, move to common.h	2024-06-15 19:55:11 +02:00
Aleksa Gordic	0f112f1b62	Fix CI complaining about default args	2024-06-15 19:39:05 +02:00
Andrej Karpathy	1e7c31eef0	add test for shuffled, permuted, multi-process dataloader	2024-06-15 16:36:43 +00:00
Andrej Karpathy	f089f90bcf	add multiprocess test simple	2024-06-15 16:21:05 +00:00
Andrej Karpathy	d4255f1d19	first draft of a dataloader test	2024-06-15 15:56:56 +00:00
Aleksa Gordic	f3195d19af	Expose deviceProp as a global var	2024-06-15 17:40:14 +02:00
Aleksa Gordic	0f6853d399	Add global norm prod kernels to dev	2024-06-15 17:29:14 +02:00
Andrej Karpathy	d7ea4da298	add scripts to export to HF and run Eleuther evals	2024-06-14 19:49:05 +00:00
Vyom Sharma	ee4b056ddf	Merge branch 'karpathy:master' into FLT_MAX-fix	2024-06-14 16:32:37 +05:30
vyom1611	e2a18d9385	Fixed missing semi-colon	2024-06-13 00:59:15 +05:30
vyom1611	dc8d8de06a	Replaced hard-coded max float with FLT_MAX	2024-06-13 00:56:53 +05:30
Vyom Sharma	afa7978227	Merge branch 'karpathy:master' into master	2024-06-13 00:19:16 +05:30
Erik Schultheis	d6793645f5	replace cublaslt with custom matmul	2024-06-12 09:42:19 +03:00
Erik Schultheis	7f3f3acf16	shared memory matmul	2024-06-11 21:03:43 +03:00
Erik Schultheis	071c977c79	vectorized memory access	2024-06-11 00:46:51 +03:00
Erik Schultheis	bb89efdcb1	register reuse	2024-06-11 00:41:08 +03:00
Vyom Sharma	8a3a29ecb7	Merge branch 'karpathy:master' into master	2024-06-10 13:54:29 +05:30
Andrej	637c1b69c4	Merge pull request #552 from karpathy/feature/streams ... Feature/streams	2024-06-08 09:25:56 -07:00
Andrej Karpathy	ee6b3c9325	take out Async copies and memsets	2024-06-08 16:17:02 +00:00
Aleksa Gordic	d3790c64a1	Change default path	2024-06-07 17:38:01 +02:00
Aleksa Gordic	4090ca8fa1	Add learning rate and grad norm logging	2024-06-07 17:34:42 +02:00
lancerts	089fdd7f23	Merge branch 'master' into fix-warning	2024-06-07 07:17:52 -07:00
vyom1611	ecb13ea74d	missing parameter in comments added	2024-06-07 16:14:59 +05:30
vyom1611	009be4dcab	Merge remote-tracking branch 'origin/master'	2024-06-07 14:21:35 +05:30
vyom1611	58c59064ec	Most updated working modal script	2024-06-07 14:21:13 +05:30
Vyom Sharma	c24ef4a4b4	Merge branch 'karpathy:master' into master	2024-06-07 14:20:55 +05:30
lancerts	4b41087c64	fix the compiler warnings and errors	2024-06-06 11:46:40 -07:00