aparler/c-llm
1
0
Fork 0
mirror of https://github.com/karpathy/llm.c.git synced 2026-07-27 20:25:09 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
1403 commits 52 branches 0 tags 21 MiB
Commit graph

1403 commits

This branch
This branch
All branches
Author SHA1 Message Date
Andrej Karpathy
5afc57064c document the imports from zero.cuh to be explicit 2024-07-15 19:09:52 +00:00
Andrej Karpathy
882a214ad4 Merge branch 'move-to-zero' of https://github.com/ngc92/llm.c into ngc92-move-to-zero 2024-07-15 18:59:22 +00:00
Andrej
a6dc64533a
Merge pull request #688 from karpathy/feature/gpt3v1 
feature/gpt3v1
 2024-07-15 11:53:05 -07:00
Andrej Karpathy
220125f39c ok let's allow T < max_seq_len because our tests use it. make warning very prominent 2024-07-15 18:21:34 +00:00
Andrej Karpathy
faf8d159a3 use int instead of bool 2024-07-15 18:01:01 +00:00
Andrej Karpathy
528b1b0e31 fix fat fingers 2024-07-15 17:56:41 +00:00
Andrej Karpathy
039904ee9f revert legacy behavior in .sh scripts for -e option 2024-07-15 17:55:26 +00:00
Andrej Karpathy
64db3ded85 refactor gpt2 gpt3 descriptor to keep legacy behavior of -e, be strict everywhere, remove interpolation for now 2024-07-15 17:51:27 +00:00
Andrej Karpathy
4e629e22a1 Merge branch 'gpt3-series' of https://github.com/ngc92/llm.c into ngc92-gpt3-series 2024-07-15 16:46:30 +00:00
Andrej
f45c2191b6
Merge pull request #684 from rosslwheeler/feature/loss_checker_ci 
Adding CI check for exceeding loss tolerance
 2024-07-15 09:30:59 -07:00
Andrej
e71c1565c9
Merge pull request #686 from indianspeedster/cuda 
Added  cudaCheck wherever missing.
 2024-07-15 09:28:02 -07:00
indianspeedster
6bb562bd88 added cudaCheck where missing for proper error checking. 2024-07-13 13:20:39 -04:00
Andrej
a6f6bc74a8
Merge pull request #682 from AnswerDotAI/master 
Add a README link under related related projects for gpu.cpp under WebGPU C++
 2024-07-13 10:12:59 -07:00
Andrej
0e2ff48440
Merge pull request #685 from planetis-m/patch-1 
Add nim port
 2024-07-13 10:08:26 -07:00
Antonis Geralis
a86a69c135
Add nim port 2024-07-13 18:57:51 +03:00
Ross Wheeler
56621bba0e Adding CI check for exceeding loss tolerance 
LLMC-26
 2024-07-13 01:14:20 -07:00
austinvhuang
ddd8aa0f34 add gpu.cpp link to related projects in readme 2024-07-12 14:42:24 -04:00
Erik Schultheis
37b6e328c1 enable model interpolation 2024-07-12 15:07:05 +02:00
Erik Schultheis
b5d5ed7f68 updated example scripts 2024-07-12 14:46:09 +02:00
Erik Schultheis
d19a6267a8 add gpt3 model series 2024-07-12 14:43:22 +02:00
Erik Schultheis
39a41fa701 move cpu float reduction to zero 2024-07-12 11:12:58 +02:00
Erik Schultheis
4d1ec6832a rename parameters to avoid name clashes with global variable 2024-07-11 19:04:02 +02:00
Erik Schultheis
09b78bc902 move set_zero_configs into zero.cuh 2024-07-11 09:55:31 +02:00
Andrej
bdb0fb5599
Merge pull request #671 from ademeure/less_slow_inference 
Faster inference by changing (B,T) to (1,t)
 2024-07-10 17:47:04 -07:00
Andrej
db2454f88f
Merge pull request #620 from ngc92/small-fixes 
small fixes based on clang-tidy
 2024-07-09 12:53:34 -07:00
Erik Schultheis
2b76155976 small fixes based on clang-tidy 2024-07-09 18:37:09 +02:00
ademeure
2fd6fee971 fix the corner case where T < 256 2024-07-07 12:35:29 +00:00
ademeure
efa6767a8a Make inference sanity checks much faster by changing (B,T) to (1,256) 2024-07-07 12:23:26 +00:00
Andrej
f0abd09ffa
Merge pull request #641 from gordicaleksa/add_close_check_socket 
Add check versions of functions
 2024-07-02 14:26:03 -07:00
Aleksa Gordic
6b62f3b9ae Fix linker issue 2024-07-02 21:54:50 +02:00
Aleksa Gordic
d27331cced Move socket header to utils.h 2024-07-02 21:17:28 +02:00
Aleksa Gordic
5645bd18b1 CI fix - add sockets header 2024-07-02 21:17:28 +02:00
Aleksa Gordic
1ae3243185 Add check versions of socket close 2024-07-02 21:17:28 +02:00
Andrej Karpathy
c6efd90a86 upload good 1558M sh script 2024-07-02 16:49:09 +00:00
Andrej
39270cc69a
Merge pull request #665 from ngc92/zero-grad 
zero-grad is async and part of backward call
 2024-07-01 16:08:07 -07:00
Erik Schultheis
486f98e3cc zero-grad is async and part of backward call 2024-07-02 00:47:41 +02:00
Andrej Karpathy
79223b950b Merge branch 'ademeure-optimize_attproj_fcproj' 2024-07-01 22:07:24 +00:00
Andrej Karpathy
cee396ab5a resolve merge conflict 2024-07-01 22:07:14 +00:00
Andrej Karpathy
48fee049cb add edu fineweb sh download script, change paths to my HF repo, add hellaswag to starter pack 2024-07-01 21:41:15 +00:00
Andrej
a876282eb8
Merge pull request #653 from ademeure/cublaslt_refactor 
Matmul refactor using only cuBLASLt + GELU Fusion
 2024-07-01 11:53:02 -07:00
ademeure
00247dca9b Disable GELU fusion by deffault & fix potential argv[] out of bounds access 2024-07-01 18:40:21 +00:00
Andrej
2e6a397d5f
Merge pull request #663 from karpathy/fix/reshuffle_zero_grad 
Fix/reshuffle zero grad
 2024-07-01 10:39:23 -07:00
Andrej Karpathy
7b92930021 move things around a bit, a few comments, and also update python script to correspond to C more 2024-07-01 17:20:46 +00:00
Andrej Karpathy
8253823432 Merge branch 'fix_periodic_spikes' of https://github.com/gordicaleksa/llm.c into gordicaleksa-fix_periodic_spikes-2 2024-07-01 17:14:22 +00:00
Andrej
942fed59fe
Merge pull request #662 from karpathy/overfit-batch-actually 
fix the overfit single batch behavior to actually overfit batch, not …
 2024-07-01 10:12:20 -07:00
Andrej Karpathy
7fd7717816 fix the overfit single batch behavior to actually overfit batch, not microbatch, we do this more cleanly by simply resetting the dataloader every step 2024-07-01 17:00:40 +00:00
Aleksa Gordic
74e5ba7260 Add micro step guard 2024-07-01 12:11:02 +02:00
Aleksa Gordic
78e7627859 Remove local args for testing 2024-07-01 11:31:55 +02:00
Aleksa Gordic
de8454d2f4 Make the grad cleaning more robust 2024-07-01 11:30:36 +02:00
Aleksa Gordic
d2f06aed01 Fix periodic spikes - clean loss after val 2024-07-01 11:24:16 +02:00