aparler/c-llm
1
0
Fork 0
mirror of https://github.com/karpathy/llm.c.git synced 2026-07-28 20:35:09 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
1022 commits 52 branches 0 tags 21 MiB
Commit graph

1022 commits

This branch
This branch
All branches
Author SHA1 Message Date
Chinthaka
58ac83f5ec
Merge branch 'master' into zero-stage1 2024-05-04 16:24:22 +08:00
chinthysl
59b66f6c07 refactor and fix CI issue 2024-05-04 16:22:44 +08:00
lancer
03b7323d1e minor fix of the include 2024-05-03 17:38:39 -07:00
Andrej
ff2fbdc2a1
Merge pull request #332 from harryjackson/llm.java 
Adding a java port of the project
 2024-05-03 13:58:59 -07:00
Erik Schultheis
67a82a611e first attempt at moving cudnn out of the main file for faster compiles 2024-05-03 23:08:44 +03:00
Andrej
aa5dd2eeab
Update README.md 2024-05-03 12:43:08 -07:00
Andrej
6260a1844b
Merge pull request #345 from ngc92/profile 
updated the profile script to be more robust, and adapted to recent k…
 2024-05-03 12:17:22 -07:00
Erik Schultheis
977d6a6c6f fix × symbol 2024-05-03 21:57:31 +03:00
Erik Schultheis
7fabb12e2e fixed instruction counts 2024-05-03 21:54:06 +03:00
Erik Schultheis
13eeeb20c8 try to detect if we need sudo 2024-05-03 21:31:54 +03:00
Erik Schultheis
325b45647f updated the profile script to be more robust, and adapted to recent kernel changes 
make it compile with cudnn
default batchsize is 24 now
 2024-05-03 21:20:06 +03:00
ademeure
a0b80920f1 Reducing profiler to only 3 steps instead of 10 given slight perf penalty (todo: make it configurable) 2024-05-03 19:03:06 +01:00
ademeure
a19fcecb79 Set cuBLAS(Lt) to use the new main_stream 2024-05-03 18:35:46 +01:00
Andrej Karpathy
6c629ac571 Merge branch 'master' of github.com:karpathy/llm.c 2024-05-03 15:54:36 +00:00
Andrej Karpathy
e780b56974 small fixes for profiling 2024-05-03 15:54:29 +00:00
ademeure
b58181079a super optimised matmul_bias + cuda streams for better parallelism + fix fused_classifier cache hint + remove remaining cooperative groups 2024-05-03 16:38:46 +01:00
Chinthaka
0fd384bb21
Merge branch 'master' into zero-stage1 2024-05-03 11:08:41 +08:00
ademeure
f456a75e3f add CUDA streams (no parallelism yet) + fix profiling init + new "blockReduce" function 2024-05-03 03:14:16 +01:00
lancer
c81adeb6b2 move gelu_backward to backward block 2024-05-02 17:36:12 -07:00
Andrej
c04c133b25
Merge pull request #335 from karpathy/feature/bias_backward2 
v1 of the new matmul backward bias kernel
 2024-05-02 16:52:34 -07:00
Andrej Karpathy
776541dac7 v1 of the new matmul backward bias kernel 2024-05-02 23:32:20 +00:00
Andrej Karpathy
6ebef46f83 ugh didn't notice this tiny rebasing mistake, introduced a bug. good candidate for a CI that we can overfit a single batch in the train_gpt2.cu script and get the exact same numbers as we expect in the test_gpt2.cu file 2024-05-02 23:23:28 +00:00
Andrej Karpathy
2c7960040d Merge branch 'ngc92-bias-backward' 2024-05-02 22:17:31 +00:00
Andrej Karpathy
937fbd8e36 small doc fixes 2024-05-02 22:17:22 +00:00
Erik Schultheis
6e48501d60 bias backward kernel that will use all available threads 2024-05-03 01:05:47 +03:00
Andrej Karpathy
6a3d63fb1d Merge branch 'PeterZhizhin-add_nvtx_ranges_and_timings' 2024-05-02 21:13:26 +00:00
Andrej Karpathy
79505bc6b3 resolve merge and small fixes 2024-05-02 21:13:08 +00:00
Andrej Karpathy
50714d2254 pragma unroll quick fix 2024-05-02 21:06:11 +00:00
Andrej Karpathy
9da2729c5a update encoder_forward with the latest and greatest Packed128 and incorporate into train_gpt2cu 2024-05-02 20:40:05 +00:00
Peter Zhizhin
41a0789a44 Added NVTX ranges for FlashAttention 2024-05-02 20:30:08 +00:00
Andrej Karpathy
8a9510f32b very minor stylistic preference let's keep the code style consistent 2024-05-02 19:48:56 +00:00
Andrej Karpathy
84dbc70160 Merge branch 'JaneIllario-patch-1' 2024-05-02 19:45:18 +00:00
Andrej Karpathy
3d1761b370 Merge branch 'patch-1' of https://github.com/JaneIllario/llm.c into JaneIllario-patch-1 2024-05-02 19:43:14 +00:00
Andrej Karpathy
01df41fbba careful to guard the overloaded warpReduceSum and small fixes 2024-05-02 19:24:04 +00:00
Andrej Karpathy
6879c36cf3 Merge branch 'JaneIllario-gelu_backwards-3' 2024-05-02 19:16:38 +00:00
Andrej Karpathy
99f51bae1c Merge branch 'gelu_backwards' of https://github.com/JaneIllario/llm.c into JaneIllario-gelu_backwards-3 2024-05-02 19:15:06 +00:00
Andrej Karpathy
8ccd05ec29 group warp reduce ops together 2024-05-02 19:09:53 +00:00
Andrej Karpathy
d05c51c7d1 Merge branch 'ChrisDryden-removing_cg_example' 2024-05-02 19:08:26 +00:00
Andrej
2583feb7a2
Merge pull request #327 from yushengsu-thu/master 
[ModuleNotFoundError]: Solution - add required package (requests) into requirements.txt
 2024-05-02 12:03:46 -07:00
Andrej Karpathy
d1771a7b59 quick fix master. there is some weirdness here in the adam update that is todo to understand better, for now i just want the master to be ok 2024-05-02 19:01:26 +00:00
Harry Jackson
c13a730149
Merge branch 'master' into llm.java 2024-05-02 11:44:44 -07:00
Andrej Karpathy
ec73086a83 Merge branch 'removing_cg_example' of https://github.com/ChrisDryden/llm.c into ChrisDryden-removing_cg_example 2024-05-02 18:14:14 +00:00
Andrej Karpathy
2202c9a51b add kernel 4 to docs. have to improve these docs more and document them better 2024-05-02 18:09:27 +00:00
Andrej Karpathy
2feb9ffcfb Fused classifier: use Packed128 and delete cg dep 2024-05-02 18:08:27 +00:00
Peter Zhizhin
bcf7d4fe93 Add NSight Compute ranges, use CUDA events for timings 2024-05-02 17:23:07 +00:00
Yusheng (Ethan) Su
31e59273e1
Merge branch 'karpathy:master' into master 2024-05-02 14:04:47 +04:00
chinthysl
632caf1ce6 Refactored zero_active var 2024-05-02 07:38:49 +00:00
chinthysl
d35d2e606b Generalized copy_and_cast_kernel and changes to cater model->master_weights 2024-05-02 07:31:34 +00:00
Chinthaka
07b8102e79
Merge branch 'master' into zero-stage1 2024-05-02 11:48:56 +08:00
“Christopher
398cdaf4cd Removed cooperative groups and added packed128 to fused classifier and prepare softmax blockwide 2024-05-02 02:33:51 +00:00