aparler/c-llm
1
0
Fork 0
mirror of https://github.com/karpathy/llm.c.git synced 2026-07-28 20:35:09 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
1022 commits 52 branches 0 tags 21 MiB
Commit graph

26 commits

Author SHA1 Message Date
Andrej Karpathy
804d0693c8 refactor part 2, moving stuff into common files so that we can later nicely separate out all the kernels as well 2024-06-03 18:10:33 +00:00
ademeure
b5e75dde8e Fully deterministic encoder backward kernels for train_gpt2.cu 2024-05-21 15:57:07 +01:00
Andrej
00c291c679
Merge pull request #434 from ngc92/profile-fix 
initialize multi_gpu_config so profile doesn't crash
 2024-05-19 20:57:11 +01:00
Erik Schultheis
299ce659ec initialize multi_gpu_config so profile doesn't crash 2024-05-19 22:25:00 +03:00
Erik Schultheis
c3a3b9daa5 fixed profile target 2024-05-18 22:47:40 +03:00
Erik Schultheis
fbd8f03eea fixup profiling 2024-05-16 14:25:06 +03:00
ademeure
876ab93c0a Add FP16 path for atomicStochasticAdd (+remove __bfloat1622float2 to work on older CUDAs) + fixes 2024-05-05 15:05:59 +01:00
ademeure
c261eecda8 Merge remote-tracking branch 'karpathy/master' into cleanup_may4 2024-05-05 01:49:25 +01:00
Andrej Karpathy
f2224f2eb2 Merge branch 'separate-compilation' of https://github.com/ngc92/llm.c into ngc92-separate-compilation 2024-05-05 00:32:31 +00:00
Andrej Karpathy
2642ffd19c Merge branch 'less_idle_more_brrr' of https://github.com/ademeure/llm.c into ademeure-less_idle_more_brrr-3 2024-05-05 00:05:38 +00:00
ademeure
34304361ba fix merge conflict 2024-05-05 00:16:49 +01:00
ademeure
bfb9c51446 refactoring & remove unused functions to reduce LOC (+wip profile.py improvements) 2024-05-04 23:40:15 +01:00
Erik Schultheis
aa5bb258b6 fixed up test and profile targets 2024-05-04 11:59:59 +03:00
ademeure
a19fcecb79 Set cuBLAS(Lt) to use the new main_stream 2024-05-03 18:35:46 +01:00
Andrej Karpathy
e780b56974 small fixes for profiling 2024-05-03 15:54:29 +00:00
ademeure
b58181079a super optimised matmul_bias + cuda streams for better parallelism + fix fused_classifier cache hint + remove remaining cooperative groups 2024-05-03 16:38:46 +01:00
ademeure
f456a75e3f add CUDA streams (no parallelism yet) + fix profiling init + new "blockReduce" function 2024-05-03 03:14:16 +01:00
Andrej Karpathy
8aa9a9ed53 and also profile lol 2024-05-01 17:56:01 +00:00
ademeure
1147983b8c Add missing cudnnDestroy() calls 2024-05-01 17:19:51 +01:00
ademeure
d9e7a0a378 Fixed profile_gpt2.cu/test_gpt2.cu for cuDNN 2024-05-01 17:03:28 +01:00
Andrej Karpathy
906d22f5b6 fix profile gpt2cu but note it currently needs the NO_MULTI_GPU=1 switch i think when calling it with make, todo fix later how we add the mpi flags 2024-04-29 23:27:55 +00:00
Rickard Hallerbäck
b522333851 Updating the CI to build different precisions 2024-04-28 22:17:47 +02:00
Andrej Karpathy
9d70d9a1b2 profile and test only use bf16. but the train script can be run with fp32 or bf16 or fp16. fp16 will error, though 2024-04-28 16:17:22 +00:00
Andrej Karpathy
d34556cf5e more profile docs 2024-04-20 01:29:14 +00:00
Andrej Karpathy
67fd37ace4 add crappy instructions for profiling basics 2024-04-20 01:06:53 +00:00
Erik Schultheis
9c1c1f0cec helper target that gets compiled with lineinfo and runs a single layer forward-backward pass 2024-04-20 03:19:23 +03:00