| .. |
|
adamw.cuh
|
small fixes
|
2024-07-28 17:10:20 +00:00 |
|
attention.cuh
|
Refactor matmul to only use cuBLASLt and optionally use fused GELU
|
2024-06-30 20:45:37 +01:00 |
|
cublas_common.h
|
Refactor matmul to only use cuBLASLt and optionally use fused GELU
|
2024-06-30 20:45:37 +01:00 |
|
cuda_common.h
|
fallback to memory allocation of m,v,master_weights on host automatically in case of OOM. will run slower but won't OOM
|
2024-08-16 17:23:42 +00:00 |
|
cuda_utils.cuh
|
fallback to memory allocation of m,v,master_weights on host automatically in case of OOM. will run slower but won't OOM
|
2024-08-16 17:23:42 +00:00 |
|
cudnn_att.cpp
|
Including unistd.h in cudnn_att.cpp for latest MSVC compiler
|
2024-07-20 11:57:10 -07:00 |
|
cudnn_att.h
|
move all kernels into a main_stream in preparation for future parallelism
|
2024-06-04 22:35:12 +03:00 |
|
dataloader.h
|
delete segmentation fault when B < 4 atm
|
2024-07-16 01:20:16 +00:00 |
|
encoder.cuh
|
Add a comment for why pass a unique seed per param
|
2024-06-18 17:01:03 +02:00 |
|
fused_classifier.cuh
|
switch losses to fp32
|
2024-06-26 20:48:08 +03:00 |
|
gelu.cuh
|
Fix guarding logic, refactor
|
2024-06-17 14:12:05 +02:00 |
|
global_norm.cuh
|
update norm computation: multi-process reduce on GPU
|
2024-06-25 21:28:52 +03:00 |
|
layernorm.cuh
|
added cudaCheck where missing for proper error checking.
|
2024-07-13 13:20:39 -04:00 |
|
logger.h
|
Add learning rate and grad norm logging
|
2024-06-07 17:34:42 +02:00 |
|
matmul.cuh
|
fix default for test_gpt2cu
|
2024-06-30 20:45:37 +01:00 |
|
mfu.h
|
disable if header is not found
|
2024-07-29 10:07:37 +02:00 |
|
outlier_detector.h
|
oops i was only using window size 16 for testing let's make it 128 or something
|
2024-06-25 01:56:04 +00:00 |
|
rand.h
|
Dataloader test and ptx/sass file generator
|
2024-06-21 17:23:03 -07:00 |
|
sampler.h
|
refactor part 2, moving stuff into common files so that we can later nicely separate out all the kernels as well
|
2024-06-03 18:10:33 +00:00 |
|
schedulers.h
|
small touchups and add the wsd schedule
|
2024-06-21 00:49:01 +00:00 |
|
tokenizer.h
|
made implicit includes of standard headers explicit
|
2024-06-04 13:22:16 +03:00 |
|
utils.h
|
Move WIN32 code into unistd.h
|
2024-07-20 12:05:20 -07:00 |
|
zero.cuh
|
move cpu float reduction to zero
|
2024-07-12 11:12:58 +02:00 |
