Default branch

master f1e2ace651 · Merge pull request #801 from ngc92/ngc92/fix-test · Updated 2025-05-10 19:24:10 -04:00

ZIP  TAR.GZ

Branches

move/fused_classifier 90a7bfecf1 · move fused classifier · Updated 2024-06-03 18:35:20 -04:00    aparler	509 0	Included	ZIP  TAR.GZ
move/attention 2186cd6fe7 · move attention · Updated 2024-06-03 18:19:47 -04:00    aparler	511 0	Included	ZIP  TAR.GZ
move/gelu d8d55e0a08 · move gelu to llmc · Updated 2024-06-03 17:57:02 -04:00    aparler	513 0	Included	ZIP  TAR.GZ
move/encoder_kernel 49b9776e50 · also move the launchers · Updated 2024-06-03 17:37:18 -04:00    aparler	514 0	Included	ZIP  TAR.GZ
feature/refactor_part2 001ad78fe1 · resolve deviceProp and cudaCheck dupes · Updated 2024-06-03 15:54:32 -04:00    aparler	529 0	Included	ZIP  TAR.GZ
try/move_cudnn_to_llmc 6434009298 · attempt to remove PATH_SEP from Makefile · Updated 2024-06-03 11:30:45 -04:00    aparler	534 0	Included	ZIP  TAR.GZ
feature/libdir 22d1607f6d · delete these files, they are now in scripts/ folder and better · Updated 2024-05-30 16:11:59 -04:00    aparler	581 0	Included	ZIP  TAR.GZ
feature/pytorch_parity 5959851d2d · small comments for pyrun124M.sh · Updated 2024-05-30 10:47:43 -04:00    aparler	596 0	Included	ZIP  TAR.GZ
feature/resume 24a8c03cb6 · Merge branch 'master' into feature/resume · Updated 2024-05-27 16:30:30 -04:00    aparler	621 0	Included	ZIP  TAR.GZ
attempt_fix_zero1 05fb642ae1 · attempt fix zero 1 · Updated 2024-05-25 19:49:16 -04:00    aparler	648 0	Included	ZIP  TAR.GZ
feature/write_checkpoints a13b2bd10b · fix bug check null · Updated 2024-05-25 09:50:56 -04:00    aparler	656 0	Included	ZIP  TAR.GZ
feature/weight_decay_2d_only 9f08882051 · add weight decay, but only for 2D tensors, as done in GPT series and in general too. this forces us to break up our adamw kernel again into one call per tensor, so there is a small throughput hit, of about 0.5% for me. but we have to break up this kernel in near future anyway · Updated 2024-05-24 20:14:10 -04:00    aparler	674 0	Included	ZIP  TAR.GZ
feature/random_init 1f91bfc442 · fix small bug on eval logging · Updated 2024-05-23 14:19:25 -04:00    aparler	707 0	Included	ZIP  TAR.GZ
feature/hellaswagc da59861d38 · make hellaswag optional eval yay · Updated 2024-05-22 18:42:37 -04:00    aparler	716 0	Included	ZIP  TAR.GZ
feature/sharded_data 099d30f814 · add a super small crappy glob for windows that only matches a single unique file. this will make CI happy but we can't train on sharded data on windows until this is improved · Updated 2024-05-22 09:08:18 -04:00    aparler	720 0	Included	ZIP  TAR.GZ
feature/organize_data 54ccbd300c · docs on master-breaking changes around dataset file representation · Updated 2024-05-21 14:31:19 -04:00    aparler	727 0	Included	ZIP  TAR.GZ
feature/hellaswag 8d55c4a6ac · fix a bug, we have to be careful to make sure we evaluate loss at the token just before the first completion token, because that is the prediction for the first token, and its accuracy · Updated 2024-05-19 11:45:28 -04:00    aparler	762 0	Included	ZIP  TAR.GZ
feature/recompute d7581fc542 · make recompute be an int instead of bool, so we can strengthen it over time just like ZeRO stages, as we recompute more and more of the model in the future possibly. and make it default on because it is awesome · Updated 2024-05-16 15:07:39 -04:00    aparler	793 0	Included	ZIP  TAR.GZ
feature/grad_accum3 8eb3a432d9 · revert all streams and synchronization events, we'll bring them back but only one at a time and very very carefully, because they cause really subtle bugs and issues · Updated 2024-05-15 17:09:46 -04:00    aparler	800 0	Included	ZIP  TAR.GZ
feature/wikitext103 3048a3a780 · oops have to use labels= · Updated 2024-05-06 14:04:35 -04:00    aparler	891 2		ZIP  TAR.GZ