aparler/c-llm
1
0
Fork 0
mirror of https://github.com/karpathy/llm.c.git synced 2026-07-25 20:05:08 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
1536 commits 52 branches 0 tags 21 MiB
Commit graph

369 commits

Author SHA1 Message Date
Andrej Karpathy
755458d0e1 fix tokenizer omg 2024-08-13 17:24:15 +00:00
Andrej Karpathy
16635d41a2 attempt to fix PR 2024-08-13 02:13:14 +00:00
Aleksa Gordic
68624cc6ac Refactor tokenize, delegate instead 2024-08-12 20:26:21 +02:00
Aleksa Gordic
e8739c89a0 Refactor magic/version header dict 2024-08-12 20:17:53 +02:00
Aleksa Gordic
eb1b359dae Replace fire with argparse 2024-08-10 22:45:46 +02:00
Aleksa Gordic
50765638c0 Add fineweb llama 3 support 2024-08-10 22:35:46 +02:00
Aleksa Gordic
f8c1ff590f Use fire module, update readme 2024-08-10 22:29:19 +02:00
Aleksa Gordic
15b76bf0fb Add tiny shakespeare 2024-08-10 22:23:33 +02:00
Aleksa Gordic
71419cbd61 Update header description 2024-08-10 21:47:09 +02:00
Aleksa Gordic
d3151c4ace Add llama 3 tokenization and data loader 2024-08-10 21:25:57 +02:00
Andrej Karpathy
1566d6b9f6 revert makefile change 2024-07-30 20:07:54 +00:00
mspronesti
574ef9b87d chore(permute): use common.h utils for validation and benchmarking 2024-07-29 00:19:31 +01:00
mspronesti
a49da7d917 make: generalize TARGETS definition 2024-07-29 00:19:03 +01:00
Andrej
b4623bc5e7
Merge pull request #712 from indianspeedster/dev-cuda 
Added permute kernel in dev/cuda
 2024-07-28 06:08:20 -10:00
mspronesti
c6e74ea322 fix(dev/cuda): memory leaks 2024-07-26 00:33:51 +01:00
indianspeedster
a80bce65e5 Added compilation steps in makefile for permute.cu 2024-07-25 18:02:06 -04:00
indianspeedster
8952c524af added kernel to permute in dev/cuda 2024-07-25 17:35:29 -04:00
Ross Wheeler
9dc920a66f Move WIN32 code into unistd.h 2024-07-20 12:05:20 -07:00
Ross Wheeler
56621bba0e Adding CI check for exceeding loss tolerance 
LLMC-26
 2024-07-13 01:14:20 -07:00
Andrej Karpathy
48fee049cb add edu fineweb sh download script, change paths to my HF repo, add hellaswag to starter pack 2024-07-01 21:41:15 +00:00
Andrej Karpathy
ea36d04e39 small cosmetic fixes to fineweb data file 2024-06-30 15:52:51 +00:00
Andrej
2daaf6ab83
Merge pull request #629 from rosslwheeler/feature/tests 
CI Dataloader test and ptx/sass file generator
 2024-06-29 15:17:31 -07:00
Andrej Karpathy
ff7a90bd8d Merge branch 'add_edu_fineweb_support' of https://github.com/eliebak/llm.c into eliebak-add_edu_fineweb_support 2024-06-28 19:35:44 +00:00
Ross Wheeler
d6d08be3c0 Addressed review comments 
Explicit precision set for tests
 2024-06-27 00:58:19 -07:00
rosslwheeler
267efc9a0e
Merge branch 'karpathy:master' into feature/tests 2024-06-26 22:38:21 -07:00
Andrej Karpathy
e12d090483 resolve merge conflicts 2024-06-26 04:05:46 +00:00
Ross Wheeler
2e6fd9b150 Add unistd.h for Windows cuDNN build 
Small fix - move WIN32 change into dev.unistd.h

Add USE_CUDNN=1 build for Windows for CI
 2024-06-25 10:36:59 -07:00
rosslwheeler
2e32e25e37
Merge branch 'karpathy:master' into feature/tests 2024-06-24 23:05:55 -07:00
Andrej Karpathy
569f9f16f9 add outlier detector, test for it, and start tracking z score of the loss, for now only print it 2024-06-25 01:37:31 +00:00
Andrej Karpathy
2543b62b88 Merge branch 'streams-io' of https://github.com/ngc92/llm.c into ngc92-streams-io 2024-06-23 00:48:05 +00:00
Ross Wheeler
5b95bff118 Dataloader test and ptx/sass file generator 
Setup new test infrastructure
 2024-06-21 17:23:03 -07:00
Andrej Karpathy
2677d82dcb add shell script that downloads the .bin files of FineWeb100B directly 2024-06-20 22:39:09 +00:00
Andrej Karpathy
0bc8753bc0 update vislog to be a bit more fancy, has GPT-2/3 perf for hellaswag 2024-06-19 01:51:09 +00:00
Andrej Karpathy
b29478e20b add starter pack .sh script for faster quickstart 2024-06-19 01:44:36 +00:00
Andrej
4577d58cba
Merge pull request #611 from rosslwheeler/feature/dataloader_test 
Bug fixes and warning fixes for test_dataloader.c
 2024-06-18 18:08:05 -07:00
Andrej
88f0543c38
Merge pull request #594 from karpathy/feature/hf_eval 
add scripts to export to HF and run Eleuther evals
 2024-06-18 18:04:14 -07:00
Andrej Karpathy
1a66151a80 small fixes and readme 2024-06-19 01:03:36 +00:00
Ross Wheeler
24923829c4 Adding suggested changes 
Ternary change to test_dataloader and header cleanup
 2024-06-18 10:38:33 -07:00
Erik Schultheis
3ebd5abe82 fixup tests 2024-06-18 19:58:03 +03:00
Erik Schultheis
188e7274e6 add unit test to CI 2024-06-18 13:23:56 +03:00
Ross Wheeler
30095bcb0b Fixes on accessing past memory buffer 2024-06-17 21:37:35 -07:00
rosslwheeler
4e0221de0d
Merge branch 'karpathy:master' into feature/dataloader_test 2024-06-17 18:18:23 -07:00
Ross Wheeler
37023dd9a1 Cleanup for data loader 2024-06-17 18:15:29 -07:00
Andrej
f104009bc7
Merge pull request #600 from gordicaleksa/add_faster_layernorm_kernel 
Use faster kernel for LayerNorm forward
 2024-06-17 14:19:22 -07:00
Andrej Karpathy
14d9cd671e resolve merge conflict in staate file between use_master_weights and DataLoader should_shuffle 2024-06-17 19:16:42 +00:00
Andrej Karpathy
de53e71729 small touchups 2024-06-17 18:55:16 +00:00
Aleksa Gordic
9b5ec8f719 Set v memory to random floats >=0 2024-06-17 17:44:00 +02:00
Aleksa Gordic
a0a8795be1 Fix guarding logic, refactor 2024-06-17 14:12:05 +02:00
Aleksa Gordic
c6040eef5f Even better kernel - modification of what we use for fused residual fwd 2024-06-17 09:53:58 +02:00
Andrej
13c893ffec
Merge pull request #561 from lancerts/fix-warning 
Fix the compiler warnings and errors
 2024-06-16 09:25:51 -07:00