288 commits

Author	SHA1	Message	Date
Andrej Karpathy	1566d6b9f6	revert makefile change	2024-07-30 20:07:54 +00:00
mspronesti	574ef9b87d	chore(permute): use common.h utils for validation and benchmarking	2024-07-29 00:19:31 +01:00
mspronesti	a49da7d917	make: generalize TARGETS definition	2024-07-29 00:19:03 +01:00
Andrej	b4623bc5e7	Merge pull request #712 from indianspeedster/dev-cuda ... Added permute kernel in dev/cuda	2024-07-28 06:08:20 -10:00
mspronesti	c6e74ea322	fix(dev/cuda): memory leaks	2024-07-26 00:33:51 +01:00
indianspeedster	a80bce65e5	Added compilation steps in makefile for permute.cu	2024-07-25 18:02:06 -04:00
indianspeedster	8952c524af	added kernel to permute in dev/cuda	2024-07-25 17:35:29 -04:00
Ross Wheeler	5b95bff118	Dataloader test and ptx/sass file generator ... Setup new test infrastructure	2024-06-21 17:23:03 -07:00
Andrej	f104009bc7	Merge pull request #600 from gordicaleksa/add_faster_layernorm_kernel ... Use faster kernel for LayerNorm forward	2024-06-17 14:19:22 -07:00
Aleksa Gordic	9b5ec8f719	Set v memory to random floats >=0	2024-06-17 17:44:00 +02:00
Aleksa Gordic	a0a8795be1	Fix guarding logic, refactor	2024-06-17 14:12:05 +02:00
Aleksa Gordic	c6040eef5f	Even better kernel - modification of what we use for fused residual fwd	2024-06-17 09:53:58 +02:00
Andrej	13c893ffec	Merge pull request #561 from lancerts/fix-warning ... Fix the compiler warnings and errors	2024-06-16 09:25:51 -07:00
Andrej	1973631c6b	Merge pull request #583 from vyom1611/FLT_MAX-fix ... Replaced hard-coded max float with FLT_MAX	2024-06-16 09:25:03 -07:00
Aleksa Gordic	d5601d16da	Fix stuff in layernorm forward	2024-06-16 09:34:01 +02:00
Aleksa Gordic	d003274b7c	Remove warp size - moved to common.h	2024-06-15 20:23:59 +02:00
Aleksa Gordic	09910acc0b	Remove blockreduce from classifier fused, move to common.h	2024-06-15 19:55:11 +02:00
Aleksa Gordic	0f112f1b62	Fix CI complaining about default args	2024-06-15 19:39:05 +02:00
Aleksa Gordic	f3195d19af	Expose deviceProp as a global var	2024-06-15 17:40:14 +02:00
Aleksa Gordic	0f6853d399	Add global norm prod kernels to dev	2024-06-15 17:29:14 +02:00
Vyom Sharma	ee4b056ddf	Merge branch 'karpathy:master' into FLT_MAX-fix	2024-06-14 16:32:37 +05:30
vyom1611	e2a18d9385	Fixed missing semi-colon	2024-06-13 00:59:15 +05:30
vyom1611	dc8d8de06a	Replaced hard-coded max float with FLT_MAX	2024-06-13 00:56:53 +05:30
Vyom Sharma	afa7978227	Merge branch 'karpathy:master' into master	2024-06-13 00:19:16 +05:30
Erik Schultheis	d6793645f5	replace cublaslt with custom matmul	2024-06-12 09:42:19 +03:00
Erik Schultheis	7f3f3acf16	shared memory matmul	2024-06-11 21:03:43 +03:00
Erik Schultheis	071c977c79	vectorized memory access	2024-06-11 00:46:51 +03:00
Erik Schultheis	bb89efdcb1	register reuse	2024-06-11 00:41:08 +03:00
Vyom Sharma	8a3a29ecb7	Merge branch 'karpathy:master' into master	2024-06-10 13:54:29 +05:30
Andrej	637c1b69c4	Merge pull request #552 from karpathy/feature/streams ... Feature/streams	2024-06-08 09:25:56 -07:00
Andrej Karpathy	ee6b3c9325	take out Async copies and memsets	2024-06-08 16:17:02 +00:00
lancerts	089fdd7f23	Merge branch 'master' into fix-warning	2024-06-07 07:17:52 -07:00
vyom1611	ecb13ea74d	missing parameter in comments added	2024-06-07 16:14:59 +05:30
vyom1611	009be4dcab	Merge remote-tracking branch 'origin/master'	2024-06-07 14:21:35 +05:30
vyom1611	58c59064ec	Most updated working modal script	2024-06-07 14:21:13 +05:30
Vyom Sharma	c24ef4a4b4	Merge branch 'karpathy:master' into master	2024-06-07 14:20:55 +05:30
lancerts	4b41087c64	fix the compiler warnings and errors	2024-06-06 11:46:40 -07:00
Aleksa Gordic	7b99ac96a5	Fix mem leaks, reduce memory, refactor	2024-06-06 19:58:50 +02:00
Andrej	d931fc0507	-std=c++17 default, merge PR #559 from ngc92/ubuntu20.04 ... set compile flag and add ci check	2024-06-06 09:23:07 -07:00
Erik Schultheis	875196b672	add c++17 flag also to dev/cuda	2024-06-06 19:02:45 +03:00
Aleksa Gordic	c116fbfa29	Fix q,k bug, move all kernels to q@k^t form	2024-06-05 22:22:55 +02:00
Aleksa Gordic	875d362381	Refactor trimat	2024-06-05 17:33:26 +02:00
vyom1611	4564a24c69	Modal script updated for cudnn and gcc version match	2024-06-04 23:39:36 +05:30
vyom1611	fbc1e4983c	Benchmark modal script revamped - profiling and external libraries	2024-06-03 17:00:10 +05:30
vyom1611	0867653aaf	Merge remote-tracking branch 'origin/master'	2024-06-03 16:48:58 +05:30
Andrej	80e0c3b8d9	Merge pull request #512 from gordicaleksa/refactor_encoder_bwd_kernel ... Remove redundant CPU computation in encoder bwd	2024-06-02 16:43:10 -07:00
Aleksa Gordic	290c00a362	Remove redundant CPU computation	2024-06-01 18:27:24 +02:00
Andrej	a26041af12	Merge pull request #495 from ChrisDryden/shared_memory ... Removed unnecesary shared memory due to blockreduce using static defined shared memory	2024-06-01 08:12:21 -07:00
Aleksa Gordic	8eafd40615	Update compile cmd in the dev/cuda README	2024-06-01 16:46:32 +02:00
vyom1611	e0abfae2bc	Benchmark modal script revamped - profiling and external libraries	2024-06-01 19:06:42 +05:30