17 commits

Author	SHA1	Message	Date
lancer	5c90845f7a	update the -lcublas -lcublasLt flag in the comment	2024-05-09 22:04:02 -07:00
Erik Schultheis	06980b6b2f	mixed precision utilities for dev/cuda	2024-05-01 22:35:55 +03:00
Erik Schultheis	44b5bb028c	one more version of attention backward for another few percent perf gains	2024-04-19 22:46:40 +03:00
Erik Schultheis	a360f104ae	some low-level optimizations	2024-04-19 22:38:13 +03:00
Erik Schultheis	7005e09450	added missing checks	2024-04-19 22:37:35 +03:00
Erik Schultheis	b2e6b6867f	separating the loops leads to yet more improvements	2024-04-19 20:47:19 +03:00
Erik Schultheis	50e105a250	further cleanup and ability to handle arbitrary sequence lengths	2024-04-19 19:12:37 +03:00
Erik Schultheis	4294820a8a	don't do stupid redundant work; the inner loop is just a block-level sum!!!!	2024-04-19 19:12:32 +03:00
Erik Schultheis	02aa8e47dc	further simplification of the loop	2024-04-19 19:12:32 +03:00
Erik Schultheis	16123c3af8	towards an even better backward attention kernel	2024-04-19 19:12:30 +03:00
Erik Schultheis	09d35a5e4d	splitting the loop to get rid of the indicator	2024-04-19 01:01:47 +03:00
Erik Schultheis	6d98af1d32	another 15% by getting rid of ifs	2024-04-19 01:01:47 +03:00
Erik Schultheis	314661f516	significantly faster kernel by reusing data in registers	2024-04-19 01:01:47 +03:00
Erik Schultheis	58a7b57f96	enable benchmarking	2024-04-18 01:39:47 +03:00
Erik Schultheis	e278112c19	coalesced reading in attention backward pass	2024-04-18 01:39:43 +03:00
Andrej Karpathy	fa4ebfa663	kernel2 for softmax backward	2024-04-17 08:14:22 +00:00
Andrej Karpathy	8a673edcc3	push my attention_backward file	2024-04-17 04:42:42 +00:00