aparler/c-llm
1
0
Fork 0
mirror of https://github.com/karpathy/llm.c.git synced 2026-07-28 20:35:09 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #625 from gordicaleksa/add_nccl_to_readme

Browse source 
Add NCCL instruction to README
This commit is contained in:
Andrej 2024-06-20 16:06:31 -07:00 • committed by GitHub
commit dad58b6800
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -134,12 +134,14 @@ sudo apt-get -y install libcudnn9-dev-cuda-12
On top of this you need the [cuDNN frontend](https://github.com/NVIDIA/cudnn-frontend/tree/main), but this is just header files. Simply clone the repo to your disk. The Makefile currently looks for it in either your home directory or the current directory. If you have put it elsewhere, add `CUDNN_FRONTEND_PATH=/path/to/your/cudnn-frontend/include` to the `make` command-line.
**multi-GPU training**. As of April 26, 2024 there is now also support for multi-GPU training using MPI and NCCL. Make sure you install MPI, e.g. on Linux:
**multi-GPU training using MPI and NCCL**. Make sure you install MPI and NCCL, e.g. on Linux:
```bash
sudo apt install openmpi-bin openmpi-doc libopenmpi-dev
```
For NCCL follow the instructions from the [official website](https://developer.nvidia.com/nccl/nccl-download) (e.g. network installer)
and then:
```bash