accelerate
https://github.com/huggingface/accelerate
Python
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to accelerate
Help out
- Issues
- using deepspeed original json config, when using bf16, get the error RuntimeError: expected mat1 and mat2 to have the same dtype, but got: c10::BFloat16 != c10::Half.
- Cuda OOM when accelerator.prepare
- Possible issue in Accelerate FSDP Documentation
- [MLU] update deepspeed-mlu dependency
- Give example on how to handle gradient accumulation with cross-entropy
- Unable to access model gradients with DeepSpeed and Accelerate
- [Bug] The clip_grad_norm of xla fsdp is not right
- accelerator.prepare() get OOM,but available in single GPU
- Distributed inference example for llava_next
- Split_batches argument in Accelerator.__init__ is available, but not used
- Docs
- Python not yet supported