Great write up on my new fav fine tuning, training + inf engine: ms-swift from Alibaba group. Supports every freaking model that matters, a massive amount of full + PEFT finetuning, has tons of backends wrapped, like lm-deploy, vLLM, unsloth, deepspeed, etc. Awesome.

https://www.shelpuk.com/post/fine-tuning-llama-3-1-with-swift-unsloth-alternative-for-multi-gpu-llm-training