Post

LLM

Posted
By Xiong Ding 1 min read

llamda.cpp

1
2
3
4
5
6
7
8
9

./build/bin/llama-server -m  ~/.cache/llama.cpp/Qwen2.5-3B-Instruct-q5_k_l.gguf --port 8012

python -m venv venv

source venv/bin/activate

pip install -r requirements/requirements-convert_hf_to_gguf.txt

python convert_hf_to_gguf.py ~/code/Qwen2.5-Coder-3B/ --outfile qwen2.5-coder-3b.gguf  --outtype f16
llm
llm
This post is licensed under CC BY 4.0 by the author.