dspy.HFClientVLLM
Usage
lm = dspy.HFClientVLLM(model="meta-llama/Llama-2-7b-hf", port=8080, url="http://localhost")
Prerequisites
Refer to the vLLM Server section of the Using Local Models
documentation.
Constructor
Refer to dspy.TGI
documentation. Replace with HFClientVLLM
.
Methods
Refer to dspy.OpenAI
documentation.