Phi 3 has a unique architecture that needed some additions to llama.cpp's conversion script. Also Phi 3 is an absolute mess, there's no reliable way to latch on to when it's done writing a message and no one wants to admit it, people are patching around it instead.
ex. I could condition on "\n\n<|assistant|>||<|system|>||<|user>", but it'd still be wrong.
Pretty much everything Phi 3 feels like it needed to all come out within 48 hours a month too early. The ONNX genai library doesn't work on Mac, at all, the mobile SDKs don't support it...sigh
ex. I could condition on "\n\n<|assistant|>||<|system|>||<|user>", but it'd still be wrong.
Pretty much everything Phi 3 feels like it needed to all come out within 48 hours a month too early. The ONNX genai library doesn't work on Mac, at all, the mobile SDKs don't support it...sigh