Can We Get WiCi Infer API use?

Some of us have big machines with lots of memory. WiCi Infer now runs a 177B parameter model on my Studio with 90+ tokens/second. It has a Local API hook at 127.0.0.1:8080/v1. Can we get this linked up to run in the Elements API window?

For those who might want to try the same thing, here’s the conversation to set it up via MCP and WiCi’s response (note that it took a bit longer to do this as I’ve got a lot of stuff running on this Studio right now and some spooling happened):slight_smile:

After a bit of memory cleanup and use, I’d say that the 177B/4-bit Qwen is running about one-tenth the speed of Claude on my Studio. Considering the huge model size, I find that respectable. It’s definitely workable. Most of the slowdown is inference (when it infers, it’s using the SSD).