DeepSeek just open-sourced its inference engine
https://github.com/deepseek-ai/DeepInfer

https://news.ycombinator.com/item?id=50163277