deepsparse

neuralmagic / deepsparse

面向 CPU 的稀疏感知深度学习推理运行时,通过利用模型稀疏性实现高效低延迟的本地部署。

Python AI 基础设施 数据科学 开发工具 稀疏推理 CPU 加速 ONNX 模型部署 深度学习

为什么值得看

编辑点评

DeepSparse 让 CPU 也能跑出接近 GPU 的推理性能,特别适合已有稀疏化模型或愿意做剪枝/蒸馏的团队。它无缝集成 ONNX,并提供稀疏微调工具链,能显著降低推理成本。但需要注意,其加速效果依赖模型本身的稀疏度,对稠密模型收益有限,且生态和硬件优化针对性较强,适合对性能敏感且可接受模型定制的场景。

Star 趋势

近 7 日
  • Star 总数3,159
  • 今日新增+0
  • 7 日增速-100%
  • Fork192

同类项目

同场景 · AI 基础设施 / 数据科学 / 开发工具

项目文档

来自 GitHub README · main 分支

  DeepSparse

Sparsity-aware deep learning inference runtime for CPUs

🚨 2025 End of Life Announcement: DeepSparse, SparseML, SparseZoo, and Sparsify

Dear Community,

We’re reaching out with heartfelt thanks and important news. Following Neural Magic’s acquisition by Red Hat in January 2025, we’ve shifted our focus to commercial and open-source offerings built around vLLM (virtual large language models).

As part of this transition, we ceased development and deprecated the community versions of DeepSparse (including DeepSparse Enterprise), SparseML, SparseZoo, and Sparsify on June 2, 2025. These tools no longer will receive updates or support.

From day one, our mission was to democratize AI through efficient, accessible tools. We’ve learned so much from your feedback, creativity, and collaboration—watching these tools become vital parts of your ML journeys has meant the world to us.

Though we’ve wound down the community editions, we remain committed to our original values. Now as part of Red Hat, we’re excited to evolve our work around vLLM and deliver even more powerful solutions to the ML community.

To learn more about our next chapter, visit ai.redhat.com. Thank you for being part of this incredible journey.

With gratitude, The Neural Magic Team (now part of Red Hat)


文档抓取自 GitHub 仓库 README,版权归原作者所有;已过滤徽章等噪音并经安全消毒后展示。