Pricing
Pricing information has not been added yet.
Overview
MLC LLM is an open-source machine-learning compilation project for deploying large language models efficiently across a wide range of hardware and software environments. It uses compiler techniques to transform models into optimized runtimes that can execute on GPUs, CPUs, browsers, mobile devices, and other supported targets.
The project provides model compilation, quantization, runtime libraries, and APIs for serving or embedding LLMs in applications. A major goal is portability: developers can use a common deployment stack while targeting platforms such as CUDA, Metal, Vulkan, WebGPU, and mobile environments instead of maintaining a separate inference implementation for each device.
MLC LLM is most useful for engineers who need local or edge inference and care about performance across heterogeneous hardware. It bridges model development and production deployment, making it possible to package open models into optimized applications beyond conventional server-only GPU infrastructure.
README
About
MLC LLM is a machine learning compiler and high-performance deployment engine for large language models. The mission of this project is to enable everyone to develop, optimize, and deploy AI models natively on everyone's platforms.
| AMD GPU | NVIDIA GPU | Apple GPU | Intel GPU | |
|---|---|---|---|---|
| Linux / Win | ✅ Vulkan, ROCm | ✅ Vulkan, CUDA | N/A | ✅ Vulkan |
| macOS | ✅ Metal (dGPU) | N/A | ✅ Metal | ✅ Metal (iGPU) |
| Web Browser | ✅ WebGPU and WASM | |||
| iOS / iPadOS | ✅ Metal on Apple A-series GPU | |||
| Android | ✅ OpenCL on Adreno GPU | ✅ OpenCL on Mali GPU | ||
MLC LLM compiles and runs code on MLCEngine -- a unified high-performance LLM inference engine across the above platforms. MLCEngine provides OpenAI-compatible API available through REST server, python, javascript, iOS, Android, all backed by the same engine and compiler that we keep improving with the community.
Get Started
Please visit our documentation to get started with MLC LLM.
Citation
Please consider citing our project if you find it useful:
@software{mlc-llm,
author = {{MLC team}},
title = {{MLC-LLM}},
url = {https://github.com/mlc-ai/mlc-llm},
year = {2023-2025}
}
The underlying techniques of MLC LLM include:
References (Click to expand)
@inproceedings{tensorir,
author = {Feng, Siyuan and Hou, Bohan and Jin, Hongyi and Lin, Wuwei and Shao, Junru and Lai, Ruihang and Ye, Zihao and Zheng, Lianmin and Yu, Cody Hao and Yu, Yong and Chen, Tianqi},
title = {TensorIR: An Abstraction for Automatic Tensorized Program Optimization},
year = {2023},
isbn = {9781450399166},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3575693.3576933},
doi = {10.1145/3575693.3576933},
booktitle = {Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
pages = {804–817},
numpages = {14},
keywords = {Tensor Computation, Machine Learning Compiler, Deep Neural Network},
location = {Vancouver, BC, Canada},
series = {ASPLOS 2023}
}
@inproceedings{metaschedule,
author = {Shao, Junru and Zhou, Xiyou and Feng, Siyuan and Hou, Bohan and Lai, Ruihang and Jin, Hongyi and Lin, Wuwei and Masuda, Masahiro and Yu, Cody Hao and Chen, Tianqi},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {35783--35796},
publisher = {Curran Associates, Inc.},
title = {Tensor Program Optimization with Probabilistic Programs},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/e894eafae43e68b4c8dfdacf742bcbf3-Paper-Conference.pdf},
volume = {35},
year = {2022}
}
@inproceedings{tvm,
author = {Tianqi Chen and Thierry Moreau and Ziheng Jiang and Lianmin Zheng and Eddie Yan and Haichen Shen and Meghan Cowan and Leyuan Wang and Yuwei Hu and Luis Ceze and Carlos Guestrin and Arvind Krishnamurthy},
title = {{TVM}: An Automated {End-to-End} Optimizing Compiler for Deep Learning},
booktitle = {13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)},
year = {2018},
isbn = {978-1-939133-08-3},
address = {Carlsbad, CA},
pages = {578--594},
url = {https://www.usenix.org/conference/osdi18/presentation/chen},
publisher = {USENIX Association},
month = oct,
}
Releases
Release information has not been added yet.
Complete your AI stack
Move from research to writing, visual communication, presentations, and automation.
Best alternatives
Compare similar tools in the same category.
