AirLLM is an open-source library for running very large language models on machines with limited GPU memory. Its core approach is layer-wise inference: model layers are loaded and executed in smaller portions instead of requiring the entire model to reside in GPU memory at once.

This design allows users to experiment with models that would normally exceed the capacity of a consumer GPU, while supporting common Hugging Face-style model workflows and quantization or compression options. The trade-off is that moving model data between storage, CPU memory, and GPU memory can reduce inference speed compared with keeping a model fully resident on the accelerator.

AirLLM is most useful for local experimentation, research, and occasional inference where fitting a large model matters more than maximum throughput. It gives developers a practical route to test oversized models without immediately moving to multi-GPU servers or hosted inference infrastructure.