AI System & AI Infra

Because large-model content has grown substantially, the newest LLM material is maintained in AIFoundation.

The written course is continuously being expanded. Course materials are available through AISys, videos are hosted on Bilibili and YouTube, and the slides are open-sourced on GitHub.

Course background

The English name of this open course is AI System (AISys). It explores system design for artificial intelligence and deep learning, based on ZOMI's accumulated work building and organizing the full AI systems stack. The goal is to learn and discuss AI systems together with the open-source community.

Course outline

The course contains five major modules:

Module	Description	Link
AI systems full-stack overview	Fundamentals and a full-stack overview of AI systems, including systematic design methods for deep-learning systems and the overall architecture of AI training and inference.	[Slides]
AI chips and architecture	Hardware architecture for AI, from CPU/GPU fundamentals to AI chip principles, design, applications, middleware, frameworks, and practical usability.	[Slides]
AI programming and computing architecture	Compiler problems in modern machine-learning system design, especially intermediate representations and backend optimization.	[Slides]
AI inference systems and engines	Practical inference systems and engines, including core algorithms and engineering considerations required to deploy AI in real products.	[Slides]
Core AI framework technologies	Automatic differentiation, graphs and operators, frontend optimization, and key technologies for distributed large-model training.	[Slides]

Course goals

The course is designed for senior undergraduates, graduate students, researchers, and AI systems practitioners. It aims to:

1. Provide a complete understanding of AI computer-system architecture and system design across the full AI lifecycle through practical problems and case studies.
2. Introduce research that combines modern system architecture and AI, together with mainstream frameworks, platforms, and tools.

Course sections

1. AI systems overview

#	Topic	Content
1	AI systems	How algorithms, frameworks, and computer architecture combine into an AI system.

2. AI chip architecture

#	Topic	Content
1	AI computing architecture	Computing patterns and architectures for neural networks and other AI workloads.
2	AI chip fundamentals	Basic architecture principles of CPUs, GPUs, NPUs, and related processors.
3	GPUs	GPU fundamentals and the evolution of NVIDIA GPU architecture.
4	NVIDIA GPUs in depth	Tensor Cores, NVLink, and NVIDIA GPU architecture.
5	International AI processors	Core principles of specialized AI processors from companies such as Google and Tesla.
6	Chinese AI processors	Core principles of specialized AI processors from companies such as Cambricon and Enflame.
7	The golden decade of AI chips	A review of AI-chip programming models and development trends.

3. AI compiler principles

#	Topic	Content
1	Traditional compilers	GCC, LLVM, and LLVM architecture.
2	AI compilers	Evolution and architecture of AI compilers, future challenges, and open questions.
3	Frontend optimization	Operator fusion, memory optimization, and other frontend compiler techniques.
4	Backend optimization	Kernel optimization and AutoTuning.
5	Polyhedral compilation	Work in progress.
6	PyTorch 2.0	The compilation technology stack introduced in PyTorch 2.0.

4. AI inference systems

#	Topic	Content
1	Inference systems	Overall inference-system concepts and inference-engine architecture.
2	Lightweight networks	Lightweight backbone networks such as MobileNet and other SOTA models.
3	Model compression	Quantization, distillation, pruning, and binarization.
4	Conversion and optimization	Converting trained models and optimizing their computation graphs.
5	Kernel optimization	Operator, memory, scheduling, and kernel-level optimization.

5. Core AI framework technologies

#	Topic	Content
1	AI framework fundamentals	The role, evolution, and programming paradigms of AI frameworks.
2	Automatic differentiation	Principles and implementation approaches for automatic differentiation.
3	Computation graphs	Graph concepts, graph optimization, execution, and control-flow representation.

Knowledge checklist

Notes

The repository is around 10 GB because the author provides source production materials and high-resolution images. A full git clone can therefore be slow; using the Releases page to download only the material you need is recommended.

Community participation is welcome. Please report errors, submit fixes through pull requests, and respect the open-source work by clearly citing the source when reusing slide content.