# Kaiyu Shi

> Kaiyu Shi is an AI infrastructure and compiler systems engineer. This site contains his professional profile and technical writing on AI infrastructure, Apple silicon, inference, training systems, accelerators, compilers, and GPU kernels.

Use the About page for a short introduction and current focus, and the CV for career history and publications. Use the writing index or Atom feed to discover posts; article pages are the canonical full-text sources. Article pages include author, publication date, topic, and description metadata.

## Profile

- [About Kaiyu Shi](https://stonesjtu.github.io/about/): Personal introduction and current focus.
- [Curriculum vitae](https://stonesjtu.github.io/cv/): Full career experience, education, skills, research publications, and patents.
- [Writing index](https://stonesjtu.github.io/): All blog posts, ordered from newest to oldest.
- [Atom feed](https://stonesjtu.github.io/feed.xml): Recent posts and summaries in a machine-readable feed.
- [Sitemap](https://stonesjtu.github.io/sitemap.xml): Canonical URLs for the site's pages and posts.

## Technical writing



- [LLM Generative watermarks](https://stonesjtu.github.io/2026/10/15/llm-watermarks/): LLM watermark deployment, generative sampling methods, and inference-system challenges.

- [Apple GPU Evolution for AI Compute](https://stonesjtu.github.io/2026/09/09/apple-gpu-evolution/): Apple GPU evolution for AI compute, from Dynamic Caching and Neural Accelerators to TensorOps and MLX Metal GEMM.

- [Rubin 3-Bit LUT Decompression](https://stonesjtu.github.io/2026/09/04/rubin-lut-decompression/): How Rubin stores 3-bit weights with shared LUT codebooks, decompresses them for MXFP8 compute, and improves LLM decoding efficiency.

- [Apple Silicon AI compute: ANE and GPU](https://stonesjtu.github.io/2026/09/02/apple-silicon-ai-compute-gpu-vs-ane/): A concise hardware and software-stack map of Apple Silicon AI compute across ANE, GPU, Core ML/Core AI, MPS, Metal, MLX, and Metal tensor operations.

- [How intelligent can the edge become? - AI-Infra Overview PART-4](https://stonesjtu.github.io/2026/07/03/ai-infra-edge-intelligence/): How memory, bandwidth, power, economics, accelerator support, and small-model quality shape AI capability on phones, AI PCs, and edge boxes.

- [Predicting the future AI infra stack - AI-Infra Overview PART-3](https://stonesjtu.github.io/2026/07/02/ai-infra-future/): A supply-chain view of future AI infrastructure: heterogeneous compute, LLM-specific hardware, memory locality, and high-bandwidth interconnects.

- [Why AI infra is fast-moving and challenging - AI-Infra Overview PART-2](https://stonesjtu.github.io/2026/07/01/ai-infra-scaling-problem/): Why model size, context, output length, agent loops, and hardware change together, widening the AI infrastructure optimization frontier.

- [From conventional infra to agentic infra - AI-Infra Overview PART-1](https://stonesjtu.github.io/2026/06/30/ai-infra-and-tokenomics/): How AI infrastructure evolved with model execution patterns and specialized CPU, GPU, TPU, and NPU hardware under real service objectives.

- [Hopper kernel for MXFP4](https://stonesjtu.github.io/2025/08/14/hopper-kernel-for-mxfp4/): A Triton Hopper MXFP4 matmul walkthrough covering packed FP4 weights, LUT and scale unpacking, BF16 conversion, and inline PTX.

- [Build PyTorch with NCCL2](https://stonesjtu.github.io/2018/01/06/build-pytorch-with-nccl2/): Install NCCL2, build PyTorch for multi-GPU communication, run all-reduce tests, and troubleshoot CUDA and network interface setup.

- [Build PyTorch with CUDA-aware MPI support](https://stonesjtu.github.io/2018/01/05/build-pytorch-with-cuda-aware-mpi/): Build OpenMPI with CUDA support, compile PyTorch against MPI compilers, and verify distributed runtime support for GPU-aware communication.

- [Trace PyTorch CUDA performance bottlenecks](https://stonesjtu.github.io/2018/01/04/pytorch-cuda-performance-profiling/): A practical PyTorch CUDA profiling workflow, from synchronized Python cProfile and line-profiler runs to nvprof and kernel-level analysis.

- [Figure out what b.add_(b) has done](https://stonesjtu.github.io/2018/01/03/pytorch-tensor-add-explained/): Follow PyTorch tensor addition from Python and ATen through BLAS, OpenMP, and AVX2 kernels, with a bandwidth-based performance explanation.

- [How PyTorch dispatches Variable.index_select](https://stonesjtu.github.io/2018/01/02/pytorch-dispatch-variable-index-select/): Trace an early PyTorch index_select call from Python bindings through ATen and C/C++ backends to CUDA kernels and autograd definitions.

- [From Python to C/C++: a PyTorch walk-through](https://stonesjtu.github.io/2018/01/01/pytorch-python-to-cpp-walkthrough/): A compact walkthrough of early PyTorch autograd, tracing Variables, Functions, the backward graph, and the imperative execution engine.