LLM Generative watermarks
Who uses text watermarks, how generative watermarking works, and what it changes in an inference system.
AI Systems
Notes on training, inference, and product co-design—connecting hardware and models to latency, cost, and user experience.
Current focus
Apple silicon
Exploring how Apple silicon architecture, compilers, and GPU kernels shape model execution.
Featured post · Sep 2026
Apple GPU Evolution for AI Compute
Apple GPU evolution for AI compute, from Dynamic Caching and Neural Accelerators to TensorOps and MLX Metal GEMM.
Read article → Featured post · Sep 2026 Apple Silicon AI compute: ANE and GPUA concise hardware and software-stack map of Apple Silicon AI compute across ANE, GPU, Core ML/Core AI, MPS, Metal, MLX, and Metal tensor operations.
Read article →Writing
15 articles · Newest firstApple GPU Evolution for AI Compute
Apple GPU evolution for AI compute, from Dynamic Caching and Neural Accelerators to the TensorOps and MLX software stack.
Rubin 3-Bit LUT Decompression
3-bit weight storage with MXFP8 compute at 3.375 bits per weight.
Apple Silicon AI compute: ANE and GPU
ANE and GPU hardware, software stacks, and runnable Core ML, PyTorch MPS, MLX, and Metal examples.
How intelligent can the edge become? - AI-Infra Overview PART-4
A hardware, economics, and model-quality view of how much intelligence can move into phones, AI PCs, and edge AI boxes.
Predicting the future AI infra stack - AI-Infra Overview PART-3
A supply-chain view of future AI infrastructure: heterogeneous computing, dedicated LLM hardware, memory locality, and interconnect.
Why AI infra is fast-moving and challenging - AI-Infra Overview PART-2
Why rapidly evolving models, request volume, context, output, agents, and hardware keep moving the AI infrastructure frontier.
From conventional infra to agentic infra - AI-Infra Overview PART-1
A history of AI infrastructure through the co-evolution of model execution patterns and CPU/GPU/TPU/NPU hardware.
Hopper kernel for MXFP4
Notes on Triton's Hopper MXFP4 matmul path, from packed FP4 unpacking into BF16 to scale conversion with inline PTX.
Build PyTorch with NCCL2
Covers NCCL2 installation, clean PyTorch source builds, multi-GPU all-reduce tests, and network interface troubleshooting.
Build PyTorch with CUDA-aware MPI support
Notes on building OpenMPI with CUDA support, compiling PyTorch against MPI compilers, and checking runtime distributed support.
Trace PyTorch CUDA performance bottlenecks
Shows a profiling workflow from synchronized Python-level cProfile and line-profiler runs down to nvprof and kernel-level investigation.
Figure out what b.add_(b) has done
Traces PyTorch tensor addition from Python bindings through ATen, TH tensor macros, BLAS paths, OpenMP splitting, and AVX2 vector kernels.
How PyTorch dispatches Variable.index_select
Explains how Tensor and Variable calls move through generated bindings, C/C++ backends, CUDA kernels, and derivative definitions.
From Python to C/C++: a PyTorch walk-through
Walks through a compact autograd example to show how Variables, Functions, the backward graph, and the imperative engine fit together.