Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU (dev.to)dev
12 points by @xbill in dev.to · 4d ago · dev
A field report on serving Gemma 4 E2B under vLLM on AWS G5g — the only aarch64 + SM 7.5 hardware there is. No published build covers that combination, AWS quietly solves half of it, and the thing that actually blocks you is 64 KiB of shared memory.