How Apple fits a LLM on iPhone with surgical efficiency(www.youtube.com)
9 pointsby lroe6 hours ago1 comment
  • paxy3 hours ago
    TL;DR

    - Quantization

    - KV caching

    - LoRA

    - Sparse MoE model

    - Dedicated AI chip

    - Offload to cloud

    Nothing really groundbreaking.