Show HN: A wc rewrite in Rust that makes it over 100 times faster(github.com)
2 pointsby CallMeAlphabet2 hours ago1 comment
  • ventanaan hour ago
    At the first glance, the 100x difference is surprising. My first thought was that the Rust code [1] uses AVX instructions, which would've explained the difference if the regular wc was a naive C code counting characters in a loop; but from what I see, at least the regular coreutils wc [2] uses AVX too, so the reasons of the 100x difference are not immediately obvious to me. Have you got any good explanation why? I don't believe it's Rust vs C, it might be some optimization that does not exist in the coreutils C code?

    [1]: https://github.com/CallMeAlphabet/fastwc/blob/main/src/main....

    [2]: https://github.com/coreutils/coreutils/blob/master/src/wc.c

    • collinfunkan hour ago
      I co-maintain GNU coreutils and wrote the Neon 'wc -l' code, for reference.

      Based on my quick look at the source code, it looks like this program can handle ASCII whitespace. GNU coreutils uses iswspace, which is locale-aware. That limits how much we can optimize. We could probably add more optimization if the current locale is ASCII, e.g., LC_ALL=C on GNU/Linux.