25 pointsby scooterbooper5 hours ago19 comments
  • internet20003 hours ago
    > Piping a single file through cat spawns an entire process whose only job is to copy bytes to a program that already knew how to read them.

    Chrome probably spawned two processes when I cmd+clicked this into a new tab. It really doesn't matter.

  • jasongi2 hours ago
    The beauty of cat is that streams are the universal interface.

    Program A might accept a file as the last positional arg. Program B might accept it as a named arg, where the name/flag could be anything from --input or -f or --file etc.

    But a program will read from STDIN, which all good unix programs do, then piping cat into it works every time. I can write the cat foo.txt part before I even know what command I'm piping it into.

    • adrian_ban hour ago
      Yes, but "cat" is superfluous anyway.

      You can start your pipeline with a file redirection: " <input_file prog1 | prog2 | prog3 | ... | progN >output_file".

    • fuzzybear39652 hours ago
      This. Sometimes I want to see what I'm looking at and then (using that dump as a reference) follow up with a corresponding filter (| jq .key, or | tail -n 30). Sure, I could use less, but then I context switch on exit; no support from the scrollback buffer.

      I've probably lost 10ms * 1E5 of my life from the extra PID. But, probably would lose more in the context switch.

      • 1-more2 hours ago
        I'm never doing just one thing to a file! I'm grepping and JQing and then piping it to JQ again because I'm kind of dumb (and it's faster to do what I do know how to do than it is to look up the perfect way to do it), then I'm outputting as a TSV and piping that to `column -ts $'\t'`. ^r reveals a decent example:

            cat expected.1376| sed '1,4d' | rg '(\t\d)\t.*' --replace '$1' | column -ts $'\t'
        
        I was figuring shit out along the way and it'd be pretty annoying to adjust which command gets the filename throughout that process.

        You know what? I'll tell you another thing I do that's similar:

            SELECT * FROM whatever WHERE true
                AND last_modified > 123
                AND otherfield NOT NULL
        
        Always bugged me that you say WHERE for the first one and AND thereafter, so if I'm poking around the database trying to create actionable insights for key stakeholders at the speed of business just as I was above with the text file, I like to be able to futz and delete/add clauses as I see fit just as I do pipeline stages.
      • oogali2 hours ago
        alias less=‘less -X’
    • hn_throwaway_992 hours ago
      Yeah, I read TFA, and my eyes were rolling the whole time. Some people really have a bee in their bonnet that "cat" is named that way because it was originally for concatenating files. Nobody fucking cares. It's the standard way for writing a file to standard out, and the general pattern of "cat file.txt | somecommand | othercommand | anothercommand" is so useful because it follows the pipeline pattern so well - read from standard in, write to standard out - that is the cornerstone of Unix shell commands IMO.

      "Oh no, it spawns another process!!" Again, nobody cares.

      • adrian_ban hour ago
        The simpler and better way to write your pipeline is:

        "< file.txt somecommand | othercommand | anothercommand"

        There is never any need for "cat".

        When such a pipeline is used repeatedly in a script, the time for executing the redundant process "cat" can easily add up to a noticeable delay.

  • MPSimmons3 hours ago
    Unless you're executing these commands in a loop over a large number of items, or the item itself is gargantuan, it's almost always harmless.

    Personally, when I'm exploring, I build a command line iteratively. Cat the file to see the content, pipe to grep to get the lines I want, sed/awk/cut/etc to finagle from there.

  • floren3 hours ago
    if this wanton abuse of cat(1) doesn't stop, we're on track to run out of PIDs by 2031! Just because Unix makes it cheap and easy to fork doesn't mean you have to!

    (who gives even a single shit, my god)

    • copperx3 hours ago
      I like piping the output of cat and the mental image of one process feeding another. It's inconsequential, but it brings an epsilon of joy.
      • adrian_ban hour ago
        You have exactly the same mental image if you start a pipeline in the right way, with a file redirection: "<input_file cmd1 | cmd2 | cmd3 ...".

        This adds the joy of doing only what is really needed, without extraneous effort.

        Moreover, in this way the start of a pipeline becomes symmetrical to its end, which frequently is an output redirection.

      • djtriptych2 hours ago
        same I just like monads lol. cat + pipe feels purer and has lower mental load for me, which dominates the efficiency of spawning an extra process for, typically, a few microseconds.
    • floralhangnail2 hours ago
      Is this a dig at IPv6?
      • collabs2 hours ago
        Everything reminds me of her (IPv6) T_T

        Edit: for context my home router is a TP Link and for some reason it has IPv6 disabled completely and I'm too scared to enable it.

      • floren2 hours ago
        Definitely not, I like IPv6.
  • lifthrasiir3 hours ago
    Don't do this:

      cat file | wc -l            => wc -l < file
      cat file | head -n 5        => head -n 5 file
      cat file | awk '{print $1}' => awk '{print $1}' file
      cat file | sort             => sort file
    
    Do this instead:

      cat file | wc -l            => <file wc -l
      cat file | head -n 5        => <file head -n 5
      cat file | awk '{print $1}' => <file awk '{print $1}'
      cat file | sort             => <file sort
    
    The front-cat abuse is all about the order. The effective solution needs to keep the relative order of arguments.
    • stouset3 hours ago
      Or just use cat and spend your brainpower on interesting, useful, and/or worthwhile topics. It boggles my mind that anyone cares about this.
      • lifthrasiir3 hours ago
        Probably, but knowing that redirection operators can be freely moved within normal arguments [EDIT: thank ButlerianJihad for pursuing me to make this more accurate] is useful.
        • ButlerianJihad3 hours ago
          They are actually not “order-independent”, and their L-R parsing/processing is why constructs such as

            cat file > /dev/null 2>&1
          
          work as intended.
          • ablob3 hours ago
            funny enough,

              2>&1 >/dev/null cat file
            
            appears to yield the same output. So i wonder where the not "order-independent" chimes in.
            • pdpi2 hours ago
              `2>&1` redirects FD2 to the current contents of FD1 (stdout), then `> /dev/null` redirects FD1 to /dev/null. That results in your errors going into stdout, and discarding regular output altogether:

                  0: stdin  -> stdin  -> stdin
                  1: stdout -> stdout -> /dev/null
                  2: stderr -> stdout -> stdout
              
              When you flip the order, `> /dev/null 2>&1` moves FD1 to /dev/null first, and then FD2 to the contents FD1 (/dev/null again), so you discard both errors and standard output:

                  0: stdin  -> stdin     -> stdin
                  1: stdout -> /dev/null -> /dev/null
                  2: stderr -> stderr    -> /dev/null
              
              In your example, `cat file` is unlikely to produce any errors, which is why you're not seeing a difference.
            • isityettime2 hours ago
              All of this depends on your specific shell and its parser. Fish doesn't let you put redirections at the beginning like that (though I wish it did), while GNU Bash does.
              • ButlerianJihad2 hours ago
                fish is not POSIX-compatible, and not Bourne-compatible, so I don't see how that really matters at all. I used the rc shell from plan9 for quite a while, and I wouldn't expect its syntax rules to match, either!
            • ButlerianJihad3 hours ago
              You're absolutely wrong!

              It does not yield the "same output", and here is why: if you cause your command to actually produce output on stderr (fd 2) it will appear as terminal output, because you have actually succeeded in "redirecting" stderr to wherever stdout (fd 1) was pointing initially.

              • 2 hours ago
                undefined
      • soraminazuki2 hours ago
        If one needs brainpower to just use the redirection operator, it shows a likely lack of understanding of basic concepts in computing like processes and files. That should be concerning.
    • mkl2 hours ago
      The order is the main thing, but it's not just that. It's common to swap cat with something else like head, tail, grep etc., and that's easy. If you have "< file command" instead of "cat file | command" you have to make edits in two places to insert the "|".
  • bram985 minutes ago
    .
  • petee5 hours ago
    > Since 1995, occasional awards for UUOC have been given out, usually by Perl luminary Randal L. Schwartz

    http://catb.org/jargon/html/U/UUOC.html

    Admittedly its taken me a long time to remember that the file is the last argument to grep, when so many other commands its the first. I'd guess common abuse is due to being easier to type cat x | than to dig up the man page

    • smelendez4 hours ago
      And also typing cat x to get a quick look at the file, hitting up, then piping that into another command and taking a look, hitting up, piping that result into a third command etc.
      • soraminazuki2 hours ago
        I suppose a lot of people use less rather than cat for looking at files though.

        `alt + .` is much more versatile. You can use it to cycle through and insert the last arguments of previous commands.

    • js23 hours ago
      It's that way so that you can grep multiple files with a single pattern. It would be odd for the pattern to come after the file arguments. It also allows the files to be optional so that it can grep stdin.
    • soraminazuki3 hours ago
      The redirection operator is consistent and requires less typing though.

      I guess the file is usually the last argument because it's the one that can be omitted.

  • 2 hours ago
    undefined
  • mkl2 hours ago
    For interactive use like these examples I think this is terrible advice. cat is very helpful because it fits into pipelines like every other command. For example:

    Take a look at the start of a file:

      head file
    
    Filter for things:

      head file | grep ... 
    
    Reformat, remove unwanted stuff, etc.:

      head file | grep ... | sed ...
    
    Do things or dry run echo based on each line:

      head file | grep ... | sed ... | while read a; do ...; done
    
    It all looks good so we change head to cat to run it on the whole file:

      cat file | grep ... | sed ... | while read a; do... ; done
    
    Yes you can technically change "head file |" to "< file" but why bother? That's changes in two places and navigating between them instead of just <alt-d> cat.

    Same is true for other workflows. E.g. if you start with one of these supposedly better commands like "wc -l < file" (why isn't that just "wc -l file"?):

      wc -l < file
    
    Oh wait, we don't want every line, just ones matching a pattern. We could change it to

      wc -l <(grep ... file) 
    
    or

      grep -c ... file
    
    which are both more work than just adding to an existing cat-based pipeline, where we just replace cat with "grep ...":

      grep ... file | wc -l
    
    If we need to also match another pattern or whatever this pipeline approach is better then too.
  • copperx2 hours ago
    I'll make a note of it in my AGENTS.md file.
  • Shadowmist4 hours ago
    I’m going to keep doing it but wouldn’t mind it if my shell auto replaced it for me.
  • jmclnx4 hours ago
    In this day an age this is still making rounds ? So this is the memory usage of cat on my system:

         VSZ   RSS    SZ CMD
        3252  1608   813 /bin/cat
    
    To me there are far more things to worry about than cat. How about your multi-gig browser for one ?

    Now for firefox:

            VSZ    RSS     SZ CMD
        3472212 395968 868053 /usr/lib64/firefox/firefox
    
    Maybe people should be looking at that ? I will not even get into modern Linux Desktops :)
  • dminvs3 hours ago
    "Don't be a catgrepper"

    - various HostGator employees, c. 2011

  • pyrolistical3 hours ago
    I like putting the stdin before the command

    < file grep abc

  • fragmede2 hours ago
    First off, stop using cat, use something modern. I recommend redpanda, rp, but I'm biased since I'm the author.

    Anyway, the reason I use useless cat is because when I'm working with large files, it starts with head, not cat, or tail, and then only after I've built the pipeline do I use cat to process the whole file.

  • sprior3 hours ago
    I raised eyebrows recently when I was working with someone and we needed to create a file and instead of starting an editor I did: cat > filename ... Ctrl-D
    • przemuban hour ago
      Oh I always do that! Send them my greetings.
    • bonsai_spool3 hours ago
      Why not touch or echo? No reason for an editor or cat
      • stouset3 hours ago
        You can type the intended file contents as-is.
      • sprior3 hours ago
        For a one line file sure, but I was creating multiple lines.
  • antonvs3 hours ago
    Presumably written by someone without much interactive shell experience.

    When you're building a pipeline, putting cat first can often be quite convenient. Essentially, it's more composable: it defines the input to the pipeline without committing to a specific tool. For example, you can up-arrow in the shell and change the part after the pipe without having to skip back past the filename.

    In fact if you don't start with cat, it's possible you're more of a script kiddie than a software developer.

  • 5 hours ago
    undefined
  • BugsJustFindMe4 hours ago
    [flagged]
    • doubled1124 hours ago
      Chances are the file stays the same and what Im grepping for changes, so after the first run, it was less work for me not needing to go back as far in the command

          cat file | grep thingone
          cat file | grep thingtwo
      
      Vs

          grep thingone file
          grep thingtwo file
      • fc417fc8022 hours ago
        If it really matters to you that much then `<file grep thing` has the desired order and is less to type out.
      • BugsJustFindMe3 hours ago
        That's fine. It's fine to do the thing that brings you joy.