E.g. removing all temporary files without the need to deal with spaces, glob limits, or silly find syntax:
csv-ls -R -c full_path . | csv-grep -c full_path -e '~$' | csv-exec -- rm -f %full_path
https://github.com/wallach-game/bashumerate/pull/1/files
to me looks like a right solution, even if I was also tired of writing for loops by hand usually
ls **/*.sh
But yeah, even in POSIX it feels redundant when we already have find, seq, grep and whatnot.
ls -- **/*.sh
Find works, but the syntax is arcane. Not bad, but unlike any other common cli tool, which makes it more difficult to remember if you haven't needed it in a while
Regarding remembering the find syntax I think its being arcane is what it made for me more easy to remember :) I now have a unique brain area dedicated to remember only that.
especially in the case of a younger profile, learning and putting dev effort, even if vibed or redundant, its still effort and a learning activity!
https://github.com/wallach-game
so I support this even if the result and problem solving can be less significant
In any case, toxicity aside, publishing a command line project and having someone dismiss the examples with shell oneliners is also a valuable lesson, and I don't think it will make him a worse developer.
But again the usefulness of the final tool should not compromise appreciation of effort and learning.
Unless the tool targets, claims or pretends to be the best, final and universal solution to such problem.
I believe it isn't, I will still fall back to my bash memories, I will not need, install and use this tool, I will not need, install or use custom shells. The effort wasted in re-aligning my brain to a tool change would superseed instantly the few second effort needed to reach the needed bash one-liner.
Also, didn't you also feel good and amazed when completing a very long and nested working one-liner? wasn't that orgasmic? :D
One uses xargs or parallel only a few times before they remember some of the quirks that but them. And then they become cautious. And then it’s muscle memory. And if it’s not an often occurrence, they learn to check the man page.
Anyways, in a world of “vibecoding” why add another “tool” to the mix when the LLMs have been trained on all our stackoverflow-posted grievances to begin with?
Inconsistent syntax native bash methods so its an additional syntax to learn.
> Template mode — single-quoted commands work as shell templates: enumerate -f '*' -- 'cat {} | head -5'
Passing commands a a single string is BAD. Now you have to think about escaping and quoting. What does {} get replaced with if the enumerant contains unsafe characters? Can it be used as part of a larger argument or only on its own? Who knows, it's not bash. Compared to a bash loop its also always a subshell with all the implications that has - even find can be piped into a normal bash loop.
> Filters — --include and --exclude with glob patterns
A fraction of what find or native loops provide. And since this can't replace them in general enumerate is an additional thing to learn on top.
> Extensible — drop a file in lib/enumerators/ to add custom sources
To extend bash loops you don't even need root access, you just add the code to the loop.
The obvious solution would be to use something more sane, like PowerShell or nushell, but instead old experts will always defend the skills they have honed for years, while criticizing anything that's different.
I really like the UNIX way of using text I/O, it has it flaws but it works for me. But that being said, I still hate bash and all its family. It has so many footguns it is an entire armory at this point, mostly related to spaces and escaping.
Something Perl-like could be a saner replacement. It is already a bit shell-like, it doesn't struggle with escaping the way bash does, and it has very powerful text processing abilities that go well with traditional UNIX tools.
People use Bash as the default shell for scripting, because people use Bash as the default shell for scripting. If you want to replace it, you should pick a winner and discourage the use of alternatives, especially when they are better than Bash. So don't say "use something more sane, like PowerShell or nushell". Say something like "use PowerShell, or Bash if you really have to for legacy purposes, but never use nushell for any reason" instead.
It's not so much about defending arcane scripting skills, but that Bash functions as a lowest common denominator and is useful because of that. If you want something that works reliably over decades, then it's best not to go for an "improved" shell as it may not still be around.
I like to think of Bash script writing as the opposite of riding a bike - you have to relearn it almost every time you write a script.
Option 1: Learn to bat and try to translate. Hmm. No. Just no Option 2: ?
Pwsh core had just come out. My immediate thought was that it would be great material for an anti-MS-ragging blog post, but then a line leapt out at me from one article I was glossing over: "... POSIX Terminal Shell Spec ...".
I still wanted that anti-MS-ragging blog material, so I decided to try and use Pwsh as a Rosetta stone until I got to a point I could convert to a real language.
But things just began to click for me. It was like going from Perl to Python - suddenly everything is an object and you can interact with everything* that way.
There's no need for grep or awk or sed in pwsh, because the output of a shell command is an object -- a string (or []byte). It has methods.
(netstat -an).replace("192.168.86.", "10.0.100.")
6 years later, pwsh is the default shell on my Mac, Ubuntu boxes, lab vms, ... everything but my docker containers unless I'm feeling feisty.
The only reasons people still use it are historical and laziness.
If it was invented today, professionals would cringe at it.
Ironically, the main friction point were not old-fashioned tools (which `from ssv` usually handled nicely) but the 'new generation' of core CLI tools like eza or fzf. They have really nice visualizations, but they do not output structured data as a middle step, so all the colours and lines only play havoc with nu's parsing.
Since I need to "ls" a lot more often than I need to do data manipulation, the tools won and I went back to zsh. Still keep nu around for the occasional config/data file wrangling though.
ls -1 ./*.sh | xargs -rd\\n sh -c 'for i in "$@" ; do ... ; done' sh
2. xargs -r just doesn't run the command if there's no input, also not strictly necessary but I'm addicted to using it because it's the sensible default to me.
3. xargs -d\\n makes it collect fields as full lines, which is what you typically want, unless you're able to generate NULs.
4. use whatever shell you want of course, but I don't use bashisms, etc., by default, /bin/sh is fine for me, even if it's dash.
5. the trailing "sh" at the end is due to a quirk of `sh -c` usage, where $0 is the first non option argument, so `printf %s\\n 1 2 3 | xargs -rd\\n sh -c 'for i in "$@" ; do printf "%s " "$i" ; done ; echo'` (note the lack of trailing "sh") would only print "2 3 " as $0 is not included in "$@" ($0 is 1, $1 is 2, $2 is 3). It's very easy to just always give the shell name itself manually as $0 instead of trying to ingest "$0" into your logic.
Of course, you could `find . -maxdepth 1 -type f -name '*.sh' -print0 | xargs -r0 ...` instead, depending on what you're up to, that may be the easiest. It's definitely the simplest -- as long as your xargs has -0 support.
I remember, though from the dim an distant past so it could be a long-fixed bug, this not working in at least one circumstance. I've explicitly included -1 in scripted calls to ls since. Of course we are breaking the best practice rule of not trying to parse the output of ls, so problems are not unexpected…
As a generally thing I like to include directives like this in scripted calls even if they happen by default anyway, because is makes my intent clear: I expect the output to being in simple single-column format and the rest will break if it isn't. I even sometimes go as far as specifying --sort=name if nothing else in the pipeline is going to enforce that.
> as long as your xargs has -0 support
I don't think I've encountered an xargs that doesn't have this support for a long time, though I don't work with embedded stuff so maybe there are cut-down versions out there still in active use for space reasons.
The problem I've hit numerous times is wanting to do something between “find -print0” and “xargs -0” and that something not supporting NUL as the item delimiter.
Just go with
foo | while read X; do bar "$X"; done grep stuff file.txt | while read key value ; do [ "$key" = "target" ] && found="$value" ; done
while read key value ; do [ "$key" = "target" ] && found="$value" ; done < <(grep stuff file.txt)
{ # code that generates one item/filename per line of output } | { while read ITEM; do # do something with "$ITEM"; done; }
enum() {
local source=$1; shift; local action=$1; shift
{ eval "$source" ; } | { while read ITEM; do eval "$action"; done; }
}
$ enum "find /tmp" "echo found: \$ITEM"
found: /tmp/.XIM-unix
found: /tmp/.ICE-unixTry this with find and grep vs xargs. There's a big difference.
* in a lot of cases the performance just doesn't matter
* xargs gets you the spaces in filenames landmine
* some commands don't even support multiple target filename arguments
For scripts in long term use, yeah, sure, figure out xargs maybe. Any other situation with a "| while read" solution, especially on an interactive session, is an oddball and simplicity wins.
While it's true not every command supports multiple targets, presumably you know if you're using one of those commands.
Always use -0. Most gnu utilities support it. It makes them put a null byte after every filename instead of a newline. Completely eliminates the problem of dealing with whitespace in the filenames.
find . -name '*.log' | xargs rm
find . -name '*.log' -print0 | xargs -0 rm find ... | tr '\n' '\0' | xargs -0 ...
That still requires your xargs to have -0 support, though, and I'd be surprised to have that without the corresponding option on find. But I've done this before when feeding xargs from something other than find.
The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs.
Parallel has an option for almost everything, it's almost too much.
But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0]
The gnu parallel book and reading materials [1] are excellent too.
[0] https://www.gnu.org/software/parallel/parallel_alternatives....
shell stuff | xargs -n 1 -I % echo real command and % args
script
script | shUtility software which has non-essential different first-run behavior is hostile to users.
Every single damn time I try parallel and decide to give it another chance, something ends up not working or causing a problem. I can never get it to just do what I want and get out of the way.
Today I foolishly thought maybe I was the one who was holding it wrong every single time in the past, so I copy pasted another command that was supposed to work, and thought surely this would be straightforward. Boy was I wrong. I got some manifesto about academic citations and plagiarism, which confused the hell out of me. After I wasted time trying to figure out how to turn off that nonsense, the app just hung there trying to figure out how long its command line can be? Literally doing nothing? What the hell? I killed it but then my terminal didn't close because every time I did this apparently some perl command was spawned in the background blocked on nothing. Why the hell was perl even relevant? Nothing I wrote used Perl. Just run the darn commands I asked in parallel, is that so hard?
I forget the details but it was some kind of surprisingly weird foot-gun behavior.
Luckily I had a backup, but it really has made me scared to try using parallel again.
That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc.
The citation thing is a little silly, I'll give you that one.
No. I did pipe to stdin. It's not my first time using Linux...
Here's a command line I ran right now, and the output I see:
$ echo "http://www.example.com" | parallel -k -j 8 curl -s "{}"
Academic tradition requires you to cite works you base your article on.
[...more nonsense...]
To silence this citation notice: run 'parallel --citation' once.
parallel: Warning: Finding the maximal command line length. This may take up to 1 minute.
So I Ctrl+C. And then the parent perl process gets killed, but the child apparently keeps running.
I press Ctrl+D to exit the terminal, and then:
$ # (Ctrl+D pressed)
logout
xargs Just Works without any of this nonsense, yet somehow I'm the one holding GNU parallel wrong?
> So I wait a few seconds [snip]
This warning is only ever printed if running in Cygwin, not Linux or macOS or elsewhere. Cygwin is notoriously slow.
# This is slow on Cygwin, so give Cygwin users a warning
if($^O eq "cygwin" or $^O eq "msys") {
::warning("Finding the maximal command line length. ".
"This may take up to 1 minute.")
}
Try reading some time. It's pretty great.
> the Ctrl+C handling is just utterly broken
That's your terminal, not parallel.
> and all of the delays
Your terminal is adding the delay, not parallel. parallel is just warning you about your broken terminal. You're shooting the messenger here.
Apparently goes on to describe being confused about parallel reading from the standard input?
parallel is written in perl.
Not everyone has the luxury to only work with their own computer, or run random software on IT/customer managed systems.
However, I also had to get comfortable enough with vi, because most customer systems that we would be required to access on a support visit, or remotely, only had ed and vi available.
Even if we nowadays stick with Linux distros instead of UNIX in general, there is still the issue of what is there by default.
Yep POSIX shell scripting can be a bit painful, which is why knowing a bit awk, perl and sed might help as well, as they tend to be available by default.
find -print0 | xargs -0 -I {} "the {} iterated command"Other brain glazing obsd wierdness is it's two argument cd command, a cryptid I am unable to wrap my head around.
find -exec the '{}' iterated command ';'While xargs will accumulate a bunch of file names, then when it as n names will invoke the command with those names, while continuing to accumulate names until n is reached or the pipe closes.
The size of n depends on the system, but is usually at least a thousand.
find -exec command '{}' '+'
Another feature missing in find that xargs has is the maximum processes to start at a time. find will run the commands in sequence, but in many situations you really want to run a bunch in parallel.
I do it because I do it, just like why I use egrep and sed in pipes along with awk.
(*35 years living and loving sh/ksh/bash/dash etc, only tried pwsh so I could write some comparisons and slag off MS a bit; now it's my default shell on everything)
Additionally the use of "--" is not what everybody expects: here it is used to introduce one argument, the command, while it's usually meant to introduce multiple arguments without worrying if they have a leading "-".
A possible revised syntax with command as the first argument followed by a list of globs optionally introduced by "--" would allow to enumerate all files with a leading "-", which the current syntax cannot:
enumerate 'whatever {}' -- '-*'
(For enumerating files, I use find or dir/b/s, possibly combined with grep, then pipe the result in.)
People moaning about avoiding non-basic use of xargs and bash (and inadvertently demonstrating in many cases why some of us think that xargs sucks) miss a large part of the point, which is that it's nice to have a tool that does exactly what you want, and works in a way that's convenient for you, and isn't so widely used that you have to worry about modifying it. If you find the tool doesn't work the way you like, you can just change it. You don't have to be answerable to anybody else. I think tptacek's quite good essay could be relevant: https://sockpuppet.org/blog/2026/05/12/emacsification/
(You don't have to use LLMs for this! I wrote my program by hand, it's only like 250 lines of Python, and you could write one too. But, whichever barriers to entry prevent you from creating your own tools for yourself, using an LLM would probably lower at least some of them.)
find . -name '*.log' -print0 | xargs -0 rm
This null termination is now a POSIX standard.
Looks like a case where reading man page would have spared writing another copycat utility.
The only times I've needed something more than `find -print0 | xargs -0` has been when I need to apply logic to decide whether to process one of the files, in a way that's not easy to express in a `find` command. Then I write a small script with a for loop and if statements inside it.
But more people should know about `-print0`. It's the answer to 95% of the problems with `find | xargs`.
If one is working with whole lines of text, setting the delimiter to newline is often desirable:
xargs -d \\n
it feel like the syntax here is odd. it still requires me to write quote my command i want to run? unfortunately i'll have to pass on this.
personally i'd want some variant where i can still auto-complete commands and have just have {} as a placeholder. (maybe time to learn how to use xargs for real?)
Fails : enumerate -f '.txt' -- 'wc -l {}'
For all use cases
Correct : enumerate -f '.txt' -- 'wc -l -- {}'
Most of this, if not all, is fixable by adding a `export IFS=$'\n'` to your bashrc. I'm not trying to disregard your project, just point out something that took me years to learn and I currently use extensively to solve this very problem. Perhaps you didn't know about it until now... :)
For example
`ps aux | grep process-name | grep -v grep | awk '{ print $2 }' | xargs -Ieach kill -9 each`
seq 1 10|xargs -I@ echo 'bash run.py @'|parallel -j 10
I know the echo is a little silly but then I can remove the |parallel and see if it's right. And if I don't want parallelism, I just pass to bash
That's not actually simpler though. Simple is removing everything unnecessary. You took commands which could already do what you wanted, and added an extra program which calls them in specific ways. This will add bugs and maintenance headaches, not be portable, etc. This is added complexity.
The reason you made this script is not because you wanted simpler, you wanted easier. There's nothing wrong with that, and I'll grant you it probably is, especially for those unaccustomed to these commands. But easier != simpler. Often you'll find that simple is hard and easy is complexity deferred.
for x (*.sh); echo "before $x after"
for x in *.sh; echo "before $x after"
for x (*.sh) { echo -n "before "; echo -n $x; echo " after" }for example
for a in `find / ` ; do echo $a ; done
https://github.com/wallach-game/bashumerate/blob/master/lib/...
`find` has `-print0` and `xargs` has the matching `-0` flag for this. So this specific argument makes me doubt the author knows their tools. Stopped reading at this point.
for f in *.txt; do
wc -l "$f"
done
> Or this?
find . -name '*.sh' -exec wc -l {} +
> These all work, but each has its own syntax, its own flags, its own quirks. I wanted something simpler — one consistent way to iterate over anything.
And therein lies the proverbial xkcd standards[0] proof.