Odd, I don't see any mention of subprocess.run, the workhorse of python scripting.
Quick rundown for the unfamiliar:
Give it a command as a list of strings (e.g., subprocess.run(["echo", "foo"]).)
It takes a bunch of flags, but the most useful (but not immediately obvious) ones are:
check=True: Raise an error if the command fails
capture_output=True: Captures stdout/stderr on the CompletedProcess
text=True: Automatically convert the stdout/stderr bytes to strings
By default, subprocess.run will print the stdout/stderr to the script's output (like bash, basically), so I only bother with capture_output if I need information in the output for a later step.
show comments
sevensor
The Python stdlib does not get enough credit. People complain about things like how its http client is dated and slow, but it’s pretty amazing that it’s just right there if you need it, no external dependencies needed. And it’s sitting right next to difflib, graphlib, pathlib, struct, glob, tkinter, and dozens of others. Sure, every one of these is limited individually, but those limitations are stable and well understood!
show comments
_jzlw
How do you handle packages? I want scripts to a be a single file with a shebang, not a repo with a requirements.txt that I need to run in a venv. To me, this is the biggest blocker to using Python for any non-trivial scripting (which is precisely the kind where I wouldn't want to use bash), but I'd like to know how others deal with it.
C# scripts let you reference packages in a comment at the top of the file, for example:
If a script is simple - I use posix sh + awk, sed, etc.
But if a script I write needs to use arrays, sets, hashtable or processes many files - I use Nim[0]. It's a compiled systems-programming language that feels like a scripting language:
- Nim is easy to write and reads almost like a pseudocode.
- Nim is very portable language, runs almost anywhere C can run (both compiler and programs).
- `nim r script.nim` to compile and run (cached on subsequent runs) or use a shebang `#!/bin/env -S nim r`
- Nim programs are fast to compile (use debug mode and tcc compiler for almost instant compile times)
- Nim scripts run very fast <10ms (something that was very annoying to me with bash and Python)
- good chances you don't need external dependencies, because stdlib is batteries included and full of goodies.
- if you need external deps - just statically link them and distribute a cross-compiled binary (use zigcc[1] for easy Nim cross-compilation).
> Python is installed on pretty much every machine
> Python 3 is installed on basically every machine out there.
> Python will work the same on all the machines you run your script on
No, no, and no.
show comments
ggm
This from the site:
/*
Oh, you're looking at my CSS. Here be dragons.
Let's bundle in a rant while you're here: I'd like to use variable width fonts
instead of fixed size fonts. I enjoy 300 (i.e. light) and 600 (i.e. semibold)
over regular and bold for Crimson Pro only. But Chrome doesn't like to show
anything but 400 and 700 with variable width fonts, so I'll have to trick Chrome
by declaring my 300 and 600 fonts as if they were 400 and 700 respectively.
... or well, I could use font-variation-settings, but that's a global override
which means I have to specify it EVERYWHERE and that's just too much effort for
poor me.
*/
kazinator
cp version.template build/version.txt
sed -i "s/@VERSION@/${COMMIT_TAG:-dev}/" build/version.txt
sed -i "s/@BUILD_DATE@/${BUILD_DATE}/" build/version.txt
Eww! Mutating a file in place, and needing a GNU extension for it.
So they suggest to write scripts in Python rather than shell because Python is stable, probably installed on the target machine, has a big standard library, and is more readable. Many people do so.
That's the bright side of Python. They should mention the dark side, or Why _not_ to use Python for scripting.
First of all, the promise of easy portability breaks as soon as the script has dependencies. Try to install some Python program on a server where you're not root and a minimal python3 is installed.
The stability isn't very good in my experience either. I've often seen programs not compatible with recent releases of Python, either explicitly in the README or implicitly at runtime. Unmaintained Python code breaks.
Unfortunately, there is no silver bullet. Posix shell or bash may be better for simple scripts; Perl or Python if you know you won't require dependencies or if you have a good control on where to install the script; languages that compile to static executables are not really "scripting", but may be a better choice for long(term usage. These past years, I tend to keep away from Python as much as I can.
show comments
yoavsha1
I strongly agree with this.
At $WORK I usually work on projects comprising many small bits in various languages - some PowerShell here, some JS there, along with a "build process" that helps minify each and combine them into the final product.
After switching from shell scripts to Just and having to deal with a ton of issues on the way (how does quoting work in each system? How does argument passing? Environment variables?) I simply wrote a simple script with Python, UV shebang and PEP723 dependencies. Typer takes care of the command line parsing, and each "build target" is a simple, composable, readable python function that takes arguments and can call other ones if it needs to. Can't be simpler than that and the LLMs love it too.
rovr138
> Finally, it should be noted that this doesn’t work in ZSH, and if you’re using ZSH, you probably are a bit more of a terminal enthusiast than the average person. And if the enthusiasts can’t use this in their favourite shell, it’s less likely that they will use it in the scripts they make.
Important thing to note here is, ZSH is the default on macOS.
Alifatisk
I like the message the article is trying to convey, Python is good alternative to complicated shell scripts in my opinion.
I do wonder, let's say the scripting file is using lots of libraries, do you have to include some kind of requirements.txt file with it aswell when you want to share it with other people?
In Ruby, there is inline bundler which makes sharing a single Ruby script very portable.
I've never liked shell scripting. Last year, I switched my build system of a Rust project over to Python (Cargo is actually quite limited as a build system). For a newer project, I'm using Rust itself with the XTask pattern. I'm not sure if I prefer the Python or Rust approach yet.
show comments
bjoli
Funny this shows up on HN now! Jean Niklas wrote his thesis about RRB trees, which is an evolution of the persistent vectors of clojure fame. I have spent many hours reading his thesis lately, because
I just spent two months porting his c-rrb (https://github.com/hypirion/c-rrb) to c#, which was a fun endeavour. I wish I would have read hus thesis better since I spent two weeks debugging issues that arise from me not enforcing the leftwise dense invariant with regards to the tail.
miggol
I think the issue with dependencies is a bit overstated. There's uv for that. And shell scripts have dependencies as well! You're not guaranteed to have bash, jq, and curl on a minimal install.
That being said, for real portability of programs that have slightly outgrown the script moniker I really like Janet.
It can compile your script to a static binary for distribution.
abhiyerra
I have been converting a lot of my makefiles to pyinvoke and fabric and it makes things so much easier to manage than bash or make. Don't know why I held on for so long.
tayo42
Pretty much anything longer then a throwaway one liner I write in python.
Would be cool if python had a pipe operator though.
The back ticks in ruby is pretty ergonomic too. Wish python had a simpler way to run commands. Kind of tedious to look up subprocess run arguments and also break things up into arrays.
I wrote another comment here about a strategy for writing portable Bash scripts without compromising on features and freely using arbitrary external commands. I wanted to give an example from the article of how I'd likely write one of his examples of a "somewhat unreadable shell script" in this style.
His ugly sh example:
morning_greetings=('hi' 'hello' 'good morning')
energetic_morning_greetings=()
for s in "${morning_greetings[@]}"; do
energetic_morning_greetings+=( "${s^^}!" )
done
and his more readable Python equivalent:
morning_greetings = ['hi', 'hello', 'good morning']
energetic_morning_greetings = \
[s.upper() + '!' for s in morning_greetings]
And I'd write the shell version in Bash something like this:
Does it still involve more syntax? Yeah. Printing arrays in Bash always involves some. But it's clearer at a glance what it does, and it doesn't involve mutating variables.
The piece this example is too simple to show (since it's focused only on data operations and not interacting with the filesystem or running external programs) is how much shorter the Bash usually ends up being than the Python equivalent.
N_Lens
Weren't we already?
binary132
I like using Go for “scripting”. To each their own I suppose.
Odd, I don't see any mention of subprocess.run, the workhorse of python scripting.
Quick rundown for the unfamiliar:
Give it a command as a list of strings (e.g., subprocess.run(["echo", "foo"]).)
It takes a bunch of flags, but the most useful (but not immediately obvious) ones are:
By default, subprocess.run will print the stdout/stderr to the script's output (like bash, basically), so I only bother with capture_output if I need information in the output for a later step.The Python stdlib does not get enough credit. People complain about things like how its http client is dated and slow, but it’s pretty amazing that it’s just right there if you need it, no external dependencies needed. And it’s sitting right next to difflib, graphlib, pathlib, struct, glob, tkinter, and dozens of others. Sure, every one of these is limited individually, but those limitations are stable and well understood!
How do you handle packages? I want scripts to a be a single file with a shebang, not a repo with a requirements.txt that I need to run in a venv. To me, this is the biggest blocker to using Python for any non-trivial scripting (which is precisely the kind where I wouldn't want to use bash), but I'd like to know how others deal with it.
C# scripts let you reference packages in a comment at the top of the file, for example:
https://devblogs.microsoft.com/dotnet/announcing-dotnet-run-...
If a script is simple - I use posix sh + awk, sed, etc.
But if a script I write needs to use arrays, sets, hashtable or processes many files - I use Nim[0]. It's a compiled systems-programming language that feels like a scripting language:
- Nim is easy to write and reads almost like a pseudocode.
- Nim is very portable language, runs almost anywhere C can run (both compiler and programs).
- `nim r script.nim` to compile and run (cached on subsequent runs) or use a shebang `#!/bin/env -S nim r`
- Nim programs are fast to compile (use debug mode and tcc compiler for almost instant compile times)
- Nim scripts run very fast <10ms (something that was very annoying to me with bash and Python)
- good chances you don't need external dependencies, because stdlib is batteries included and full of goodies.
- if you need external deps - just statically link them and distribute a cross-compiled binary (use zigcc[1] for easy Nim cross-compilation).
[0] - https://nim-lang.org
[1] - https://github.com/enthus1ast/zigcc
> Python is installed on pretty much every machine
> Python 3 is installed on basically every machine out there.
> Python will work the same on all the machines you run your script on
No, no, and no.
This from the site:
So they suggest to write scripts in Python rather than shell because Python is stable, probably installed on the target machine, has a big standard library, and is more readable. Many people do so.
That's the bright side of Python. They should mention the dark side, or Why _not_ to use Python for scripting.
First of all, the promise of easy portability breaks as soon as the script has dependencies. Try to install some Python program on a server where you're not root and a minimal python3 is installed.
The stability isn't very good in my experience either. I've often seen programs not compatible with recent releases of Python, either explicitly in the README or implicitly at runtime. Unmaintained Python code breaks.
Unfortunately, there is no silver bullet. Posix shell or bash may be better for simple scripts; Perl or Python if you know you won't require dependencies or if you have a good control on where to install the script; languages that compile to static executables are not really "scripting", but may be a better choice for long(term usage. These past years, I tend to keep away from Python as much as I can.
I strongly agree with this. At $WORK I usually work on projects comprising many small bits in various languages - some PowerShell here, some JS there, along with a "build process" that helps minify each and combine them into the final product. After switching from shell scripts to Just and having to deal with a ton of issues on the way (how does quoting work in each system? How does argument passing? Environment variables?) I simply wrote a simple script with Python, UV shebang and PEP723 dependencies. Typer takes care of the command line parsing, and each "build target" is a simple, composable, readable python function that takes arguments and can call other ones if it needs to. Can't be simpler than that and the LLMs love it too.
> Finally, it should be noted that this doesn’t work in ZSH, and if you’re using ZSH, you probably are a bit more of a terminal enthusiast than the average person. And if the enthusiasts can’t use this in their favourite shell, it’s less likely that they will use it in the scripts they make.
Important thing to note here is, ZSH is the default on macOS.
I like the message the article is trying to convey, Python is good alternative to complicated shell scripts in my opinion.
I do wonder, let's say the scripting file is using lots of libraries, do you have to include some kind of requirements.txt file with it aswell when you want to share it with other people?
In Ruby, there is inline bundler which makes sharing a single Ruby script very portable.
https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...
Xonsh is perfect for this: https://xon.sh
I've never liked shell scripting. Last year, I switched my build system of a Rust project over to Python (Cargo is actually quite limited as a build system). For a newer project, I'm using Rust itself with the XTask pattern. I'm not sure if I prefer the Python or Rust approach yet.
Funny this shows up on HN now! Jean Niklas wrote his thesis about RRB trees, which is an evolution of the persistent vectors of clojure fame. I have spent many hours reading his thesis lately, because I just spent two months porting his c-rrb (https://github.com/hypirion/c-rrb) to c#, which was a fun endeavour. I wish I would have read hus thesis better since I spent two weeks debugging issues that arise from me not enforcing the leftwise dense invariant with regards to the tail.
I think the issue with dependencies is a bit overstated. There's uv for that. And shell scripts have dependencies as well! You're not guaranteed to have bash, jq, and curl on a minimal install.
That being said, for real portability of programs that have slightly outgrown the script moniker I really like Janet.
https://janet.guide/scripting/
It can compile your script to a static binary for distribution.
I have been converting a lot of my makefiles to pyinvoke and fabric and it makes things so much easier to manage than bash or make. Don't know why I held on for so long.
Pretty much anything longer then a throwaway one liner I write in python.
Would be cool if python had a pipe operator though.
The back ticks in ruby is pretty ergonomic too. Wish python had a simpler way to run commands. Kind of tedious to look up subprocess run arguments and also break things up into arrays.
Dupe: https://news.ycombinator.com/item?id=46176113
> By the way, don’t add a comma after the elements in the list, ...
Python:
Python for scripting honestly is Xonsh :)
I wrote another comment here about a strategy for writing portable Bash scripts without compromising on features and freely using arbitrary external commands. I wanted to give an example from the article of how I'd likely write one of his examples of a "somewhat unreadable shell script" in this style.
His ugly sh example:
and his more readable Python equivalent: And I'd write the shell version in Bash something like this: Does it still involve more syntax? Yeah. Printing arrays in Bash always involves some. But it's clearer at a glance what it does, and it doesn't involve mutating variables.The piece this example is too simple to show (since it's focused only on data operations and not interacting with the filesystem or running external programs) is how much shorter the Bash usually ends up being than the Python equivalent.
Weren't we already?
I like using Go for “scripting”. To each their own I suppose.
Same, use Javascript for Scripting. https://github.com/gutenye/script.js
Why not use Perl?
why not use perl?
[dead]