this post was submitted on 02 Sep 2023
81 points (89.3% liked)

Programming

20075 readers
28 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 2 years ago
MODERATORS
 

why?

Because bash feels clunky to write and work with for anything non-trivial, especially compared to other scripting languages.

Why not another scripting language (no compile necessary)?

Because bash and sh are installed nearly everywhere. Any other scripting language means the user is required to have that installed, and that is far less likely to be the case.

If I could write my scripts in a nice syntax, but be sure my users will be able to use it effortlessly by distributing to them compiled versions, then that would make both of our lives easier!

Thoughts? Are there any languges that do this?

top 50 comments
sorted by: hot top controversial new old
[–] ndotb@programming.dev 54 points 2 years ago* (last edited 2 years ago) (2 children)

python is usually the next step up in admin land

python is a pretty standard install on linux systems since so many things like you're talking about use it

[–] crunchpaste@lemmy.dbzer0.com 17 points 2 years ago (1 children)

Not only that it's basically everywhere, but even if it's not, you can compile it using something like nuitka and still use it.

[–] wick@lemm.ee 5 points 2 years ago (2 children)

Huh, why doesn't python just ship this? Managing python installs is annoying as hell.

[–] crunchpaste@lemmy.dbzer0.com 5 points 2 years ago

I would guess mostly because python interpretes are just about everywhere.

Also the binaries compiled with nuitka end up being much bigger in size. A simple script of a few kb can and up in the hundreds of mb when you start compiling the dependencies, so it's not a perfect solution.

This is about python packaging, like making/getting libraries/apps rather than compiling binaries, but it's pretty relevant here:

https://chriswarrick.com/blog/2023/01/15/how-to-improve-python-packaging/

[–] RoadieRich@midwest.social 6 points 2 years ago

Python or Perl. IIRC, I've seen systems with a Perl install by default, but not Python.

[–] jeffhykin@lemm.ee 52 points 2 years ago* (last edited 2 years ago) (9 children)

I write a lot of bootstrapping scripts, and I have a solution thats probably something you and others in this thread have never seen before. You can write a single script in a full/normal language, no compilation step, and it works on systems that only have bash/sh. It doesn't compile to bash, or at least not in the way you might think/expect it to, but it should do what you want.

(guillotine because it's a universal executor) https://github.com/jeff-hykin/deno-guillotine

This^ one uses Deno/JavaScript, but in principle it might be possible to do with other languages. It definitely requires some explanation, so I'll try to give that here;

As another person said, shells are not nearly as standardized as we need them to be. Mac uses zsh, Ubuntu uses dash, neither store a posix bash exectuable in the same place, and both have ls and grep differences that are big enough to crash common scripts. Even if you're super strict on POSIX compliance, common things will still break if you write a big script (or trying to compile a big program to bash).

I hate JS as much as the next guy, but it is possible to write a single text file that is valid bash/dash/zsh/powershell and valid JavaScript all at the same time. It sounds impossible, but there is enough overlapping syntax that actually any javascript program can be converted into a valid bash script without mangling the JS code. It might be possible to do for python as well.

POSIX is good enough for making a small, carefully-crafted well-tested OS-detecting caveat-handling script. So that's exactly what we do; use a small shell script at the top to ensure that the JS runtime you want is installed (auto install if missing). Then the script executes itself again using the JS runtime. It wasn't easy but I a made a library that explains how it's possible and gives a cli tool that automates it for the Deno runtime (the link I posted above).

After that, I just recreated tools that feel like bash, but this time they are actually cross platform. Ex:

let argWithSpaces = "some thing"
run`echo hello ${argWithSpaces}`

I picked Deno because it auto installs libraries (imports directly from URL so users don't have to install anything)

[–] aport@programming.dev 43 points 2 years ago (2 children)

its possible to write a single text file that is valid bash/dash/zsh/powershell and valid JavaScript all at the same time. It sounds impossible, but there is enough overlapping syntax that actually any javascript program can be converted into a valid bash script without mangling the JS code.

I'm both impressed and horrified

[–] UlrikHD@programming.dev 24 points 2 years ago* (last edited 2 years ago) (1 children)

It'd honestly the funniest thing I've read on this instance. Puts programmer humour to shame. Love it when developers finds the jankiest/unconventional way to solve problems.

load more comments (1 replies)
load more comments (1 replies)
[–] keegomatic@kbin.social 15 points 2 years ago

Okay at first I was pretty convinced that this was just the wrong way to accomplish what I thought your goal was. But now, after reading the StackOverflow post and your README, I think this is fascinating and frankly really awesome. What a clever and strange thing, using multiline comments that way, and string no-ops. I think just knowing this exists will cause me to find reason to use it.

[–] onlinepersona@programming.dev 11 points 2 years ago (1 children)

Bro, make a video and put it up on peertube please then link it in the README. I need to see this shit in action. It sounds awesome, but it's 10am and my eyes are just opening, so reading through everything and testing it isn't happening on my phone rn.

[–] jeffhykin@lemm.ee 2 points 2 years ago (1 children)

Actually I've been thinking of starting a Youtube/Peertube channel for a while so this will be a good place for me to start!

I'll come back and post a response once I've uploaded it! It'll probably take a week or two.

load more comments (1 replies)
[–] snowe@programming.dev 10 points 2 years ago (1 children)

Not only do you solve the problem op is trying to solve, but you also made the most horrifying and hilariously ingenious thing at the same time.

[–] Natanael@slrpnk.net 2 points 2 years ago

Polyglot runtime lmao

load more comments (5 replies)
[–] aport@programming.dev 30 points 2 years ago* (last edited 2 years ago) (3 children)

Unfortunately shell script is not as portable as you might be anticipating. Different distro run different shells, with different settings, and also different tools. Think BSD grep vs GNU grep.

[–] scottmeme@sh.itjust.works 7 points 2 years ago* (last edited 2 years ago)

I've seen multi distro scripts that are also able to bootstrap their own assets for each distro/architecture. Don't see why you wouldn't be able to check that considering /etc/os-release exists in pretty much every unix like environment.

And having it run on a specific shell type could also be an option.

[–] BatmanAoD@programming.dev 2 points 2 years ago

I discovered a fun one the other day: there is literally no way to represent word-boundary anchors that's valid in both GNU sed and BSD sed. https://unix.stackexchange.com/a/393968/38050

[–] wgs 2 points 2 years ago* (last edited 2 years ago)

That's what the POSIX spec is for. BSD and GNU commands may differ, but they both support what's specified by POSIX. By limiting your calls to it, you can write portable script with no problem (I've been doing that for the last few years without issue).

[–] forked_bytes@lemm.ee 29 points 2 years ago

There is Batsh, a simple language with c-style syntax that compiles to bash and batch.

[–] monobot@lemmy.ml 26 points 2 years ago (4 children)

This thread reminds me of stackoverflow, most people are just convincing you in something else and it is obvious they have never been in your (and mine) situation.

Just answer question if you have some idea, yes we know python exist, that's nice, but not an answer to this question.

load more comments (4 replies)
[–] ShittyRedditWasBetter@lemmy.world 23 points 2 years ago (1 children)

Just use Python. If you absolutely need it on every system use go and compile it down to a binary.

load more comments (1 replies)
[–] dack@lemmy.world 12 points 2 years ago (1 children)

I don't see why bash would be used at all here. If you want something that doesn't need another interpreter, then just compile a binary.

[–] cyclohexane@lemmy.ml 7 points 2 years ago (3 children)

Possible use case: scripts that are found in a codebase for doing... Codebase things. Like setting up dev environment for example.

[–] dack@lemmy.world 4 points 2 years ago

But then building it still requires whatever scripting tool you use. Including the bash-ified version would not for practice, as it wouldn't be very human readable and would have to be kept in sync with the source script. It's much cleaner and simpler to just require python for your build environment.

[–] anzo@programming.dev 3 points 2 years ago

look into devenv.sh for that constrained use-case

[–] match@pawb.social 2 points 2 years ago

That's a use case for Docker

[–] mundane@feddit.nu 12 points 2 years ago (1 children)

Why not use a compiled language that compiles to fat binaries (rust, go etc)?

[–] Dark_Arc@social.packetloss.gg 4 points 2 years ago* (last edited 2 years ago)

It's worth noting any compiled language can make a "fat" binary (e.g., C++), you just need to use static linking.

[–] AI_toothbrush@lemmy.zip 11 points 2 years ago (1 children)

Giga brain move: make a bash script that compiles some c code then runs it

[–] nxdefiant@startrek.website 2 points 2 years ago

Still requires the compiler though.

I don't have a good answer, but I've embedded python in bash scripts before after having the bash script install python. Take that as you will.

[–] DrDeadCrash@programming.dev 8 points 2 years ago (1 children)

Not exactly what you asked for, but take a look at Nushell

[–] cyclohexane@lemmy.ml 4 points 2 years ago (1 children)

I'm familiar with NuShell and looks very nice. But unfortunately yeah, not what I'm looking for. It would require installation by user.

[–] maxbossing@feddit.de 1 points 2 years ago (1 children)

Install nushell vie bash and then pipe your commands into it?

load more comments (1 replies)
[–] sim642@lemm.ee 7 points 2 years ago (3 children)

Why not another scripting language (no compile necessary)?

But you're describing compiling that new language to bash...

[–] cyclohexane@lemmy.ml 4 points 2 years ago

Yes, I'm answering why I am not taking a completely other scripting language

In other words, I am making the case for a compiled language by answering the question of why i am not considering a language that doesn't need it.

load more comments (2 replies)
[–] EvanHahn@bigshoulders.city 5 points 2 years ago

https://github.com/tdenniston/bish is one such language.

I'd also recommend Shellcheck which helps prevent many problems with shell scripts.

[–] theherk@lemmy.world 4 points 2 years ago

Although it doesn’t crosspile to bash, I think a good middle ground is bitfield/script. Basically you can do many things you would normally script very simply with nice syntax and distribute a binary.

[–] vzq@lemmy.blahaj.zone 4 points 2 years ago

The way I used to solve this problem back in the day was with a statically compiled Perl binary.

[–] swordsmanluke@programming.dev 3 points 2 years ago

I'm gonna pile in with yet another option that isn't a language that compiles to bash...

Consider Ruby for easier shell scripting. With its back tick syntax for executing shell commands, it's quick to use when you want to glue together a series of commands. ...and then you get to use a sane syntax for your script's logic.

Ex:

# Check my history for all usages of the xsv command
# and extract the filename
csv_files = `grep ~/.zsh_history "xsv" | awk '{print $3}'`.
                     split("\n").
                     map(&:chomp)

# list any csv with my phone number
csv_files.select { | filename|
  `grep #{filename} "555-1234"`.chomp != ""
}.each {|filename| puts filename}
[–] cobra89@beehaw.org 2 points 2 years ago

There's always Golang which compiles to an x86 binary. But that's probably not what you want since you probably want something readable and editable on the end system.

[–] roo@lemmy.one 2 points 2 years ago (3 children)

Doesn't powershell do this? I've been learning powershell, and they keep making a tech agnostic claim along these lines, but I haven't tested it on Linux yet.

[–] Gurfaild@feddit.de 4 points 2 years ago

Not quite - even in PowerShell 7 there are some features that only work on Windows and Windows only comes with PowerShell 5.1 by default.

[–] alokir@lemmy.world 2 points 2 years ago

I used it on a Mac and on Windows, for me it feels very modern when compared to bash (although I never was a bash expert).

However, the problem is that it's not installed by default on Linux (at least on most distros as far as I know) and Mac, and Windows machines might have an outdated version which you'll have to take into account.

So unfortunately it doesn't meet OP's criteria that it should "just work" without installing an interpreter.

[–] manapropos@lemmy.basedcount.com 2 points 2 years ago

I haven’t used it, but this node package looks enticing

[–] waspentalive@beehaw.org 1 points 2 years ago

And when it is powerful enough you can make it self hosting.

load more comments
view more: next ›