Why "collapse" (not "rot") is the way to think about software problems

First of all, I guess I should say that I am the author of the article we are discussing.

Next, the article was written for an audience of computational scientists, whose computing environments and ways of working are probably quite different from yours.

The dominant OS is Linux (with its many variants), with macOS coming second and Windows far behind. Software is shared as source code, with dependencies typically listed either without any version reference or with a quote such as “2.3 or higher”. Also, practically all software is libraries, compiled ready-to-run executables are the exception. The top-level code that people run is scripts or increasingly Jypyter notebooks. Users regularly update their complete software stack to the latest version, though update frequency varies enormously, ranging from “weekly” to “whenever I get a new laptop”.

In that context, running every script in the exact environment that its author used is practically impossible given today’s state of the art. It would also be a management headache, given that a typical scientist has a tens to hundreds of scripts for more or less regular use.

This is a much-discussed problem these days because people have come to realize that computational science is not reproducible, meaning that claims cannot be checked. Many support tools are currently under development to restore reproducibility at least for common simple situations.

Windows executables or Java JAR files are very very different. The developer has full control over all dependencies at compilation time. But that’s not a viable deployment model for scientific computing.

5 Likes

I have never understood the COBOL hate.

It’s a language. Yes, it’s old, but it’s stable and it’s maintained; and you can get compilers for it for almost anything. (I mean, I am fairly sure my shoes have a COBOL compiler available. :wink: )

Poor code hacks have one language that they worship and they crap on older languages (older languages being defined as “languages that existed before they went to college”).

Good programmers don’t care what language the code is written in. Learning a new syntax takes a couple of weeks. Going to a new language family (like from a Procedural language like COBOL to an Object-Oriented language like Python) takes a little bit more, but even that isn’t a really big deal.

Hacks learn how to cobble together code in one language. Coders learn how to program and can do it in whatever language is needed.

3 Likes

Well. They are supposed to anyways.

You’d be surprised at how often configuration drift is present on build machines. And how often developers cannot provide a concise answer as to what needs to be present on build machines when (say) a CM team is provisioning a new one.

2 Likes

Yes and everyone hates both the process and the results. A lot of the bureaucracy in things like modern urban planning is exactly that. It is a system of keeping a complex web of dependencies intact in a fluid environment.

1 Like

python brings these important issues to the forefront

3 Likes

First off, thanks for taking the time to respond here!

I think I get what you are saying, in that you are addressing how a script that works with one environment may stop working when the environment itself changes. Evolves. Shifts. Whatever. Interfaces no longer work like the original code expected. Parts that once seemed reliable no longer fit.

Which is different from actual rot, physical bit rot, when the physical bits in the computer’s memory get lost or corrupted.

So yeah, we come from really different worlds, or at least I do from yours. I write front end code for web sites, and for the last five years have been working exclusively on GUI’s for web applications. And I have seen in recent years how dependencies on libraries in the package.json has exploded, and how it can feel like a Jenga tower. Something that might evoke similarities to rotted wood causing a structure to lose integrity, and collapse as you use it is not in the sense of a building collapsing, but an ecosystem?

2 Likes

Well, I actually do have issues with languages that don’t support halfway decent structured programming. Like, for instance, local variables in procedures/functions. Global everything is like wearing a sign that says, “Hurt me. Then do something worse.” Just begging for stuff to break at scales larger than a page or five.

Just remember: you can write FORTRAN programs in any language, as people prove every day.

3 Likes

You can use python code to interact with binary files, but it’s a great way to discover strange and interesting bugs.

2 Likes

The Case for not using software at all – Permanent backward compatibility.

light-switch-003

1 Like

it depends on what you mean by component.

it’s been literal decades since ive seen an entirely standalone application. ( even excluding the os. entirely independent app that’s also portable? only in extremely low level components. )

maybe people rely less on things like dlls and microsoft ole or com - but people, if anything, seem to rely more and more on third party libraries. ( png, zlib, video codecs, json parsers, physics engines, orm interfaces, jquery, an infinite list. )

and then there’s whole categories of software entirely built on the platforms of other software well above the os level. from games like unreal or unity; to services like aws, databases or cdns; to plugins for your browser; to the web itself: html5 and css ( talk about shifting sands. yikes. )

and even that’s not taking into account modern production pipelines. tools to build the app. not just code compilers, source control management, test harnesses, and build machines but also wikis, doc files, email and slack. photoshop or maya, the list goes on.

if something exists, the tendency where im from is to use it. there’s just no time to write it all yourself any more. everything is so bootstrapped into glittering towers of rvs im surprised somedays anything works at all.

( and please don’t get me started on how often apple deprecates ios apis. my spell check wanted to change “deprecate” to a bodily excretion, which is probably just about right. )

2 Likes

sorry, now i’ve got my pendant hat on.

unplugging is really the only way. even turning on a light switch is a software mediated experience. ( okay, maybe you need to turn on a few light switches. ) the power company can’t make ( much ) more power than what’s demanded because there’s no where to dump the excess. electric production has to be spun up and down, and adjusted minute by minute to keep things safe.

there are teams of machines and people working to balance production with demand every hour of the day. and software is key.

( i think in the us they’re called balancing authorities. )

1 Like

This clip is funny because he’s all worked up about tabs (like someone from the olden days), and then his final word is “one tab saves you eight spaces”, like people are writing code with eight spaces per indent lol

I know you see the irony there

I use tabs. Actual Tabs. I understand that you can have your editor send multiple spaces for every press of a tab key, but it’s more than not bothering to turn this feature on. Some of my python code comes from GUIs and those are hard coded to output ‘\t’ characters. In a sense, I’ve invested in tabs.

I’ve also run into situations where the python interpreters get confused when both tabs and spaces are used to indent code. Those are a real pain to debug because the relevant information is hidden from my eyes.

1 Like

My point was that they were trying to write “really nerdy coder” but then had him say that tab stops are eight characters wide, when of course a tab is equivalent to two spaces in code.

1 Like

Ooh, now I don’t like you at all, Mister Jerwin! No, sir, not at all! One of the reasons why I leave invisible characters on is because I keep seeing mixed tabs and spaces! All right, I do my work in JavaScript/TypesScript, HTML and (S)CSS, but also YAML, and yes, even PHP at times. Working with legacy stuff where there was one developer who was invested in tabs, and I could see tab tab space space space tab space in a file, because he insisted upon tabs and so years later his tabs still come up, and I clean it up, and risk losing his commit messages… which were worthless most of the times anyways, but it is annoying.

Actually, I am just kidding about not liking you. For literary purposes, you see. In reality I’d still buy you a beer, you happy mutant, you.

3 Likes

im like the bruce banner of pendants. the only way to hulk out is to always be hulking out. on the inside.

Thanks for writing the article. Interesting.

I’d say that the first and third points above are somewhat mitigated by using VMs. Once you can find or create a VM that supports the stack you need, you just snapshot it and use it forever. The second point, though, is truly horrific. Or, at least it has been in my experience. I’ve had to monkey patch old libraries that we couldn’t upgrade due to the age of the system using them.

This topic was automatically closed after 5 days. New replies are no longer allowed.