Behind the scenes, "plain" text editing is unbelievably complex and weird

Originally published at: https://boingboing.net/2019/10/29/backwards-in-heels.html

4 Likes

I’m editing text right now

19 Likes

This is why, as a designer or developer, you should try as hard as possible to use native widgets instead of implementing your own.

9 Likes

but don’t computer makers wrap up basic tools like this into the OS tool box for app programmers to access? Its not like every piece of software re-invents text editing…?

1 Like

I’m a huge fan of plain text and base as many workflows as I can on it. But it’s shocking how many sophisticated tools that are supposed to make our lives easier struggle with the simplest file format in existence. It should be trivial to dump plain text into a Word template, but it’s not, at least not if the template is more than minimally formatted (though pandoc is getting increasingly better at that).

2 Likes

It always amazes me how badly my rtf resumes get mangled by the semantic recognition scripts when I’m submitting for jobs.

They should either ask for a finished pdf resume to display to the recruiter, or to fill out the forms. But asking for a resume, then filling in the forms automatically from that, then asking the applicant to fill in a blank copy of the forms is just stupid.

3 Likes

It’s not standard in Linux, I know little of Windows.

Decades ago I did play with some code to avoid word wrapping. As in I thought about it and worked at implementing it. It struck me as being more complicated than how it appears on screen. Even a full screen editor was hard, trying to visualize how to do it until I ran a text editor while printing something, which slowed things down so much that I saw the cursor move slowly around the screen. What you see hides what goes on under the hood.
Nowadays it’s long been done, and ooen source means the code us out there, so it’s easy to find such things as a foundation if yiu really think yet another text editor is needed.

I’d like to add a leap key like in Raskin’s Swyftcard.

This is one of the reasons why Linux-ported applications look so different than native Windows applications; Windows does provide tools for input and user interface, and there are standards for appearance and functionality that Microsoft highly recommends for common behavior across applications from different vendors. Obviously, Linux application code bases do not follow the Microsoft suggestions religiously (why would they?); and the nature of the OS tends to attract people who are more interested in re-inventing the wheel.

Generally speaking, on the MS platforms, the larger the corporation and the more corporate the application’s user base, the closer to the Microsoft Standards the product will probably stay. For some reason, smaller companies and companies that make more consumer products have a tendency to try to make things “simpler”, which makes things harder. (If you have to accomplish the same thing in 15 different programs (like save a file), it’s better to have one way of doing it that works for all 15 programs, even if you think you can do it a little bit better, than to have 15 different ways of doing it, each one a bit different than the others.)

And yep, I know there is a standard for interfaces in Linux, it could just be my limited experience that suggests that it isn’t followed as closely; since my experiences are mostly with consumer Linux applications.

(The most annoying thing to me is the order of the buttons on pop-ups. Windows uses one order for the buttons, Mac uses the other, and Linux seems to float between the two with a weak preference to Mac order.)

4 Likes

Some fun esoteric details about text editors are the internal algorithms on how to represent and operate on the data in memory. The buffer gap algorithm is one of the famous data structures for this and while a textbook analysis of the algorithmic complexity would imply it is not an ideal, it tends to fit the usage patterns of text editors (mostly you edit for a while in one area either appending or inserting text a character at a time).

I strongly recommend borrowing the book The Craft of Text Editing by Craig A. Finseth for some very old school details of how a traditional text editor works inside. (the book is now available as PDF or print-on-demand on Lulu. I bought my copy from a library sale many years ago)

4 Likes

I took a graduate course in linguistics (for the fun of it! lol) and realized early on how difficult language is to parse. The brain makes wonderful work of it, but trying to parse a complex sentence is impossible without nuance and context…

2 Likes

GTK, Qt, and EFL provide full featured text input boxes. I think GTK even supports emacs keybindings.

1 Like

Safari on Apple products – at least at some point – relied on its own native text rendering libraries. I’m fairly certain that is still the case (as supporting multiple text rendering engines would use up a lot of space on an iPhone).

The main reason that the other browsers roll their own text rendering is in order to provide consistent rendering across all supported platforms.

1 Like

These problems all existed before. The proliferation of “code page” character encodings for different scripts meant that they manifested in a variety of different ways. Unicode has at least managed to gather together all the complications within one standard.

The linked video, by Jonathan Blow, was interesting. I find the argument more compelling when it comes to how hardware interfaces work (especially video hardware), the complexity of software development environments and frameworks, and such things. With text editing, there’s irreducible complexity coming at us from the immense realm of human language. Programming this stuff is just not going to be fun.

Plain old text editing is still easy - the issue (as in the callout) is that nobody uses plain old text any more and even ‘simple’ texting now requires various forms of markup and context. Like Z̤ḁ̻̖̼̤̌̎ͧ̄͋lg͓̦̈́̅o̱̰͔̹͍̓̊ͥ̎̚ ͚̺ͫ̍T̝̩͍̻̼̭e͈ͫx̞̎t̥̤̘̱̤̙͔ͩ͑ͪ́̒̃̎, which suddenly involves suffix modifiers, and then the color suffix modifiers for emoji.

Back in the Old Days we could just show you the raw formatting codes and all ambiguity would be eliminated - remember Word Perfect, WordStar, or Sprint? That would never fly now, so you have to guess user intent and that will never end well. All you can do is make things work well enough 99% of the time, and that’s more than people expect on the internet.

2 Likes

You shouldn’t be amazed

Those who can, do
Those who can’t, teach

Those who can’t even teach, recruit

(note: I lubs teachers and the teaching industry. Despite the doggerel above, I do NOT view it as a second best option … I was using that old joke to make a point about recruiters, not teachers.)

To be fair, implementing a text editor is probably a pretty good project to work on as you teach youself coding. Does the world need another text editor? No, clearly not. Will I learn a shitload about coding by trying to write my own editor? Yes, of course. Will ‘my’ editor have features that I am the first to think would be useful in a text editor? Yeah, probably.

2 Likes

Yes, for learning. When I played with word wrapping decades ago, I had no end purpose, just wanted to explore, when I was doing a lot of that. Someone mentioned the matter of inserting text when there’s no space. That reminded me of a book I did have that explained how to do a lot,including that problem. Sadly, I remember none of the details, though I know where the book is. This is 6502-era.

I entirely agree.

Anybody remember Turbo Pascal, from back around the mid-80s? One of the coolest things that Borland did when they released it was to include sample source code for a simple text editor, comparable to what was in their IDE or their “Sidekick” accessory program.

I’d been programming less than 10 years at that point, and had always thought editors must be incredibly complicated and difficult to write. Reading their source code gave me that “Aha!” moment where I realized that really there was a very conceptually simple control loop - read incoming keys, identify commands vs. text input, apply command or insert text, refresh screen - and yes, I could really write something like that myself.

Yes, this was all ASCII text, pre-Unicode, a vastly simpler case than what’s under discussion here, but IMO it’s still a very valuable thing to do to grasp the concepts.

2 Likes

Remember this fine old chestnut?

The other popular string is “this app can break”. Can we update this for the modern age? “Toss the bad Trump”?

2 Likes

Clearly a reason to embrace the simple purity and old-fashioned values of an 80 column terminal with a (singular) monospaced bitmapped font that understands that there are really only 128 characters, some of them not even printable.

What values are we raising our children with when they are being indoctrinated that a zero width joiner is not only an acceptable form of relationship; but isn’t just for decent characters. We’ll have emoji zero width joining with dogs!

(I sincerely hope the sarcasm is evident)

2 Likes