In Jason Pargin’s I’m Starting To Worry About This Black Box Of Doom, a manic pixie dream girl cajoles a shut-in incel loser to drive her and her mysterious box cross-country. The further they drive, the more evidence starts to build that she is a terrorist and her box is a nuke. As our protagonist becomes increasingly desperate to turn around and return to his comfortable world of social media feeds and psych meds, she pleads with him to come out of his shell, learn to trust people offline, and have a sense of adventure. The book’s dramatic tension comes from our simultaneously rooting for his character development and worrying that it might be a ruse to manipulate him into blowing up Washington, DC.

This book is not shy about its moral, delivered in approximately one soliloquy per state by our author mouthpiece character (the girl). Although there is a literal black box of doom - the suspected nuke - the real danger is the metaphorical “black box” of Internet algorithms, which make us waste our lives “doom” scrolling instead of connecting to other human beings. Or the “black box” of fear that the algorithms trap us in, where we feel like the world is “doomed” and there’s nothing we can do. She urges us to break out of our boxes and feel optimism about the state of society. Quote below, Ether is the girl, Abbott is the loser, and he’s just ventured the opinion that it’s unethical to have children in a world as doomed and dystopian as ours:

“My grandfather,” continued Ether, “who I basically never talk to anymore, one hundred percent believes Christ is going to return to earth at any minute to bring about the apocalypse, due to mankind’s sinfulness. He believes everything he watches on the news is a sign: encroaching Communism, the Satanic conspiracy to allow gays to marry, race-mixing, debauchery, pornography, drag queens, the QAnon child sex cult, the climate change ‘hoax’ he says has fooled the world. He has a TV on every minute he’s awake, tuned to these ultra-right-wing news outlets ranting about depravity.”

“I know old guys like that,” said Abbott. “My dad works with a couple. They’re nuts. You can’t even talk to them.”

“So we can agree that, purely via the carefully filtered media a person consumes, they can come to fully believe in an apocalypse that is not, in fact, occurring?”

“I mean, the world is on fire, just not in the way your grandpa thinks.”

“Are you one hundred percent sure, Abbott, that you haven’t fallen into the exact same trap, just from the other side?”

“Ah, you’re about to tell me climate change isn’t real.”

“I am not. I’ve seen the melting ice with my own two eyes. But let me ask you this: When I met you, I asked if you felt like you were cursed to be born when you were, if you felt like you had arrived just in time to see the world end. So I’m guessing that you think the world is collapsing because of the feminization of society, something like that? That we’re killing masculinity?”

“I mean, that’s definitely part of it. Men are scared to date; no babies are being made.”

“Okay, and in my corner of the internet, the harbingers of doom were the opposite: savage patriarchal governments crushing women’s rights, taking us back to the dark ages while overpopulation destroys the environment. So that’s two groups who both believe the world is ending, but for totally opposite reasons. Some say runaway capitalism, some say runaway socialism. Some say it’ll be chaotic lawlessness, some say iron-fisted authoritarianism. It’s like I have one panicked neighbor saying there’s an impending drought and another screaming that we’re all about to drown in a flood. Somebody has to be wrong.”

“That wouldn’t make them both wrong.”

Ether groaned and put her head in her hands. “Okay,” she said, trying again. “How about this: What do you think the world will look like in the future, post-collapse?”

Abbott thought for a moment as if picturing it. “Uh, terrified people scrounging for food and running from bandits. Rampant disease, infrastructure breakdown. All the stuff from the movies, I guess.”

“No internet?”

“I wouldn’t think so.”

“No electricity? No running water, no sewage? No hospitals?”

“Probably not.”

“Got it. So, what I’m about to say isn’t an opinion, it’s not a matter of personal philosophy or politics. It is an objective fact that what you’re describing is how virtually all humans have lived through all of history. Until, that is, about thirty years ago. Just in the time I’ve been alive, somewhere between two and a half and three billion people got their first access to clean water and toilets. That’s billion, with a B. About that same number got electricity in their homes for the first time in their lives. Worldwide, infant mortality has been cut in half, illiteracy has dropped almost as much. Suicides are going up here in the US, but worldwide, they’ve dropped by a third—again, that’s all just in my lifetime. Basically, every positive category has skyrocketed: access to communication, paved roads, motorized transportation, international travel, climate control, medicine…”

“Okay, it sounds like you’re talking about a bunch of good stuff that happened in China and India and—I don’t know. A bunch of poor countries I’ll never visit.”

“I’m talking about how your entire life span has been spent in a literal reverse apocalypse. I’m talking about billions of people who lived in what you would consider post-collapse conditions have had those conditions remedied, gaining roofs and lights and safety. A human’s chances of dying from famine or natural disasters are as low as they’ve ever been, ever, in the history of the species. It’s been nothing short of a worldwide miracle that makes everything Jesus supposedly did in the Bible look like party tricks. And people like you and me and others in our demographic describe that state of affairs as the world being ‘on fire.’ I think that’s a bizarre mass delusion and that there’s a very specific reason for it: we’ve been trained to cling to a miserable view of the world to the point that we think that not seeing the world as miserable makes us bad people. When I spent those months doing hallucinogens, I didn’t suddenly see the beauty and harmony of nature; I saw that humans everywhere were working really hard to make life better for other humans and that almost none of us appreciate it. I’m not crediting this miracle to capitalism or socialism or any other kind of ism but to the fact that it’s what humans do, because humans are amazing. And it’s all invisible to us because the progress occurs behind these dark walls of cynicism, outside the black box of doom.”

“That’s nice. And again, nothing you said means anything considering the world’s scientists have agreed that climate change will wipe out civilization.”

“If we don’t fix it, yeah. Climate change is a huge deal; it’s terrifying. And also, it is objectively true that if we do fix it, the media will only report it as bad news. All the headlines will be about the oil and coal workers who lost their jobs, birds dying to windmills — they’ll only focus on the negative side effects. And don’t tell me we never clean up our messes. There used to be oil slicks on our rivers that would literally catch fire. Sulfur dioxide used to choke the air — when’s the last time you’ve heard about acid rain? Or the hole in the ozone layer? Go read about how previous generations all had lead poisoning or how food contamination used to be a nightmare. I’m not saying everything will be fine; I can’t predict the future. I’m saying that it is a one hundred percent certifiable guaranteed fact that it can be fine. But people like us have decided that we’re never allowed to even acknowledge the possibility.”

“Or maybe it’s hard for people to care about toilets in India when another maniac is shooting up a school every week.”

“You think that happens every week?”

“I bet you have a whole bunch of stats to dump on me about that, too. I’m sure the parents of those dead kids would love to hear them.”

“And there’s the anger. People hate it when you threaten their nihilism! That’s the black box, drawing you back in. Can’t you see that it wants you to be afraid to do anything but cower in front of your screens? It only has one trick, one card to play, which is this idea that bad news is the only news you can trust. I’m telling you, if you just allow yourself to step outside of it, you’ll see it for what it is: a prison where the walls are made of nightmares.”

Here’s another of her descriptions of the Black Box as she understands it. Phil is her mentor, Cammy a random friend:

“Social media algorithms are a twenty-four-seven humiliation machine. That, Phil believed, is how a population is primed for authoritarian rule. And that’s just one example; we’re essentially teaching machines how to hack human insecurity...If you relentlessly attack people’s self-image, they’ll scramble for something, anything to preserve it. Every cultural faction has their own scapegoats—the government, their childhood trauma, their mental illness, the evil billionaires, immigrants — and it doesn’t matter the degree to which any of them are valid, because all the system cares about is that you surrender your own agency. ‘I cannot be blamed for the state of my life, because I am at the mercy of this other, more powerful thing.’ Phil’s theory is that people want that powerful thing to exist, to take over their lives. At that point, we will have finally surrendered the entire concept of free will, the one thing that makes us human.”

“So that will make them vote for a dictator?” asked Cammy. “I think they’re already doing that now.”

“It will, and they are. But Phil didn’t think even that would be enough. What the people want is a cruel, all-powerful being that they can simultaneously obey and also endlessly complain about...Look around you. How many people out there are addicted to internet gambling, or games, or porn, or outrage headlines they compulsively click and share? See, [it works] on the back end, too, dialing in on exactly what pixels on a screen will subdue the human animal. And we go along willingly because we want to be subdued. The whole appeal of being in a media-induced flow state is that you block everything else out. We want to be zombies. Puppets.”

Here’s an uncharitable summary of the theses of these two sections:

1. The Black Box has scared us into believing that everything is dystopian and getting worse. The bad news we are so relentlessly exposed to is trapping us in a prison made of our own pessimism and fear.

2. The Black Box is destroying everything that makes us human, causing our society to spiral into dictatorship, and turning us into zombies/puppets.

Or, to be even less charitable:

1. We must reject doomerism, where we treat the problems of today as unprecedented crises that risk destroying us.

2. …except for the problem of doomerism, which really is an unprecedented crisis that risks destroying us. You cannot possibly imagine how bad this one is, and we must treat it as an absolute emergency which requires us to uproot everything about our lives.

I’m not attacking Pargin for this. His book is great, and it’s the prerogative of great artists that we treat any apparent contradiction in their works as grist for the mill - if not done intentionally to provoke us, then at least enacted through some trickster urge of the subconscious. But Black Box Of Doom is hardly the only place where we find this contradiction.

Peter Thiel recently gave a lecture on the End Times, described as “portraying the Antichrist as a technocratic leader exploiting fears of catastrophe to impose global control.” Thiel suggested that maybe the Antichrist would use worries about global warming, or inequality, or AI safety, to frighten people into accepting some kind of evil surveillance state. His moral was that we need to stop living in fear of people’s scare stories.

But isn’t the idea that if we try to regulate things, it will summon the literal Antichrist and plunge the world into eternal darkness, kind of a scare story? Isn’t Thiel using this scare story to frighten people into accepting the, uh, evil surveillance state he’s enabling? Thiel seems to have the same blind spot as Pargin’s characters - you need to stop letting scary stories ruin your life, except the scary story about how scary stories can ruin your life, which you should let ruin your life as quickly and decisively as possible.

Tyler Cowen had a recent post China Understands Emotional Contagion, on China’s policy of censoring negative speech online - “punishing bloggers and influencers whose weary posts are resonating widely in a country where optimism is fraying”. He seemed oddly enthusiastic about this - no condemnation, just “If you are spreading negative emotional contagion, there is a very good chance that, no matter what you are saying, that you are part of the problem.”

But isn’t the idea of an epidemic of negative emotional contagion, bringing in its wake collapsing state capacity and stagnant economies, and so threatening that we must arguably suspend our usual liberal values in order to crush it before it spreads - itself a form of negative emotional contagion? If China banned criticism of climate projections, because global warming was too much of an emergency to allow debate or dissent, wouldn’t that be a classic example of doomerism gone too far?

In Internet slang, the opposite of a doomer is a “bloomer”. I recently got a chance to talk to the bloomers at the Progress Studies conference. They were great and I learned a lot. But as far as I could tell, the semi-official philosophy was “We need to be forward-looking rather than obsessed with some mythical better past - you know, like we were in the good old days of the 1920s, back when society could actually accomplish things.”

None of this is logically contradictory. This is a real way the world could be: all crises are overreactions, except the crisis of overreaction to fake crises, which is worse than you can possibly imagine. The present is better than the past in every way, except that the past got the question of is-the-present-is-better-than-the-past right and the present doesn’t. Totally possible, nothing says it can’t happen.

But would the bloomers be equally charitable to other people making this claim for other pet causes? Some would: many are smart people. For the rest, this situation should provide a lesson in humility. A strong view of the “crisis of doomerism” is incompatible with a worldview in which strong crises are impossible, or should never be mentioned because the overreaction to them will always be worse than the crisis itself, or must always be the tool of sinister interests trying to divide us. Rather, it forces us back to the normal position where optimism is a heuristic and nothing more: some crises will be overblown, and we may want a slight bias against taking them seriously, but this bias can yield to evidence like anything else.

And how strong is the evidence for the “crisis of doomerism”? Nobody has proven its existence with a p < 0.05 study. There is no universal scientific consensus on its existence. And there is no shortage of stories about how bad people might be using it to accumulate power (I’ve given you Thiel and China for free). So whatever evidentiary bar bloomers set for “a real crisis” cannot hold these as absolute demands.

My own view is that we have many problems - some even rising to the level of crisis - but none are yet so completely unsolvable that we should hate society and our own lives and spiral into permanent despair. We should have a medium-high but not unachievable bar for trying to solve these problems through study, activism and regulation (especially regulation grounded in good economics like the theory of externalities), and a very high, barely-achievable-except-in-emergencies bar for trying to solve them through censorship and accusing people of being the Antichrist. The problem of excessive doomerism is one bird in this flock, and deserves no special treatment.

Yesterday, the NY Times published an article about Donald Trump’s threat to take away citizenship from a US-born citizen: Trump threatens to strip Rosie O’Donnell of U.S. citizenship. The Times Bluesky account posted a link to the article accompanied by this text:

President Trump said on Saturday he was considering revoking Rosie O’Donnell’s U.S. citizenship. Trump has feuded with the comedian and actress since before he became president. Experts said the president does not have the power to take away the citizenship of a U.S.-born citizen.

Large media companies, and the NY Times in particular these days, like to use the phrase “experts said” instead of simply stating facts. The thing is, many other statements of plain truth in that brief Times post lack the confirmation of expertise. To aid the paper in steering their readers away from notions of objective truth, here’s a suggested rewrite of that Bluesky post:

Donald Trump, who experts said is president of the United States, which experts said is a sovereign state on the planet Earth, which experts said is an oblate spheroid and revolves around the Sun, which experts said is a G-type main-sequence star about 93 million miles from us, said on what experts said was Saturday that he was considering revoking (which experts said is a process of making invalid) the U.S. citizenship of a person with the last name of O’Donnell, who experts said is a living human person and U.S. citizen with the first name of Rosie (which experts said is a diminutive of Roseann). Trump, who experts said has feuded with the person who experts said is a comedian and actress since, experts said, before he became president (again, experts said this, that Trump is the president and that also there exists a time (which experts said i— {Do we really need to cite someone on the concept of time here? Surely, time is just time and everyone kinda sorta gets that? -ed}) before he was president). Experts said the president, who experts said doesn’t simply float away into the cosmos because of the mutually attractive force of gravity between him and the Earth, does not have the power (which experts said is whatever Robert Caro said it was in that heavy book about Robert Moses; the experts honestly did not make it through the whole thing) to take away the citizenship of a U.S.-born citizen.

Or, I guess you could do it the easy way:

President Trump said on Saturday he was considering revoking Rosie O’Donnell’s U.S. citizenship. Trump has feuded with the comedian and actress since before he became president. Experts said The president does not have the power to take away the citizenship of a U.S.-born citizen.

Thank you for your attention to this matter!

PS. Rolling Stone did waaay better with this story: Trump Thinks He Can Take Away Citizenship From Anyone He Doesn’t Like. And see also the NY Times Pitchbot if you are unaware of its existence.

PPS. The image is a political cartoon from 1894 — you can see the full version at the Library of Congress.

Tags: Donald Trump · journalism · NY Times · politics · Rosie O’Donnell

The typical neoliberal defense of self-made billionaires goes: entrepreneurs and other businesspeople create a lot of value. EG an entrepreneur who invents/produces/markets a better car has helped people get where they’re going faster, more safely, with less pollution, etc. People value that some amount, represented by them being willing to spend money on the car. The entrepreneur should get to keep some of that value, both because it’s only fair, and because it incentivizes people to keep creating value in the future.

How much should they keep? The usual answer is that the surplus gets distributed between the company and the customers. So suppose that this new type of car makes the world $200 billion better off. We could have the company charge exactly the same price as the old car, in which case customers get a better car for free. We could have the company charge enough extra to make a $200 billion profit, in which case customers are no better off than before (they have a bit less money, and a bit better car). Or they could split it down the middle, and customers would end up better off than before and the company would make some money. Which of these distributions happens depends on competition; if there’s no competition, the company will be able to take the whole surplus; if there’s a lot of competition, all the companies will compete to lower prices until they’ve handed most of the surplus to the customers. Then once the company has some portion of the surplus, it divides it among capital and labor in an abstractly similar way, although with lots of extra complications based on whether the labor is unionized, etc.

This seems to me pretty hard to argue with - if someone creates a surplus, who doesn’t want them getting to keep some large fraction of it as a reward?

Most arguments against involve a claim that the company’s capitalists must be oppressing labor. Regardless of whether capital generally oppresses labor, this doesn’t seem like a strong argument against billionaires getting rich off of innovation. Suppose the old mediocre car company paid its workers $50,000 per year. Now someone invents a new better car company, and its workers do the same job as the workers at the old car company (ie their advantage isn’t more skilled workers, it’s equally-skilled workers making a better-designed car). It seems pretty fair to also pay their workers $50,000, which means that the big surplus created by the better car should mostly go to the capitalists.

(as an intuition pump, if Google and Bob’s Tools produce the same amount of value per employee in 2000, and the janitors at both get paid the same, and then in 2020 Google produces 1,000x more value per employee, should a janitor at Google get paid 1,000x the amount?)

But here’s an anti-billionaire argument that makes more sense to me.

Suppose Amazon creates $1 trillion in extra value for the world, it gets split 50-50 with consumers, Amazon makes $500 billion, that gets split 50-50 with labor and other stockholders, and Jeff Bezos ends up with $250 billion. The standard argument would say that this is fair compensation for the $1 trillion Amazon provided to the world.

But suppose that we go back in time and prevent Jeff Bezos from ever being born. Does this mean Amazon wouldn’t exist today? Probably not by that name. But does it mean that we wouldn’t be buying things online today? That we would have to walk to the brick-and-mortar store every time we wanted a book? Does it mean that Internet retail would be split across a hundred different storefronts, none of which had a good selection or was easy to use?

It might mean that. But I think “Internet retail giant which dominates the market through economies of scale” is a natural niche which Jeff Bezos won the race to fill. When I say “natural niche”, I don’t want to discount Bezos’ accomplishment - I certainly didn’t notice that niche in 1994, and even if I had, I wouldn’t have had the business acumen to fill it effectively. I just mean that, probably sometime between 1994 and today, someone with business acumen would have noticed that niche and filled it successfully. Maybe not quite as successfully as Bezos. But successfully.

(It’s like that old joke about how if Thomas Edison had never existed, we’d all be browsing the Internet in the dark; no, we would have waited another few years, and then some other genius would have invented electric light.)

Suppose that if Jeff Bezos had never existed, someone would have founded pseudo-Amazon two years later. That means Bezos gets credit for Amazon being two years more advanced than it otherwise would have been. That’s actually still worth quite a lot of surplus value - maybe still enough to make him a billionaire many times over! But probably not enough for him to have $200 billion or however much he has right now.

The problem with the neoliberal argument is that it gives the first person to fill a niche credit for the niche’s entire existence, not just for filling it earlier than it otherwise would have been filled. Just because Jeff Bezos solved Internet retail two years earlier than the person who would have done it if he was never born, he gets to collect rent on all transactions forever, while that other guy gets nothing.

(as an intuition pump, consider an archaeologist who spends years narrowing down the location of a pirate treasure to a beach in Florida, then digs there and finds it. Whatever the finders-keepers law, we would feel like in some sense she had earned the treasure. But consider another world where God announces on 1/1/2023 “THERE IS A TREASURE ON THIS BEACH IN FLORIDA!”, and everyone rushes there, and whoever ran the fastest got it. Again, regardless of the actual finders-keepers law, we would feel like that person hadn’t really earned the treasure, just beaten the next person by a couple of seconds. Or maybe there are some weak qualifications - you have to be a strong person who owns a shovel - but there are many strong people with shovels and it’s mostly luck that one of them was closer than another. )

This . . . maybe still isn’t a good argument to tax Jeff Bezos or distribute his money to his employees? Taken seriously, it implies that the only person who Bezos has “stolen” any money from is the second-best entrepreneur.

But that isn’t exactly right. Suppose that if Bezos hadn’t existed, the next potential Amazon founder would be two years later, the next founder one year after that, the next founder six months after that, and so on (we assume Amazon becomes more obvious and easier to found as the Internet revolution continues). A few years after Amazon was founded, the number of counterfactual founders reaches the thousands - at some point the tech space is so saturated, and the idea of founding a big store is so obvious, that each counterfactual-founder is only shaving days or hours off of the next one’s time.

So one way of thinking about this is that of the $250 billion portion of the surplus that the system has allocated to Bezos, Bezos “deserves” two years’ worth, the second-best entrepreneur “deserves” one years’ worth, and then a whole bunch of other people “deserve” a few dollars. But this doesn’t really make sense, because all the other entrepreneurs didn’t (non-counterfactually) do any work.

(you could argue that Bezos is being paid not just for his idea, but for the hard work of serving as CEO of Amazon over many years. I’m skeptical of this because it seems like the fair wage for that would be the going rate for top-tier CEOs, which is very high but not hundreds of billions of dollars. So fine, subtract that small amount out, but the rest still seems to be some sense of paying Bezos for his idea.)

But another way of thinking about this might be to imagine these people in some kind of abstract counterfactual competition with Bezos; in a world where they were all able to found their giant retail sites (because there was no natural monopoly), they would have driven down Amazon prices and given more of the surplus to consumers - they might also have competed for workers and driven wages up. So in this sense, maybe Bezos’ extra portion of the surplus is in some sense coming from ordinary people. This would be surprising, because Amazon already has very low prices. But given that Jeff Bezos has $200 billion, there must be some world in which $200 billion more could have gone to consumers through lower prices.

I don’t know what a fair economic system that takes this model into account would look like. It wouldn’t look like “billionaires shouldn’t exist”, because it might be that pushing Amazon forward by two years is indeed worth several billion dollars. It wouldn’t even look like “higher taxes on all billionaires in the hopes of having them have less money and maybe that will bring them closer to the fair level”, because plausibly some billionaires aren’t replaceable - I’m not sure anyone else would have started SpaceX if Musk hadn’t. But this model convinces me that “taxing billionaires a lot” and “taxing billionaires not at all” are at least two different unfair failure modes with their own advantages and disadvantages from a desert point of view.

I’ve been confused about what’s going on with terminals for a long time.

But this past week I was using xterm.js to display an interactive terminal in a browser and I finally thought to ask a pretty basic question: when you press a key on your keyboard in a terminal (like Delete, or Escape, or a), which bytes get sent?

As usual we’ll answer that question by doing some experiments and seeing what happens :)

remote terminals are very old technology

First, I want to say that displaying a terminal in the browser with xterm.js might seem like a New Thing, but it’s really not. In the 70s, computers were expensive. So many employees at an institution would share a single computer, and each person could have their own “terminal” to that computer.

For example, here’s a photo of a VT100 terminal from the 70s or 80s. This looks like it could be a computer (it’s kind of big!), but it’s not – it just displays whatever information the actual computer sends it.

Of course, in the 70s they didn’t use websockets for this, but the information being sent back and forth is more or less the same as it was then.

(the terminal in that photo is from the Living Computer Museum in Seattle which I got to visit once and write FizzBuzz in ed on a very old Unix system, so it’s possible that I’ve actually used that machine or one of its siblings! I really hope the Living Computer Museum opens again, it’s very cool to get to play with old computers.)

what information gets sent?

It’s obvious that if you want to connect to a remote computer (with ssh or using xterm.js and a websocket, or anything else), then some information needs to be sent between the client and the server.

Specifically:

• the client needs to send the keystrokes that the user typed in (like ls -l)
• the server needs to tell the client what to display on the screen

Let’s look at a real program that’s running a remote terminal in a browser and see what information gets sent back and forth!

we’ll use goterm to experiment

I found this tiny program on GitHub called goterm that runs a Go server that lets you interact with a terminal in the browser using xterm.js. This program is very insecure but it’s simple and great for learning.

I forked it to make it work with the latest xterm.js, since it was last updated 6 years ago. Then I added some logging statements to print out every time bytes are sent/received over the websocket.

Let’s look at sent and received during a few different terminal interactions!

example: ls

First, let’s run ls. Here’s what I see on the xterm.js terminal:

bork@kiwi:/play$ ls
file
bork@kiwi:/play$

and here’s what gets sent and received: (in my code, I log sent: [bytes] every time the client sends bytes and recv: [bytes] every time it receives bytes from the server)

sent: "l"
recv: "l"
sent: "s"
recv: "s"
sent: "\r"
recv: "\r\n\x1b[?2004l\r"
recv: "file\r\n"
recv: "\x1b[?2004hbork@kiwi:/play$ "

I noticed 3 things in this output:

1. Echoing: The client sends l and then immediately receives an l sent back. I guess the idea here is that the client is really dumb – it doesn’t know that when I type an l, I want an l to be echoed back to the screen. It has to be told explicitly by the server process to display it.
2. The newline: when I press enter, it sends a \r (carriage return) symbol and not a \n (newline)
3. Escape sequences: \x1b is the ASCII escape character, so \x1b[?2004h is telling the terminal to display something or other. I think this is a colour sequence but I’m not sure. We’ll talk a little more about escape sequences later.

Okay, now let’s do something slightly more complicated.

example: Ctrl+C

Next, let’s see what happens when we interrupt a process with Ctrl+C. Here’s what I see in my terminal:

bork@kiwi:/play$ cat
^C
bork@kiwi:/play$

And here’s what the client sends and receives.

sent: "c"
recv: "c"
sent: "a"
recv: "a"
sent: "t"
recv: "t"
sent: "\r"
recv: "\r\n\x1b[?2004l\r"
sent: "\x03"
recv: "^C"
recv: "\r\n"
recv: "\x1b[?2004h"
recv: "bork@kiwi:/play$ "

When I press Ctrl+C, the client sends \x03. If I look up an ASCII table, \x03 is “End of Text”, which seems reasonable. I thought this was really cool because I’ve always been a bit confused about how Ctrl+C works – it’s good to know that it’s just sending an \x03 character.

I believe the reason cat gets interrupted when we press Ctrl+C is that the Linux kernel on the server side receives this \x03 character, recognizes that it means “interrupt”, and then sends a SIGINT to the process that owns the pseudoterminal’s process group. So it’s handled in the kernel and not in userspace.

example: Ctrl+D

Let’s try the exact same thing, except with Ctrl+D. Here’s what I see in my terminal:

bork@kiwi:/play$ cat
bork@kiwi:/play$

And here’s what gets sent and received:

sent: "c"
recv: "c"
sent: "a"
recv: "a"
sent: "t"
recv: "t"
sent: "\r"
recv: "\r\n\x1b[?2004l\r"
sent: "\x04"
recv: "\x1b[?2004h"
recv: "bork@kiwi:/play$ "

It’s very similar to Ctrl+C, except that \x04 gets sent instead of \x03. Cool! \x04 corresponds to ASCII “End of Transmission”.

what about Ctrl + another letter?

Next I got curious about – if I send Ctrl+e, what byte gets sent?

It turns out that it’s literally just the number of that letter in the alphabet, like this:

• Ctrl+a => 1
• Ctrl+b => 2
• Ctrl+c => 3
• Ctrl+d => 4
• …
• Ctrl+z => 26

Also, Ctrl+Shift+b does the exact same thing as Ctrl+b (it writes 0x2).

What about other keys on the keyboard? Here’s what they map to:

• Tab -> 0x9 (same as Ctrl+I, since I is the 9th letter)
• Escape -> \x1b
  
• Backspace -> \x7f
• Home -> \x1b[H
• End: \x1b[F
• Print Screen: \x1b\x5b\x31\x3b\x35\x41
• Insert: \x1b\x5b\x32\x7e
• Delete -> \x1b\x5b\x33\x7e
• My Meta key does nothing at all

What about Alt? From my experimenting (and some Googling), it seems like Alt is literally the same as “Escape”, except that pressing Alt by itself doesn’t send any characters to the terminal and pressing Escape by itself does. So:

• alt + d => \x1bd (and the same for every other letter)
• alt + shift + d => \x1bD (and the same for every other letter)
• etcetera

Let’s look at one more example!

example: nano

Here’s what gets sent and received when I run the text editor nano:

recv: "\r\x1b[Kbork@kiwi:/play$ "
sent: "n" [[]byte{0x6e}]
recv: "n"
sent: "a" [[]byte{0x61}]
recv: "a"
sent: "n" [[]byte{0x6e}]
recv: "n"
sent: "o" [[]byte{0x6f}]
recv: "o"
sent: "\r" [[]byte{0xd}]
recv: "\r\n\x1b[?2004l\r"
recv: "\x1b[?2004h"
recv: "\x1b[?1049h\x1b[22;0;0t\x1b[1;16r\x1b(B\x1b[m\x1b[4l\x1b[?7h\x1b[39;49m\x1b[?1h\x1b=\x1b[?1h\x1b=\x1b[?25l"
recv: "\x1b[39;49m\x1b(B\x1b[m\x1b[H\x1b[2J"
recv: "\x1b(B\x1b[0;7m  GNU nano 6.2 \x1b[44bNew Buffer \x1b[53b \x1b[1;123H\x1b(B\x1b[m\x1b[14;38H\x1b(B\x1b[0;7m[ Welcome to nano.  For basic help, type Ctrl+G. ]\x1b(B\x1b[m\r\x1b[15d\x1b(B\x1b[0;7m^G\x1b(B\x1b[m Help\x1b[15;16H\x1b(B\x1b[0;7m^O\x1b(B\x1b[m Write Out   \x1b(B\x1b[0;7m^W\x1b(B\x1b[m Where Is    \x1b(B\x1b[0;7m^K\x1b(B\x1b[m Cut\x1b[15;61H"

You can see some text from the UI in there like “GNU nano 6.2”, and these \x1b[27m things are escape sequences. Let’s talk about escape sequences a bit!

ANSI escape sequences

These \x1b[ things above that nano is sending the client are called “escape sequences” or “escape codes”. This is because they all start with \x1b, the “escape” character. . They change the cursor’s position, make text bold or underlined, change colours, etc. Wikipedia has some history if you’re interested.

As a simple example: if you run

echo -e '\e[0;31mhi\e[0m there'

in your terminal, it’ll print out “hi there” where “hi” is in red and “there” is in black. This page has some nice examples of escape codes for colors and formatting.

I think there are a few different standards for escape codes, but my understanding is that the most common set of escape codes that people use on Unix come from the VT100 (that old terminal in the picture at the top of the blog post), and hasn’t really changed much in the last 40 years.

Escape codes are why your terminal can get messed up if you cat a bunch of binary to your screen – usually you’ll end up accidentally printing a bunch of random escape codes which will mess up your terminal – there’s bound to be a 0x1b byte in there somewhere if you cat enough binary to your terminal.

can you type in escape sequences manually?

A few sections back, we talked about how the Home key maps to \x1b[H. Those 3 bytes are Escape + [ + H (because Escape is \x1b).

And if I manually type Escape, then [, then H in the xterm.js terminal, I end up at the beginning of the line, exactly the same as if I’d pressed Home.

I noticed that this didn’t work in fish on my computer though – if I typed Escape and then [, it just printed out [ instead of letting me continue the escape sequence. I asked my friend Jesse who has written a bunch of Rust terminal code about this and Jesse told me that a lot of programs implement a timeout for escape codes – if you don’t press another key after some minimum amount of time, it’ll decide that it’s actually not an escape code anymore.

Apparently this is configurable in fish with fish_escape_delay_ms, so I ran set fish_escape_delay_ms 1000 and then I was able to type in escape codes by hand. Cool!

terminal encoding is kind of weird

I want to pause here for a minute here and say that the way the keys you get pressed get mapped to bytes is pretty weird. Like, if we were designing the way keys are encoded from scratch today, we would probably not set it up so that:

• Ctrl + a does the exact same thing as Ctrl + Shift + a
• Alt is the same as Escape
• control sequences (like colours / moving the cursor around) use the same byte as the Escape key, so that you need to rely on timing to determine if it was a control sequence of the user just meant to press Escape

But all of this was designed in the 70s or 80s or something and then needed to stay the same forever for backwards compatibility, so that’s what we get :)

changing window size

Not everything you can do in a terminal happens via sending bytes back and forth. For example, when the terminal gets resized, we have to tell Linux that the window size has changed in a different way.

Here’s what the Go code in goterm to do that looks like:

syscall.Syscall(
    syscall.SYS_IOCTL,
    tty.Fd(),
    syscall.TIOCSWINSZ,
    uintptr(unsafe.Pointer(&resizeMessage)),
)

This is using the ioctl system call. My understanding of ioctl is that it’s a system call for a bunch of random stuff that isn’t covered by other system calls, generally related to IO I guess.

syscall.TIOCSWINSZ is an integer constant which which tells ioctl which particular thing we want it to to in this case (change the window size of a terminal).

this is also how xterm works

In this post we’ve been talking about remote terminals, where the client and the server are on different computers. But actually if you use a terminal emulator like xterm, all of this works the exact same way, it’s just harder to notice because the bytes aren’t being sent over a network connection.

that’s all for now!

There’s defimitely a lot more to know about terminals (we could talk more about colours, or raw vs cooked mode, or unicode support, or the Linux pseudoterminal interface) but I’ll stop here because it’s 10pm, this is getting kind of long, and I think my brain cannot handle more new information about terminals today.

Thanks to Jesse Luehrs for answering a billion of my questions about terminals, all the mistakes are mine :)

Security-Enhanced Linux (SELinux) is a type of Mandatory Access Control (MAC) in the Linux kernel. It can prevent software from performing unexpected — such as abusive or malicious actions — on your Linux systems. However, … it’s also an unmanageable mess, and I have a much greater understanding of why people recommend that people disable it.

Read more …