Terminal telekinesis

le 30/05/2020 par Léo Jacquemin
Tags: Software Engineering

Tl;dr

In my experience, software developers’ skills regarding tasks in a terminal range from desperately slow to impressively swift. I believe that, for a large part, this is caused by an abundance of articles not discussing the real problems software developers are dealing with daily. Namely, how to feel productive on a terminal, after using omnipotent integrated software environments for so long.

This article aims at bridging this gap by sharing carefully selected tips that can, in my opinion, dramatically increase the terminal productivity of those who have never heard of them.

All the following tips are either natively present or very easy to install. I think they are of particular relevance to junior developers who are usually overwhelmed by the seemingly endless stream of new technologies they have to learn, although this list has proved helpful with seasoned terminal users too.

  1. Maxing out the cursor speed
  2. Instant search with backward incremental search
  3. Terminal teleportation with z
  4. Finding files with fd and words with rg
  5. Short man pages with tldr
  6. Perfect recall with history

Introduction

As Nietzsche put it:

If you stare long enough into your terminal, your terminal stares back into you.

To be honest, I’m not convinced those were his exact words. But whether they were or not, one can hardly argue against the fact that the terminal is not equipped with a friendly user interface. As far as I’m concerned, I won’t be denying it any longer. For a long time, my terminal and I had a very conflictual relationship.

I was mad at my terminal. I was mad because it made me feel slow and dumb every time I tried to perform even the most basic cd:

cd  ../where/is/it/located/again/omg

or searched  for a specific option:

curl -X ’header? method? hostname?’

I seeked help from external sources, such as Internet articles. Alas, I failed at finding the material I required to get my self-esteem back.  Every piece of advice I found was missing the mark.

Many tutorials throw unrelated Unix tips at you, as if there was any added value in aggregating all of them in one page, something Google has been doing since 1998.

Far too many tutorials try to teach you Unix Piping Art. If you have never heard the term before, don’t feel embarrassed, I just made it up. It is the art of combining seemingly endless sequences of grep, sed, sort, cut and awk so one can rename all his summer pictures from .png to .jpg  and email them properly filtered and gzipped to his best friend.

All jokes aside, most of these articles are great. I mean it. However, they don’t really help with day-to-day programming tasks, such as spinning up multiple containers in the background, or quickly editing a configuration file far away from where you stand.

I was full-blown frustrated for a long time. Until one day, one of my colleagues suggested that I brew install a little utility that would save me a lot of typing, thereby sparing me a lot of nuisance. All of a sudden, the ops inside of me was reborn. And so began my quest: fierce, restless, endless. The search for a lightning fast terminal experience. So here we go, dear reader. If you are frustrated at feeling slow, brace yourself. For I promise you this is the last time you will ever feel slow in your terminal.

1/ Maxing out the cursor speed

Many web developers pride themselves at being KISS experts. In the spirit of simplicity, we shall start this productivity check-list with a genuinely simple advice: increasing the cursor speed.

Switching back and forth from IDEs and terminals, one quickly realizes that the major developer experience paradigm shift is the absence of a mouse to fly from line to line. Keyboard-based navigation is thereby constrained by two things: your knowledge of epic time-saving shortcuts, and your cursor speed.

Learning shortcuts can tremendously increase one’s terminal agility, at the expense of learning them, remembering them, and trying not to rely too much on them. Topping the cursor speed, on the other hand, will instantly increase your swiftness with the sole cost of figuring out how to do it on your operating system.

This will have two interesting benefits as well. As a system-wide setting, doing so will give you a faster experience everywhere your computer requests you to fill out a text input. Besides, you will be able to cope with all these insane terminal-based editors like vim and emacs by just pressing an arrow key and waiting.

Increasing the speed of the cursor - or of anything else in fact -  can be considered solid advice when it comes to productivity, after all. Believe it or not, it took me 2 years to realize it. In the aftermath, I set out on a complex calculation to estimate the time I spent looking at my cursor moving at the default speed rate in my terminal.

I should never have done that.

2/ Instant search with backward incremental search

This second command is probably my favorite. I think of it as the quintessential of the time-saving commands. Is your shell’s current configuration file filled with tons of aliases, half of which you don’t even remember setting up in the first place ?

  • alias mkp=mkdir -p
  • alias gcm=git commit -m
  • alias loginpg=pqsl -h localhost -U postgres mydb
  • alias mysrv=ssh there@142.93.98.118 -i ~/.ssh/mysrv

It probably is. In fact, various popular shell frameworks will provide plugins, nicely packaged with their fair share of aliases, confusing the terminal apprentice into thinking that tons of aliases are actually a good idea.

I’m going to make a bold claim here. Possibly along with making myself a few enemies in the process, but that’s fine. So mark my words for they may cause me trouble...

Shell aliases are an anti-pattern.

To illustrate all their pettiness, I could tell you about the time it took me 4 hours to understand why a slightly customized vagrant box was working correctly on everyone’s laptop but was failing silently on mine. But doing so would probably shed more light on my weak debugging skills than anything else, so I’ll pass.

In my opinion, shell aliases are essentially redundant global variables that encourage indirection. They allow us to map a fully qualified and explicit long shell instruction into a random sequence of letters of our choice, in order to save us the trouble of typing them, remembering them, or both.

Truth be told, they remind me of the infamous Singleton pattern, which has the somewhat confusing ability to be both a pattern and an anti-pattern at the same time, thereby making it the very first quantum design pattern in software’s history.

Although they can actually come in handy in some situations. But what you're probably looking for is a way to quickly execute commands you have already executed before, and there is a much better alternative, natively present in almost every shell called backward incremental search (or reverse search history).

  • Distinctive sign: bck-i-search: _
  • Summoned by pressing: ^+r

When entering this mode, your shell will quickly guess which command you’re looking for by displaying the most recent matches as you type.

For instance, if you’re working on a configuration file that you opened with vim:

vim ~/.mydir/config

and typed a few other commands afterwards, what would be the fastest way to open up the same file again ?

Many may have the reflex to use the arrow keys to quickly display the latest commands one after another. This works well. But one can accomplish exactly the same within a fraction of the time by invoking some incremental search goodness by pressing just 3 keys:

> vim ~/.mydir/config

bck-i-search: v

How cool is that !

3/ Terminal teleportation with z

The cd command has been around for as long as computers had terminals - external ones - and screens. But a 70-year old built-in command should tip the wise developer off. Indeed, all odds are that a way more modern version has been developed since. In this case, the modern version exists. It is called z.

z is a tool that tracks where you’ve travelled using cd and is able to guess where you want to go, allowing you to instantly jump into a directory with just a few keywords, provided you have visited it before. Prioritization is based on what they call “frecency”, which takes into account both the most frequent and most recent directory that match a given pattern.

z works extremely well on its own. However, it will sometimes fail to get you exactly where you want to go in one go. Occasionally, you will land somewhere else. No tool is perfect. However, if you help it just a little by naming and ordering your directory with just a zest of z  in mind you will be able to jump right into any directory with just 3 key strokes at most! When you do, thank cd for his decades of service, but don’t forget to put it in the attic where it belongs!

4/ Finding files with fd, words with rg

Searching for files and words in a project is something software developers do everyday. The developer experience for doing both in an IDE is hard to match. This is because IDEs index every source file we write. Although with great powers come… great limitations. Indeed, everything must be indexed first and this can be very slow, which is why we need something else.

We need something that lets us find symbols such as parts of a word inside a directory. We need it to traverse each directory recursively by default, use regex patterns by default, and do all of this almost instantly by default. We need ripgrep.

Ripgrep is the fastest command line search program I have ever seen in action. It is so fast that search results appear almost instantly on stdout. And even when they don’t, the speed is so impressive that I enter a weird hypnosis state from which I wake up when the job is done anyway. Either way, my consciousness perceives it as instantaneous.

Oddly enough, ripgrep falls short when it comes to quickly finding a path inside a directory. For the sake of illustration, let us assume that there was a problem going on, involving a configuration file. You wish to quickly retrieve all the files that look more or less like a configuration file inside your project’s directory. How would you do it efficiently ?

Finding paths inside a directory is a valid use case for a native command called find. One could for instance, write the following:

find . -name “*conf*”

One can go a long way with this command. Especially when paired up with the backward incremental search we discussed before. However at some point, one must inevitably address the elephant in the room, and have the courage to ask: what is wrong with the way the find accepts its options ?

The command’s API does not seem to care for any of the unix standards most other commands duly respect. That alone should be sufficient to look for a more conventional alternative. If we strip out the in from find, it becomes fd. It is twice faster to type and works twice as fast. Plus, the output is colored by default.

As for the find command, we must acknowledge that part of the reason for its unconventional usage is because its options are in fact not options, but predicates instead. Truth always come out. It has now. Let go of find and use fd instead!

5/ Short man pages with tldr

Backward incremental search can do wonders for you most of the time but it suffers one major drawback: it cannot help you with commands you have never typed before.

As it turns out, when one wants to know to get a quick documentation answer about a specific shell command, Google may not be the fastest way from point I to point K. Man pages are, and you can type man man to have more information about them if you’re that type of person.

Man pages are truly great. Once you start getting used to them and are used to your $PAGER like less, you can retrieve relevant material very quickly without ever leaving your terminal. Furthermore, you’re always guaranteed to look at the documentation of the specific version you’re using, if your system is correctly set up.

However, man pages suffer a major drawback themselves. They’re just static pages. They lay a program’s options down without any consideration of any sort for the relative importance of each of them, the way autojump and z do with recently visited directories. Alas, tldr summaries were not a thing then. Fortunately for us, they are now.

tldr maintains an online database of the most useful program options for you to consult any time you’re sitting idly in your terminal. I don’t use it as often as bck-i-search: _ (which I use dozens of times a day), although when I do, it allows me to quickly find the 5 most popular usages of a program I’ve never used before. And I must give it some credit because up to this point, tldr almost never failed me.

One of my favorite use cases is to pair it with Git, a very popular VCS software. From its humble beginnings, GIt API has been notoriously hard to understand. This is perhaps the reason why so many developers don’t take the time to appreciate it. To appreciate the power of tldr on the other hand, I can’t think of a better example than one of Git’s most confusing parts: the undoing of some work with the git reset command.

tldr git-reset

- Unstage everything:   git reset

- Unstage specific file(s):   git reset path/to/file(s)

- Undo the last commit, keeping its changes in the filesystem:         git reset HEAD~

- Reset the repository to a given commit, discarding all since then:         git reset --hard commit

I honestly think that this short abstract covers 10% of what this command can do and yet 90% of what it is used for. Pretty straightforward, as opposed to the 445 git-reset man pages.

Another neat trick with this command is to use it as a learning tool. As a community-driven command, typing tldr will always give you what other people are using this command for, which will instantly give you a pretty good idea of its most relevant usages: something neither the man pages nor a Google search can give you with that accuracy.

Try using tldr on both fd and rg to see more of it in action, and in the process, give this article the opportunity to refer back to itself, in a kind of meta-way.

6/ Perfect recall with history

The Pareto principle states that 80% of the effects come from 20% of the causes. It is an empirical law that is confirmed by experience time and time again, with deep implications in many areas. Backward incremental search and tldr are already secretly piggybacking on the 80% part of it. But what about the other 20% ?

From time to time, you will have to roll up your sleeves and write a specific command. One that you won’t be needing anytime soon. Perhaps even a master-piece of unix pipes of your own. However, one that will be tremendously useful at the time you’re crafting it, and therefore probably also useful another day in the future.

History saves the last thousands unique entries you have typed. It is not to be confused with a journal log of everything you have typed chronologically. When piped into grep, it allows you to quickly pull out of the distant past something you know you have solved before. You don’t have to remember the exact command, or even the beginning of it. Just a tiny specific detail about it for grep to do its pattern filtering magic.

What does it mean ? It means the terminal power-user that you are becoming is natively geared up with perfect recall abilities. Therefore, it is always a good investment to take the time to create a complex powerful shell command. For you can bring it back anytime you want by invoking history, and use a slightly different version fitting your slightly different problem.

Conclusion

In this article, I’ve shared a few tips I wish I had known when I started leveraging my terminal every day. All these tips proved invaluable to me, and I have yet to see a terminal power-user that would discard them all.

Picking one of these tips up will definitely improve your terminal experience on some occasions. But if you pick them all, and add your own to the mix, you may come to realize that feeling fast is only half of the equation.

What’s really important in the end, is not wasting time and energy on repetitive tasks, so we can focus on what truly matters in our field.  Rock solid quality code that solves challenging problems for our users. And even if you’re new to the game, my guess is you already know that it is a full time job.

I’m not advocating to look for better ways to do things every second of the day. Sometimes, self-improvement must be put on hold and things are good enough the way they are. But after a while, it is a healthy practice to step out of your comfort zone, in search of lost time. You never know, you might stumble upon the time-saver of your life.

Telekinesis is the faculty to move things around by the will of the mind. Well, I might not be able to execute shell commands without ever touching my keyboard yet, but one thing is for sure. I’m getting closer every day.

Takeaways

  • Increase the cursor speed
  • Use ctrl+r to search through your shell’s history
  • Use cd when you visit a directory for the first time, and then use z
  • Use fd and rg instead of find
  • Use tldr first and then man
  • Use history with grep to retrieve anything you have typed before

Drawings

Courtesy of my colleague Aryana Pezé