Navigation
Home
Home
RSSGit ServerRainchan ImageboardBrowse Site on Tor
Backups - (Tor) (I2P)
Home > Technology

YouTube Ecosystem Wholly in a Terminal

There's a reason for which I sudden got really into RSS as a format. This all started when quite recently, the 'primary' web mirror for the software Invidious, was going down - that being, invidio.us (now hosts a list of other instances). Of course, this didn't mean the other instances would be going down, so I thought little of it. I preferred to use Invidious rather than the normal YouTube front-end, for various reasons related to internet privacy:

Alongside the fact that the default YouTube front-end is an unnecessarily bloated CPU-eater. I believe they have removed the legacy front-end now, though I've not checked in a long time.

Invidious was convenient, as rather than use a Google account for subscriptions - it is effectively just a GUI front-end for an RSS feed. It can also play videos without the need for JavaScript.

But Invidious has its own problems. Hosting tends to be highly unreliable. I can't tell how how easy or difficult it is to host an Invidious instance as I've never done it, but particularly in offshoot instances, they tend to break often and are generally unreliable, and sometimes inaccessible.

Returning to the Original Story

So, before long, invidio.us went 'down', as expected. But I noticed something strange begin to happen. As I used other instances instead, it seemed like they were breaking even more often. If that were true, it would be highly conspicuous - it would imply that in some way, these offshoot instances depended on the primary instance, and were now running into problems. That's quite troublesome design. Of course, I can't statistically prove that is true without taking a serious look at the codebase, which I don't plan to - it certainly could just be cognitive bias. But it got the ball rolling to make me begin to consider my alternatives.

YouTube offers an RSS feed to channels by default. You can in fact subscribe to YouTube channels in an RSS feed reader - this is exactly how Invidious works. There are RSS feed readers - such as newsboat - where you can read RSS feeds from a terminal. Put 2 and 2 together - I can see my YouTube subscriptions from the terminal.

So I went ahead with that

It all worked perfectly well. I added a few macros, which you can run via ",<key>" in newsboat, to do things such as - play audio, play video, download audio, download video, and open in Qutebrowser. You can see what I've done below:

macro v set browser "mpv --ytdl-format=22\/18 --ytdl %u" ; open-in-browser ; set browser "lynx %u"
macro V set browser "youtube-dl --add-metadata -f 22/18 %u" ; open-in-browser ; set browser "lynx %u"
macro a set browser "mpv --ytdl-format bestaudio --ytdl %u" ; open-in-browser ; set browser "lynx %u"
macro A set browser "youtube-dl --add-metadata -x --audio-format mp3 %u" ; open-in-browser ; set browser "lynx %u"
macro q set browser "qutebrowser %u" ; open-in-browser ; set browser "lynx %u"

This was great. And then I wanted to search for a song I didn't have a copy of, and didn't quite remember the name of. And then I thought, "Wait, how am I supposed to search?"

A TUI client for YouTube

The first thing I did, rather than reinvent the wheel, was to look to see what else others had done. But it was all pretty poor.

Generally they were written in Python. Almost all of the time they didn't work. The few that did 'work' packaged a bunch of other things I didn't need - like a video player, when I already had mpv. I just needed the ability to query.

And so, I wrote my own

The program, 'ytsearch', offers you this functionality in selecting queried videos:

Data showed in search queries:

Actions on selected videos:

Preferences which can be set as parameters or in a config file:

Beyond a standard GNU/Linux system with coreutils - this program depends on mpv, youtube-dl, and to a minor extent on xclip (which makes video sharing more convenient - no functionality is broken by editing it out).

At first, I tried to use the functionality packaged with youtube-dl - 'ytsearch' - which can be used to get one result, 5 (ytsearch5), or however many you can (ytsearchall). But I found this to be an extremely slow process, that was difficult to format and work with. I also think it is 'unfriendly' to YouTube, as it makes many video requests in formatting a single list of search results. This led to me getting banned very quickly.

So I pretty quickly ditched that idea. I couldn't curl YouTube either though - their results don't appear without JavaScript enabled. So, I settled for curling Invidious instances for links (their format is pretty universal across instances, I've not noticed any problems yet), and then passing those links to download/stream from YouTube itself. This method is much 'friendlier', and by far much faster to query - even for playlists with hundreds of videos (which are displayed in pages you can scroll through).

Drawbacks and Concessions

It has some drawbacks, of course. It is very scrappy scripting - this program evolved as I needed it to - and as such the code is an absolute shambles. It at least does work. Naturally, it's not even close to POSIX compliant, and depends heavily on Bash. I could likely have made a better program if I bothered learning ncurses, but I don't use YouTube often enough to justify that kind of effort.

It also only works as well as your Invidious instance works - if the instance has problems, so will the script. To mitigate this, I've made it very easy to test and change instances via parameters and config files. It reports reasonably well on what the issue could be if it fails to fetch videos.

Companion Programs

'ytsearch' has two companion programs - 'ytformat' and 'ytchannel'.

'ytformat' just takes links pointing to an Invidious instance, and makes them a YouTube link, if you are looking to share them as such. Fairly straightforward.

'ytchannel' is a fairly hacky script, that downloads and deletes a low-quality audio copy of a video, in order to get the YouTube channel it belongs to and to give you it in RSS feed compatible format. Thus, with the 3 programs, it is entirely possible to run the full YouTube 'ecosystem' from a terminal, and to never need to leave a terminal, and still have advanced search features and channel subscriptions.

Getting the Scripts

You can find all of the aforementioned scripts on my Git server, in a repository of their own, here. I hope they work for you. I'd also love to see - if anyone is interested in it - improvements to it, and probably written in something better than Bash.

2020.09.25

Send me something interesting: nixx@firemail.cc