Enter the Matrix

Hey all, hope you’ve been well! As I outlined in my last post, I’ve gotten around to some of the things I’ve been working on, specifically setting up a Matrix Homeserver for THP. I’m going to split up posts between a more general overview and more technical one. This time around expect an explanation of what Matrix is and why I’ve set up a server running it.

But before we get started, there’s something very important to state: you can connect to this chat service right now.Simply go to https://chat.thp.reisen if you want to use the web client I’ve set up (alternatively, https://cin.thp.reisen also exists on an experiment basis—Cinny is less feature-rich but more Discord-like). Or, if you want a proper client for your phone or desktop, look here for one—just make sure to set your homeserver as thp.reisen (or matrix.thp.reisen, but that shouldn’t be necessary!). I recommend Element on whatever device, as it is the most complete and actively-developed client. Hope to chat with you soon!

Now, on with the show. Just what is Matrix? It’s a stupidly-named protocol (searching for it will give you either math or movies) but a cool communication protocol. It’s meant to be federated by design, meaning that a bunch of decentralized servers can come together on a voluntary basis to share users and information. It’s an open spec and anyone can contribute to it and connect it with other services or adapt it to their needs.

Matrix, despite being open and decentralized, also makes it central to its vision to provide privacy via encryption and to make sure that users are in control of how they want their data to be shared. It’s no wonder then that even state actors, like the French Government or German Ministry of Defense have adopted the protocol to develop their own secure but easy-to-use programs. NGOs, tech enthusiasts and privacy advocates have all vested interests in Matrix succeeding.

It’s a protocol that, despite being ready for use, is still under active development, with the 1.3 release being made just last week, while our server had already been deployed. A further 1.4 release of the spec is expected by the end of the year, in addition.

So, then, I’ve already mentioned the decentralized and self-hosted aspects of it. That’s a very appealing reason to use anything because it means that we’re only subject to the rules of our host and the country where things are hosted, instead of further bound to inconsistently applied rules like Discord’s Guidelines.

Surely, similar liberation could be obtained using something like IRC (v3 spec in particular) or something like a self-hosted instance of Revolt? Well, I’ll thank you to not call me Shirley and no, not really. IRC is great and the v3 extensions are really promising but it’s nowhere near ubiquitous enough in clients and even in servers that can be deployed easily. Sure, Twitch uses some version of it as do a few other networks but it’s still a little while again. Revolt is similarly pretty cool but it’s very niche and actively encourages users to use their centralized server—self-hosted instances require me to manually distribute clients with pre0modified domains, which makes it impractical to do so securely and expediently.

So then, in order to get the combination of reliability, freedom from proprietary software, modern features like voice chat, image and video sharing and avatars (seriously guys this was never cool, even in the 90s when we only had text forums) that pretty much leaves us only with Matrix. And so down the rabbit hole we go.

The implementation of the Matrix spec is up to homeserver software. The most feature-rich and actively-maintained that’s open source is Synapse which is what we’re using. There are others that are more resource-efficient or, well, “faster”, but I preferred to go with something well-known and reliable. The thing is that Synapse implements the spec rather quickly and often has experimental features available as well. So something as fundamental to a self-hosted instance, such as granular media retention rules (how long files should remain on the server’s storage) is something that’s appeared first in Synapse. In fact, another missing piece was added just this last week and we’re already benefiting from this new feature. So the trade off for increased memory usage is well worth it.

This does mean, however, that configuring things for Matrix does not have a universal approach. I’ve spent a lot of time reading the Synapse documentation and only some of that is transferable to other homeserver implementations like Dendrite.

I won’t bore you with all the details—for today at least—but then there’s a third layer of complexity that is necessarily introduced: the client itself. It’s up to the client to recognize many of these features as well and display things to the user in a coherent manner. This means, in practical terms, that sometimes features don’t work universally across clients. Especially as things like the standards for what a “room” means and what it can support (defined by “versions”) need to be implemented on by the server software and then applied by the administrator of the particular server to their rooms.

In practice it’s not so bad and basic functionality is fairly consistent. Old, or lesser-developed, clients will still work and do most things, they just won’t be able to enjoy features like reply threads or knocking into restricted rooms for access.

That said, sometimes clients do implement their own features. Especially Element that, while being open source software, is backed by commercial interests as well. So you’ll get Element-only implementations for things like polls, specific types of voice chat or ways to display data. There is a strong relationship between them and the Matrix Foundation, however, and some of their extensions are added to the spec in due time. Still, because Matrix’s spec sometime moves slowly to implement features, it is often up to the clients and the homeservers to provide users with what they want. Things like Stickers and Custom Emojis are important but apparently not important enough to be high atop the Matrix Foundation’s list of priorities.

Most of these things are more of headaches for server administrators than users and the active and technologically-savvy community usually has plenty of workarounds and solutions in place. This is just the way it is with open protocols and free software; it’s a superior product in the end because the users are in control but, because money is not the primary motivating factor, deliberation and correctness with consensus is valued.

Well, I hope that that gave you a good overview of what Matrix is and why we’ve deployed our own server. Our homeserver has been working well for a bit less than a fortnight and I don’t foresee any radical changes needed. There are still configurations to do and tweaks to make, but I think that we have a nice foundation for a future home. Hope to see you there.

Next time I’ll be talking more about the technical aspects and challenges I’ve faced during this ordeal. Until next time, take it easy!