What's Full-text Search Anyway?

About a month ago, Mastodon announced that v4.2.0 would contain the addition of federated full-text search—a long requested feature by Mastodon users. Federated full-text search is the ability for users to search through any posts that reach their instance—not just local ones, but from anywhere in the Fediverse. You know that useless little rectangle in the top left of the web client? Yeah, that does something now.

Like anything new that ever comes to the Fediverse though, this caused panic and confusion among users. Was it opt-in? Was it opt-out? Why do women go to public bathrooms in groups? The answer to most of these questions soon became clear. Later today, Mastodon fully releases v4.2.0. So let's talk about how we got here.

To opt or not to opt?

The biggest topic of debate in the new release was the possibility of the feature being opt-out instead of opt-in. One side argued that opt-in was the obvious choice, as this gave users more control over their privacy by default. The other side argued that opt-out was the way to go, because a lack of post searching would be frustrating to new users. These arguments between the opt-sides became so intense that it may have been dangerous if one saw an opt on the street.

Here's something that the opt-in side may not have considered: Searching for posts is a feature that people expect when coming from mainstream microblogging platforms. Search is just there for them, and tinfoil hats aside for a sec because let's be real, most people don't care about any of this.

Of course, we know since v4.2.0-beta3 that they chose to go with opt-in. I think this does more harm than good, however I'm not taking the opt-out side either. Why can't we have both?

What I think should've been done

Personally, I think something similar to this was a better way to do it:

• A new "reach" frame is added to the onboarding process after registration
• Users see a toggle that is set to the "on" position by default, but can easily click it to opt-out
• The user's account is then activated, and they can begin making posts

This in my opinion would make both sides happy. It first informs the user, gives the user an in-your-face control over the setting, then sends them on their way. A method like this would be way, way more effective than every individual needing to discover the existence of the opt-in toggle at some point down the line. I have no doubt that the current system will struggle with adoption of indexed accounts, which leads to a heavily incomplete database and plenty of "hey, can you turn on indexing" private mentions.

How Full-text Search currently works

As of v4.2.0, releasing in a matter of hours from this post, it works like this:

• The user opts into being indexed in Settings > Public profile > Privacy and reach > Include public posts in search results (Note: As the name suggests, unlisted, followers-only, and private mentions are never in the index)
• On their instance: If the user's instance has ElasticSearch enabled, their posts will be indexed and displayed in their instance's search results
• On other instances: If a user's post reaches an instance via federation, with the receiving instance having ElasticSearch enabled, it will be indexed and displayed in the receiving instance's search results (Note: Yes, even if the user's own instance has ElasticSearch disabled)

So there you have it. Full-text search was announced, faced backlash, and is in a rather ineffective state at the moment. I do think it's still much better than nothing overall. Personally, though, I won't be enabling ElasticSearch on Orchard. The reason is that it consumes a significant amount of CPU power, and our rig isn't beefy enough at the moment to run it while also having the majority of resources on standby (as it should be). Who knows, maybe one day we'll get the funds to a get a powerhouse server and set it up.