I recently came across a proposal to create a marketplace of sorting algorithms for Lemmy, allowing users to browse, select, and even create their own custom sorting algorithms. This would provide more advanced users with additional ways to customize and sort content, and give the community the ability to share their creations. I’m curious to know if this is actually possible within the Fediverse and what your thoughts are on the idea. Would this be a valuable addition to Lemmy, or do you see any potential drawbacks?
Sorting algorithms run on the server.
No way a server can allow arbitrary remote code execution.
You could implement something that uses a server sort and has the client do custom sorting on the results the server told it about.
How you do it is a hybrid of how you described.
One of the first optimisations the lemmy devs did with the Reddit exodus was to precompute the “hot score” of a post and store it in a field rather than trying to compute it “live” in every query. So the score gets updated every hour and also whenever there is some comment or change to the post.
And the server simply sorts by this value.
There is no reason similar sorting fields can’t be exposed via the API allowing the client (or some client eg with mod access or admin access) to store some arbitrary value to a column and then the server can happily sort by this column.
If the server sends posts, and the client computes the ordering, the client has to do the sorting.
The “hot” thing is just server side caching of scores for a server side sort.
What you seem to be suggesting is that the server support storing post scores sent from clients so it can feed that sort back to the client. I don’t think that would scale for millions of users storing their preferred post sorting order on the server.
I was the guy who checked the performance issues on those queries and suggested the caching algorithm to the Devs so I know how it works and have a fairly reasonable understanding of the Lemmy schema.
Every single vote you make is already stored. Adding some “ranking vote” is not a big deal. It only has to scale on your own home instance. It’s not a matter of calling for millions of users.
If you run your own instance it will have to scale for 1 user.
The difference is the server doesn’t need to scan the giant vote table for ordinary polling or ranked posts.
Unless there was significant optimization work done on the DB schema, this approach would require joining the large posts table to the very large user-post-ranking table for each request for sorted posts.
This will kill DB performance without some sort of clustering or sharding.
Bluesky has custom feeds with arbitrary selection and sorting algorithms, although it’s more Twitterish than forumish
The feeds don’t run on the host server, they run on separate feed generation servers
I guess that answers it. It is a neat idea.
With fixed sorting algorithms, the server can do caching and indexing to speed it up. (Emphasis on “can”. IDK if Lemmy servers actually do this.)
With custom sorts, that goes out the window.