652

Addressing the Exponential Growth of Communities (lemmynsfw.com)

submitted 2 years ago* (last edited 2 years ago) by OptimusPrime@lemmynsfw.com to c/lemmyworld@lemmy.world

214 comments fedilink hide all child comments

Over the past few days, I've witnessed a remarkable surge in the number of communities on browse.feddit.de. What started with 2k communities quickly grew to 4k, and now it has reached an astonishing 8k. While this exponential growth signifies a thriving platform, it also brings forth challenges such as increased fragmentation and the emergence of echo chambers. To tackle these issues, I propose the implementation of a Cross-Instance Automatic Multireddit feature within Lemmy. This feature aims to consolidate posts from communities with similar topics across all federated instances into a centralized location. By doing so, we can mitigate community fragmentation, counter the formation of echo chambers, and ultimately foster stronger community engagement. I welcome any insights or recommendations regarding the optimal implementation of this feature to ensure its effectiveness and success.

you are viewing a single comment's thread
view the rest of the comments

[-] PriorProject@lemmy.world 33 points 2 years ago* (last edited 2 years ago)

I see very little discussion about the implications of this for moderation, and it feels to me like they get very sticky. With traditional human-curated multi-reddits, you as a subscriber must engage with the idea that you are choosing to aggregate multiple communities into a single feed, which is intuitive enough, the subscribed feed already works that way.

But by making it automatic, the software hides the fact that it's pulling together discrete feeds from communities with different rules and different moderators. This feels very awkward to me. I'm all in favor of traditional multi-reddits, which can be used to create this sort of feed for yourself. I'm still on the train of "duplicate communities will sort themselves out if community discovery is made much easier and popular communities reliably show up at the top community searches, mostly irrespective of what instance the search was performed on" (obviously defederation takes precedence here).

[-] Ryumast3r@lemmy.world 6 points 2 years ago

I’m still on the train of “duplicate communities will sort themselves out if community discovery is made much easier and popular communities reliably show up at the top community searches, mostly irrespective of what instance the search was performed on”

Honestly, we've all seen this happen in every community that fragments for some reason. Whenever reddit banned a subreddit, 1,000 mini-subs would spawn, eventually coalescing into 1 or 2 main communities. Whenever a migration happens (Tumblrinos moving to reddit, redditors moving to various reddit-like things, forums moving to reddit, fark migrations, usenet, IRC, etc) they always fragment initially only to end up with a relatively few coalesced groups that hold similar ideas.

c/news might end up with 10,000 news subreddits (even one for each local newspaper!) but eventually it comes out to something like: c/news, c/leftnews c/rightnews c/ihateallnews with the most active (by far) being one of the generic ones. The others might technically exist, but if you search "news" and get one that has 10,000,000 subs and another that has... 3, well, you're just not going to bother.

[-] MentallyExhausted@reddthat.com 2 points 2 years ago

I think it would need to be voluntary; mods on different instances have to agree to be a joint mod group. If an instance decides to break up, they take their mods with them, etc. Might he complicated, but it seems doable.

[-] PriorProject@lemmy.world 2 points 2 years ago

If the mods were willing to agree to this, why would they not also agree to simply merge the communities? Frequently, when mods choose to maintain overlapping/competing communities/subs... it's because there's conflict about culture or moderation policies. This mechanism feels very duplicative of existing patterns, and limited in much the same ways when cooperation breaks down between mods of different communities.

[-] MentallyExhausted@reddthat.com 5 points 2 years ago

There’s an advantage to splitting the load amongst multiple instances. If community@host1.com has issues, the subscribers to community can still access community@host2.com.

Further, preventing centralization is a core focus of the fediverse, with all the pros and cons that come with it. The reddit migration is a testament to that philosophy.

If an in-fight happens between two instances’ communities, host1 won’t be able to boot host2’s entire existence, just their agreement to be in sync. Voluntary syncing of communities seems to me a natural extension of the federated philosophy.

[-] PriorProject@lemmy.world 1 points 2 years ago

There’s an advantage to splitting the load amongst multiple instances.

I don't have links handy, but the devs have said multiple times that federation replication load is not a limiting factor. It's the browsing load that matters, which is already spread across the instances that subscribers hail from. There isn't a performance issue here to fix, at least around the current network size.

If community@host1.com has issues, the subscribers to community can still access community@host2.com.

Again, user-account federation already provides this. If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance. I don't see significant further benefits in splitting the community hosting.

Voluntary syncing of communities seems to me a natural extension of the federated philosophy.

I don't disagree with this, the challenge is that the federated philosphy makes everything it touches very complicated. A sprinkle of federation in the core of an ecosystem brings enormous resilience against the ecosystem getting coopted against the will of its users. A heaping spoonful of federation makes it an unusable mess. The complexity of federated design needs to bring very big benefits to "pull its weight" in complexity. The tradeoff looks positive for me for federated replication even though the cost in complexity is large, but I don't for meta-communities. They seem like cosmetic improvements over the existing capabilities.

But fair enough, I see where you're coming from. It's not madness, on-balance it's just not something I see providing value in proportion to its conceptual complexity.

[-] MentallyExhausted@reddthat.com 2 points 2 years ago

If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance.

Can you help me understand how that all works? I ran into this today where an instance wouldn’t let me submit due to performance issues, and I jumped instances to be able to provide some (potentially) important breaking news to “community.” If they were fully in-sync and one instance didn’t “own” the community, when things stabilized, my content would have replicated instead of being stranded on a lonely instance. As it stands, that content will never be part of the “real” community.

But fair enough, I see where you’re coming from. It’s not madness, on-balance it’s just not something I see providing value in proportion to its conceptual complexity.

Right on. I’m still new to the fediverse and am probably missing some foundational concepts still. Love where it’s headed though.

[-] PriorProject@lemmy.world 2 points 2 years ago

If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance.

Can you help me understand how that all works?

I'll preface this by saying I haven't read the code and it's certainly possible there are errors in my mental model, but my belief is:

When the community-instance is up and it gets a post from a user local to the community... It write that data to its db and asynchronously sends federated messages to instances with users that sub the community. Those instances write it to their local DB.
When the community-instance goes down and someone on a user-instence tries to read the post, it gets read from the local-db cached copy.
While the community-instance is still down and user-a on a user-instance posts or comments, again it goes into the local DB and async sends a federation message to the community-instance... the message doesn't immediately get through though.
While the community-instance is down, if user-b on the same user-instance browses the community, they see user-a's content from step 3 in the user-instances-db. If user-c on a different user-instance checks, they won't see user-a's content until the community-instance comes back up to process the federation messages from step 3.
When the community-instance comes back up, it accepts the federation messages from step 3 and updates all subscribed instances so user-a's content is viewable everywhere.

It's possible I'm mistaken about 3-5, but if you had trouble posting... my guess would be that your own user-instance was struggling and slow. But if you've rules that out maybe the update behavior when the community-instance is down works differently than I think it does. The reading in step 2 I'm quite certain about though.

[-] MentallyExhausted@reddthat.com 3 points 2 years ago

I'm pretty sure you can't post to community-instance if they're down/having issues to the local DB. The instance I'm on is very stable and every time I think it may be an issue, it turns out to be the other instance.

It sounds like a lot of the "sync" behavior I was thinking of is already built-in if we can just expand on it a bit.

[-] PriorProject@lemmy.world 2 points 2 years ago

I'm pretty sure you can't post to community-instance if they're down/having issues to the local DB. The instance I'm on is very stable and every time I think it may be an issue, it turns out to be the other instance.

I can't disprove this. My experience when lemmy.ml was struggling was that I could comment-reply and vote consistently even though many of the communities I'm active in are on that struggling server. But maybe I was just lucky. I can't explain the behavior you observed, and it may be that certain operations are more directly passed through to the community-hosting instance than I previously believed.

But irrespective of the details, plain federation does provide some significant resilience to transient instance failure, and there's probably room to tune it further in at least some ways.

Interesting discussion, though.

[-] MentallyExhausted@reddthat.com 1 points 2 years ago

Ah, yeah I should clarify that I was able to comment on existing posts, but could not submit a new post. Fun discussion, agreed!

[-] PriorProject@lemmy.world 2 points 2 years ago

That's the difference then, I do A LOT more commenting than posting. Perhaps there's some essential reason they're handled differently or hopefully an accidental one and posts could be made to behave more like comments someday.