851
submitted 1 year ago* (last edited 1 year ago) by nutomic@lemmy.ml to c/lemmy@lemmy.ml

This site is currently struggling to handle the amount of new users. I have already upgraded the server, but it will go down regardless if half of Reddit tries to join.

However Lemmy is federated software, meaning you can interact seamlessly with communities on other instances like beehaw.org or lemmy.one. The documentation explains in more detail how this works. Use the instance list to find one where you can register. Then use the Community Browser to find interesting communities. Paste the community url into the search field to follow it.

You can help other Reddit refugees by inviting them to the same Lemmy instance where you joined. This way we can spread the load across many different servers. And users with similar interests will end up together on the same instances. Others on the same instance can also automatically see posts from all the communities that you follow.

Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements. That way the server will only go down sooner.

top 50 comments
sorted by: hot top controversial new old
[-] bilb@lemmy.ml 64 points 1 year ago

I'm going to set up a general purpose instance tomorrow with the intention of handling a relatively large number of users. The main problem is choosing a domain!

[-] TerrorBite@meow.social 34 points 1 year ago

Naming things is one of the two most difficult issues in IT, alongside cache validation and off-by-one errors.

load more comments (8 replies)
[-] Nicarlo@lemmy.ca 24 points 1 year ago

I was also contemplating setting up a new instance for this. I have 100s of gigs of unused ram, CPUs on idle and a 10gbit connection looking for something to do. The only issue I couldn't figure out was the name. I own itjust.works was thinking of something clever subdomain to use with it. I'm glad I'm not the only one with this issue

[-] autisticaudioguy@lemmy.ml 39 points 1 year ago* (last edited 1 year ago)
load more comments (6 replies)
[-] 7eter@feddit.de 18 points 1 year ago

can't wait for fedd.itjust.works to go online!

load more comments (3 replies)
load more comments (2 replies)
[-] ruud@lemmy.world 55 points 1 year ago

Lemmy.world is a new server, accepting signups. You're welcome there.

[-] anji@lemmy.anji.nl 51 points 1 year ago

Sadly, I feel like the Fediverse, based on ActivityPub, was fundamentally designed wrong for scaling potential. I do like Fedi and I like ActivityPub, but I think instances should not have to be responsible for all of this:

  • Owning user accounts
  • Exclusively host communities
  • Serving local and remote users webpages and media
  • Never going down, as this results in users and content becoming unavailable

Because servers "own" the user accounts and communities it's not trivial for users to switch to a different instance, and as instances scale their costs go up slightly exponentially.

I wish the Fediverse from the beginning was a truly distributed content replication platform, usenet-style or Matrix-style, and every instance would add additional capacity to the network instead of hosting specific communities or users.

I guess it's a bit too late for a redesign now... Perhaps decentralized identifiers will take us there in some form in the future.

[-] poVoq@slrpnk.net 24 points 1 year ago* (last edited 1 year ago)

Not sure why you reference Matrix, which has even worse scaling issues as it indeed tries to replicate nearly the entire network on every server.

The Fediverse is really just working how the general web does, just with some standardized API for websites to interact. It's not perfect, but it works and has proven to be relatively scalable.

It sounds a bit like you had a bit too much of the Bluesky cool-aid, which indeed replicates nearly all of the mistakes of Matrix and makes it impossible to scale via small community owned servers instead of big company owned data-centers (which might be by design?).

[-] anji@lemmy.anji.nl 19 points 1 year ago* (last edited 1 year ago)

Well yeah, point taken that replicating everything everywhere and forever might be impossible. But I do believe at a minimum my identity should be portable and accessing Fedi (ie. in microblogging: posting and viewing a feed of the latest posts of my follows) should be decoupled from which instance I pick to access the Fediverse.

I don't particularly like how owners of instances which grew are now essentially locked in to having to spend 100s or 1000s of dollars a month keeping their now expensive instances running and providing service. This is a bad place to be for a platform ran by volunteers. Letting instance owners scale their service down as well as up would be ideal. But this requires at least decentralized identity, and at best some form of content hosting redundancy...

It's easy to say the current architecture of Fedi works when it's still small. Your instance has 139 users.. That's not intended as a slight. Hosting instances is good and I applaud you for it! But I wish it were easier to more equally share the load once the platform becomes more popular.

load more comments (7 replies)
load more comments (4 replies)
[-] comfy@lemmy.ml 49 points 1 year ago* (last edited 1 year ago)

@nutomic@lemmy.ml It might be a good idea to default the Communities page to All instead of Local, to help push users into discovering other instances and promote them.

load more comments (6 replies)
[-] aksdb@feddit.de 48 points 1 year ago

I think lemmy will be bitten in the ass by not having considered clustering/horizontal scaling from the start. Federation alone as a scaling mechanism is only feasible for "nerds". But if the network wants to grow, we will need a few scale-able large hosted instances. And if their only choice is to scale vertically, there will be a hard limit (unless we put a good old Mainframe somewhere ^^).

Another downside of this design is: you can't run it with high availability. If there's only one process per instance, updating it will mean the whole instance is down. Sure, if all goes well this downtime is under a second. But if it doesn't go well or if a migration is needed, this might quickly become hours.

[-] PriorProject@lemmy.world 19 points 1 year ago

I think you probably underestimate how far one can get with "vertical" scaling. Here's the dockerfile: https://raw.githubusercontent.com/LemmyNet/lemmy/release/v0.17/docker/prod/docker-compose.yml

  • It includes 4 different containers... so there's a way to scale out to 4 machines right away. Maybe not every container is doing an equal amount of work... but there's some amount of immediately available machine-splitting.
  • I'm no expert, but I believe that at least the lemmy and lemmy-ui containers are stateless. If so, they're horizontally scalable already.
  • Postgres then would likely be the main bottleneck. But postgres offers read-replicas, so again the write-load and the read-load can be hosted on separate machines. And if there's enough read-load, you can have many replicas.

Other comments from the admins have shown that lemmy.ml today is running on a single eight-core box and it's currently hosting 30k registered users and over 1k active. So how much more compute capacity can we throw at "vertical" scaling on the current software architecture?

  • Just by going to a bigger single box, we can get 128 cores with no problem, a 16x bump in capacity. Does that get us to at least to 300k registered + 10k active?
  • Splitting the containers onto 4 separate machines. Does that get us 2x more?
  • Adding PG read-replicas and additional lemmy/lemm-ui containers would allow us to expand our instance footprint to maybe 6 physical machines should get us another 2x or more in performance.

Conservatively, that's 100x the computing capacity of the current hardware and could potentially support 1m registered users and 50k active. Now, I don't REALLY expect this to be possible today, there will be many software bottlenecks found along the way to scaling a single instance this large. But my point is that there's already a medium amount of horizontal scalability built into lemmy, and if the software doesn't fall over for algorithmic reasons (which is will at first), the current infrastructure architecture allows quite a lot of growth. There's plenty of time between now and a federation of million user instances to adopt a truly distributed storage backend if needed.

load more comments (6 replies)
load more comments (18 replies)
[-] lightrush@lemmy.ca 43 points 1 year ago

Point us to where the coin slot is. E.g. Patreon. We insert coin 🪙, you upgrade.

[-] nachtigall@feddit.de 44 points 1 year ago* (last edited 1 year ago)

Here they list them:

https://join-lemmy.org/donate

EDIT: shoutout to the madlad who donated $1,000!

load more comments (3 replies)
[-] Copio@lemmy.ml 43 points 1 year ago* (last edited 1 year ago)

Over at https://join-lemmy.org/ , when someone clicked on "Join a Server", they are presented with a list of instances, it's not that obvious that these are cross-accessible (yes, the homepage mentioned it, but not here), and people are bound to look for one with the most users.

Perhaps, add a simple TLI5 explanation/diagram explaining how Lemmy works on https://join-lemmy.org/instances .

(The documents are also too wordy for most people to care.)

[-] nutomic@lemmy.ml 31 points 1 year ago

Pull requests welcome, the code is here: https://github.com/LemmyNet/joinlemmy-site

[-] therealfooza@lemmy.ml 42 points 1 year ago* (last edited 1 year ago)

I have been wondering how cumbersome the Lemmy design will become for some. I love the idea that it is federated and decentralized however these are also major drawbacks for most average users (i.e not multi account users.

Multiple accounts needed for maximum uptime on different instances. What if I really like my username and its taken on another instance? If one instance is down and i comment with my other account will i then need to manage replies etc through different profiles? What happens if something spins up another instance of a similar domain so that they can get a username of someone to imitate them? I am sure these can be blocked after the fact or will other federated instances be automatically blocked.

What happens when someone gets bored of their instance and stops it, or it gets blocked, or they start getting unwanted attention. Does this mean all that content then goes into the ether?

Will this go down the route of whomever provides the instance with the most resources, best load balancing becoming the one, blocking other instances and controlling it as if it were private and independent?

There are a lot wait and see things, but I am excited to help and see what this great project becomes.

[-] Undearius@lemmy.ca 22 points 1 year ago* (last edited 1 year ago)

I've experienced a taste of this already. I checked the instance list a couple days ago, and didn't see one that stood out for my interests, so I created an account on the main lemmy.ml instance.

I just registered the same username on another but as far as I can tell, there is no way to merge or link these two accounts. So all the setup I've done and all the communities I've subscribed to, I have to do over again.

------

Another "issue" (a bug or feature?) I'm seeing is there are a lot of duplicate communities between the instances. I guess one will eventually "prevail" and become the defacto instance for that community.

load more comments (13 replies)
load more comments (10 replies)
[-] dessalines@lemmy.ml 39 points 1 year ago

Another thing:

We do need more site admins to help us handle the applications and moderation.

For obvious reasons, we prefer ppl who have been here for a long time, and post / comment consistently. If you'd like to help us out, so that nutomic and I can focus on coding, that would be splendid.

load more comments (6 replies)
[-] LoreleiSankTheShip@lemmy.ml 35 points 1 year ago

I tried like 4 or 5 instances before coming to lemmy.ml, but none of them were taking applications anymore. Finding even those was a hassle, since all I got was a list of domains without any details as to what the instance is about or if they allowed newcomers.

Now that I've setup everything, Lemmy does seem like nice alternative to Reddit, but as someone from the outside, all of this is daunting.

load more comments (3 replies)
[-] UnpopularBrainRot@lemmy.ml 33 points 1 year ago

I sent my registration yesterday, because I signed in another instance, one from my country, but I couldn't see all the post and no comments from lemmy.ml even thought is supposedly linked, so thank you for approving my account.

Even if I'm a tech savvy person I found the whole experience of joining lemmy pretty bad, I like the concept of federation, but I think it's too confusing to normal people, it really needs to be more seamless if you want to grow, how? idk, I was thinking some sort of replication, when you sign up, you are registered to the main instance (this) and given the choice to select other instances, automatically selecting let's say another 3 based on your location, then your account is synced in all the registered and linked instances, when you login if an instance is experiencing overload then it switches to another one. I don't know if this is realistic or out of the scope of Lemmy, or maybe against the philosophy of it. I'm just rambling.

I'm just glad that there is an open alternative for anonymous social interaction in this day of walled internet services such as discord, twitter, facebook etc. and I wish you all the success.

[-] Deiv@lemmy.ml 23 points 1 year ago

Agreed, someone needs to create an easy "sign up here" with a default option (maybe just randomize across various instances, not sure)

load more comments (6 replies)
load more comments (5 replies)
[-] Generator@lemmy.pt 33 points 1 year ago

If now is struggling then on June 12 will be a nightmare.
Reddit will go dark in protest, many messages to join Lemmy, most instances will be overloaded or even DDoS with so many users, like what happen with Mastodon.

load more comments (3 replies)
[-] darkkite@lemmy.ml 29 points 1 year ago

I wonder if a longer term solution would be to auto rotate the server list to bump less popular ones.

[-] nutomic@lemmy.ml 32 points 1 year ago

Pull requests welcome, here is the repo:

https://github.com/LemmyNet/joinlemmy-site

[-] beto@lemmy.studio 26 points 1 year ago

I just created https://lemmy.studio/ for music-related discussions.

[-] Barbarian@lemmy.ml 26 points 1 year ago

Users are likely going to see this as it's the "official" Lemmy instance when trying to join for the first time.

Any admins of instances that are accepting people, give your best elevator pitch!

load more comments (1 replies)
[-] papasfritas@lemmy.ml 26 points 1 year ago* (last edited 1 year ago)

Crashing with 994 users online?

that's not a good look

[-] 0xc0ba17@lemmy.ml 45 points 1 year ago

Well when you host your own website with your own funds, you try to not pay for unneeded perfs. But then a few hundreds people join, register, create content, upload images and videos, and suddenly your small VPS can't handle the load. And you're hesitant to scale up because it costs money, sometimes it costs time too because you need to migrate stuff, and maybe in a few days (or a few hours?) that load will disappear and now you're on a more expensive tier for nothing.

[-] Two9A@lemmy.ml 18 points 1 year ago

Can attest: I have a small VPS (docean's lowest tier, in fact) and in the deep past I had occasions where a blog post was popular enough to throw it offline for a while. I wasn't about to lock myself into paying more money forever, to keep the site running for a day's peak traffic.

In the case of lemmy.ml, this isn't helped by the frontend being Websockets-driven, which holds open a connection for every concurrent user; I hear Dessalines and crew are working on cutting that dependency out and dropping down to something more static, which should help with load.

load more comments (5 replies)
[-] Bicyclejohn@lemmy.ml 26 points 1 year ago

Is there any way to migrate instance or do I make a new account entirely?

[-] Neuromancer@lemmy.ml 22 points 1 year ago

You have to make a new account unfortunately.

[-] FaceDeer@lemmy.ml 21 points 1 year ago

Seems like this should be a high priority feature. I did try joining a different instance but at the time only lemmy.ml was functioning and accepting applications. Now that I have subscribed to a bunch of communities starting afresh would be a bit of a hassle.

load more comments (4 replies)
[-] 1337admin@1337lemmy.com 21 points 1 year ago

Got my instance running earlier today thanks to some helpful people in the Matrix chat. Intention at the time was to have some friends of mine join but I'm not opposed to having signups as well. Long live open source, federated software!

load more comments (1 replies)
[-] Rhabuko@feddit.de 20 points 1 year ago* (last edited 1 year ago)

I guess every bigger instance needs to constantly promote a monthly support fee to keep the servers runnin. Something that most people already use and don't have to register first. Like PayPal or Patreon.

Edit: And I support Lemmy now on Patreon.

load more comments (2 replies)
[-] vmaziman@lemmy.ml 20 points 1 year ago

If server crashes again the devs should just stop all registrations to this instance, people always follow path of least resistance and the first dev instance is just too easy to be stuck onto

load more comments (4 replies)
[-] cecirdr@lemmy.ml 18 points 1 year ago* (last edited 1 year ago)

I'm a noob. I created an account on beehaw and on lemmy.ml. That's because I see communities on one instance that I'm interested in and a different community on another instance. So if there's a technology community on both, how do I get to see all the technology posts without having to have two accounts?

This is really confusing for noobs like me. I'd just like to see one community to technology, one for Science, one for nintendo etc. I don't care it it's spread out amongst different servers to divvy up the load, but from the user side, it needs to be seamlessly integrated.

I'm still learning how all this works though. But I don't know how many folks that are more casual than me will be willing to figure it out. I hope they do though! It'll be worth it to leave reddit in the rearview mirror!

Edit: lawdy, I just figured it out. Local vs all on the communities list. It was right in front of my face. good grief!

load more comments (6 replies)
[-] ilikebagels@lemmy.ml 18 points 1 year ago

I just created https://lemmy.film if that would be useful for anyone.

[-] tmpod@lemmy.pt 18 points 1 year ago

I know I'm being a bit pushy at this point, but distributing instance load can be helped in some part by merging this PR and deploying the latest changes (including more languages and recommended instances as well) :)

load more comments (1 replies)
load more comments
view more: next ›
this post was submitted on 04 Jun 2023
851 points (100.0% liked)

Lemmy

12544 readers
94 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 4 years ago
MODERATORS