95
submitted 1 year ago by jonah@lemmy.one to c/meta@lemmy.one

As you reading are undoubtedly aware now, the Lemmy.one instance experienced a massive failure this weekend. Unfortunately the data was not in a recoverable state, so the server was restored from a July 26th backup, and data after that time is likely lost.

Since this occurred while I was out of town, I haven't had the time to collect all the details of what happened, so I will have to post more information at a later time.

all 45 comments
sorted by: hot top controversial new old
[-] ViciousTurducken@lemmy.one 23 points 1 year ago

My biggest concern was that there was no way to find information about what was going on. Your Mastodon had no information.

There was a thread on Lemmy support about it and some people thought the instance was gone forever.

[-] EliasChao@lemmy.one 6 points 1 year ago

I’m in the same boat as you. Although I was told to check http://lemmy-status.org/ to see the status of the instance, truth is I didn’t know if Lemmy.one was ever going to be up again.

[-] ViciousTurducken@lemmy.one 3 points 1 year ago

I had no idea this existed!

[-] DidacticDumbass@lemmy.one 5 points 1 year ago

I got confused, and ended up making an alternative account in another instance. Glad Ido not have to worry about it, but redundancy is a virtue.

[-] NightOwl@lemmy.one 2 points 1 year ago

With each day with no news I started considering making another lemmy account in case this one didn't come back. I didn't know which one to make my new home though, since I liked privacyguides run one the best.

[-] DidacticDumbass@lemmy.one 2 points 1 year ago

I think this is a wonderful instance and I am happy to be in it.

I hope it stays stable.

[-] dill@lemmy.one 4 points 1 year ago

Yeah really would be good to have a separatly hosted status page or something, but I understand that is a luxury. Considering Jonah has freely volunteered to host it's hard to complain.

[-] lauha@lemmy.one 15 points 1 year ago

Thank you for your work! No one is expecting you to be always available, considering this is voluntary work you are doing.

[-] SocialEngineer55@lemmy.one 10 points 1 year ago

@jonah can this post be unpinned? Don’t think I need it at the top of my feed anymore :)

[-] jb007gd@lemmy.one 8 points 1 year ago

Why is this still pinned?

[-] Johnnypneumoniac@lemmy.one 7 points 1 year ago

Just another appreciation comment here. Thank you! Glad it was just temporary, and hope it doesn't cause you too much stress.

[-] Kalkaline@lemmy.one 7 points 1 year ago

Thanks for getting it back up.

[-] Catpuccino@lemmy.one 7 points 1 year ago

Can we please remove this as a pin?

[-] user@lemmy.one 6 points 1 year ago

ty 4 ur work 🙏 look after yourself.

[-] minthenry@lemmy.one 5 points 1 year ago

Thank you for your work! I'm glad everything is back up and running.

[-] Melody@lemmy.one 5 points 1 year ago

Thank you for your hard work none-the-less.

I had a feeling it was just that. I heard rumbles about some type of issue in the lemmy software itself; although I don't know the exact details. It appears similar problems happened to other instances as well; so don't take it too hard...several massive instances did in fact crash.

[-] agent_flounder@lemmy.one 2 points 1 year ago

I noticed lemmy.world started displaying the same server error as lemmy.one a day or so after (I switched over to that instance after this one had issues).

[-] agent_flounder@lemmy.one 2 points 1 year ago

...And lemmy.world just went down again.

[-] dill@lemmy.one 4 points 1 year ago

Thanks for the hard work and expense of hosting. Appreciate it

[-] CyanFen@lemmy.one 4 points 1 year ago

Unpin this. It's 3 months old.

[-] monty@lemmy.one 3 points 1 year ago

Does anyone know why this is still showing up at the top of my "active" feed?

[-] lemann@lemmy.one 1 points 1 year ago

I think that's because it's pinned by the instance admin - if you unsub from this meta community it should disappear from your feed

[-] s_s@lemmy.one 3 points 1 year ago* (last edited 1 year ago)

Glad to be back; Thanks, man!

What's up with browsing lemmy.world? Did it de-federate from us here or is it a syncing issue?

[Edit]Fine now

[-] Bandicoot_Academic@lemmy.one 5 points 1 year ago

It seems that lemmy.world is expirencing server issues so thats likely it

[-] Tyfud@lemmy.one 2 points 1 year ago

Thank you for your hard work! I love this instance the best.

[-] jordanlund@lemmy.one 2 points 1 year ago

Welcome back! We were all terribly worried about the outage and the radio silence. Thanks for posting up an explanation!

[-] CCatMan@lemmy.one 2 points 1 year ago

I never used mastodon so much in my life.

[-] ojmcelderry@lemmy.one 2 points 1 year ago

Same! 😅 huge thanks to @jonah@lemmy.one for bringing lemmy.one back online. I wasn't ready to mourn the loss of my fediverse homepage.

[-] FleshPrison@lemmy.world 2 points 1 year ago* (last edited 1 year ago)

Uptime robot can provide both monitoring and status page for the user. Could be an option?

https://uptimerobot.com/status-page/

If something where to happen I think a message could be added to the status page as well.

[-] Iwasondigg@lemmy.one 2 points 1 year ago

Thanks for your work on this instance!

[-] Sentau@lemmy.one 2 points 1 year ago* (last edited 1 year ago)

So I have a question. Since the last backup using which the server was on the 26th, there are several posts and comments missing. But since most of these comments and posts originate from other instances and are still there, why can't lemmy.one pull these comments/posts from the instances¿?

Also Jonah thanks for all the hard work.

[-] DrVortex@lemmy.one 1 points 1 year ago

And there is lies the point of failure. If we generate posts and so on...will they also disappear if a particular lemmy server disappears?

[-] lemann@lemmy.one 2 points 1 year ago

No - the posts and comments you make are copied to other federated servers

[-] gridleaf@lemmy.world 1 points 1 year ago

Are new signups not being accepted? It's been nearly a week since I applied. I thought I answered the questions appropriately. Other instances accepted relatively quickly, within a couple of days.

[-] lemann@lemmy.one 1 points 1 year ago

Life's probably happened I think, he hasn't uploaded to his YT or been on the Techlore Talks podcast in a while

[-] diskmaster23@lemmy.one 1 points 1 year ago* (last edited 1 year ago)

Is it me, or are some of our comments missing? I feel like I made some comments on another server, but they are not there. I guess I don't understand how this works.
By the way, thanks for your help.

[-] accesslog@lemmy.one 3 points 1 year ago* (last edited 1 year ago)

restored from a July 26th backup

[-] diskmaster23@lemmy.one 2 points 1 year ago

I don't underhand why the comments disappeared from a different server.

[-] PeachMan@lemmy.one 1 points 1 year ago

Maybe because that server synced with this server, assuming this is your home instance. If the comments aren't on any record here, they might get deleted from the other server.

Option 2: you attempted to make those comments but they never went through because this server was already having problems. So they never truly existed on the other server?

I'm just guessing, I don't know how the hell this fediverse stuff works.

[-] BrainisfineIthink@lemmy.one 1 points 1 year ago* (last edited 1 year ago)

Thanks for all you do! Can someone (either you or someone that knows) post a link to your funding page? I've been meaning to set up a donation since this is my preferred instance but haven't got around to it. Missing it all. Weekend is a good reminder!

[-] ojmcelderry@lemmy.one 2 points 1 year ago

@jonah@lemmy.one’s links are here:

https://jonah.omg.lol

Or a direct link to his Ko-Fi:

https://ko-fi.com/jonaharagon

[-] spaceghoti@lemmy.one 1 points 1 year ago

Are we having a server failure today? I seem to have no ability to post comments.

[-] HotsauceHurricane@lemmy.one 0 points 1 year ago* (last edited 1 year ago)

I’m just glad it’s back up. Thanks for the hard work.

this post was submitted on 31 Jul 2023
95 points (100.0% liked)

Meta (lemmy.one)

1004 readers
1 users here now

A place to discuss or ask anything about lemmy.one's instance or moderation.

For discussion about Lemmy (the software) itself, visit !lemmy@lemmy.ml

founded 1 year ago
MODERATORS