Would there be any benifit to host another instance? (reddthat.com)

submitted 1 year ago by Mirriad@reddthat.com to c/fediverse@lemmy.world

56 comments fedilink hide all child comments

As the question states. I have the skiils and know how to setup a new instsnce and from looking through the documentation its relatively straight forward. Im just wondering would it actually help the fediverse and lemmy as a whole? There seems to already be plenty of instances to choose from. What are your guys thoughts?

top 45 comments

sorted by: hot top controversial new old

[-] Candelestine@lemmy.world 32 points 1 year ago

There's always going to be people that want to be on a smaller Instance, because it's frankly an objectively superior experience unless you get unlucky with your admin. There's really no major advantages to being on a large Instance. That I know of anyway, and I'm on one.

So, running a smaller Instance is probably always going to soak up some demand, so long as you advertise it a little bit.

[-] smitten 16 points 1 year ago

One downside that I’ve run into is discovering communities. Your instance only gets updates from communities someone on that instance is subscribed to, so you’ll have to go to other instances to discover them.

[-] NettoHikari@social.fossware.space 13 points 1 year ago

My instance uses a seeder script that'll do exactly that, but automated. It'll check the most popular communities on the most popular servers and use an account to subscribe to them.

Boom, /all feed populated.

[-] smitten 4 points 1 year ago

Oh cool, didn’t know that existed. I’ll look into that

[-] Socialphilosopher@lemm.ee 4 points 1 year ago

wav perfect solution

[-] JustAHouseCat@lemmy.world 2 points 1 year ago

Did you guys make that script or is there a repo for it cause that would be very useful.

[-] NettoHikari@social.fossware.space 6 points 1 year ago

There are several scripts working like that. I use lcs (lemmy community seeder). I'm going to switch to something else or make my own, because lcs doesn't handle it well if an instance is overloaded or not available.

[-] shinjiikarus@mylem.eu 3 points 1 year ago* (last edited 1 year ago)

Not OP, but I’m using this one: lemmony on GitHub

EDIT: deleted link, I feel like I made a mistake, see below.

[-] PriorProject@lemmy.world 3 points 1 year ago

Of the 3 subscription bootstrappers listed in this thread, lemmony is by far the worst of them because it subscribes to EVERYTHING by default.

Lcs forces you to pick a number of communities to subscribe to, and the other one has default threshold heuristics that pick a limited number of active communities. Lemmony signs you up for the entire firehose of the threadiverse which both makes instances using it pretty bad fediverse citizens in terms of generating a 50x-100x the federation load of a "normal" single-user instance that subs maybe a hundred communities... and also exposes novice single-user instance owners to legal liability by subbing all the small under-moderated communities full of questionably illegal stuff.

I would recommend one of the better designed tools, and to review the resulting subscription list manually to ensure you're not signing up for some sketchy stuff.

[-] shinjiikarus@mylem.eu 3 points 1 year ago

Yeah, I let it run by night and saw what I did today. You are totally right, I will probably need to clean up a few things.

[-] PriorProject@lemmy.world 2 points 1 year ago

I'm not experienced at cleanup, and I don't think lemmony offers an undo. But if you used an account other than your primary... I think deleting that account will nuke its subs and you can start over. If I'm wrong, this may leave a bunch of orphaned subs with no user and no way to write an api-script to unsub.

In either case, I do recommend doing your automated subs on a different account and viewing them in all rather than directly subbing your main account to all that junk. It definitely give you more options for cleanup.

[-] shinjiikarus@mylem.eu 2 points 1 year ago

I created a bot account for this, so all green on that front, will experiment with it, thanks!

[-] throwaway_OT05wZjv@lemmy.world 2 points 1 year ago* (last edited 1 year ago)

[-] PriorProject@lemmy.world 2 points 1 year ago

Awesome feature. It's great to see you thoughtfully trying to balance the community discovery problem with the federation overhead caused by mass subscription.

While I continue to favor and recommend manual subscription, I've bookmarked lsb and lcs to recommend to folks looking for automated approaches. I think they're looking pretty good right now and seeing you proactively address feedback gives me faith that they'll get even better.

[-] Candelestine@lemmy.world 2 points 1 year ago

You still have the same search capabilities and the same all feed though. Just not a local feed worth anything.

[-] Bishma@social.fossware.space 18 points 1 year ago

IMO the best communities serve a niche. Then you get a bunch of like minded members on one server and you end up with a local feed that is likely to be full of personally interesting stuff.

[-] iopq@lemmy.world 8 points 1 year ago

Yeah, for example, I'm on fosstodon and I like the local feed because it's all people talking about coding and open source software, it's nice to see the local feed even though I'm not subscribed to too many people on Mastodon

[-] dfc09@lemmy.world 11 points 1 year ago

Would it make any sense to stand up my own instance, federated with what I want, follow the communities I want from other instances, and just ignore the rest? Is there any reason to not do that?

[-] flashmedallion@lemmy.nz 8 points 1 year ago

If you're happy to pay whatever it costs to host then there's no other disadvantage to this.

The only thing I can think of is if you somehow end up at odds with an admin of a big instance and they defederate your instance. Or perhaps in the future major instances decide to only federate with instances of a certain size, for some as of now unpredictable reason

[-] Nougat@kbin.social 1 points 1 year ago

Really, if your instance is just for you and your account (maybe a few friends, family), and you're not running any communities on it (all your posts and comments are on other instances), there's really no reason anyone would defed you. Unless you're a dick.

I have to think that if your very small private instance was federated with a large public instance, your instance would pull down content from the public one. That's bandwidth, and that's costing the large instance some amount of money/performance. Because your small private instance isn't "giving back" a level of content that the large instance would want or need, that could be the unpredictable reason you mentioned.

Now, I don't see that being a real issue unless there are so many low-content instances that Large Instance reviews their usage and finds out that an unacceptably high portion of their bandwidth is being used by "leeches," and wants to control cost.

[-] PrimaCora@lemmy.fmhy.ml 0 points 1 year ago

Could even run the instance from your phone or whatever device used to look at Lemmy

[-] NotAPenguin@kbin.social 2 points 1 year ago

Yeah no, it's not that simple lol

[-] OptimisticPrime@lemmy.fmhy.ml 0 points 1 year ago

Has anyone done that and documented it? I know android phones are sort of linux, but still, does need some modifications.

[-] eric5949@lemmy.cloudaf.site 1 points 1 year ago

That's what I do as of a couple days ago, it's working pretty ok so far.

[-] herrvogel@lemmy.world 0 points 1 year ago

What does it realistically take to host that? Would one of them smaller Digitalocean droplets be enough, for example?

[-] eric5949@lemmy.cloudaf.site 1 points 1 year ago

MIne's on a 1 core 2gb Linode, but I am also the only user.

[-] BlueEther@no.lastname.nz 0 points 1 year ago

I run an instance on a 2 core 4gb vps, I think the disk space will be the thing that I'll run out of

30 reg'd accounts and about 10 active users

[-] herrvogel@lemmy.world 1 points 1 year ago

How fast is your storage filling up with 10 active users?

[-] BlueEther@no.lastname.nz 1 points 1 year ago

about 8GB over the last 4 weeks

[-] oleorun@lemmy.world 7 points 1 year ago

I like to tinker with software and thought a standing up an instance just for myself. Then you can keep your account, domain, communities, etc relatively static.

I think having various instances geographically distributed is also beneficial so popular instances can be offloaded a bit.

[-] bionicspud@lemmyverse.org 6 points 1 year ago

I created mine because I am a systems engineer by trade and I moved on from Reddit. Nobody else has used my instance except for me, but I think it's worth it. I like being in control of my own account and can be assured my instance wont impose rules that i dont agree with or worse, just disappear. If other people find it useful, then that's even better.

[-] Cat@kbin.social 1 points 1 year ago

@bionicspud, I've been thinking of doing the same thing for the same reasons. How much storage does your instance use? I haven't looked in to how asset caching works yet. It seems like it could easily get out of hand.

[-] DarkIrata@lemmy.gwa.app 2 points 1 year ago* (last edited 1 year ago)

Running the Instance for 3 Days now. Added multiple new Subs and cleared pictrs (but not Postgres) folder yesterday. My instance is mostly used by me.

87M ./pictrs

335M ./postgres

Edit: Small update. Original Post was posted ~9AM. Now its ~11AM

112M ./pictrs

351M ./postgres

As far as i know, images shouldn't be stored locally if not posted from the local instance. This looks to be a proxing problem as far as i understood.

[-] Die4Ever@programming.dev 2 points 1 year ago* (last edited 1 year ago)

what happens when you clear pictrs? you mean just deleting the folder? does that break stuff?

for comparison, my instance is only communities and no users (FAQ here: https://programming.dev/post/442419 )

it's about 5 days old but not much activity

# du -h --max-depth=2 ./
73M     ./volumes/postgres
7.8M    ./volumes/pictrs
8.0K    ./volumes/lemmy-ui
81M     ./volumes
81M     ./

the latest backup is only 7.4 MB zipped, I use this backup script

#! /bin/bash
# https://join-lemmy.org/docs/administration/backup_and_restore.html#a-sample-backup-script
now=$(date +"%Y-%m-%d_%H.%M.%S")

cd ~/lemmy && (docker-compose exec -T postgres pg_dumpall -c -U lemmy 1> dump.sql 2> dump.errors)
cd ~/lemmy && zip -r9 ~/bak-lemmy-$now.zip ./ --exclude "volumes/postgres/*"
rm -f ~/lemmy/dump.sql

like I said though, not much activity

and here's how my backup zip files have grown in size over time

6.6M bak-lemmy-2023-07-08_03.00.01.zip
6.6M bak-lemmy-2023-07-08_06.00.01.zip
6.6M bak-lemmy-2023-07-08_09.00.01.zip
6.6M bak-lemmy-2023-07-08_12.00.01.zip
6.6M bak-lemmy-2023-07-08_15.00.01.zip
6.6M bak-lemmy-2023-07-08_18.00.01.zip
6.6M bak-lemmy-2023-07-08_21.00.01.zip
6.6M bak-lemmy-2023-07-09_00.00.02.zip
6.6M bak-lemmy-2023-07-09_03.00.01.zip
6.6M bak-lemmy-2023-07-09_06.00.01.zip
6.6M bak-lemmy-2023-07-09_09.00.01.zip
6.6M bak-lemmy-2023-07-09_12.00.01.zip
6.6M bak-lemmy-2023-07-09_15.00.01.zip
6.6M bak-lemmy-2023-07-09_18.00.01.zip
6.6M bak-lemmy-2023-07-09_21.00.01.zip
6.7M bak-lemmy-2023-07-10_00.00.01.zip
6.7M bak-lemmy-2023-07-10_03.00.01.zip
6.7M bak-lemmy-2023-07-10_06.00.01.zip
6.7M bak-lemmy-2023-07-10_09.00.01.zip
6.8M bak-lemmy-2023-07-10_12.00.01.zip
6.8M bak-lemmy-2023-07-10_15.00.01.zip
7.0M bak-lemmy-2023-07-10_18.00.01.zip
7.0M bak-lemmy-2023-07-10_21.00.01.zip
7.0M bak-lemmy-2023-07-11_00.00.01.zip
7.0M bak-lemmy-2023-07-11_03.00.01.zip
7.0M bak-lemmy-2023-07-11_06.00.01.zip
7.0M bak-lemmy-2023-07-11_09.00.01.zip
7.0M bak-lemmy-2023-07-11_12.00.01.zip
7.0M bak-lemmy-2023-07-11_15.00.01.zip
7.4M bak-lemmy-2023-07-11_18.00.01.zip

[-] DarkIrata@lemmy.gwa.app 2 points 1 year ago* (last edited 1 year ago)

Tested deleting the pictrs/files/001 which was at that point at ~150mb. And yes, it broke stuff. For example my Avatar and a local test community logo. To "purge" it, something would need to compare it to the database i guess. My size probably comes from /c/memes 😅
I thought remote images aren't cached localy 🤔

[-] Die4Ever@programming.dev 1 points 1 year ago

I think it might just be thumbnails that are cached locally? And obviously your own banner and local images

[-] DarkIrata@lemmy.gwa.app 1 points 1 year ago* (last edited 1 year ago)

Looks like random? Both images are from meme@lemmy.ml didn't post on any of these 2 threads. It would be nice to know why sometimes its get copied. https://lemmy.ml/pictrs/image/b2e8d1e4-b638-4d04-9710-9fa28756261d.jpeg?format=webp https://lemmy.gwa.app/pictrs/image/962b1734-d524-468c-b3f9-4d615f91f094.png?format=webp

The interesting thing is, both posts originated from .world and when opened from the original sources are also hosted the.

https://lemmy.world/post/1318708

https://lemmy.gwa.app/post/1437

https://lemmy.world/post/1348802

https://lemmy.gwa.app/post/2310

[-] Crow@lemmy.world 0 points 1 year ago

How much storage does it take to host your own instance that’s federated with large servers? I am really curious about hosting my own instance but I don’t really understand how the storage works and don’t want my computer filled with thousands of strangers comments.

[-] bionicspud@lemmyverse.org 2 points 1 year ago

In the month I've been running my server, I have accumulated about 8GB of data. I've subscribed to about 30 communities or so, but my server only has be actively using it.

[-] curiosityLynx@kglitch.social 1 points 1 year ago

If you're the only user on your instance, your server will only know about and fetch from communities you're personally subscribed to. If there's a second user, add the communities that user is subscribed to, etc.