274

I'm Starting A Search Engine For The Fediverse (lemmy.ca)

submitted 10 months ago by lautan@lemmy.ca to c/fediverse@lemmy.world

65 comments fedilink hide all child comments

Hey everyone,

This isn't an announcement, just wanted peoples thoughts on this.

I think everyone knows searching the fediverse can be better. Googling doesn't work too well, etc. So I wanted to do my part and help out.

Indexing all posts, etc is quite a lot to handle, so I wanted to start small and just focus on video search. I've started indexing videos from Peertube and other video websites. (Even YouTube but this could be removed to just focus on independent sites)

I know Peertube has their own search engine for videos. I will be reaching out to them. Compared to my site I'm planning it'll have other video sources and be easier to use.

So that leads to feedback from you guys.

What do you think about indexing videos posted on the fediverse and other independent platforms?
Are there similar services?
Am I just wasting my time?

you are viewing a single comment's thread
view the rest of the comments

[-] gabe@literature.cafe 35 points 10 months ago* (last edited 10 months ago)

Well, please make sure it respects post privacy at least but also realize that on the microblogging side of the fediverse, they may not take kindly to this prospect at all. People who start these kinds of projects are often harassed or at least receive passive hostility. Making it opt in instead out of opt out in some capacity is best.

[-] scrubbles@poptalk.scrubbles.tech 42 points 10 months ago* (last edited 10 months ago)

I disagree. Post privacy sure, but the internet is by definition public. Anything you put out there can be used for pretty much everything, the original rules of the internet apply. I'd be happy to see an easy opt out on the engine to remove yourself, but if everything is opt in it'll never get off the ground.

[-] gabe@literature.cafe 7 points 10 months ago

That's not how the fediverse functions and approaching it that way is a problem waiting to happen. I'm stating so as a warning to be mindful of the culture of the way the fediverse itself functions. This is not Reddit, we share the fediverse with other software with different uses and features and we need to be mindful of that especially when building these kinds of tools. Making it opt out not only places a burden on smaller instances but presents a potential harassment risk for instances with vulnerable people on other fediverse platforms. As well, it is contrary to the entire way specific other activitypub instances operate. The fediverse is like a city we share with others, if Lemmy is not mindful of that city's culture then people will promptly give them the boot.

I'm not saying user by user opt in either, but instance by instance. Lemmy needs a tool of archiving especially. There is already cultural clashes I see occurring with the rest of the fediverse. Post like these of potential tools when it seems like the creator doesn't know the messy history behind previous projects like them in the fediverse make me fearful of the clashes coming to fruition.

[-] lautan@lemmy.ca 13 points 10 months ago

Well that’s why I’m asking for input. And I won’t launch this on every instance without letting them know. Baby steps.

[-] gabe@literature.cafe 5 points 10 months ago* (last edited 10 months ago)

My matrix is open if you want/are actually interested in doing this in a way that won't make the rest of the fediverse flip shit. I support this tools creation especially for lemmy, but if it isn't done the right way it'll be received poorly. Making it behave differently on lemmy compared to other software as well might be an idea too.

[-] Kierunkowy74@kbin.social 4 points 10 months ago

Mastodon since 4.2 version supports allows its users to opt into appearing into search results. Just respect this flag with Mastodon users, and you will be fine, IMHO