131
submitted 1 month ago by jordanlund@lemmy.world to c/world@lemmy.world

We've had some trouble recently with posts from aggregator links like Google Amp, MSN, and Yahoo.

We're now requiring links go to the OG source, and not a conduit.

In an example like this, it can give the wrong attribution to the MBFC bot, and can give a more or less reliable rating than the original source, but it also makes it harder to run down duplicates.

So anything not linked to the original source, but is stuck on Google Amp, MSN, Yahoo, etc. will be removed.

you are viewing a single comment's thread
view the rest of the comments
[-] Rooki@lemmy.world 8 points 1 month ago

MBFC API is free as they gave us access for us as a Non Profit.

We already had in mind adding these sources to our bot but we didnt had the time and knowledge how to scrape that. Personally i would like to host it on our own server so that we dont require you to use your own money just for one bot, in what programming language did you write it?

Thanks a lot!
Rooki

[-] PhilipTheBucket@ponder.cat 10 points 1 month ago

On a different topic: It sounds like jordanlund is saying that if he tried to remove the MBFC bot from the politics sub, he might be removed as a moderator, and replaced with someone else, and the bot would come back.

https://lemmy.world/comment/12825768

Is that true? Is the admin team mandating the use of this bot, and if so, why?

[-] Rooki@lemmy.world 6 points 1 month ago* (last edited 1 month ago)

No, i dont get it from where he would get that idea, because see c/politics mods wanted the bot gone and we removed it no question asked.

@jordanlund@lemmy.world if you really dont want the bot here we can remove the bot and shut the bot down ( please consult other c/world mods too )

[-] goferking0@lemmy.sdf.org 6 points 1 month ago

You mean news? Cause it's still running on politics

[-] PhilipTheBucket@ponder.cat 10 points 1 month ago

Here you go:

https://ponder.cat/wp/wp-sources.zip

It's in python, suitable for sticking directly into the bot if the bot is in python. There are docs. It's a first cut. How did you envision this working? I can make a real API, if for some reason that makes things easier, but it's not immediately obvious how it would get integrated into things.

Running it on the last 50 articles posted to /c/politics, we see:

It's more complex to use this than MBFC, because there's a lot more depth to the rankings, and sometimes human judgement is needed to assign scores. There's a category "needinfo," meaning it's necessary to know what topic is being discussed or when an article was written, because of an ownership change or similar factor. I've applied that judgement above. That, to me, is a good thing. It means the bot is grounded in something, and not just blithely spitting out arbitrary scores without bothering to ground them in any reality.

In practice, I think it would be realistic to assign a single reliability ranking to most of the "needinfo" sources. You can manually edit the .json data to do so. Almost all of the posts are going to fit into one of Wikipedia's categorizations or another. Newsweek is unreliable, The Guardian is reliable, and so on.

I think most of the mixed-consensus sources can be used without a second thought. Mostly, the questions about them boil down to open partisanship of the source, which for a political community is perfectly fine as long as they're trustable factually.

If you want me to boil this down further, so that it gives a single "yes" or "no" score to each source, I can do that and probably keep almost all of the accuracy of the rankings, now that I've looked at it for a little while.

When you talk about "adding" this to the bot, are you proposing to still have MBFC be the main source, with this as a footnote? A lot of the criticism of the bot is on the grounds that MBFC is a very bad source for judging reliability, so I would question the idea of keeping it on as the primary source.

[-] NOT_RICK@lemmy.world 4 points 1 month ago

Nice work, thanks for contributing!

[-] Rooki@lemmy.world 3 points 1 month ago

By "adding" i mean adding it into the field higher than MBFC ( as i personally think wikipedia is a little bit better for that ).

new:

Wikipedia: Reliability consensus is mixed.....l ( whatever the scrapper scrapes ) MBFC: Right-Center - Credibility: High - Factual Reporting: Mostly Factual - United States of America
Search Wikipedia about this source

I would like to implement your code into the bot myself so i can learn how you would do it. If you are willing to share your code, please send me a github link ( or invite me if you want it to be private between you and me ) or if its super simple just send it in the dms.

[-] PhilipTheBucket@ponder.cat 7 points 1 month ago* (last edited 1 month ago)

I already sent it. It's here:

https://ponder.cat/wp/wp-sources.zip

Edit: You don't need to do the import initially, since there's already a sources file with some small modifications. The import is the only complicated part. Use categorize.py to categorize a source, or lookup.py to run a quick command-line test.

[-] Rooki@lemmy.world 4 points 1 month ago

Ok i will look into it, thanks i thought it was just the sources not the code.

[-] Rooki@lemmy.world 2 points 4 weeks ago

Ok i implemented it into the bot and it took about 1 hour and 6 minutes to fetch all links and i am now implementing the part where it is inserted into the new text.

[-] PhilipTheBucket@ponder.cat 2 points 4 weeks ago

Sounds good. If you redid the import, I think you’ll want to make some manual fixes to the .json. Off the top of my head, I think you just need to add bbc.co.uk and aljazeera.com to the URL lists for those sources.

[-] nmtake@lemm.ee 8 points 1 month ago

Since it's a MediaWiki page you can get Markdown source of the page with appending action=raw query to the URL.

this post was submitted on 10 Oct 2024
131 points (100.0% liked)

World News

39000 readers
2053 users here now

A community for discussing events around the World

Rules:

Similarly, if you see posts along these lines, do not engage. Report them, block them, and live a happier life than they do. We see too many slapfights that boil down to "Mom! He's bugging me!" and "I'm not touching you!" Going forward, slapfights will result in removed comments and temp bans to cool off.

We ask that the users report any comment or post that violate the rules, to use critical thinking when reading, posting or commenting. Users that post off-topic spam, advocate violence, have multiple comments or posts removed, weaponize reports or violate the code of conduct will be banned.

All posts and comments will be reviewed on a case-by-case basis. This means that some content that violates the rules may be allowed, while other content that does not violate the rules may be removed. The moderators retain the right to remove any content and ban users.


Lemmy World Partners

News !news@lemmy.world

Politics !politics@lemmy.world

World Politics !globalpolitics@lemmy.world


Recommendations

For Firefox users, there is media bias / propaganda / fact check plugin.

https://addons.mozilla.org/en-US/firefox/addon/media-bias-fact-check/

founded 1 year ago
MODERATORS