188
scraperule (lemmy.zip)
submitted 2 years ago by Monologue@lemmy.zip to c/196
you are viewing a single comment's thread
view the rest of the comments
[-] b3nsn0w@pricefield.org 12 points 2 years ago

fun fact: on the r-site, you can still append .json to the end of any path (before the query params) to get the formatted data

fun fact 2: on the same site you get a similar json if you grab the script that says id="data" (trivial with jsdom if you run nodejs), eval it in a sandbox (node's built-in vm package), and look for your passed global object's $.___r param

fun fact 3: also on the same site, if you use the old interface it's full of data tags intended for css, jsdom goes brrr

fun fact 4: even if they stopped all of this you could use a headless browser and grab the data in flight from the api calls (virgin dom scrubber vs chad api capturer)

i don't know much about the t-site and can't check right now because you can't even access it the normal way, lol

this post was submitted on 04 Jul 2023
188 points (100.0% liked)

196

17520 readers
1169 users here now

Be sure to follow the rule before you head out.


Rule: You must post before you leave.



Other rules

Behavior rules:

Posting rules:

NSFW: NSFW content is permitted but it must be tagged and have content warnings. Anything that doesn't adhere to this will be removed. Content warnings should be added like: [penis], [explicit description of sex]. Non-sexualized breasts of any gender are not considered inappropriate and therefore do not need to be blurred/tagged.

If you have any questions, feel free to contact us on our matrix channel or email.

Other 196's:

founded 2 years ago
MODERATORS