188
scraperule (lemmy.zip)
submitted 2 years ago by Monologue@lemmy.zip to c/196
you are viewing a single comment's thread
view the rest of the comments
[-] b3nsn0w@pricefield.org 12 points 2 years ago

fun fact: on the r-site, you can still append .json to the end of any path (before the query params) to get the formatted data

fun fact 2: on the same site you get a similar json if you grab the script that says id="data" (trivial with jsdom if you run nodejs), eval it in a sandbox (node's built-in vm package), and look for your passed global object's $.___r param

fun fact 3: also on the same site, if you use the old interface it's full of data tags intended for css, jsdom goes brrr

fun fact 4: even if they stopped all of this you could use a headless browser and grab the data in flight from the api calls (virgin dom scrubber vs chad api capturer)

i don't know much about the t-site and can't check right now because you can't even access it the normal way, lol

this post was submitted on 04 Jul 2023
188 points (100.0% liked)

196

16790 readers
2220 users here now

Be sure to follow the rule before you head out.

Rule: You must post before you leave.

^other^ ^rules^

founded 2 years ago
MODERATORS