4
submitted 7 months ago* (last edited 7 months ago) by BluePower@mastodon.social to c/perchance@lemmy.world

[Bug] Downloading revisions not working

I made a similar post about this one (https://lemmy.world/post/11134086) and it happened again, and hopefully the Dev can notice. (ping please)

Here's the error that Perchance throws when I try to download revisions from https://perchance.org/power-generator-manager-10k-milestone:

Uncaught URIError: URI malformed
at decodeURI (see image)

EDIT: After some later investigation, I've found out that this does NOT affect earlier revisions (see this comment: https://lemmy.world/comment/8770044)

@perchance

you are viewing a single comment's thread
view the rest of the comments
[-] BluePower@sh.itjust.works 1 points 7 months ago* (last edited 7 months ago)

That's interesting. I didn't know the hub also has the same revisions problem. I've been tinkering with that code for hours and this is what I made:

function checkForMalformedRevs(patch, loglabel, startFrom) {
  let text;
  let dateRegex = [/^.* ([0-9][0-9]:[0-9][0-9]:[0-9][0-9]) GMT+.*/g, "$1"];
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Started.');
  patch.forEach((a,i) => {
    if (i < startFrom) return;
    let pi = diffStuff._patchesToRevision(patch, i);
    let x = 0;
    let offset = 0;
    try {
      for (x = 0; x < pi.length; x += 100) {
        offset = (!!pi.substring(x, x + 100).match(/%.$/g) * 1) + (!!pi.substring(x, x + 100).match(/%$/g) * 2); // checks if there's any "chopped" URI characters at the end of the chunk to prevent `URI malformed` errors just because of that
        text = decodeURI(pi.substring(x, x + 100 + offset));
        text = '';
        x += offset;
        offset = 0;
      }
      console.log('[' + loglabel + ']', Date().replace(...dateRegex), 'Checked:', i);
      x = 0;
    } catch (err) {
      console.error('[' + loglabel + ']', Date().replace(...dateRegex), 'Error:', i, 'at chunk', x + '-' + (x + 100 + offset), 'of', pi.length, err);
    }
  })
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Complete.');
}
checkForMalformedRevs(diffStuff.modelTextPatches, "modelTextPatches");
checkForMalformedRevs(diffStuff.outputTemplatePatches, "outputTemplatePatches");

There we go! It's a greatly enhanced version of the checker tool. To reduce all the wait, the checker first processes a chunk of data, then if that chunk of data has been checked successfully, then the next chunk will be checked. (But still, if the revision data has around tens of thousands of characters it's going to take seconds to process)

The process roughly looks like this:

"generatorStats%20=%20%7Bimport:generator-stats-plugin%7D%0%0Atitle%0A%20%20Power%20Generator%20Manager%0A%0A%0A%0A\n\n"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ < checking

The checker will take a part of the data and then decodeURI's them. Then if it's successful, another chunk is also checked:

"generatorStats%20=%20%7Bimport:generator-stats-plugin%7D%0%0Atitle%0A%20%20Power%20Generator%20Manager%0A%0A%0A%0A\n\n"
                                                  ~~~~~~~~!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ < checking

And there, once it's URI malformed time, the checker stops for checking further chunks. Additionally, the logger not only throws the error, but also locates the chunk location where the error is throwed and shows when the check happened in time. You can also set on which index the checker should start checking with startFrom for easier debugging.

Additional note: While testing, I also found that other "suspicious" (invalid) characters (including %9F) and the single percentage % are also affecting this whole revisions problem.

(Also I might create a whole "utility" generator out of this so more people can contribute to this ๐Ÿ˜Š)

[-] VioneT@lemmy.world 1 points 7 months ago* (last edited 7 months ago)

On that note, I duplicated the hub and deleted the old one, it did resolve the revisions problem (though the view count reset).

[-] BluePower@sh.itjust.works 1 points 7 months ago

What? With a hub that had over a thousand views? That's remarkable.

Also @perchance@lemmy.world, if there's a way to backup and erase all the revisions somehow ๐Ÿค”

[-] VioneT@lemmy.world 2 points 7 months ago* (last edited 7 months ago)
function checkForMalformedRevs(patch, loglabel, startFrom) {
  let text;
  let errors = []
  let dateRegex = [/^.* ([0-9][0-9]:[0-9][0-9]:[0-9][0-9]) GMT+.*/g, "$1"];
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Started.');
  patch.forEach((a,i) => {
    if (i < startFrom) return;
    let pi = diffStuff._patchesToRevision(patch, i);
    let x = 0;
    let offset = 0;
    try {
      for (x = 0; x < pi.length; x += 100) {
        offset = (!!pi.substring(x, x + 100).match(/%.$/g) * 1) + (!!pi.substring(x, x + 100).match(/%$/g) * 2); // checks if there's any "chopped" URI characters at the end of the chunk to prevent `URI malformed` errors just because of that
        text = pi.substring(x, x + 100 + offset)
        let decoded = decodeURI(text);
        decoded = '';
        text = '';
        x += offset;
        offset = 0;
      }
      console.log('[' + loglabel + ']', Date().replace(...dateRegex), 'Checked:', i);
      x = 0;
    } catch (err) {
      console.error('[' + loglabel + ']', Date().replace(...dateRegex), 'Error:', i, 'at chunk', x + '-' + (x + 100 + offset), 'of', pi.length, err, 'Chunk Text:', text);
      errors.push({date: new Date(diffStuff.patches[i].creationTime), log: loglabel, revisionId: i, chunkRange: `${x}-${x + 100 + offset}`, revisionLength: pi.length, text: text});
      
    }
  })
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Complete.');
  console.table(errors, ['date', 'text']);
}
checkForMalformedRevs(diffStuff.modelTextPatches, 'modelTextPatches')
// checkForMalformedRevs(diffStuff.outputTemplatePatches, 'outputTemplatePatches')

Here is a code that also outputs the Date of the Revision as well as the Text that was throwing the error neatly in a table. It seems ONLY the modelTextPatches are throwing the malformeds.

[-] BluePower@sh.itjust.works 1 points 7 months ago

Oooh nice! I'll test that later, and I'll improve it even more when needed.

this post was submitted on 24 Mar 2024
4 points (100.0% liked)

Perchance - Create a Random Text Generator

449 readers
11 users here now

โš„๏ธŽ Perchance

This is a Lemmy Community for perchance.org, a platform for sharing and creating random text generators.

Feel free to ask for help, share your generators, and start friendly discussions at your leisure :)

This community is mainly for discussions between those who are building generators. For discussions about using generators, especially the popular AI ones, the community-led Casual Perchance forum is likely a more appropriate venue.

See this post for the Complete Guide to Posting Here on the Community!

Rules

1. Please follow the Lemmy.World instance rules.

2. Be kind and friendly.

  • Please be kind to others on this community (and also in general), and remember that for many people Perchance is their first experience with coding. We have members for whom English is not their first language, so please be take that into account too :)

3. Be thankful to those who try to help you.

  • If you ask a question and someone has made a effort to help you out, please remember to be thankful! Even if they don't manage to help you solve your problem - remember that they're spending time out of their day to try to help a stranger :)

4. Only post about stuff related to perchance.

  • Please only post about perchance related stuff like generators on it, bugs, and the site.

5. Refrain from requesting Prompts for the AI Tools.

  • We would like to ask to refrain from posting here needing help specifically with prompting/achieving certain results with the AI plugins (text-to-image-plugin and ai-text-plugin) e.g. "What is the good prompt for X?", "How to achieve X with Y generator?"
  • See Perchance AI FAQ for FAQ about the AI tools.
  • You can ask for help with prompting at the 'sister' community Casual Perchance, which is for more casual discussions.
  • We will still be helping/answering questions about the plugins as long as it is related to building generators with them.

6. Search through the Community Before Posting.

  • Please Search through the Community Posts here (and on Reddit) before posting to see if what you will post has similar post/already been posted.

founded 1 year ago
MODERATORS