Jump to content
  • optimization finale: "fuck i gotta stop doing this"


    andracass

    hi

    remember me

    i haven't posted in, like, a month- probably the longest period of time that i've gone without posting since the start of this whole optimization fuckfest.


    this post is a little difficult for me to write because i really don't feel like i'm done. over the course of this summer, i've probably spent more time coding than i've spent sleeping ("definitely. not even a contest." ame says to me as i write this) and i still do not remotely feel like i'm done. there is just so much work left and so much more that could be done on all of this that stopping now feels wrong.

    but i also, like, can't anymore.

    ame and i have kind of taken to calling this little bouts of mine, my Apeshit of the Month ™️. it's basically just rips on the fact that when i usually work on the game i just go nonstop for a long period of time before i burn out and do nothing for a little bit before just going off on another Apeshit of the Month. i think i'm now reaching the end of my energy for that.

    originally, this month was just supposed to be a mini-continuation of last month- little bit more optimization in the AI to speed up how long a turn takes to process. it then kinda blew up. some other section of the code would end up catching my attention and i'd end up getting distracted, or i'd see something and it'd annoy me so i'd start working on it- not to mention that 90% of my coding experience has come in the past few months, so i'd occasionally see something i did a few weeks ago and think "wow, this is shit" and then i'd end up redoing it.... and i think i've just reached the point where i've coded my heart out and need to do something else now.

    this post is a mix of a catalog of what i've done for the sake of actually making a record of it and also a deep-ish dive into what the code does and how it does it for those of you who are interested in coding things yourselves.

    (and likely some angy for those who enjoy that too.)

    gonna be a loooooooooong fuckin post.

     

    How many levels in a meech?'

    for my opening act, we have something that i actually forgot about until right before i started writing this post. this fix comes courtesy of our dear mr. perrence, who did this way back in june, which i then immediately forgot about because there's just been so much happening. this came up when i was running a profiler to see what functions were being called during the AI phase of a four-person double battle. four-person doubles are great for these. since you have 3 AI trainers in the fight, as well as multiple targets that each of them need to choose from, it creates a situation where the scripts are basically being stressed-tested to the greatest extent possible during normal gameplay.

    so i ran this profile, and saw that there were 244265 calls to the pbGetExpInternal function. 

    that's a lot of calls!

    so let's take a peek at what's going on here.

    (pictures with a white background are from essentials v17.2, and pictures with a black background are reborn code, for reference.)

    image.png

    what you are seeing is the calculation for EXP. most of this isn't actually relevant here, since levels under 100 just need to check an exp table. it's also fairly inoccuous- if this function is being called a fuckload of times, the true culprit is the function that is calling it:

    image.png

    HAHA, i've got you NOW, essentials!!!!! just wait until i tell everyone about how BAD you a- wait, no, actually, this function seems fine. the for loop explains why it's getting called so much too, i mean, look at meech:

    unknown.png

    meech so stronk it's gotta run 100 times for her.

    ...but then why is it running 244265 time?

    and that brings me to the TRUE culprit:

    image.png

    the problem here is not about what the code is doing, but rather, what it is not. and the thing that it is not doing is saving your level.

    this means that your level is manually calculated up based on exp (like it says right there) every time it's needed.

    this can be fixed by making a class variable:

    image.png

    and just like that: poof. the game has obtained new knowledge. never again will unsuspecting pokmans be level checked more than once.

     

    The Compil - nope, nope, not yet, too scary

     

    uhhhh let me see what else i've got in here...

    *blows dust off the code

     

    Set HW_XYZ

    so all the way back in june again, we noticed that a bulky sprite refresh function would be called every time a sprite's attributes were changed.

    here it is now, the privrefresh function:

    image.png

    there it is!

    it's 350 lines long, quite big, does a lot of math (you know how much i hate math).

    now, i'm not going to sit here and tell you to remove this function and replace it with sharktale.

    unknown.png

    (pictured here: me, working on the code)

    that would be silly.

    but there are certainly times when the function doesn't need to be called, like when you're setting a bunch of variables before the sprite is even displayed on-screen. those calls aren't necessary.

    which brings me to the setHW_XYZ function.

    image.png

    (pay no mind to the variable names. hopefully no one will throw rocks at me for this.)

    this neat little function only runs privrefresh once for five whole variables wow crazy.

    this is boring. like, this is definitely an optimization, but...god i just hate graphics. i think we ended up just putting it in the initialization functions and left it at that.

     

    let's get to the good stuff.

    grab your hazmat suit, remember the buddy system, and send some prayers to The Exalted Lord Arceus (praise be) and get ready to dive into

    The Compiler

    (ok. i can do this.)

    the compiler is-

    9k.png?width=354&height=703

    the compiler is just...

    unknown.png

    don't forget about move memory. we'll come back to that. also shout out again to the Fairy of Toothpaste for patching up the battle factory. lad's killin it.

    the compiler is just....... terrifying.

    i mean, look at this!

    image.png

    those aren't even words!

    image.png

    how does anyone manage to look at this and not immediately have a brain aneurysm?? 

    image.png

    O_O

     

    the compiler is the one place

    that no one should go

    it is such a terrible place

    or, at least,

    it was

    now? i am beeg cass. i have fought the AI. i have fought the battle system. 

    and now?

    fite the compiler.

    let's go.

     

    last time, on Dragon Blog Z:

    unknown.png

    we had THIS.

    hooooooo i was mad. i was all out here like 😠 thats how mad i was.

    also it turns out this wasn't the real culprit either.

    like, it was a culprit, but arresting this guy would be like giving someone a parking ticket when there's a murderer on the loose.

    the true real final ACTUAL culprit times infinity

    was this.

    image.png

    this rat bastard. 

    this piece of shit was behind it all.

    lemme tell you about what this function does.

     

    FIRST

    it opens a file.

    wow. i'm so smart.

     

    BUT THEN

    it reads the file.

    incredible. how do i do it

     

    while this is, of course, a perfectly acceptable way to save and load move memory, it wasn't good enough for me.

    there are lots of ways to load and process data. this function here uses a bitstring.

    imagine a very, very, very long line of numbers, stretching off into space.

    that's what moves.dat looks like.

    the actual data that you want from moves.dat is located somewhere in that string. to find it, this function shifts to a certain section of the string (determined based on the move ID), reads the data out piece by piece, and then ends the read.

    this isn't an inherently bad way to process memory. it does work, after all. but it's definitely not efficient. the series of function calls, coupled with the frequency at which this function itself is called, means that string processing becomes a lot of what the battle system does in the background.

    so i changed it.

    image.png

    feast your eyes.

    now this is how to read memory.

    moves.rxdata is basically identical to moves.dat, but instead of having data in a bitstring, it has it in an array. not only is this a lot faster to process, but it's also really easy to read.

    want the function code? it's in [id][0]. there ya go. ez.

    if, for some reason, you just randomly want the effect of move 482 in some oddball script, you can just grab it with $pkmn_move[482][6]. simple, speedy, clean.

     

    abilities wasn't off the hook, though.

    image.png

    561640006658293761.png?v=1

    ok compiler. let's go and have a ch-

    image.png

    nope nope nope im out byeeeeeee

     

    so the compiler as a whole is a little bit too large to post here; while this is a blog post about coding and script changes, i don't really want to blind you with more blind text than my lawyers recommend.

    but i also think i spent a few hours staring at this trying to understand what it did, and i don't want anyone to have to go through that.

    so i'll take pieces and explain them and we'll go from there.

    let's start with this.

    image.png

    this is from essentials 17, so it won't be what our code looks like. i just don't want to get hurt this time.

    this section defines how the PBS data gets processed (pokemon.txt in particular for this). it defines: 1.) what section this pertains to, 2.) what byte the data gets written to, and 3.) what type of data gets written.

    take, for example, "Color". whatever is entered in the "Color" field in the PBS will be written to byte 6 and processed as type "e". (it also requires PBColors in order to function.)

    the problem i have with this process is pretty much the same problem that i had with moves.dat- it's a lot of data processing for very little (if any) gain. the difference here is that patching move compilation was easy.

    for moves, you just take this:

    image.png

    and tell it to stop.

    image.png

    "why is it moves.rxdata?" you might be wondering. and, well, idk. i'm not sure that the file extension matters a whole lot for this and it seemed like a simple way to differentiate between the two.

    dexdata compilation, on the other hand, is like what if the move compilation was tied into a knot. and written in dothraki. and you had to read it with your chin.

    instead of rewriting it, i managed to get away with just repurposing the current code to throw everything into an array rather than a bit string:

     

    image.png

    pg-13 warningL there is a swear word here. if you find such language offensive, then how did you make it on a fangame for pokemon reborn

    (as you can see i was getting rather frustrated with the code.)

    this array is all of your pokemon data. you'll notice that there are a handful of nested arrays inside of it- they're placed where they are to match up with the old order that dexdata was stored in. it's super arbitrary.

    the order that data gets stored in is determined by the same parameter that determined what byte data was stored in.

    image.png

    color would be the first item, type 1 would be the third item, base stats would be the fifth (and is, itself, an array) and so on. it's not really in a very intuitive order, but that's mostly because i'm not really sure an intuitive order exists for this.

    the section that allocates data to specific bytes is a little spread out, so i'm going to skip the explanation there (i didn't really do a whole lot to it and also holy shit this post is massive already)  and jump to implementing this.

    because it's a fuckin pain.

    i'll walk you through it.

    image.png

    this is an example of a call to dexdata. the calls are all structurally similar: dexdata is opened, the bitstring is offset to the data you want, the data is pulled, and dexdata is closed.

    pbDexDataOffset is what's really running the show here. the number at the end of the function corresponds to the specific data it needs- in this case, the 8 refers to Type1, which you can confirm by checking the original code.

    fgetb and fgetw read a specific number of bytes of data. it reads 1 byte for the type and 2 bytes for height/weight. 

    with the new system, that code becomes this:
    image.png

    $pkmn_dex is preloaded at game start, so there isn't a check to see if it exists or not. if you somehow manage to get in-game without it preloaded, then, well, i'm impressed! also please don't do that.

    you can see that it works pretty much the same as move initialization did earlier. you put in the pokemon, you put in the index of data you want, and boom. you have it.

    we are moving off some antiquated data structures one step at a time.

     

    now, for me, patching dexdata was about me looking at how this worked and going "hhhhhhhhhhhh i hate this", thus sparking a strong desire to fix

    but this code is also, like, actually bad.

    infinite fusion had to weigh adding more pokemon (its main feature) against the game grinding to a halt whenever dipping into dexdata

    rejuv can't add gen 8 without compiler edits because the compiler effectively limits the total number of abilities to 256. gen 8 has more than that.

    it's kinda bad.

    that's enough on the compiler- i've been writing this post for five hours and we're still not even on the ai.

    spoiler alert

     

    The AI

    so i spent, like, three days on the compiler?

    the rest of the apeshit of the month went in to this fuckin thing.

    there is just... so much to cover here.

    over the course of the last month, we trimmed the ai from 42,000 lines to just a bit under 25,000.

    some of this was just code cleanup

    some of this involved offloading work off into individual functions

    some of this was to fix some bugs.

    the rest..... i don't even remember. it was so long ago.

     

    first let's cover some background stuff.

    Importing Hidden Classes aka "i hope you liked talking about the compiler earlier"

    during compilation, hidden classes are made that hold the IDs of everything that you've compiled. they're hiding off to the side, in that constants.dat (or rxdata. one of them.) since they're off to the side, though, they aren't always accessible to the main script classes, which occasionally means that direct calls to them will fail. i manually imported them by modifying the compiler to yeet them out entirely instead of adding them to the constants file and then copying them into the main scripts myself.

    is this a good way to do it? no. i already know that there's a better way, but i just have really burned out hard on this and don't plan to get to it immediately. but it involves having the scripts import themselves. 's fun.

    PBStuff

    this one was me. i did this.

    i made PBStuff to store large arrays of constants that frequently get called/checked together. this was mostly for the sake of the AI, which tended to have these very long lists of moves that stretched out well beyond the incredibly small rmxp script window, but it also came in handy for making sure that ability-changed moves like role play and entrainment would, like, actually fuckin work correctly. 

    image.png

    wow! comments! that explain things!

    so this was pretty important for code cleanup for two reasons.

    the first is that it makes the code much more readable if you're just calling an array of things rather than just rattling off a massive list of moves/abilities. 

    the second is that it ended up fixing a lot of issues with role play/etc, which had some abilities missing because those moves are really picky.

    in the end, it allows for stuff to go from this: 

    image.png

    to this:

    image.png

    i do not know for sure if it correctly gets called as an array with this syntax.

    which is just so much clearer.

     

    okay. let's get started with the real stuff.
    isConst? and hasWorkingAbility

    this'll get me some hate mail.

    i opened this post by talking about how exp/leveling functions work, and that they were called 60 billion times.

    well, these two functions were the next two worst offenders.

    image.png

    (this is from one AI phase.)

    so what do these functions do?

     

    isConst?

    this one you've already seen from the Role Play screencap. isConst? takes 3 parameters: an object, the class you want to check, and the value you're checking for. isConst? puts them together like so:

    image.png

    if you're still with me, this far into my gigantic post, then you might see my issue here.

    isConst? checks to see if a value exists before it gets that value.

    it has a rescue statement if the value does not exist.

    so why not just directly get the value?

    the rescue statement would catch it the same way.

    but then, later on, I realized something.

    why does this function exist?

    you can see in PBStuff that i have arrays of values that are all called like this: [CLASS]::[VALUE], which is functionally identical to what isConst? does.

    and the reason it doesn't work is because it can't directly access the hidden classes.

    fixing this is what originally prompted me to import the hidden classes into the scripts; that's what enables the use of the class/value call. nearly every single instance of isConst?(val,mod,constant) can just be replaced with

    [val] == [mod]::[constant]

    and skip out on several lookups and function calls.

    hasWorkingAbility? faces a similar situation:

    image.png

    it's effectively a glorified isConst? call. 

    the checks that are made here are necessary, but they are also being run a lot

    these calls can be replaced by running a single check at the start of a large function, and then replacing the remainder of the calls with (abilityworks && object.ability == PBAbilities::Ability), which can then be condensed further with some organizing. (i'm also considering just making a class variable, but i only thought of it this morning. work is never done.)

     

    AI Move Memory Utilities

    remember this from earlier? the AI move memory system is part of what helps the AI just kick your ass. 

    it also takes up a lot of space in the AI. i mentioned that there were about 17k lines of code that are gone now. 10k of that was through ai memory functions.

    every AI move memory check has two basic components: it first pulls the memory, and then it pulls certain information from it. there's a lot of things that are done with the memory, but the majority of it falls into four categories.

    the memory puller is here:

    image.png

    and all of the utility functions are here:

    image.png

    astute readers will notice that there are five functions here. however, we all know that in ruby, the 5th element in an array is index 4. so thus i'm still correct and not just too exhausted to change that number.

    this, in combination with the PBStuff arrays allows these massive chunks of code to be condensed into single lines.

    before:

    image.png

    after:

    image.png

    imagine that happening hundreds of times and you know what's happened to the AI code.

     

    *checks watch

    i'm almost 8 hours into this post.

    last big thing and y'all can go home.

    AI restructure

    so i'm sitting here talking up a big game about the ai. but did you know, that our perfect ai, that even it has flaws? it's true! i was also surprised.

    when making decisions, the AI first decides if it should switch, then it decides if it should use an item, and finally it picks a move.

    every single step, the AI checks the scores of all of its moves.

    why

    unknown.png

    pictured above: death.

    to fix this, i cut pbChooseMoves in half and put the actual movescore grabbing bit in front.

    image.png

    this is a much simpler change than the others, but it slices a large chuck of the AI processing time off.

     

     

    ok. we did it. actual dev content over. y'all can go home.

    imma end this out on a more personal note.

    frequent readers of my posts (thank you! and also: why?) will recall that last time i posted, it consisted of direct screencaps of the dev server where i was Very Angry about code inefficiency.

    frequent readers of the comments (why? and also: no seriously why would you do that) will also recall that a Big Fan of essentials showed up to call bullshit on the work i've done over the past few months (despite not really understanding what i do), before calling me a memer rather than a programmer.

    this hurt me.

    likewise, this post was kind of a chore to write.

    that's never happened before.

    these posts have always been written in a certain style that i hope conveys a feeling that i am speaking to you, rather than simply putting words on a page that you then read and process. it's a stylistic choice on my part that i understand some people appreciate and some people don't.

    writing in an informal, open way, only to have someone- a rival community admin- show up on your devblog to tell you that the work you've put into your game is shit kinda fucks you up. 

    i've spent a lot of time checking over my shoulder for everything i've written. "oh hey, is this code absolutely perfect?" "if i say this, is someone from relic castle going to tell me that i'm stupid and should just upgrade my version of essentials?" (the latter here is why i've been very careful to only include code that is in essentials 17)

    but what is most troubling about this is the fact that i was bashed by someone who claimed to represent essentials for offering optimizations for other devs to use in their own games.

    i was mocked for using essentials 15. but insurgence uses 15. deso uses 15. rejuv uses 16. games that have been built on older versions often can't just upgrade to newer ones because it would require reimplementing everything they've already done. meanwhile, the changes between versions are often opaque, with little information to help developers who are stuck on older versions to get any of the benefits of upgrading.

    to blame someone who is trying to improve an older version of the engine for being stuck on an older version is not only callous, but it also shows that there is a massive disconnect between the people making tools to make games and the people who actually make the games.

    and i should be clear here- i'm not upset with the actual developer of essentials at all. when we talked he seemed pretty open to some of the things that i mentioned we had implemented.

    the problem that i see is two-fold.

    the first part of it is that there is a community that seems to be entitled to essentials to the point where they will bash people who speak out negatively regarding it. i have spent the better part of three months grinding away at the scripts. i spent more time coding than i've spent sleeping. and i want nothing to do with that place. this is a unique circumstance for me because i'm generally really happy to offer my work to people. i like making these posts where i talk about what we've been doing. i like handing out my work to people because it means that everyone can make higher quality stuff without also needing the technical knowledge to tinker with the engine. it's practically a form of elitism- we have the new version, and that makes us better than you with your old one. i suspect this matters less to newer devs, as they can just pick up the latest version and start working with it, but i feel like the fact that this situation has happened at all should still be concerning.

    the second part of it is just that...this is so much work. i have been going non-stop for months and i still don't feel like i'm anywhere close to having the code be in a state where i can consider it to be "done". and this has only included work on parts of the engine; there are still entire sections that i haven't even begun to think about, let alone look at.

    my point is that this is all just too much work for any one person to do, and it doesn't make sense for multiple people to end up doing the same work over and over again. it doesn't make sense for me to fix the bugs in my version that are still present in the current version, meanwhile there's another version that's being developed and i sure dunno if it's being fixed in there or not. and then, even if it is being fixed, there's no telling if anyone else would be able to use it in their games since there's not a lot of guidance on what the actual changes are.

    it also means that, as a result of everyone doing the same things independently of everyone else, that there's a lot of wasted scripting talent being spent on re-doing things that have already been done. for example: how many gen 8 mods are there in development right now? you've seen my scripts. gen 8 is already there. we helped out with rejuv on developing it. i've assisted deso with optimizations, and am helping rejuv implement some of this as well. when marcello heard that infinite fusion was having slowdowns due to the sheer number of pokemon in their game, i sent the compiler changes to see if that would help solve things. even aside from the major fangames, time and time again there have been fans and community members who will show up out of nowhere with their own fixes for things. i've been getting some additional mkxp support from aeodyn, who just messaged me one day saying "hey you know you could be using this better tilemap, right?" and then fuckin waynolt is out here making the entire modding system for reborn/rejuv, and then fuckin toothpastefairy fixed the goddamn battle factory before he was even on the team!

    my point here, is this:

    essentials needs to be open source.

    thanks for coming to my ted talk i'm going to go do anything else now because i have been typing for nine hours and i can't anymore

    • Like 37
    • Thanks 14
    • Hmm 1
    • Upvote 5


    User Feedback

    Recommended Comments

    .....wooooow i wouldnt even have the patience to do a little of that script😂

     

    also...Ice face and gulpmissille?? arent they gen 8 abillites?

    Share this comment


    Link to comment
    Share on other sites
    Just now, Gengar768 said:

    .....wooooow i wouldnt even have the patience to do a little of that script😂

     

    also...Ice face?? isnt that like a gen 8 abillity?

    Quote

    you've seen my scripts. gen 8 is already there. we helped out with rejuv on developing it.

    it's okay i don't have the patience to read my posts either

    i never wanna see this shit again

    who does this bitch think she is

    • Like 3

    Share this comment


    Link to comment
    Share on other sites

    honestly bless you cass. you've done so much incredible work and deserve nothing but love for it. if anyone wants to disagree they can feel free to contact me directly and I'll be happy to tell them to eat shit 

     

    With all the cool stuff that the fangame community as a whole has managed to collectively produce, essentials could become a mighty god indeed with a unified development front....
     

    • Like 5

    Share this comment


    Link to comment
    Share on other sites
    On 8/4/2020 at 9:23 AM, andracass said:

    you've seen my scripts. gen 8 is already there. we helped out with rejuv on developing it

    Oh yeah I forgot you guys also help with rejuv my bad😅

    also i was kinda meaning the actual scripts but your right that shits long as hell to read as well 

    Share this comment


    Link to comment
    Share on other sites
    Just now, Dreamblitz said:

    essentials could become a mighty god indeed with a unified development front....

    well isn't that just

    the

    dream

    • Haha 1

    Share this comment


    Link to comment
    Share on other sites
    Just now, Gengar768 said:

    Oh yeah I forgot you guys also help with rejuv my bad😅

    also i was kinda meaning the actual scripts but your right that shits long as hell to read as well 

    i knew i said something about it in there but when i tried to find where it was i thought i was just gonna die

    • Haha 2

    Share this comment


    Link to comment
    Share on other sites
    1 minute ago, andracass said:

    i knew i said something about it in there but when i tried to find where it was i thought i was just gonna die

    I think it was somewhere around the part where you were mad at the code

    • Like 4

    Share this comment


    Link to comment
    Share on other sites
    1 minute ago, Dreamblitz said:

    I think it was somewhere around the part where you were mad at the code

    please excuse me while i collapse on the floor

    • Like 3

    Share this comment


    Link to comment
    Share on other sites

    thank you cass for your bravery and hard work. i understand only about half of any of this because i'm smooth brain and blank the moment i stare at any code, but it's very, very clear that you've put a ton of effort into this and that you're doing god's work tbh. you're a blessing to all of the ppl who run their games on older versions.

     

    godspeed you magnificent bastard

    • Like 3
    • Upvote 1

    Share this comment


    Link to comment
    Share on other sites

    You could combine all of these optimisation posts into an actual thesis and get a research publication out of this literal madness

    • Like 5
    • Upvote 1

    Share this comment


    Link to comment
    Share on other sites
    Just now, Caz said:

    You could combine all of these optimisation posts into an actual thesis and get a research publication out of this literal madness

    i have to scroll through the entire post to check all the comments

    0/10 would not recommend

    not worth the phd either

    • Like 3

    Share this comment


    Link to comment
    Share on other sites

    I appreciate this post, both for detailing all your hard work

     

    And also because you politely did not point out how much of the deeply inefficient code you optimised was written by me oops

     

    ❤️

    • Like 5
    • Haha 3

    Share this comment


    Link to comment
    Share on other sites

    thank you for your hard work! personally I enjoy your writing. keep it up!

     

    ...and wow, I never worked with essentials but some parts of the code look hideous. I totally get why one would hide from the compiler 😄

    • Like 3

    Share this comment


    Link to comment
    Share on other sites

    I'm one of those people who reads all your blog posts - partially because it's interesting (even if I only understand a fraction, I often end up learning something!) and partially because of your style 🙂, someone "talking" to me about programming is a lot easier and more fun to read than more formal stuff 🙂 

     

    thank you very much for your hard work! and for the extra effort of telling us about it

    • Like 5

    Share this comment


    Link to comment
    Share on other sites

    I, for one find the Blog Posts fascinating. and they remind me why i'm a cook, and not a programmer. Godspeed, devs.

    • Like 4

    Share this comment


    Link to comment
    Share on other sites

    That AI streamlining is soooo satisfying. And yay for memoization!

     

    TBF for the tilemap, they had to rewrite it cuz the RMXP version just doesn't support some things. Though why someone thought that meant they should sometimes render tiles as individual sprites is beyond me.
    Luckily ancurio & inori's mkxp-z (game-z.exe) is open-source, so I was able to implement/copy-paste what the built-in tilemap was missing.

    • Like 2

    Share this comment


    Link to comment
    Share on other sites

    Honestly, I'm amazed at how much work you've been doing, cass. Your commitment is definitely something to be admired.

    • Like 1
    • Upvote 1

    Share this comment


    Link to comment
    Share on other sites

    Honestly Cass and the rest of those helping her have been really working hard over these past few months tackling these things and I can imagine what she shares with us is only the tip of the iceberg. Hats off to everyone involved in this massive task, someone pay for some therapy sessions for those who have had to deal with this code.

     

    Everyone go give Cass a hug.

    • Like 3
    • Upvote 2

    Share this comment


    Link to comment
    Share on other sites

    I think I started getting a headache reading this once symbols were used in variable names or something. Dear lord, I'm glad you have the patience to work these through, bc I doubt I'd be that virtuous 😅

    Still don't understand why anyone would use bitstrings to load and process data. Is this a Ruby-thing or code relic or anything? It seems so much harder to adjust, call or troubleshoot and an overall pita to work with.

    I do wonder why essentials isn't open source, given its suboptimal coding piling up adjusting to multiple pokemon generations. Is this because of legal issues or something?

     

    #GiveCassAHug

    • Like 1

    Share this comment


    Link to comment
    Share on other sites

    I think I'll just honestly start by saying your changes have not gone unnoticed and I will say this one did show interest in me because I've actually dealt with some issues because of the things you're trying to fix. Turns out when calculating every move 1,000 times for a certain battle does tend to brick the AI causing it to halt. I can't really add much since you kind of answered everything in your post.

     

    But...can I just say something about the last part:

     

    I don't think any of us would argue that guy had any right to really make an account to make a tangent about the dev blogs. Maybe he was a long time fan of Reborn and these posts sort of released a bunch of bottled up frustration. There's better ways to handle it but sometimes our emotions get the best of us. Maybe he's just an ass who can't take a slight bit of criticism. The point is you don't know. And rarely do things get resolved with petty arguments as it just makes them angrier.

     

    I'm not going to say what to do or how to handle them. That's up to you. Maybe you should be the bigger person or maybe give them a piece of their own medicine. That won't be the first and that certainly won't be the last person to say something that puts you down. It's something you accept will happen when putting yourself into the public with work. I and probably a number of other people have had their own experiences and even light hearted stuff can feel a jab in the gut.

     

    I get it. I really do. People make it sound really easy to ignore or get over. Everyone learns how to handle it differently. I've learned how to find amusement in it. Helps put in perspective how harmless it really is and how silly it was getting so worked up about it. Also makes me realize people are reading the things I say, but that certainly is a me problem.

     

    This got a bit too sentimental. Especially because the OP is using Shark Tale pics. That's a movie that should be dropped to the deepest deaths of the earth to be forgotten for all eternity. No person should ever be exposed to such horrors.

    Share this comment


    Link to comment
    Share on other sites

    These pictures of code might as well be Shark Tale for as much as I understand what I'm looking at.

    But it was a fun journey to follow, and the writeups made it possible for thus code-illiterate person to follow.

    • Like 1

    Share this comment


    Link to comment
    Share on other sites
    9 hours ago, Marcello said:

    I appreciate this post, both for detailing all your hard work

     

    And also because you politely did not point out how much of the deeply inefficient code you optimised was written by me oops

     

    ❤️

    i meeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeean.............................................................................................................................................

    • Like 1

    Share this comment


    Link to comment
    Share on other sites

    I had a stroke trying to read most of this but I get the jist, you absolutely HATE essentials. Most of this code was bad and you made it good, thank you for improving the game so much more than anyone ever expected, it truly shows your passion and commitment to making this game great. 

    Share this comment


    Link to comment
    Share on other sites

    It's great to see new ways to improve game speed, and I'm glad you're still giving updates about this. The level thing with saving it as a new variable so it's not called thousands of times seemed pretty simple, so I tried adding it in my copy of essentials, but for some reason, the Pokemon's level doesn't increase when it levels up. I guess there are some other places that have to be updated to account for this variable as well?

    Share this comment


    Link to comment
    Share on other sites

    I think it goes without saying how thankful we are for all of these improvements in the below average code that andracass spent so fucking long improving, no wonder speeding up the game would slow it down so much, I didn't realize how inefficient the code was until I checked some of these optimization posts. I hardly know any programming yet even I can tell that strings are so long that if there's a better way to program a function, I'd do it rather than make the program run 20-25% longer. I really wish I could play the latest Reborn update or Desolation right now but I'm experiencing some technical difficulties. Good luck with the rest andracass, imagine how much better these kind of games will be with the enhanced program you've been working on

    Share this comment


    Link to comment
    Share on other sites
    Quote

    the Pokemon's level doesn't increase when it levels up. I guess there are some other places that have to be updated to account for this variable as well?

    I was just going to comment about this. You need to do @poklevel = nil when adding experience.

    Share this comment


    Link to comment
    Share on other sites
    1 hour ago, silverlime said:

    It's great to see new ways to improve game speed, and I'm glad you're still giving updates about this. The level thing with saving it as a new variable so it's not called thousands of times seemed pretty simple, so I tried adding it in my copy of essentials, but for some reason, the Pokemon's level doesn't increase when it levels up. I guess there are some other places that have to be updated to account for this variable as well?

    there are! it's also tossed into the initialization function for initialization like so:

    image.png

    Share this comment


    Link to comment
    Share on other sites

    This looks absolutely amazing, cass. Thanks a lot for it all, code modifications, code cleanups, and the dev posts (and please, take care of yourself). 
     

    I’m a math guy, but I like coding (although my stuff is much, much smaller-scale), so your posts make me code vicariously. 
     

    Just a stupid question: what exactly does the compiler do? Does it gather Pokemon, move, and ability data from .dat/.rxdata files (as I think I understood from your post, which already sounds really not-straightforward)? Does it do something else? 

    Share this comment


    Link to comment
    Share on other sites
    1 hour ago, Mindlack said:

    Just a stupid question: what exactly does the compiler do? Does it gather Pokemon, move, and ability data from .dat/.rxdata files (as I think I understood from your post, which already sounds really not-straightforward)? Does it do something else? 

    basically, the compiler is what takes the PBS and puts it into a data structure that is easier to use/process by the scripts. PBS data is way easier to access/edit than the scripts themselves since you only need a text editor to work with them. the compiler then imports what you wrote in the text files and puts it into an array.

    • Like 1
    • Thanks 1

    Share this comment


    Link to comment
    Share on other sites

    The fact that there was so much work that needed to be put into making the whole program run faster worries me, imagining having a project like that in a programming class would make me lose my mind. I hope I don't have this kind of project in the near future, doing something on a much smaller scale already made me reluctant to learn more about programming but this is fucking OD.

    TLDR: I'm praying my next few years don't include projects that require A+ programming skills like this because I'm not up for it

    Share this comment


    Link to comment
    Share on other sites
    On 8/4/2020 at 8:58 PM, Commander said:

    But...can I just say something about the last part:

     

    I don't think any of us would argue that guy had any right to really make an account to make a tangent about the dev blogs. Maybe he was a long time fan of Reborn and these posts sort of released a bunch of bottled up frustration. There's better ways to handle it but sometimes our emotions get the best of us. Maybe he's just an ass who can't take a slight bit of criticism. The point is you don't know. And rarely do things get resolved with petty arguments as it just makes them angrier.

     

    I'm not going to say what to do or how to handle them. That's up to you. Maybe you should be the bigger person or maybe give them a piece of their own medicine. That won't be the first and that certainly won't be the last person to say something that puts you down. It's something you accept will happen when putting yourself into the public with work. I and probably a number of other people have had their own experiences and even light hearted stuff can feel a jab in the gut.

     

    I get it. I really do. People make it sound really easy to ignore or get over. Everyone learns how to handle it differently. I've learned how to find amusement in it. Helps put in perspective how harmless it really is and how silly it was getting so worked up about it. Also makes me realize people are reading the things I say, but that certainly is a me problem.

     

    This got a bit too sentimental. Especially because the OP is using Shark Tale pics. That's a movie that should be dropped to the deepest deaths of the earth to be forgotten for all eternity. No person should ever be exposed to such horrors.

    If anything, Cass, know your worth is far more than what people jabbing your work proclaim it to be. The responses here are just one testament to that.

     

    Also Commander, if one wants to pollute the ocean even further with such abominations, please do not evoke imagery similar enough to that vile thing's content. 1/10 did not like that flashback 😅

    Share this comment


    Link to comment
    Share on other sites

    makin eyes emojis at Gulp Missile and Ice Face ngl

    or i cant read
    BUT STILL
    eyes emojis

    Share this comment


    Link to comment
    Share on other sites

    These posts actually entertain me. The one with calculating level each time is such a barbaric method that I wonder how it is like that in the first place. And Yes, arrays are the way to go if you know the index. Beats searching a string. Also, regexes are unreadable most of the times if they are complex 😑. Trust me. I have had cases where I looked at regexes I myself created and wondered what the hell it does lol. Keep doing the good things you are and here's to getting a better game.

    PS: I always wondered(not when I was a kid obviously) how the Pokemon code worked and how all the different abilities, moves etc fit together. Well, now I know after reading all the dev posts. I assumed they were stored as jsons haha.

    Share this comment


    Link to comment
    Share on other sites

    My goodness.  A lot of the code stuff flies over my head but it is an enlightening(and entertaining!) read every time.  On a more serious note, please take some time to yourself to just relax if you need it Cass.  We all know how hard you(and the rest of the dev team!)work, and we appreciate it.   Game wouldn't be worth it if it ends up causing actual pain(aside from essentials headaches which are part and parcel of dev)to y'all!

    Share this comment


    Link to comment
    Share on other sites

    I'm just here to say thank you,


    I still don't understand why you and your Fanatics keeps hurting yourselves that much,

    But recently i remembered how dark the story was, and i understood that you need that much pain to keep the essence of the game.

     

    Keep up the good work and the suffering i guess...

    Share this comment


    Link to comment
    Share on other sites

    okokokok wow.

     

    I just gotta say, as a person whos been here a while, wow.

     

    I didn't know how much damn work went into making Reborn the way it is. Before I came back here in may, I always thought that e19 not being out yet was a product of the project as a whole being dead and discontinued. I stand corrected, and although I have no coding experience whatsoever, and probably very little in the way of saying something meaningful here, I know effort and dedication when I see it. I believe in Reborn, I believe in this community, and I believe in you, Cass.

     

    Don't let any naysayers bring you down, take all the time you need to make e19 something truely unforgettable. I'll be there every step of the way!

     

    (all this code jargon still makes me go  👁👄👁 lmao)

    • Like 1

    Share this comment


    Link to comment
    Share on other sites
    5 hours ago, Sin! said:

    I always thought that e19 not being out yet was a product of the project as a whole being dead and discontinued.

    no............................................................it's just............................................................................................................................ poor planning..............................................................................................

    Share this comment


    Link to comment
    Share on other sites

    I took one python class in grad school, and withdrew after 4 weeks.
     

    you are strong. Stronger than I will ever be. Everything in these posts hurt my brain 🧠 

    Share this comment


    Link to comment
    Share on other sites

    The amount of effort and mental power you have put into the code is truly astounding and as a community we are deeply greatful for it. The fact that you also are assisting other games, are pushing for others to use the tools and also calling for a community solely for essentials sole improvement is deserving of even higher praise.

     

    For someone to call u out is either seeking attention or just have their own problems that they are throwing at you. I just want to say that there will be always people like this, one bad person for every 10 good people that has the potential to bring everything down. Its hard but you really just got to ignore them if they have nothing helpful to say and are just trying to scream sounds (and not words) at you. You also need to give them the middle finger which is represented by all the work you have been able to do and how far you have come with the program. ofc this is all just preaching and easier said than done. but do know i understand this annoyance.

     

    I tried dabbling in coding a few times but never made much progress. the fact that you pushed and went so far is amazing. i know you hate the fact of going through the comments for a phD but i think your work is something you should try and put on your CV or something and have the evidence to back it up.

     

    I always try to read and check up on these development blogs and while i havent in a bit (due to my current situation) do know that myself and many more are supporting you (and we also want you to take care of urself and sleep more)  so u can just ignore that one annoying boi.

    Share this comment


    Link to comment
    Share on other sites

    Game Freak lies: "dlc will feature level scaling" But there was NOT

     

    Reborn lies: "update will NOT feature gen 8" But there was

     

    I was kinda expecting it for some reason but I LOVE IT THANKS CASS you are all of us

    Share this comment


    Link to comment
    Share on other sites
    On 8/19/2020 at 11:19 AM, Lorisaur said:

    Reborn lies: "update will NOT feature gen 8" But there was

    it's in the code. 

    it's not in the game.

    Share this comment


    Link to comment
    Share on other sites
    On 8/20/2020 at 11:20 AM, andracass said:

    it's in the code. 

    it's not in the game.

    Oh. My bad

    Share this comment


    Link to comment
    Share on other sites

    It's clear you do an immense amount of work, and it's nice to see you post so much about it. I've taken inspiration from some of the things you've talked about and applied them to Essentials. It's just a shame that I can't help you out as much as would be decent, given your massive modifications to what is now an older version of Essentials. I don't support older versions of Essentials simply because working with just one version is quite enough for me (and because I forget how older versions worked).

     

    The Compiler is definitely a terrifying thing. It was made to do its job and nothing more. I recently pushed myself to figure out exactly what it actually does (even if I still don't quite know how or even why) - in addition to converting the plain text PBS files into data files, it also scans through the maps and makes various changes, such as turning events into trainers or item balls. I also changed, in much the way you described, how a bunch of the data files are saved and loaded, to eliminate that fgetb mess and to put information in nice arrays where it belongs (and it all gets cached too). I still haven't tackled the compiling/data files for pokemon.txt, though, just because it's that daunting.

     

    The original creator of Essentials seemed very keen to make it work exactly like the GBA games, which means data being stored in byte streams and relying on calculations rather than setting variables as much as possible. This isn't reasonable or useful for a PC game, but it'll take quite an effort to get everything improved.

     

    After the next version of Essentials has come out (it's currently in beta testing), I intend to put it on GitHub or GitLab or somewhere. My experience with developing this version (primarily a huge overhaul of the battle system) showed me that the current development process for Essentials is untenable. It needs a revolution.

    Share this comment


    Link to comment
    Share on other sites
    2 hours ago, Maruno said:

    After the next version of Essentials has come out (it's currently in beta testing), I intend to put it on GitHub or GitLab or somewhere.

    :OOOOOOOOOOOOOOOOOOOOOOOO

    2 hours ago, Maruno said:

    I don't support older versions of Essentials simply because working with just one version is quite enough for me (and because I forget how older versions worked).

    i am incredibly sympathetic to this. trying to backport changes into e18 and having to work off of several different working copies has just been a confusing mess.

    2 hours ago, Maruno said:

    The original creator of Essentials seemed very keen to make it work exactly like the GBA games

    this is something that i've noticed a lot, not only with regards to the compilation and data access, but also the fact that essentials uses [HP,ATK,DEF,SPD,SATK,SDEF] as the default order of stats.

    i've mentioned this before in my posts, too, but it really seems like the author was very much steeped in a C++ background, and while that's not necessarily inherently a problem, it means that the code is really not well-designed for an interpreted languages.

    3 hours ago, Maruno said:

    My experience with developing this version (primarily a huge overhaul of the battle system) showed me that the current development process for Essentials is untenable. It needs a revolution.

    okay, so this is the big thing.

    from where i'm standing, right now it seems like there are two major versions of essentials- there's the base one that you're working on, and there's what reborn runs, which is effectively just a branch of essentials 15. somehow we've gotta bring them up to date with each other.

    i don't know if you keep any records on specific changes (i sure don't) but it seems like the best way to merge the two is to bring reborn up-to-date, make sure that everything still functions (at least to as much of an extent as is reasonably possible), and then start integrating reborn's stuff into main essentials.

    i imagine the best time to do this would be after essentials 18 is done (since it sounds like you're pretty close to that anyway), though I don't really want to hold back any major optimizations/changes that may be useful until after that. i'd be open to coordinating some of that stuff in advance if you're open to it.

     

    i appreciate that you're open to this! hopefully changing how things are done will improve the engine for everyone's sake.

    Share this comment


    Link to comment
    Share on other sites
    39 minutes ago, andracass said:

    okay, so this is the big thing.

    from where i'm standing, right now it seems like there are two major versions of essentials- there's the base one that you're working on, and there's what reborn runs, which is effectively just a branch of essentials 15. somehow we've gotta bring them up to date with each other.

    i don't know if you keep any records on specific changes (i sure don't) but it seems like the best way to merge the two is to bring reborn up-to-date, make sure that everything still functions (at least to as much of an extent as is reasonably possible), and then start integrating reborn's stuff into main essentials.

    i imagine the best time to do this would be after essentials 18 is done (since it sounds like you're pretty close to that anyway), though I don't really want to hold back any major optimizations/changes that may be useful until after that. i'd be open to coordinating some of that stuff in advance if you're open to it.

    I'm afraid I don't keep notes other than the change logs I post when a new version comes out. Upgrading will definitely be a harder task if you don't have some kind of record about what code you've changed. You can at least compare Reborn's code with vanilla Essentials v15 to get an idea (if you need a copy of that, let me know). (I recommend upgrading by starting with a clean copy of Essentials and porting code changes from Reborn into it, rather than backporting new code into Reborn.) While you're upgrading (or at any time), if you find anything worth contributing to Essentials itself, I'm always willing to listen.

     

    When Essentials gets put onto a GitThing, all its script sections will be split up into separate .rb files for easier editing. This will also make it easier to spot changes, not to mention having multiple people working on them at once and being able to immediately make small changes rather than saving them all up for a big release. I don't have experience working in this way, but it definitely sounds like a good thing to do and get used to.

     

    I can't say for sure when Essentials v18 will come out, but I'd be surprised if it was more than a couple of weeks from now. That's not too long to wait, is it?

     

    43 minutes ago, andracass said:

    i appreciate that you're open to this! hopefully changing how things are done will improve the engine for everyone's sake.

    Definitely! It's been on the To Do list for months, but I didn't want to do it in the middle of a version's development (and this latest version has been in development for an awfully long time, which is one of the reasons why the current process is not sustainable). Apparently Pokémon SDK benefits a lot from a more open source approach, so hopefully Essentials will too.

    • Thanks 1

    Share this comment


    Link to comment
    Share on other sites
    18 minutes ago, Maruno said:

    I can't say for sure when Essentials v18 will come out, but I'd be surprised if it was more than a couple of weeks from now. That's not too long to wait, is it?

    nope. i can actually get some sleep.

    • Haha 1

    Share this comment


    Link to comment
    Share on other sites

    @Maruno Something you may be interested in is that mkxp-z can be compiled to use any version of Ruby, and at least with Reborn, getting it to work with Ruby 2.7 is relatively straightforward. ~50% performance increase.

    There's also the Tilemap improvements/fixes I implemented in mkxp-z (used in Reborn 18.4.3), which means you can just completely toss out CustomTilemap. (Feel free to PM me about either, if you want.)

    Edited by Aeodyn

    Share this comment


    Link to comment
    Share on other sites
    13 hours ago, Aeodyn said:

    @Maruno Something you may be interested in is that mkxp-z can be compiled to use any version of Ruby, and at least with Reborn, getting it to work with Ruby 2.7 is relatively straightforward. ~50% performance increase.

    There's also the Tilemap improvements/fixes I implemented in mkxp-z (used in Reborn 18.4.3), which means you can just completely toss out CustomTilemap. (Feel free to PM me about either, if you want.)

    It's on the list. A lot of things are on the list.

    Share this comment


    Link to comment
    Share on other sites
    13 minutes ago, Maruno said:

    It's on the list. A lot of things are on the list.

    what a giant mood.

    Share this comment


    Link to comment
    Share on other sites


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Paste as plain text instead

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • hello. the progress bars are in storage. you can read about why here.

    and, no, the game is not cancelled. it's okay.

    (Updated 8/25)

  • 16-4.png
    16-5.png
    16-6.png
    16-7.png
    16-8.png
    16-11.png
    16-12.png

×
×
  • Create New...