optimization overtime

1 hour ago, Acetic said:

These optimization posts are always super interesting to me as an aspiring computer scientist. Love it!

 

Is it possible to (somewhat easily) apply some of these Essentials engine changes to other fangames? Pokemon Insurgence, for example, would likely benefit greatly from the work you've done here.

sometimes, yes! i passed off a few of our less dangerous improvements to deso. the need_update code from from the prior post is a good example of something that's reasonably safe. other times the optimizations are pretty reborn-specific and targets features that we don't use.

i'm generally fine with passing stuff around if they end up being interested in it.

It's like a spinoff movie from the animation loading!

7 minutes ago, Dreamblitz said:

It's like a spinoff movie from the animation loading!

maybe one day this series will end

1 hour ago, andracass said:

maybe one day this series will end

Oh there's already a 10 year plan for the Essentials Cinematic Universe in production

5 hours ago, Dreamblitz said:

Oh there's already a 10 year plan for the Essentials Cinematic Universe in production

fuck

24 minutes ago, Venonaut97 said:

Am I the only one that remembers that there are Pokemon with 4 abilities? Both Rockruff and Chandelure have 4 abilities. How will they work with these changes? Can I never get dusk Lycanroc anymore?

so

chandelure does not have 4 abilities

and rockruff doesn't get own tempo in reborn in the first place

so neither of these things are affected

hi guys!

 

so yeah uhhhh
i will say in the first place i get what why you're bothered by this. it isn't our intention to disparage the work y'all have done on the engine and fangame community. i hope that it's plain that i really appreciate all the effort y'all have put into this
but, personally when i shoot the shit about essentials i am saying that with the understanding that

• this is a code base that has been maintained for, what, over a decade
• has changed hands multiple times in that time, including people of various skill levels (especially on our parts)
• and has to keep pace with one of the most convoluted, bloated battle systems in existence
• which has, itself, repeatedly changed over the years and (unfortunately) shows no signs of stopping

thinking of it like this, i feel like there is literally no conceivable way that despite all human effort this will ever not be at least a little bit of a disaster.

 

what my team and i are saying here is not meant to be an attack on the engine or anyone who maintains it, but rather sort of just venting frustrations/making light of the reality of the clusterfuck that this scriptset is by its very nature

 

i recognize, however, that to many people it may not look like that, and i am so sorry if this has ended up spilling over into your communities or has led to you having to put out fires. personally i'm definitely willing to make an effort to clarify in the future so that that isn't an issue. 

 

we are well aware that a lot of the changes we're making here are reborn-specific. i have mixed feelings about some of those* but ultimately us doing those changes are something we're doing for our game, and talking about here to our community. most players of the game don't necessarily care about how efficient essentials itself is, but lag is a common complaint for reborn specifically (probably because some jerk decided to go and make unreasonably large/busy maps in a game with way too much content already who the fu--).
we know that we're running on an old version, but the issues still exist for us, so the improvements we're making for our game here is relevant to the audience we're posting to.
but like if we were going off and posting all this same shit in pokecommunity, i'd definitely be mega concerned too. 

 

* some of those mixed feelings are because i'm aware there are projects out there that base off of reborn directly rather than essentials. however, for that same reason, the optimizations we're doing here may be particularly relevant to those developers too. that's also why we've been posting them frequently. 

 

and yes, a lot of this is still in testing. i appreciate the catch on the ability issue! we'll make sure to take a look at that! there's still a lot of testing on all of this stuff that has to be done, but i think many people enjoy reading along for the journey-- especially since we do try to have some fun with it. we are in the entertainment business here, games or otherwise

 

so i hope there's not too many hard feelings here. cass is gonna follow up with some more specific points i think.

o my. hello there.

I'll put on my serious face for this.

Before getting into the more active quote/explanation back-and-forth that usually emerges from this, I want to clarify two things that seem to generally apply to a lot of your comments: first, that there's a lot of stuff I've worked on that is very much reborn-specific and I fully realize that it won't apply for a starter-kit meant to apply for all fan games; second, we're also running on an older version of essentials, and I'll clarify whether what I've worked on is in the newer version or not. Regardless, none of this is meant to be a personal attack on you or anyone else involved in the development of the engine itself- if this eases some of the hostility at all.

Anyway, into the nitty-gritty.

59 minutes ago, M3rein said:

Essentials supports 4 hidden abilities, which is why it loads four individual abilities.

Essentials is built around the whole "dexdata" approach. species/abilities/forms and all that are loaded by opening the relevant data file, reading from the relevant section at the desired offset. This is an archaic data structure, but removing three string read points isn't going to make a difference. A real difference would be switching to a Hash-based data structure, like how item data is stored ($ItemData). This would actually have a noticeable impact on performance, unlike what is suggested here.

This is going to require some additional clarification.

After some later probing at the system, it appears that this was not the primary source of the issues I was seeing. The real issue was PBMoveData.new. I'll consider your comments towards dexdata to be effectively the same as what you may say towards this, as the two functions work very similarly.

You may say that cutting the number of abilities read by 3 is insignificant. In many cases, I would agree. Inefficiencies of that sort are so minor as to not be worth fixing. However, my assumption going into this fix was that this function was called a lot. Our data showed that we had 5250 calls to hasWorkingAbility and I had (wrongly) assumed that every call corresponded with a call to the ability function. Had that been the case, the decrease in CPU load caused by this would've been substantially greater.

The real culprit, PBMoveData.new, instead does... 11 string reads? each time it's called. And it ends up taking a significant amount of times to process this. Changing this function lead to a noticeable speedup during AI processing.

1 hour ago, M3rein said:

I also can't help but comment on your older optimization claims. The idea that "something*2/4" versus "something/2" makes any difference or what-so-ever, is ridiculous.

So, sure, if the function was called once, I'd agree. And, in general, fixing some minor math functions is not the focus of what our optimizations have been about.

I mentioned them because I find them irritating.

It really puts the cherry on top of the "why is this here" feeling that I often have while working in the code. It certainly doesn't help that there's a noticeable lack of comments.

I also can't help but notice there's more of this in 17.2's in_range function.

1 hour ago, M3rein said:

This would take one processor cycle, of which there are roughly 3,000,000,000 on modern computers. Even if it did run some 76000 times per second as per your claim, that would use 0.0025% of your processor's capacity in one single second. And then you claim that it makes an impact.

But this isn't quite how interpretive languages work. Nor is it quite how CPUs work. Pure machine code instruction for multiplication can take 1-4 cycles depending on number size. Division takes at least 10. This is further complicated depending on how exactly Ruby decides to send the instructions in the first place (which, considering we run 1.8, is likely not done very well). This is a bit more of a technical gripe, but the point I'm trying to make is that this isn't quite as clear-cut as you may want it to be.

1 hour ago, M3rein said:

You also talk about using constants for tile width/height and subpixel calculations, and ridicule Essentials for it. I think you've forgotten that Pokémon Essentials is a kit that was built to suit everyone's needs, and those may including variable tile width and height. As such, this needs to be reflected in the math. Perhaps Reborn, or most other fangames don't use this, but that doesn't make it Essentials' fault for including it.

In our version, this was set as a specific number and never changed (aside from a random dungeon generator that, I believe, doesn't work), which thus incurred the same ire as normal excessive multiplication.

1 hour ago, M3rein said:

Then how in the world can you use said profiler to measure performance increases? After all, addition, subtraction, muliplication and division and all other mathematical operator all call functions too. As for the profiler, does it take into account averages, or do you just run it 10 times and pick the biggest speed difference to advertise your "improvements"? Based on some of the percentual increases you've shared, I wouldn't say you're using a particular reliable profiling method at all. Note that the context of quote was for event reflections, which was a good improvement.

So, I say this about the profiler a lot, but it's probably worth repeating again since I'm sure a lot of this was specifically on the dev server:
It's bad.

It was designed for RMVXA which has a much faster version of ruby. It adds a shitload of overhead to every function, and the ms/call readouts that it produces need to be taken with a grain of salt.

That said, some of this seems... mildly insulting?

Quote

do you just run it 10 times and pick the biggest speed difference to advertise your "improvements"?

No? That would be dishonest?

I'm not here to flex on essentials or some bullshit. I'm here because people regularly complain about the performance. It has been a thorn in my side for a very long time. Lying about improvements does no one any good. If you'd like to see some actual evidence of improvements, I have a lot of it. Perhaps it would be worth making a devblog post to better document this.

Quote

If you want to tackle real performance issues, 

I feel like this highlights a pretty fundamental difference between how you and I view this problem. You're here for the engine, I'm here for the game. If these performance issues are fake, as you imply, then why am I seeing improvements?

1 hour ago, M3rein said:

But from what I've seen, you prefer memeing over programming.

Yikes.

Uh.

Okay.

So.

Firstly, as, well, self defense, this is a devblog post for the game, not the engine. I write with a very informal style that people seem to find entertaining. The optimizations I've discussed are not exhaustive, they do not represent everything I've worked on, they are specifically chosen because of the blend of information/enjoyment value I feel they offer the reader. If you read the optimization posts and think there's a lot of memes in there....that's the point. If you think that is all I do, then, well, that's insulting. 

2 hours ago, M3rein said:

You're claiming that Pokémon Essentials is the offender in most cases here. However, what you've shared is largely to be attributed to your ancient version of Essentials, or to things that are highly specific to Reborn. And that's not even to mention all the factual inaccuracies and exaggerations. Yes, Pokémon Essentials runs poorly, and yes, it needs improvements. But what you're doing is oftentimes wrongfully alienating Pokémon Essentials to your playerbase, and giving other fangame developers the impression that these changes are meaningful. I have read some decent improvements, like assuming a sprite does not warrant a reflection, as opposed to assuming it does. Or checking less input keys (although, if this is only called once during Input.update, you're going to have a hard time convincing me that it made any difference at all).

I... look, I don't want to break this down here, because overall it seems like this is just a knee-jerk reaction to a series of posts that disparage essentials. And, like, I understand that.

But I feel like there's a disconnect here between what you're talking about and what's really happening.

I'll put my money where my mouth is. 

I implemented three changes into Desolation: the needs_update variable for events; the updated in-range? function; and the new controls checker.

50% improvement in FPS on the overworld.

If you still doubt me, I can personally show you what I did as I did it. We can use vanilla scripts and everything.

But I also really don't think that it's worth fighting this out.

I'd like to call a truce on this and, perhaps, work on making the next release of essentials more efficient.

We can backport some other improvements to our version and push some improvements to yours.

5 hours ago, M3rein said:

But I disagree with the way you criticized it in these recent optimization posts.

you could have at least apologized, but maybe that's asking too much....