Current squish formula, Math, and Kyber1: Here is how to fix the leak
tl;dr version: developers need a separate squish formula for kyber 1 and possibly kyber 2, and a corrective positive squish this month. Analysis of the squish devs are using explains why!
Problem: Kyber1 is down to 6% of kyber, and 1.1% of player base. This has happened through squish.
Discussion and background:
Currently, squish is being applied as a linear relationship of squish value to skill rating across each league. This works for most of the player base, since there is free flow of players from above and from below the division such that over the month, the players "spread" from the middle of each league out to the edges.
However, it fails in kyber, since kyber 1 is at the edge of the player base. At the top of kyber, players can only come from the bottom, they can't be brought in from "the league above" since that doesn't exist.
Because of this unique property of kyber 1, we shouldn't push players out of kyber 1 through the squish. Over all, skill rating points won and lost in combat is zero sum; any skill rating points won by one player are lost by another. In the context of kyber 1, that means each player who enters kyber 1 does so by pushing someone else out. So kyber1 can't grow over the course of the month the way the other league "div 1" can.
In fact, natural movement of skill rating in kyber 1 would be net negative even without squish, since anyone in kyber 1 who forgets to register is pushed down in skill rating, while combat again will be net neutral.
I would like devs to carefully consider this! For the next month or two, we need a positive rather than negative squish to get kyber1 back to 10% of kyber. After that, rather than a linear formula for skill rating to squish across all of kyber, you will want to apply a separate formula for kyber 1 - while I recognize the need to reduce the range of rating variation within kyber1, that has to be tempered by consideration of how to maintain the size of the division.
Thank you for coming to my talk!
15
Replies
I’ve been in Kyber 1 since nuGAC launched and each time the squish sends me closer and closer to Kyber 2. I have a (marginally) better than 50/50 record since it all began yet still I get squished closer and closer to falling out of Kyber 1 each GAC.
Fingers crossed the devs fix it. But I’m not holding my breath.
1) apply your normal squish
2) look at the results and assess what it does to the number of players in kyber 1.
3) Apply a secondary "reverse squish" on top of the first one that adjusts kyber 1 back to 10% of kyber
That way you still collapse the kyber 1 skill ratings, and get the benefit of the squish throughout the rest of the player base.... but you don't hurt the top of the player base either!
I extended this out to the top 10 in kyber 1 by pulling from leaderboard data just in case anyone I might be missing some kind of edge effect..... Linear projection is applied all the way.
After the squish when the season was done and headed into 3v3, I ended up with a lower SR than I had when I started the season... Doesn't even make sense at this point.
Yeah, it's pretty demoralizing :-/ I understood when they pushed kyber down to 20%. But continuing to cut kyber1 on top of that just hurts. But hey, at least we can look at all the data charts and realize that a few months ago, our ranking overall would have been a kyber1 one?
One last comment on this - don't forget that since we don't get relegated until AFTER the first battle of the next season, after squish happens, you have to project forward to where skill rating will land you at that point. So people are "technically" still kyber 1 even when their skill ratings got dropped below 3610. This is why the squish effects in kyber 1 don't show up until week 2 or 3 on the bracket counts.
There’s always been a decent balance of skill vs money spent, but this has changed that. I hope this is taken seriously and addressed asap.
Yet I am still down 78 skill rating from where I where I was at end of last season, with a winning record. In fact, I started 5 wins and 2 losses, and was still 9 SR below the previous rating.
The skill squish is squishing players out of K1 for no logical reason that I can see. Being above .500 shouldn't see your rating go into a freefall.
Remember the first cycle after the introduction of the new system? That was fun and exciting, playing 12 rounds of increasing difficulty and sharing the excitement with some of the top players that were on track for the elusive 12:0 that only 3 people in the entire community achieved. Now we are again at a point of staleness, with a lot of people facing the same opponents over and over again which can be also demoralizing, exhausting and boring.
I want that fresh GAC 2.0 back, go the easy route and reset the leaderboards CG! Regularly!
Why do these values decrease as the season progresses? Why would my "final" in the 3rd bracket be worth so much fewer points than the first match in my 1st bracket?
And why is the amount won or lost exactly the same for both players? Wouldn't the problem with non-participants falling so far so fast be at least partially addressed if the SR reduction when you lose was smaller than the SR gain when you win?
The only explanation I can see is because it prevents players who are likely to win all their matches from becoming uncatchable. So if you whale, you can’t reach a skill rating where you can lose multiple matches and still be significantly ahead of the pack. i.e. go inactive and maintain number 1. So this, combined with skill squish, keeps the top competitive.
Also, for the other posts, I would point out that division shrinking is most likely due to population shrinking, not skill squish. Fewer spots in any particular division is not because skill rating got squished. As was pointed out, Kyber 1 is an edge division; as the division shrinks, there is only one direction for the accounts that don’t fit to go. Stop the division shrinkage and fix the problem.
I think I'd go for Hanlon's razor as the most likely explanation though.
Perhaps, but given CG's history, I'd go with it being more of a feature than a bug. It's been no secret that K1 has been shrinking since nuGAC's inception.
In fact it is EXACTLY because skill rating got squished. The correction pushes all skill ratings downward but the boundary doesn't change.
We lost 30k players from Kyber between May and August 2022. In the same time period, the total GAC player base changed by 10k. At the exact same time, we had longitudinal data on players showing that their skill rating was pushed across the boundary because of the squish - they had winning records, but lower skill ratings.
Look at the shape of the squish in the original figure. Top leagues are being pushed downward; bottom leagues are pushed upwards. If they get the squish "right" the spread would balance that out. But another version of "right" is deliberately changing league shapes to match their view of what the distribution should look like across the board, without a consideration of what it does to an individual player.
The reason it really matters is because the difference is whether someone who used to be kyber 4 and is now aurodium (because of the loss of kyber slots) is a "worse player".... or the same place in the player ranking, but just ends up with a worse score now.
I'm pretty sure k1 is a bug. But I do think the shifting of kyber last summer was a "feature".
You can't believe anything they say, so have you tracked the drop rates to verify?
Your math is impeccable, and you are definitely a better mathematician than I am.
However the logical contradiction needs to be resolved. You say skill squish does this, CG says it can’t change a player’s rank. Contradiction. And since your goal is to convince CG, you need hard, incontrovertible proof. Because it doesn’t matter how many of us stand in a circle and say “I agree” if we are basing that on speculative or circumstantial proof.
And if you say you have hard evidence, I’m fine with accepting your word on it.
So I often wondered this myself, and at some point I thought maybe it was an attempt to prevent rematches, at least at the middle ranges. If in week 1, my first opponent has the same SR as me, even if we go the same record that week, we will end with different SRs. If I had won our match, then any round I lose where my previous opponent won would still put me slightly ahead of them. If I lose the next two and my opponent wins the next two, now they are way ahead of me. Any time we win or lose in the same round, we shift together. So it will be almost impossible to face each other again in week 2 unless there are insufficient numbers of players with our exact SRs, or we end up on the edges of an SR range, etc. We probably won't even face each other in week three. It would take a specific pattern of wins/losses for each of us over time to end up with the same SRs as each other again.
Now I'm wondering if it is some kind of attempt at a convergence algorithm. If you are attempting to assess a latent variable, skill, and you won or lost against someone with the same skill estimate, you would want to be paired next with someone of higher or lower estimated skill, respectively. Then if you win or lose, you move in the appropriate direction, but not overshooting a previous skill estimate if you moved back in that direction. Eventually you get to a point where you are moving very little, and then pairings against you and someone of the same SR result in a 50% win rate. This is an estimate of your "true" skill.
Now, because your opponents' SRs also need to be estimated, you need to have a more sophisticated procedure, and in certain contexts this is a type of maximum likelihood procedure. Additionally, CG have settled into an algorithim of being matched against someone with the same SR (and this indeed may be part of an attempt to estimate all SRs simultaneously). So instead of matching you against someone of different skill, they bump you into an extremely different range at first, and narrower skill ranges as time goes on. So it may be an attempt at converging on your true skill level.
However, I'm not really sure this is actually an appropriate method of approximating this, or if the numbers they've chosen even make much sense given such an algorithm. Indeed, the numbers reset every season! So that seems counter intuitive. Additionally, there's the problematic formula for the skill squish, which is the subject of this thread. In my mind, instead of "skill squish," I would've called it, "recentering." In that context, I think every season is too frequent for it to be happening anyway.
Additionally, I think if they are basing the matching on ELO models or chess or something like that, then there is a degree of model misfit for this game. I think roster size is an additional variable to take into consideration, as is participation factor. That last is difficult to nail down, as people who score 0 or even just 10/65/75 are easy to catch, but what about the people who attack, then stop once they've realized they've won or lost? Not even bothering to attack a fleet zone, for instance, because it wouldn't influence the outcome of a match? I see a lot of those.
At any rate, it may be the current model is working exactly as CG intends. There are numbers math nerds look at for the sake of an accurate matchmaking/skill estimation algorithm, and then there are numbers bean counters look at. Having a system that makes players feel as though they are on a treadmill with no upward mobility may be of interest to the latter group.
Still, I wonder if the skill squish being a blanket calculation that doesn't take into account the extremes is indeed just a mistake/oversight. And it needn't be. It seems like you could do a pretty straightforward z-score transformation and get the percentages lined up where they need to be.
Of course there is. It's just data. Track it.
However, the percentages in each league should stay the same, contrary to what is presented here. However, it occurs to me that new, previously unscored players entering the system is an issue. If they are all given the same score, then this could affect the distribution, and I'm not sure yet what that would do to the percentages of kyber 1 players. With small numbers of new players, it may be a non-issue. However, a large enough spike at the bottom of the distribution could have a drastic effect.
Do you have any numbers for the percentages? Both the percentage of the total signed up players by league/division and the percentages of division by league? That would be interesting to look at.
This graph certain seems to fluctuate with regards to the percentages of each. But the color lines seem to roughly follow the same trend, and I'm wondering if we just need more data. I'm wondering how much the "noise" in this graph is due to and influx of new players with the same score into the system, and the rejoining of people who skipped a season or partial season. If people leave the system or forget to join up in the middle, I wonder how this affects things combined with the players coming in, especially if this is not random, but disproportionate by league/division.
We might find that over time, there are some lumps, but the general long term trends are within some margin.
In addition to the numbers, it might be interesting to see a bar graph like this but dividing up all the different leagues instead of just kyber.