Current squish formula, Math, and Kyber1: Here is how to fix the leak

Prev13
hp25gjrs408c.png

tl;dr version: developers need a separate squish formula for kyber 1 and possibly kyber 2, and a corrective positive squish this month. Analysis of the squish devs are using explains why!

Problem: Kyber1 is down to 6% of kyber, and 1.1% of player base. This has happened through squish.

Discussion and background:
Currently, squish is being applied as a linear relationship of squish value to skill rating across each league. This works for most of the player base, since there is free flow of players from above and from below the division such that over the month, the players "spread" from the middle of each league out to the edges.

However, it fails in kyber, since kyber 1 is at the edge of the player base. At the top of kyber, players can only come from the bottom, they can't be brought in from "the league above" since that doesn't exist.

Because of this unique property of kyber 1, we shouldn't push players out of kyber 1 through the squish. Over all, skill rating points won and lost in combat is zero sum; any skill rating points won by one player are lost by another. In the context of kyber 1, that means each player who enters kyber 1 does so by pushing someone else out. So kyber1 can't grow over the course of the month the way the other league "div 1" can.

In fact, natural movement of skill rating in kyber 1 would be net negative even without squish, since anyone in kyber 1 who forgets to register is pushed down in skill rating, while combat again will be net neutral.

I would like devs to carefully consider this! For the next month or two, we need a positive rather than negative squish to get kyber1 back to 10% of kyber. After that, rather than a linear formula for skill rating to squish across all of kyber, you will want to apply a separate formula for kyber 1 - while I recognize the need to reduce the range of rating variation within kyber1, that has to be tempered by consideration of how to maintain the size of the division.

Thank you for coming to my talk!

Replies

  • Options
    This is brilliant.

    I’ve been in Kyber 1 since nuGAC launched and each time the squish sends me closer and closer to Kyber 2. I have a (marginally) better than 50/50 record since it all began yet still I get squished closer and closer to falling out of Kyber 1 each GAC.

    Fingers crossed the devs fix it. But I’m not holding my breath.
  • Options
    Feedback is that I need to be more direct. My proposal:
    1) apply your normal squish
    2) look at the results and assess what it does to the number of players in kyber 1.
    3) Apply a secondary "reverse squish" on top of the first one that adjusts kyber 1 back to 10% of kyber

    That way you still collapse the kyber 1 skill ratings, and get the benefit of the squish throughout the rest of the player base.... but you don't hurt the top of the player base either!
  • Options
    xpnsz40morzb.png

    I extended this out to the top 10 in kyber 1 by pulling from leaderboard data just in case anyone I might be missing some kind of edge effect..... Linear projection is applied all the way.

  • Options
    Please follow up with Taliana’s work!
  • Options
    I had a "winning" season last 5v5. Was pretty excited about it to get back to Kyber 1, or at least closer to it and hop back over during 3v3 potentially.

    After the squish when the season was done and headed into 3v3, I ended up with a lower SR than I had when I started the season... Doesn't even make sense at this point.
  • Options
    I had a "winning" season last 5v5. Was pretty excited about it to get back to Kyber 1, or at least closer to it and hop back over during 3v3 potentially.

    After the squish when the season was done and headed into 3v3, I ended up with a lower SR than I had when I started the season... Doesn't even make sense at this point.

    Yeah, it's pretty demoralizing :-/ I understood when they pushed kyber down to 20%. But continuing to cut kyber1 on top of that just hurts. But hey, at least we can look at all the data charts and realize that a few months ago, our ranking overall would have been a kyber1 one?
  • Options
    Thank you for your research, Taliana!
  • Options
    Thanks for posting the data, hope the devs actually take a look at this so it doesn't get too far out of whack.
  • Options
    Great analysis. I agree with your proposed fixes.
  • Options
    One last
    Taliana wrote: »
    Feedback is that I need to be more direct. My proposal:
    1) apply your normal squish
    2) look at the results and assess what it does to the number of players in kyber 1.
    3) Apply a secondary "reverse squish" on top of the first one that adjusts kyber 1 back to 10% of kyber

    That way you still collapse the kyber 1 skill ratings, and get the benefit of the squish throughout the rest of the player base.... but you don't hurt the top of the player base either!

    One last comment on this - don't forget that since we don't get relegated until AFTER the first battle of the next season, after squish happens, you have to project forward to where skill rating will land you at that point. So people are "technically" still kyber 1 even when their skill ratings got dropped below 3610. This is why the squish effects in kyber 1 don't show up until week 2 or 3 on the bracket counts.
  • Options
    Great share, Tal. The fact that this has been identified by a fellow player, and not a paid dev is kind of shocking to be honest. This unfortunate “oversight” has cost many players to be frustrated with this game mode, genuinely. I understand the desire to force less gems on the player base, but the mentality you’re showing us is that you must pay to remain in K1. Some of my fellow CAW guild mates have recently fallen into K2, and I can vouch for all of them that they’re some of the best strategists to play the game, and this is the first time that they’re in K2.

    There’s always been a decent balance of skill vs money spent, but this has changed that. I hope this is taken seriously and addressed asap.
  • Options
    Personal example, I dropped 121 SR from squish transition to the Nov 7th 3v3 season. I had a record of 5-4, which is not spectacular but should be a net positive in my skill rating, having won more than I lost.

    Yet I am still down 78 skill rating from where I where I was at end of last season, with a winning record. In fact, I started 5 wins and 2 losses, and was still 9 SR below the previous rating.

    The skill squish is squishing players out of K1 for no logical reason that I can see. Being above .500 shouldn't see your rating go into a freefall.
  • Tohu
    81 posts Member
    Options
    There would be no need for a squish formula at all if CG would reset the leaderboards regularly.

    Remember the first cycle after the introduction of the new system? That was fun and exciting, playing 12 rounds of increasing difficulty and sharing the excitement with some of the top players that were on track for the elusive 12:0 that only 3 people in the entire community achieved. Now we are again at a point of staleness, with a lot of people facing the same opponents over and over again which can be also demoralizing, exhausting and boring.

    I want that fresh GAC 2.0 back, go the easy route and reset the leaderboards CG! Regularly!
  • Options
    The thing I understand least about the Skill Rating system is the gradual decrease in the SR points awarded / removed if you win or lose.

    Why do these values decrease as the season progresses? Why would my "final" in the 3rd bracket be worth so much fewer points than the first match in my 1st bracket?

    And why is the amount won or lost exactly the same for both players? Wouldn't the problem with non-participants falling so far so fast be at least partially addressed if the SR reduction when you lose was smaller than the SR gain when you win?
  • Options
    The thing I understand least about the Skill Rating system is the gradual decrease in the SR points awarded / removed if you win or lose.

    Why do these values decrease as the season progresses? Why would my "final" in the 3rd bracket be worth so much fewer points than the first match in my 1st bracket?

    And why is the amount won or lost exactly the same for both players? Wouldn't the problem with non-participants falling so far so fast be at least partially addressed if the SR reduction when you lose was smaller than the SR gain when you win?

    The only explanation I can see is because it prevents players who are likely to win all their matches from becoming uncatchable. So if you whale, you can’t reach a skill rating where you can lose multiple matches and still be significantly ahead of the pack. i.e. go inactive and maintain number 1. So this, combined with skill squish, keeps the top competitive.

    Also, for the other posts, I would point out that division shrinking is most likely due to population shrinking, not skill squish. Fewer spots in any particular division is not because skill rating got squished. As was pointed out, Kyber 1 is an edge division; as the division shrinks, there is only one direction for the accounts that don’t fit to go. Stop the division shrinkage and fix the problem.

    Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for life.
  • Iy4oy4s
    2950 posts Member
    Options
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.
  • Options
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.
  • Options
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Perhaps, but given CG's history, I'd go with it being more of a feature than a bug. It's been no secret that K1 has been shrinking since nuGAC's inception.
  • TVF
    36860 posts Member
    Options
    They probably looked at how many more crystals they were paying out after moving them from arena.
    I need a new message here. https://discord.gg/AmStGTH
  • Options
    nfidel2k wrote: »

    Also, for the other posts, I would point out that division shrinking is most likely due to population shrinking, not skill squish. Fewer spots in any particular division is not because skill rating got squished. As was pointed out, Kyber 1 is an edge division; as the division shrinks, there is only one direction for the accounts that don’t fit to go. Stop the division shrinkage and fix the problem.

    In fact it is EXACTLY because skill rating got squished. The correction pushes all skill ratings downward but the boundary doesn't change.

    We lost 30k players from Kyber between May and August 2022. In the same time period, the total GAC player base changed by 10k. At the exact same time, we had longitudinal data on players showing that their skill rating was pushed across the boundary because of the squish - they had winning records, but lower skill ratings.

    Look at the shape of the squish in the original figure. Top leagues are being pushed downward; bottom leagues are pushed upwards. If they get the squish "right" the spread would balance that out. But another version of "right" is deliberately changing league shapes to match their view of what the distribution should look like across the board, without a consideration of what it does to an individual player.

    The reason it really matters is because the difference is whether someone who used to be kyber 4 and is now aurodium (because of the loss of kyber slots) is a "worse player".... or the same place in the player ranking, but just ends up with a worse score now.





  • Options
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Perhaps, but given CG's history, I'd go with it being more of a feature than a bug. It's been no secret that K1 has been shrinking since nuGAC's inception.

    I'm pretty sure k1 is a bug. But I do think the shifting of kyber last summer was a "feature".

    qxo6zalg1d9s.png


  • Iy4oy4s
    2950 posts Member
    Options
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Not in this industry. There are numerous patents that play to people’s impulses, drop rate adjustments and other stuff. CG has said they don’t use them, but it’s been proven that you can’t believe anything they say, so….there’s that. The last time they said “we do t adjust drop rates”, they could have has a meeting and adjusted drop rates when you get close to the next star “after internal review….”

    I used to think this with MSF….MSF is playing this game hardcore…and it’s making them money by the dump truck.
  • TVF
    36860 posts Member
    Options
    Iy4oy4s wrote: »
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Not in this industry. There are numerous patents that play to people’s impulses, drop rate adjustments and other stuff. CG has said they don’t use them, but it’s been proven that you can’t believe anything they say, so….there’s that. The last time they said “we do t adjust drop rates”, they could have has a meeting and adjusted drop rates when you get close to the next star “after internal review….”

    I used to think this with MSF….MSF is playing this game hardcore…and it’s making them money by the dump truck.

    You can't believe anything they say, so have you tracked the drop rates to verify?
    I need a new message here. https://discord.gg/AmStGTH
  • Options
    Taliana wrote: »
    nfidel2k wrote: »

    Also, for the other posts, I would point out that division shrinking is most likely due to population shrinking, not skill squish. Fewer spots in any particular division is not because skill rating got squished. As was pointed out, Kyber 1 is an edge division; as the division shrinks, there is only one direction for the accounts that don’t fit to go. Stop the division shrinkage and fix the problem.

    In fact it is EXACTLY because skill rating got squished. The correction pushes all skill ratings downward but the boundary doesn't change.

    We lost 30k players from Kyber between May and August 2022. In the same time period, the total GAC player base changed by 10k. At the exact same time, we had longitudinal data on players showing that their skill rating was pushed across the boundary because of the squish - they had winning records, but lower skill ratings.

    Look at the shape of the squish in the original figure. Top leagues are being pushed downward; bottom leagues are pushed upwards. If they get the squish "right" the spread would balance that out. But another version of "right" is deliberately changing league shapes to match their view of what the distribution should look like across the board, without a consideration of what it does to an individual player.

    The reason it really matters is because the difference is whether someone who used to be kyber 4 and is now aurodium (because of the loss of kyber slots) is a "worse player".... or the same place in the player ranking, but just ends up with a worse score now.

    Your math is impeccable, and you are definitely a better mathematician than I am.

    However the logical contradiction needs to be resolved. You say skill squish does this, CG says it can’t change a player’s rank. Contradiction. And since your goal is to convince CG, you need hard, incontrovertible proof. Because it doesn’t matter how many of us stand in a circle and say “I agree” if we are basing that on speculative or circumstantial proof.

    And if you say you have hard evidence, I’m fine with accepting your word on it.
    Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for life.
  • crzydroid
    7419 posts Moderator
    edited December 2022
    Options
    The thing I understand least about the Skill Rating system is the gradual decrease in the SR points awarded / removed if you win or lose.

    Why do these values decrease as the season progresses? Why would my "final" in the 3rd bracket be worth so much fewer points than the first match in my 1st bracket?

    And why is the amount won or lost exactly the same for both players? Wouldn't the problem with non-participants falling so far so fast be at least partially addressed if the SR reduction when you lose was smaller than the SR gain when you win?

    So I often wondered this myself, and at some point I thought maybe it was an attempt to prevent rematches, at least at the middle ranges. If in week 1, my first opponent has the same SR as me, even if we go the same record that week, we will end with different SRs. If I had won our match, then any round I lose where my previous opponent won would still put me slightly ahead of them. If I lose the next two and my opponent wins the next two, now they are way ahead of me. Any time we win or lose in the same round, we shift together. So it will be almost impossible to face each other again in week 2 unless there are insufficient numbers of players with our exact SRs, or we end up on the edges of an SR range, etc. We probably won't even face each other in week three. It would take a specific pattern of wins/losses for each of us over time to end up with the same SRs as each other again.

    Now I'm wondering if it is some kind of attempt at a convergence algorithm. If you are attempting to assess a latent variable, skill, and you won or lost against someone with the same skill estimate, you would want to be paired next with someone of higher or lower estimated skill, respectively. Then if you win or lose, you move in the appropriate direction, but not overshooting a previous skill estimate if you moved back in that direction. Eventually you get to a point where you are moving very little, and then pairings against you and someone of the same SR result in a 50% win rate. This is an estimate of your "true" skill.

    Now, because your opponents' SRs also need to be estimated, you need to have a more sophisticated procedure, and in certain contexts this is a type of maximum likelihood procedure. Additionally, CG have settled into an algorithim of being matched against someone with the same SR (and this indeed may be part of an attempt to estimate all SRs simultaneously). So instead of matching you against someone of different skill, they bump you into an extremely different range at first, and narrower skill ranges as time goes on. So it may be an attempt at converging on your true skill level.

    However, I'm not really sure this is actually an appropriate method of approximating this, or if the numbers they've chosen even make much sense given such an algorithm. Indeed, the numbers reset every season! So that seems counter intuitive. Additionally, there's the problematic formula for the skill squish, which is the subject of this thread. In my mind, instead of "skill squish," I would've called it, "recentering." In that context, I think every season is too frequent for it to be happening anyway.

    Additionally, I think if they are basing the matching on ELO models or chess or something like that, then there is a degree of model misfit for this game. I think roster size is an additional variable to take into consideration, as is participation factor. That last is difficult to nail down, as people who score 0 or even just 10/65/75 are easy to catch, but what about the people who attack, then stop once they've realized they've won or lost? Not even bothering to attack a fleet zone, for instance, because it wouldn't influence the outcome of a match? I see a lot of those.

    At any rate, it may be the current model is working exactly as CG intends. There are numbers math nerds look at for the sake of an accurate matchmaking/skill estimation algorithm, and then there are numbers bean counters look at. Having a system that makes players feel as though they are on a treadmill with no upward mobility may be of interest to the latter group.

    Still, I wonder if the skill squish being a blanket calculation that doesn't take into account the extremes is indeed just a mistake/oversight. And it needn't be. It seems like you could do a pretty straightforward z-score transformation and get the percentages lined up where they need to be.
  • Iy4oy4s
    2950 posts Member
    edited December 2022
    Options
    TVF wrote: »
    Iy4oy4s wrote: »
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Not in this industry. There are numerous patents that play to people’s impulses, drop rate adjustments and other stuff. CG has said they don’t use them, but it’s been proven that you can’t believe anything they say, so….there’s that. The last time they said “we do t adjust drop rates”, they could have has a meeting and adjusted drop rates when you get close to the next star “after internal review….”

    I used to think this with MSF….MSF is playing this game hardcore…and it’s making them money by the dump truck.

    You can't believe anything they say, so have you tracked the drop rates to verify?

    I’m not taking about the overall drop rate, but the drop rates when you’re close to the next star. There’s really no way to verify this. There are patents that do just this.
  • Options
    Iy4oy4s wrote: »
    TVF wrote: »
    Iy4oy4s wrote: »
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Not in this industry. There are numerous patents that play to people’s impulses, drop rate adjustments and other stuff. CG has said they don’t use them, but it’s been proven that you can’t believe anything they say, so….there’s that. The last time they said “we do t adjust drop rates”, they could have has a meeting and adjusted drop rates when you get close to the next star “after internal review….”

    I used to think this with MSF….MSF is playing this game hardcore…and it’s making them money by the dump truck.

    You can't believe anything they say, so have you tracked the drop rates to verify?

    I’m not taking about the overall drop rate, but the drop rates when you’re close to the next star. There’s really no way to verify this. There are patents that do just this.
    I’m going to have to use Hitchen’s Razor on this idea.
  • TVF
    36860 posts Member
    Options
    Iy4oy4s wrote: »
    TVF wrote: »
    Iy4oy4s wrote: »
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Not in this industry. There are numerous patents that play to people’s impulses, drop rate adjustments and other stuff. CG has said they don’t use them, but it’s been proven that you can’t believe anything they say, so….there’s that. The last time they said “we do t adjust drop rates”, they could have has a meeting and adjusted drop rates when you get close to the next star “after internal review….”

    I used to think this with MSF….MSF is playing this game hardcore…and it’s making them money by the dump truck.

    You can't believe anything they say, so have you tracked the drop rates to verify?

    I’m not taking about the overall drop rate, but the drop rates when you’re close to the next star. There’s really no way to verify this.

    Of course there is. It's just data. Track it.
    I need a new message here. https://discord.gg/AmStGTH
  • crzydroid
    7419 posts Moderator
    Options
    Keeping in mind that any squish they do will result in a graph like the one at the very top of this thread. Looking at this, I think the scoring method will result in the skill score range ballooning. I haven't yet looked at, simulation-wise, how this would affect league groupings, but any centering technique would bring the scores of carbonite 5 and kyber 1 in drastically.

    However, the percentages in each league should stay the same, contrary to what is presented here. However, it occurs to me that new, previously unscored players entering the system is an issue. If they are all given the same score, then this could affect the distribution, and I'm not sure yet what that would do to the percentages of kyber 1 players. With small numbers of new players, it may be a non-issue. However, a large enough spike at the bottom of the distribution could have a drastic effect.
  • crzydroid
    7419 posts Moderator
    Options
    Taliana wrote: »
    Iy4oy4s wrote: »
    Less people in kyber 1-2 means less “free” crystals given out and more crystal spent trying to stay in kyber 1-2.

    I think I'd go for Hanlon's razor as the most likely explanation though.

    Perhaps, but given CG's history, I'd go with it being more of a feature than a bug. It's been no secret that K1 has been shrinking since nuGAC's inception.

    I'm pretty sure k1 is a bug. But I do think the shifting of kyber last summer was a "feature".

    qxo6zalg1d9s.png


    Do you have any numbers for the percentages? Both the percentage of the total signed up players by league/division and the percentages of division by league? That would be interesting to look at.

    This graph certain seems to fluctuate with regards to the percentages of each. But the color lines seem to roughly follow the same trend, and I'm wondering if we just need more data. I'm wondering how much the "noise" in this graph is due to and influx of new players with the same score into the system, and the rejoining of people who skipped a season or partial season. If people leave the system or forget to join up in the middle, I wonder how this affects things combined with the players coming in, especially if this is not random, but disproportionate by league/division.

    We might find that over time, there are some lumps, but the general long term trends are within some margin.

    In addition to the numbers, it might be interesting to see a bar graph like this but dividing up all the different leagues instead of just kyber.
Sign In or Register to comment.