Raid gear rewards are NOT random - is this deliberate or a massive bug?
TLDR - raid gear types do not drop equally for individual players and most people seem to get a surplus of one type of gear and a shortage of the others. This shouldn't happen if the gear drops equally randomly. So is the system broken deliberately or is there a massive bug?
Detailed post:
So we all know that by completing the raids we get a chance to win some raid gear that we all need. There are 4 types - furnace, droidcaller, nubian tech, and nubian scanner.
Everyone finishing a raid will get salvage (or a full item) for one of these gear types. The amount you get depends on the raid level and where you finish.
Now everyone assumes that we have equal chance of winning one of these 4 items. This would make sense as they are all roughly equally needed if we are collecting all the characters in the game (maybe scanners are needed slightly more than droidcallers but it is close enough).
However, after I have completed 69 raids, most of them at heroic level with a few early ones at tier 5 and 6, it is emphatically clear to me that the raid gear does NOT drop randomly.
In fact, it seems clear that people are more likely to get a single gear type only, with only a small chance of the other 3 types.
As an example, for my first 50 or so raids I mainly got scanners - the distribution was roughly 60-70% scanners, and 30-40% the 3 other gears (I didn't keep exact records back then but have been doing so more recently). This pattern has got even more extreme the more raids we have done - in my last 12 raids, I have received scanners 11 times.
If we assume the gear drops equally (so there is a 25% chance of receiving each gear type), then it is easy enough to do a statistical hypothesis test. In my case this test is rejected so emphatically that there is (statistically speaking) zero chance of this pattern of rewards occurring unless the original assumption is wrong. This means that gear does NOT drop randomly.
(I teach University level statistics and probability so know what I am talking about. I have calculated the hypothesis test and it is rejected so strongly that the effective probability of the original assumption of equal gear type drops being true is 0).
Now, my results could be an anomaly just specific to me, but speaking to my guild-mates and following the discussion on the forums, it is clear that although my example is extreme, it is part of a common pattern - most people tend to have a big surplus of one or two types of raid gear and a big shortage of the other types. Not everyone is short of the same gear types though so across all players, the gear types do drop at about 25% each. But for an individual player it seems that this isn't at all true.
Am I wrong?
If not, then I want to ask the question - is this gear drop anomaly deliberate? Or a massive bug in the game?
After 70 raids getting increasingly frustrated I think it is worth getting some insight into this question.
Detailed post:
So we all know that by completing the raids we get a chance to win some raid gear that we all need. There are 4 types - furnace, droidcaller, nubian tech, and nubian scanner.
Everyone finishing a raid will get salvage (or a full item) for one of these gear types. The amount you get depends on the raid level and where you finish.
Now everyone assumes that we have equal chance of winning one of these 4 items. This would make sense as they are all roughly equally needed if we are collecting all the characters in the game (maybe scanners are needed slightly more than droidcallers but it is close enough).
However, after I have completed 69 raids, most of them at heroic level with a few early ones at tier 5 and 6, it is emphatically clear to me that the raid gear does NOT drop randomly.
In fact, it seems clear that people are more likely to get a single gear type only, with only a small chance of the other 3 types.
As an example, for my first 50 or so raids I mainly got scanners - the distribution was roughly 60-70% scanners, and 30-40% the 3 other gears (I didn't keep exact records back then but have been doing so more recently). This pattern has got even more extreme the more raids we have done - in my last 12 raids, I have received scanners 11 times.
If we assume the gear drops equally (so there is a 25% chance of receiving each gear type), then it is easy enough to do a statistical hypothesis test. In my case this test is rejected so emphatically that there is (statistically speaking) zero chance of this pattern of rewards occurring unless the original assumption is wrong. This means that gear does NOT drop randomly.
(I teach University level statistics and probability so know what I am talking about. I have calculated the hypothesis test and it is rejected so strongly that the effective probability of the original assumption of equal gear type drops being true is 0).
Now, my results could be an anomaly just specific to me, but speaking to my guild-mates and following the discussion on the forums, it is clear that although my example is extreme, it is part of a common pattern - most people tend to have a big surplus of one or two types of raid gear and a big shortage of the other types. Not everyone is short of the same gear types though so across all players, the gear types do drop at about 25% each. But for an individual player it seems that this isn't at all true.
Am I wrong?
If not, then I want to ask the question - is this gear drop anomaly deliberate? Or a massive bug in the game?
After 70 raids getting increasingly frustrated I think it is worth getting some insight into this question.
0
Replies
I'd go as far as to say there are dynamic filters in place to detect when a person buys gear from the shop, then seemingly less of that gear types drops on farmable gear encouraging spending as well. Time and time again I've seen this happen
except that i definitely did have scanners previously. i just used them. can't really recall how many were bought by crystals though. I distinctly recall there were times all i wanted was a caller, or furnance...
all in all over the course of a few months, it's relatively even, I'd say.
sadly, in the last 5 raids when i've been #1, not even a single piece of scanner, much less a full one...
it's kinda funny sad to me. When i used to be middle of the pack or in the teens in raid ranks, i felt like i got some scanner salvages. now that i can't get any higher on the ranks, somehow i never get the pieces i need....
From this wouldn't it make sense that older guilds which have been building and growing together having a need for the same gear and a surplus of the same gear. This in my mind would squew an analysis of what useful gear you have and what you have that is "surplus".
They also put different values on raid gear any time you see it in a shop, wouldn't that imply that they have a different "value" or drop rate?
Has anyone added up the full lifetime gear for all toons, maybe there is just a higher need for certain gear that if they all drop at the same rate you would see a surplus of not used gear.
Just my thoughts, but as always the rng in this game is.....funny to say the least.
My point is simply that you will likely not get many posts that have actual relevant statistics. You will more likely see anecdotal "evidence" without much substance. Much of which might have to do with roster preferences instead of drop rates.
In game guild: TNR Uprising
I beat the REAL T7 Yoda (not the nerfed one) and did so before mods were there to help
*This space left intentionally blank*
However 10 of my last 11 raids have been droid callers.
In turn this caused me to have to spend all my guild currency to craft scanners at their ridiculous prices. In turn that caused me to fall behind farming the characters in guild shipments.
Contrary to what many people think about RNG... I honestly don't believe there is actually very much that is truly random in this game. I believe much more time is spent programming the code to draw money out of people.
Trade you 170 Nubian tech salvage for droid caller. Lol
In game guild: TNR Uprising
I beat the REAL T7 Yoda (not the nerfed one) and did so before mods were there to help
*This space left intentionally blank*
Conclusion: yes they are.
Dont need to be a statistics professor for this.
Deal! I have 5 complete droid caller right now
In game guild: TNR Uprising
I beat the REAL T7 Yoda (not the nerfed one) and did so before mods were there to help
*This space left intentionally blank*
I need scanners like crazy now but when I had no toons in need of them I had enough salvage to fully craft 2 of them.
After u put those two and realized I needed 7 more, I stopped getting them until the last T7 raid (where I ended up getting 1st for the first time ever) where I got 40 scanner salvage.
I was getting droid callers but now that iced used up almost all of the ones I have and am "looking" for more I won't be surprised if it doesn't show up again.
I'm not a statistics guru but I DO know that when I am just playing the game and getting and equipping gear as it becomes available (even going so far as to look and see what a toon needs before I start gearing them) I notice far less drop rate frustration.
A watched pot never boils as they say.
Though when I look in my arena and see my competitors with all gear 10 and 11 on their arena squad and have only done 12 raids or less and then look at MY own toons and that I have done almost 50 raids and only have 3 toons past gear 9... THAT is frustrating, lol.
There's another fallacy, the secundum quid, where you draw conclusions about the distribution of rewards for all players, based on a very small set of data which you gathered from personal experience (again, not verifyable). Even if you get 22 scanners in 22 raids, the next 70 may very well be the other 3 types in roughly equal amounts and you'd still have a normal distribution. Your sample is just much too small compared to the number of reward types, it's bordering cherry picking.
And, another thing: wouldn't you need to know how often such a series of identical rewards occurs on average before drawing conclusions about the statistical probability? You'd need a real big set of data before you could conclude that 11 scanners in a row is indicating something fishy. Maybe it happens to a mere 0.1% of the players and you're one of them.
I'm not saying you're wrong, because your data is not presented and I haven't got any of my own, so I cannot draw conclusions. But all you really said to support your claim was: "My sample was very small, I used my memory as the main source, but I know what I'm talking about". That's really not a strong position.
I got nothing but scanners or Nubian design techs for the last 11+ raids, so if I could've just assumed that the game hates me. But lo and behold, 2 full furnaces the last 3 raids in a row! Rng is rng, man.
Firstly the data - my first 15 raids I didn't track the data (why would you) but after getting mainly scanners in these raids I started to record to see if it was really random (so the skew was already there early on). I have tracked the data for the past 54 raids (69 in total completed now), and that is what gives the figures. The tracked data i have for 54 raids is 57% scanners (31/54). My statement of 60-70% is based on my recollection of the first 15 raids but we can discount these early raids if it makes things easier.
Note that these are the actual raid rewards received. It makes no difference which characters I have or have not equipped. There is clearly a skew towards scanners being received (surplus) and much less of the other raid gear types.
So we have 31 drops for scanners out of 54 raids (the actual recorded data). This is a perfectly reasonable sample size to undertake a statistical hypothesis test. You can do such a test for ANY sample size - it is just with low sample sizes your test will almost never reject the hypothesis because of the small sample.
We can do a binomial test under the assumption that I have a probability of 0.25 to receive a scanner (the null hypothesis assuming gear drops equally). We have total 'successes' of 31 from 54 trials, with p = 0.25.
Putting these figures into a binomial test calculator such as this one: http://vassarstats.net/binomialX.html gives a probability of receiving 31 (or more) scanners from 54 raids of 0.000000411526, or 1 in 2.5 million. This is such a low value that it is clear that the hypothesis test is rejected. In other words, statistically speaking, we have to reject the idea that raid gear drops equally (at least in my own case).
People are quite right that my results have no bearing on other people, and all I can say is that the statistical test above suggests that raid gear doesn't drop equally for me. Now, it could be that I am the one guy out of all the players who has just had a really bad run of luck. This is why I asked if other people have the same or similar patterns, as I see anecdotally other people saying the same thing.
Well, the game actually shows a quite obvious behaviour answering your question: the 1 hour that it needs to "calculate" rewards.
Well, I highly doubt random drops would need an hour to be calculated.
This more than obviously is a ROI-algorithm that optimizes rewards to the point were we always get what we actually DO NOT NEED.
Not only are the rewards not random, they are tuned to reward you just enough to give you an incentive to buy crystals.
Did you ever get 40 scanner pieces when you allready had 30-45? Any stun cuffs if you needed a couple more?
I buy furnaces (2 full ones on stock currently) and scanners (50 pieces from raids on stock) in shipments when I see them, the only gear I do not need is raid discs: I have 177 plus a full one.
Guess what I am getting from t7 raids all the time? Yes, discs 3/4 times, 0,5/4 times furnace pieces, 0,5/4 times droid caller pieces.
The reward system is working like a charm and thr timer at the end of each raid laughs in our faces.
I just want to ask, why are you assuming they would drop evenly? That is a major assumption that has never been stated by a dev and without knowing the base odds any data is just data.
Also that is an extremely small data set and that makes it hard to understand the system as a whole.
There is also the point of you are not stating your inventory, if it is intentional then what you have in you inventory should be a factor....
It would be a genius bit of coding to do this and overall not that hard, but it would seem a little taxing on the system for very little reward to them.
In my inventory I have 11 scanners equipped and 286 scanner salvage spare. I have 8 furnaces equipped (3 of which were bought with crystals or from the guild store), 4 droidcallers equipped (1 of which was precrafted), and 4 nubian tech equipped. For the latter 3 gear types I have less than 20 salvage for each spare. So if we discount precrafting or buying from shipments / guild store, then I have received (nearly) 17 complete scanners / salvage, but only 5, 4 and 3 of the other gear types. That is clearly massively skewed.
You can check the details here: https://swgoh.gg/u/sarryen/collection/
And it is true that the devs have never stated that the gear will drop equally, but if we are rewarded with gear that we don't need with a higher probability then that would be a big issue for many people.
The data set isn't that small - the sample of 54 raids is enough to complete a robust hypothesis test - but based on a single individual. It is certainly true that it may just be me affected like this, but it seems there are others in similar situations going by what people are saying here.
Don't need Furnaces but have had 3 whole pieces in last 3 raids
Here's my own stack where I have no excess of the other 3 raid gear: