No matter how much they test, even if they added test servers, something odd will happen when you 170 characters with 6 mod slots per character, with multiple types of mods. You just can't get the variety in a limited data set that you will have in a live environment.
Should they have caught something with the more popular characters? Sure. But this is only going to get worse as they add more stuff, because it's impossible to figure out all of the interactions.
My prediction? At some point you'll be forced to only use like characters with like characters. So no more running old republic with galactic republic, for example. I hope I'm wrong, but that's how I saw things play out in a game I used to play.
I’m sure the “thousands of hours” was in reference to the total amount of time they’ve spent testing since the game was released 3.5 years ago. The 12 minutes that they spent testing JKA’s rework factors into that number
Doubt "thousand of hour" were ever spent across the games whole lifespan.
No matter how much they test, even if they added test servers, something odd will happen when you 170 characters with 6 mod slots per character, with multiple types of mods. You just can't get the variety in a limited data set that you will have in a live environment.
Should they have caught something with the more popular characters? Sure. But this is only going to get worse as they add more stuff, because it's impossible to figure out all of the interactions.
My prediction? At some point you'll be forced to only use like characters with like characters. So no more running old republic with galactic republic, for example. I hope I'm wrong, but that's how I saw things play out in a game I used to play.
Would be valid if the problems they keep occuring didn't have anything to do with the 5 most popular squads .
It's one of those things. It looks so obvious in hindsight, but easily missed at the time. I've worked in software development and seen far worse things than this.
With all the interactions, it speaks to me of either simplifying the game or beta testing. As a player, I'd infinitely prefer the latter, but it will lose cg the ability to have surprise events as stuff is going to leak out of beta testing almost instantly.
Also, I wince every time I see these 'fire everyone, heads on sticks' type of comments. Try a bit of empathy, that sort of stuff is really not on!
No matter how much they test, even if they added test servers, something odd will happen when you 170 characters with 6 mod slots per character, with multiple types of mods. You just can't get the variety in a limited data set that you will have in a live environment.
Should they have caught something with the more popular characters? Sure. But this is only going to get worse as they add more stuff, because it's impossible to figure out all of the interactions.
My prediction? At some point you'll be forced to only use like characters with like characters. So no more running old republic with galactic republic, for example. I hope I'm wrong, but that's how I saw things play out in a game I used to play.
You don’t have to take all combinations of mods. You just need to artificially boost stats of testing toon when testing. Simple maths. Remember STR and AAT when it first released? EA even didn’t expect it will take several months to finish it!
I think they don’t use testers at all. Just computer testing. That’s rather bad move taking into account how long is the description of the abilities now.
So once again and again and again they are failed in testing.
And it will be worse and worse with the long list of ability description.
I don't think I have heard a bigger pile of baloney. If thre were 1000s of hours testing why did they have to nerf Sith against Malak after several days, surely that was a standard counter to the team.
Funny how that works isn't it? It take the community a day or two, some times hours to get the results but CG, the people who actually make the kits cannot figure it out in "thousands of hours"
Funny how that works isn't it? It take the community a day or two, some times hours to get the results but CG, the people who actually make the kits cannot figure it out in "thousands of hours"
You should read eric's post again.
OT: yea, thousand hours and not testing the extremely obvious team choice is dubious for sure, haha.
Funny how that works isn't it? It take the community a day or two, some times hours to get the results but CG, the people who actually make the kits cannot figure it out in "thousands of hours"
within the first hour of a character going live the community has eclipsed the total number of hours that we’ve been able to test - and many times over.
Reading the entire post is also important. Within an hour of release, several thousands of man-hours of Malak play are already completed by the community. What seems obvious to us after someone has figured it out, does not mean it was necessarily obvious to the play-testers. Hindsight is always 20/20, after all.
It's also possible that those thousands of man-hours testing were on different variations of the kit - and for early iterations of Malak's kit, Palpatine was non-viable.
Unless we know exactly what was tested, how many times, what the results were, and what Malak's kit looked at the time, we can't know if the testers are incompetent, or if the scenario set them up for failure.
Remember when Carrie tried to assuage hurt feelings about the underwhelming droid rework (glad they can be used against Malak but at that time they were underwhelming) by saying that Grievous is great in HSith and then people discovered within 5 seconds that Droideka could solo P3 in one hit...despite saying that Grievous was great in the raid...as though they tested Grievous in the raid...
Even before Anakin was reworked people were evaluating his kit mostly on the viability of him being on a Revan team. Galactic Republic was second or a complete after thought.
Also, how exactly does a public test server benefit the community...? Theory crafters should create new strategies to beat CG's newest character so that CG can buff up that newest character based on counters for the final release? It's not like CG is fixing bugs, their just nerfing counters that are presented
They probably do commit thousands of hours to testing. Not all testing is team comps or dinner but humans. The large majority of their testing is probably automated and checks for functional bugs. It probably spits out a bunch of metrics for them to review as well. It probably even ran JKR + JKA vs DR + Malak. Without a human controller on either side it probably looked normal due to AI inefficiencies. The effort to setup and execute automation is significant, and the result reviews can be as well. Many teams include this in the "hours of testing" or as CG said, "man hours of testing."
Where you can reasonably criticize them is in the manual review and play testing they should have been doing in parallel. They clearly missed something here. And that doesn't mean they're lying about the thousands of hours of testing.
Source: I test complex software and distributed systems for a living.
+1 for this.
The most important testing they do is to make sure the game still actually works.
e.g, having characters out of balance is an issue, but pretty insignificant compared to making sure the game doesn't crash when launched.
That isn't an excuse for them to miss some obvious play balance testing mechanics they should have hit, but thousands of hours of testing can easily be filled with regression testing of all the game functionality pieces.
They probably do commit thousands of hours to testing. Not all testing is team comps or dinner but humans. The large majority of their testing is probably automated and checks for functional bugs. It probably spits out a bunch of metrics for them to review as well. It probably even ran JKR + JKA vs DR + Malak. Without a human controller on either side it probably looked normal due to AI inefficiencies. The effort to setup and execute automation is significant, and the result reviews can be as well. Many teams include this in the "hours of testing" or as CG said, "man hours of testing."
Where you can reasonably criticize them is in the manual review and play testing they should have been doing in parallel. They clearly missed something here. And that doesn't mean they're lying about the thousands of hours of testing.
Source: I test complex software and distributed systems for a living.
+1 for this.
The most important testing they do is to make sure the game still actually works.
e.g, having characters out of balance is an issue, but pretty insignificant compared to making sure the game doesn't crash when launched.
That isn't an excuse for them to miss some obvious play balance testing mechanics they should have hit, but thousands of hours of testing can easily be filled with regression testing of all the game functionality pieces.
Agreed. But that’s not what we re talking about. At my old job I can easily say I work 40 hours a week.
But a good 38 of them I was on the forums or playing swgoh lol
Of course there are bigger issues in terms of testing but we re talking about all of these newer toons getting fixes and why. It’s annoying for everyone as I’m sure cg doesn’t want to have to do this.
Still worries that as we are now in the prequel and clone war error this will continue to happen.
@Kyno .... Elaborate to us again how the community is just so much more massive so any issues will obviously be caught by the community much easier than the devs.
They forgot to test literally the very first lineup that people automatically congregated towards?!?
Major key- when updating or releasing a new toon. Take that toon and put them in the factions best lead - even more so if that leader is a journey or legendary toon- even more so if it was a dominant meta.
Test against old meta and emerging one.
Test against p3 sith.
I got y’all
What took you 10 minutes to post has eluded CG for 3 years....
In game name: Lucas Gregory FORMER PLAYER - - - -"Whale blah grump poooop." - Ouchie
In game guild: TNR Uprising I beat the REAL T7 Yoda (not the nerfed one) and did so before mods were there to help
*This space left intentionally blank*
This is the first time Ive read anything about CG's testing procedures. Erik just said that theyre putting thousands of hours of testing into every new character interaction. Which is great!
The only part im not getting is how its done. Is it done in an alphabetic order?
My recommendation to CG would be to do the testing in the order of character force. For example, if they release a jedi (anakin) , then start the testing by running him under the most powerful jedi leader (jkr). If its an attacker character, then also put some buffers in the team, like hoda or bastilla.
Had CG managed to done that during those thousands of hours, it would have been quickly noticed that Anakin's aoe can easily kinstakill anything below 100k, like 4 members of an average squad. Maybe thats not right.
Lol. I thought something similar.
How can you not test them fully against darth revan/malak and JKR meta teams.
You would think the newest meta the last meta and maybe 2 or 3 former metas would be the norm
Also, 'thousands of hours of testing'.. Thousands. So, like, at least 2000 hours.
Thats weird because new characters are released/character kits are changed like every 2-4 weeks.
So, essentially, Erik claims they're testing 2000+ hours every 2 weeks.
...
A full time employee works 80 hours every 2 weeks. (in reality, even less, due to holidays etc)
So, they have at least 2000/80 = 25 full time testers, constantly working on character tests?
They tested... G12+ against G12+... and I am sure.... if there are enough G12+ Malaks, no JKR will use Anakin anymore.
And why should they test a reworked toon against a 5* not fully geared toon (no matter if a new toon or an old one)
Major key- when updating or releasing a new toon. Take that toon and put them in the factions best lead - even more so if that leader is a journey or legendary toon- even more so if it was a dominant meta.
Test against old meta and emerging one.
Test against p3 sith.
I got y’all
What took you 10 minutes to post has eluded CG for 3 years....
In a perfect world they would pair him with every character, every lead and run a mock sim battle, then they can look at the damages and stuff. Then test it in every phase of raids...etc. Yes it would be a lot but I feel there are ways to do it that aren't too difficult.
I work on upgrades to our system at my work. You have to test everything.
I’m sure the “thousands of hours” was in reference to the total amount of time they’ve spent testing since the game was released 3.5 years ago. The 12 minutes that they spent testing JKA’s rework factors into that number
It’s simple: their testers are probably a couple of degenerates, having already a performance improvement plans assigned to them. Can’t get rid of them though because they know how to do the work, there’s a hope they will get better, and it will be harder to train somebody new than just sticking with them and hoping they get better. It comes down to people, just like every other business in the world.
This explains a lot (screen shot from ea.com/studios). Ahnald or Cubs Fan showed it in one of their videos.
Sharing a workstation is a new age thing for eco friendly woke conscious companies. My company pays for everyone to have their own stuff but I did read about this new trend in college. it doesn't seem to be working well for them seems like every release since Rancor Han Solo has been bugged in some way.
Thousands of hours of testing and they couldn't even realise that palpatine would have totally stomped a team that revolves around self-inflicted debuffs and that has no tenacity boost whatsoever if you get to start before enemy bastila inflicts corrupted BM.
Thousands of hours of testing, and they couldn't realise that Anakin, with that ability setup, would have been almost an hard counter for DR/DM.
Thousands of hours of testing, and they couldn't realise that Finn and c3 would have done crazy damage in the raids.
Thousands of hours of testing, and they couldn't realise that some time ago paper zombie was more effective than geared zombie.
The list goes on.
Tl;dr: I don't believe a single word of that post.
It's one of those things. It looks so obvious in hindsight, but easily missed at the time. I've worked in software development and seen far worse things than this.
With all the interactions, it speaks to me of either simplifying the game or beta testing. As a player, I'd infinitely prefer the latter, but it will lose cg the ability to have surprise events as stuff is going to leak out of beta testing almost instantly.
Also, I wince every time I see these 'fire everyone, heads on sticks' type of comments. Try a bit of empathy, that sort of stuff is really not on!
This pretty much. Making computer games can be a complicated process. People should cut CG some slack, at least when they deserve it, like now. There was an oversight, so what? They're fixing it. All is well.
Replies
Should they have caught something with the more popular characters? Sure. But this is only going to get worse as they add more stuff, because it's impossible to figure out all of the interactions.
My prediction? At some point you'll be forced to only use like characters with like characters. So no more running old republic with galactic republic, for example. I hope I'm wrong, but that's how I saw things play out in a game I used to play.
Doubt "thousand of hour" were ever spent across the games whole lifespan.
Would be valid if the problems they keep occuring didn't have anything to do with the 5 most popular squads .
With all the interactions, it speaks to me of either simplifying the game or beta testing. As a player, I'd infinitely prefer the latter, but it will lose cg the ability to have surprise events as stuff is going to leak out of beta testing almost instantly.
Also, I wince every time I see these 'fire everyone, heads on sticks' type of comments. Try a bit of empathy, that sort of stuff is really not on!
You don’t have to take all combinations of mods. You just need to artificially boost stats of testing toon when testing. Simple maths. Remember STR and AAT when it first released? EA even didn’t expect it will take several months to finish it!
I think they don’t use testers at all. Just computer testing. That’s rather bad move taking into account how long is the description of the abilities now.
So once again and again and again they are failed in testing.
And it will be worse and worse with the long list of ability description.
You should read eric's post again.
OT: yea, thousand hours and not testing the extremely obvious team choice is dubious for sure, haha.
Reading the entire post is also important. Within an hour of release, several thousands of man-hours of Malak play are already completed by the community. What seems obvious to us after someone has figured it out, does not mean it was necessarily obvious to the play-testers. Hindsight is always 20/20, after all.
It's also possible that those thousands of man-hours testing were on different variations of the kit - and for early iterations of Malak's kit, Palpatine was non-viable.
Unless we know exactly what was tested, how many times, what the results were, and what Malak's kit looked at the time, we can't know if the testers are incompetent, or if the scenario set them up for failure.
These seem to me as no brainers.
Even before Anakin was reworked people were evaluating his kit mostly on the viability of him being on a Revan team. Galactic Republic was second or a complete after thought.
Also, how exactly does a public test server benefit the community...? Theory crafters should create new strategies to beat CG's newest character so that CG can buff up that newest character based on counters for the final release? It's not like CG is fixing bugs, their just nerfing counters that are presented
+1 for this.
The most important testing they do is to make sure the game still actually works.
e.g, having characters out of balance is an issue, but pretty insignificant compared to making sure the game doesn't crash when launched.
That isn't an excuse for them to miss some obvious play balance testing mechanics they should have hit, but thousands of hours of testing can easily be filled with regression testing of all the game functionality pieces.
Agreed. But that’s not what we re talking about. At my old job I can easily say I work 40 hours a week.
But a good 38 of them I was on the forums or playing swgoh lol
Of course there are bigger issues in terms of testing but we re talking about all of these newer toons getting fixes and why. It’s annoying for everyone as I’m sure cg doesn’t want to have to do this.
Still worries that as we are now in the prequel and clone war error this will continue to happen.
They forgot to test literally the very first lineup that people automatically congregated towards?!?
What took you 10 minutes to post has eluded CG for 3 years....
In game guild: TNR Uprising
I beat the REAL T7 Yoda (not the nerfed one) and did so before mods were there to help
*This space left intentionally blank*
Lol. I thought something similar.
How can you not test them fully against darth revan/malak and JKR meta teams.
You would think the newest meta the last meta and maybe 2 or 3 former metas would be the norm
Thats weird because new characters are released/character kits are changed like every 2-4 weeks.
So, essentially, Erik claims they're testing 2000+ hours every 2 weeks.
...
A full time employee works 80 hours every 2 weeks. (in reality, even less, due to holidays etc)
So, they have at least 2000/80 = 25 full time testers, constantly working on character tests?
That doesnt look too reasonable to me..
And why should they test a reworked toon against a 5* not fully geared toon (no matter if a new toon or an old one)
Surely I will be nerfed soon
I work on upgrades to our system at my work. You have to test everything.
Quality rant post. Here, take my like.
Sharing a workstation is a new age thing for eco friendly woke conscious companies. My company pays for everyone to have their own stuff but I did read about this new trend in college. it doesn't seem to be working well for them seems like every release since Rancor Han Solo has been bugged in some way.
Thousands of hours of testing, and they couldn't realise that Anakin, with that ability setup, would have been almost an hard counter for DR/DM.
Thousands of hours of testing, and they couldn't realise that Finn and c3 would have done crazy damage in the raids.
Thousands of hours of testing, and they couldn't realise that some time ago paper zombie was more effective than geared zombie.
The list goes on.
Tl;dr: I don't believe a single word of that post.
This pretty much. Making computer games can be a complicated process. People should cut CG some slack, at least when they deserve it, like now. There was an oversight, so what? They're fixing it. All is well.