Thousands of testing hours - forgot to test under jkr?

Prev1
This is the first time Ive read anything about CG's testing procedures. Erik just said that theyre putting thousands of hours of testing into every new character interaction. Which is great!
The only part im not getting is how its done. Is it done in an alphabetic order?
My recommendation to CG would be to do the testing in the order of character force. For example, if they release a jedi (anakin) , then start the testing by running him under the most powerful jedi leader (jkr). If its an attacker character, then also put some buffers in the team, like hoda or bastilla.
Had CG managed to done that during those thousands of hours, it would have been quickly noticed that Anakin's aoe can easily kinstakill anything below 100k, like 4 members of an average squad. Maybe thats not right.

Replies

  • They were too busy running him as lead with Kit Fisto and Eeth Koth by his side against the Tuskens.
  • They were testing under General Kenobi lead so the jedi killer Grievous can kill him.
    >:)So what? I want Krell!
  • This is the staggering part of this whole thing, although it’s not like it’s new. I continue to be amazed at what doesn’t get caught in testing. Certain bugs are to be expected, and they absolutely won’t catch everything. Same with team comps, I get that. But shouldn’t the expectation be to get the major points right. Or catch the obvious bugs. For example, who didn’t think to test B1 with the new droid “bonuses” in TW? Isn’t that the very first character you would look at as a tester and think hey, I wonder if I can screw something up there?
  • This is the staggering part of this whole thing, although it’s not like it’s new. I continue to be amazed at what doesn’t get caught in testing. Certain bugs are to be expected, and they absolutely won’t catch everything. Same with team comps, I get that. But shouldn’t the expectation be to get the major points right. Or catch the obvious bugs. For example, who didn’t think to test B1 with the new droid “bonuses” in TW? Isn’t that the very first character you would look at as a tester and think hey, I wonder if I can screw something up there?

    Also great example.

    We may be able to put together a manual for CG on the basics of testing.
    If we could have those resources actually being productive, maybe this game would quickly perform much better.
  • Clearly they only run computations on expected revenue gains.
  • Jarvind
    3920 posts Member
    Erik just said that theyre putting thousands of hours of testing into every new character interaction.

    Yeah, I just straight up don't believe that. Thousands of hours against what? Clearly not against "the most dominant team in the game" which probably ought to be, y'know, top of the list. Likewise, previous lolcounters of this magnitude have been found almost immediately by players, i.e. Nest holding off JKR teams until they tweaked JKR's kit specifically to prevent it.

    They're just saying whatever they think will generate the least backlash.
    u58t4vkrvnrz.png



  • Yeah, this puzzles me as well.

    There are issues with Revan/Malak that are unanticipated, like running Nightsister Zombie in the leader spot to abuse Revan's Deathmark mechanic, or the bad Malak AI that keeps stealing 1 health from B1 rather than someone with actual health to steal. I understand missing stuff like this in the development phase, as they are outlier cases.

    But for the life of me I can't comprehend a play tester not immediately plugging JKA into a JKR team to see how his kit interacts, nor can I understand missing the obviousness of the Palpatine/Vader leadership as a counter. I mean, I thought of the latter when Darth Revan's kit was revealed and he wasn't even in the game yet!

    I don't have a lot of sympathy, as these problems are self-inflicted. The kits have become so complicated that it's difficult to design something that slides in somewhere between OP and useless. The JKR/Jolee interaction is forever going to dominate the course of this game unless it's remedied - the obvious solution is to limit Jolee's revive mechanic a bit, but the community uproar for messing with the kit is going to be ugly.

    Someone earlier in this thread commented that they should slow down character releases and test them more; I'm not sure this is possible. The release cadence is already at its slowest and CG revenue is down significantly from last year; they desperately need to release something for people to buy. There's currently nothing for mid-spenders who couldn't or didn't want Malak. If you don't care about finishing top of arena, there's currently no game content that can't be finished using easily accessible F2P characters.
  • Jarvind wrote: »
    Erik just said that theyre putting thousands of hours of testing into every new character interaction.

    Yeah, I just straight up don't believe that. Thousands of hours against what? Clearly not against "the most dominant team in the game" which probably ought to be, y'know, top of the list. Likewise, previous lolcounters of this magnitude have been found almost immediately by players, i.e. Nest holding off JKR teams until they tweaked JKR's kit specifically to prevent it.

    They're just saying whatever they think will generate the least backlash.

    They probably do commit thousands of hours to testing. Not all testing is team comps or dinner but humans. The large majority of their testing is probably automated and checks for functional bugs. It probably spits out a bunch of metrics for them to review as well. It probably even ran JKR + JKA vs DR + Malak. Without a human controller on either side it probably looked normal due to AI inefficiencies. The effort to setup and execute automation is significant, and the result reviews can be as well. Many teams include this in the "hours of testing" or as CG said, "man hours of testing."

    Where you can reasonably criticize them is in the manual review and play testing they should have been doing in parallel. They clearly missed something here. And that doesn't mean they're lying about the thousands of hours of testing.

    Source: I test complex software and distributed systems for a living.
  • Kailen
    90 posts Member
    Yea. This wasn't some obscure combo that some theorycrafter managed to pull out of their bums.

    This was simply slotting the new char into the most popular and dominant team on the market.

    If they can't test THAT, they aren't testing.
  • Fahrius
    620 posts Member
    They were too busy running him as lead with Kit Fisto and Eeth Koth by his side against the Tuskens.

    Same as the nest nerf, everybody beated her with jkr... Their testing team couldnt...
    https://swgoh.gg/u/fahrius/ - Discord - Fahrius#1194
  • Roahn
    231 posts Member
    I'm not really sure what is worse. The fact that CG's testing team maybe doesn't know how to test new characters or that CG expects each person in the community to shell out $1,000 plus. And what's maybe worse than that is the game has made millions since its release and all we get in return is a few new characters each year. For the amount of cash CG expects each person to pay a year amounts to 6 new games and a new gaming console a year.
  • The last few weeks have been a whirlwind for everyone associated with this game. Where to begin with this mess. I guess we can start with the thousands of hours spent testing, but didn't bother to try plugging JKA into any JKR team. Seriously, just fire the testing team and get a team that can do the job right. You make millions of dollars a month, you can at least have an adequate team to test new content and how it will interact with current content. I know we will just save that money and let the community test for us....

    JKA isn't broken, the obvious problem is the leadership from JKR and DR. You have one leader that boost your teams offense, paired with other stat boosting party members then you pair off against a team that lowers their own defense, it doesn't take a genius to know that JKA will hit like a truck.

    Even with all of these things in JKA favor, not everyone can plug and play him into a JKR team and win even half the time. If they dont have the right mods, they are not winning. Most of the people doing this have seriously invested in mods for that whole team. Even then it doesn't hold on defense.

    It seems that the real issue is that JKR set the bar too high. Doesn't seem like DR's kit and team synergy are working very well. Something is obviously wrong when you have had to already make multiple changes to keep DR on top. It almost seems like Malak was rushed in order to keep ftp players from getting him, since DR didn't net as much money or again not enough testing was done to see how he and DR would mesh.

    I don't think changing JKA is the answer. If Malak and the sith empire are suppose to be great, then do the work and testing to make them work as you wanted. Of course your player base are going to be upset when situations like this arise. Keep in mind free to play players, may walk away and you are not loosing any money, however they do own PC's and consoles and can stop buying EA titles.
  • Fitz
    302 posts Member
    It's just PR ****. It's obvious they don't test anything. Focus is on monetizing the game. Then coding the latest money making scheme. Testing does not bring in cash.
  • Amateurs :/
  • I think he means the testing made available by the player base as he says per release
  • five2zero
    512 posts Member
    edited April 2019
    They dont test! No matter what they are saying. It cannot be true to forget the newest characters.
  • zegene
    249 posts Member
    They were too busy running him as lead with Kit Fisto and Eeth Koth by his side against the Tuskens.

    This!
    Check my collection. Contact me at Discord: zegene#1773
  • Kanzi
    45 posts Member
    Our intention is that Jedi Knight Anakin should not outperform Darth Malak.
    1. Because? Only p2w is allowed? f2p peasants should suffer?
    2. What means outperform? JKA is not a sith tank, he's not outperforming Malak in this role. JKA shouldn't be able to kill Malak, or damage him, or do powerful AOE, or what?
    This is just ridiculous
  • leef
    13458 posts Member
    The whole first part of that post by erik is hilarious if you put it in context.
    At first glance it looks like he makes a great point about the playerbase getting more "testing" done in the first hour after release than they could ever do with their team, and then (like you pointed out) you realize the most obvious team composition is the reason why they're re-evaluating the rework, haha.
    Save water, drink champagne!
  • Nauros
    5429 posts Member
    leef wrote: »
    The whole first part of that post by erik is hilarious if you put it in context.
    At first glance it looks like he makes a great point about the playerbase getting more "testing" done in the first hour after release than they could ever do with their team, and then (like you pointed out) you realize the most obvious team composition is the reason why they're re-evaluating the rework, haha.

    And the exact same thing happened with Palp - it was speculated as soon as the DR kit was known. Hopefully they really let the community do some of their testing, what they do themselves is obviously lacking.
  • Maybe the only expecred jka to go in GR teams? And only tested him in those type of teams so gk kead ani lead ect they might only test characters in teams that they expect the use from them and probably had all 3 in every test
  • My guess is that JKA and probably some of the other era specific toons needed the boost for the upcoming geo TB map. With the focus on how these era specific toons work on the new map they likely just kinda forgot about how it might affect other aspects of the game. People can get hyper focused on their projects when trying to meet deadlines. And they likely have a deadline we're just unaware of it.

    I usually busy CG balls but I think it's important to take everything into account
  • Nauros
    5429 posts Member
    Dagonsith wrote: »
    My guess is that JKA and probably some of the other era specific toons needed the boost for the upcoming geo TB map. With the focus on how these era specific toons work on the new map they likely just kinda forgot about how it might affect other aspects of the game. People can get hyper focused on their projects when trying to meet deadlines. And they likely have a deadline we're just unaware of it.

    I usually busy CG balls but I think it's important to take everything into account

    This excuse would hold if there was some unexpected interaction with Aayla and Luminara under Eeth lead, but those are the most prevalent meta teams we are talking about...
  • JKA normally doesn't even aoe for 100k. It's only against DR teams due to the low defense and when he has masters training, buffs, etc. Against other toons its like 30-40k. The point of DR was to sacrifice defense for offense and then they are surprised when it makes them easier to kill? ****?
  • Naw
    969 posts Member
    JKA normally doesn't even aoe for 100k. It's only against DR teams due to the low defense and when he has masters training, buffs, etc. Against other toons its like 30-40k. The point of DR was to sacrifice defense for offense and then they are surprised when it makes them easier to kill? ****?

    Exactly this.

    Can’t wait to see how useless he’ll become again.
    ”CG” wrote:
    Our intention is that Jedi Knight Anakin should not outperform Darth Malak.

    It’s your own fault. Can’t wait the next OP character and their extremely complex kit. I read this as an advice to not bother with any other chaeacters than those that may require money to be spent.

    Well, I’m not going to spend, so there we are...
  • Nauros wrote: »
    Dagonsith wrote: »
    My guess is that JKA and probably some of the other era specific toons needed the boost for the upcoming geo TB map. With the focus on how these era specific toons work on the new map they likely just kinda forgot about how it might affect other aspects of the game. People can get hyper focused on their projects when trying to meet deadlines. And they likely have a deadline we're just unaware of it.

    I usually busy CG balls but I think it's important to take everything into account

    This excuse would hold if there was some unexpected interaction with Aayla and Luminara under Eeth lead, but those are the most prevalent meta teams we are talking about...

    They haven't released any of the upcoming Jedi updates so it may well be that they've screwed the pooch on some interaction there but what I'm saying is that they likely overlooked the Revan interaction because they're tailoring the Jedi and other era specific toons for the new TB. Is that aggravating? Of course it is. But it's also understandable. CG does need to up it's game.in regards to play testing and I will be the first to say so but if it's because they're delivering new content I'll give them a.break on this one.
  • I’m sure the “thousands of hours” was in reference to the total amount of time they’ve spent testing since the game was released 3.5 years ago. The 12 minutes that they spent testing JKA’s rework factors into that number
  • Major key- when updating or releasing a new toon. Take that toon and put them in the factions best lead - even more so if that leader is a journey or legendary toon- even more so if it was a dominant meta.

    Test against old meta and emerging one.

    Test against p3 sith.

    I got y’all
  • Tell EA to open their coin purse and get you some live action test servers CG. You would save face and avoid the clusters we face now.
Sign In or Register to comment.