PDA

View Full Version : The Problem with the state of balance Data

Seeroftruths
09-28-2018, 02:50 PM
TL;DR: Win-rate isn't enough

So after reading the State of Balance post I have to say, That's not enough data.
If this is the only data set you are collecting, then you have to widen your scope to include the actual meat and potatoes of competitive game Data collection

So you now have the What is going on with the win-rate but you need the why.

The why includes:
Average games played of the characters.
Win-rate with Games played
Pick % (Which you got)
Average Round length
Win-rate based on Round length

If someone has a high Win-rate and you see they have a high Average round length you can see they might be stalling out games and winning through running away (LB Kite strat) This would be something you look for if those to Data points are high.

If somebody has a Low Win-rate but a high Pick-rate (Raider) what does the win-rate look like compared to time played. Maybe they are hard to learn and people lose often with them when they first play them, or maybe they just suck.

If somebody has a High Win-rate but a low pick-rate (Shaman) that doesn't mean they are good, it could just mean the people who play her are good at her, or it could mean they are sleeper OP. Until we see more data, we don't know..

Examples from another game
Looking at the Win-rate in League of legends
https://champion.gg/statistics/#?sortBy=general.winPercent&order=descend

Sorry I couldn't figure out pictures

Kayle top is number 19 out of 192 with a 52.44% win-rate but does that mean she is good right now?
She has a 0.22% play-rate compared to the people a rounder her with a 2% or higher play-rate that is really low.
Alas, we can take a closer look.
https://champion.gg/champion/Kayle/Top
Here we can see
that her win-rate goes up with longer games (She likes to stall out)
and that her Win-rate sky rockets why you have played 50-100 games of her raising around 4%
and that 1.41% of people played 50-100 games of her

It looks like the reason she has a high win-rate is because the people who play her are good at her

Nami Support is 9 out of 192 with a 43.05% win-rate
she has a 10.30% play-rate that is absurdly high
She has a stable Win-rate compared to games played with about 1% deference
But she has a 2.14% of people with 50-100 games of her
https://champion.gg/champion/Nami/Support
It seems she is just good no matter how much you play her and is actually strong right now, though more research would have to be done.

I did cherry pick these examples to make a point.

Other needed information
Win-rate on console vs PC (Because there is a difference)
and what the Lower skill levels look like (Because they are the majority of your game)
You can't ignore the lower skill levels and just hope they get good, cause that's not how that works.
If they are up against toxic things that are ruin their experience that high levels can deal with, they just might leave (Light spam)
Also console has problems PC doesn't (Light spam)
I play on PC and don't have a problem with light spam, but that doesn't mean I think it should be ignored.
Over all i think it's good, but maybe it needs a little bit of a look at anyways just to see if you can do something to make it not so oppressive.
I play a lot of fighting games so the idea you cant react to everything makes sense to me.

In Conclusion
More data is needed to even start understanding why things are the way they are, you can't make informed decisions why you don't have the information
also don't just outright lower the Sample size like some people have been suggesting, get all the information before you start to cut some out.

Hope that helps,
Seer of Truths

DefiledDragon
09-28-2018, 06:09 PM
They seem to focus on overall winrate whilst ignoring the huge discrepancies between matchups. That's a pretty glaring oversight. If their philosophy is 50% overall winrate all round is balanced then they really do have no ****ing clue.

Seeroftruths
09-28-2018, 06:50 PM
Honestly, I wan't to give them the benefit of the doubt and say they collect and analyze more data then just those 2 data points, but I have no evidence to back that up.

I hope they do look at more then just Win-rates
and I hope they don't have a "50% overall win-rate is balanced" Philosophy