Announcement

Collapse

Forum Rules

  • No flaming or derogatory remarks, directly or through insinuation.
  • No discussion, sharing or referencing illegal software such as hacks, keygen, cracks and pirated software.
  • No offensive contents, including but not limited to, racism, gore or pornography.
  • No excessive spam/meme, i.e. copious one liners in a short period of time, typing with all caps or posting meme responses (text/image).
  • No trolling, including but not limited to, flame incitation, user provocation or false information distribution.
  • No link spamming or signature advertisements for content not specific to Dota 2.
  • No Dota 2 key requests, sell, trade etc.
  • You may not create multiple accounts for any purpose, including ban evasion, unless expressly permitted by a moderator.

  • Please search before posting. One thread per issue. Do not create another thread if there is an existing one already.
  • Before posting anything, make sure you check out all sticky threads (e.g., this). Do not create new threads about closed ones.
  • It is extremely important that you post in correct forum section.

  • Balance discussion only in Misc.
  • All art related (such as hero model) feedbacks go to Art Feedback Forum.
  • All matchmaking feedback should go here: Matchmaking Feedback
  • All report/low priority issues should go here: Commend/Report/Ban Feedback
  • No specific workshop item feedback. These should go to workshop page of that item.
  • When posting in non-bugs section (such as this), use [Bugs], [Discussion] or [Suggestion] prefix in your thread name.



In case you object some action by a moderator, please contact him directly through PM and explain your concerns politely. If you are still unable to resolve the issue, contact an administrator. Do not drag these issues in public.



All rules are meant to augment common sense, please use them when not conflicted with aforementioned policies.
See more
See less

Different Way For The MMR System To Work

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Originally posted by DarkLite View Post
    While that's a fair point, I think it's important to remember that we're talking about a small bonus to an already small change. For example, let's say you were given 3 * (your ally healing / team ally healing) * (game time / 2 * average game time) extra points in each game. Assume that people don't know this happens (avoiding the issue of gaming it).
    Your example is bad in its own right. For example, I'd say that any game Juggernaut is in, he does more healing than anyone else with almost no exceptions. Witch doctor / necrophos likewise gets a massive boost. Favouring heroes is obviously not a fair way to judge skill. Even if you limit it to only items there are other problems. It would then just arbitrarily reward whoever got the mek at that point.

    Also, you say "Let's make it a small number of points". What's the point then? The original goal of such a system is to get people to their true MMR faster, but such a small amount would hardly be fast. Either it's large enough to be a bad idea, or it's small enough to be so trivial it doesn't matter, and thus still a bad idea.

    However, your example brings up an even bigger problem: If you account for other factors in MMR, you now change what it means. Right now MMR (basically ELO) is purely a determination of your ability to win. Nothing else. If you add more factors, MMR becomes a determination of a combination of how much you win and how much you fit those factors. To continue with your example: Let's take two equally skilled players of MMR 1000. One of them typically plays Crystal Maiden while the other typically plays Witch Doctor. They both sometimes get mek (say half the time), but even in games where WD doesn't get mek, he tops the healing chart. His MMR moves towards 1003 because he's the top healer always while the CM is only the top player when she gets a mek and isn't matched with WD/Juggs, so she moves to about 1001. Is this fair? Is his "true" MMR really 1003, not 1000? Obviously not in the scenario I set up. They both still have a completely equal chance to win, yet one has a higher MMR. A world where MMR does not exactly map to "chance to win" is a bad world - the point of matchmaking is not to have "equal chances to either lose and heal a lot or win and heal less" on each team.

    I didn't verbalize this in my last post, but I think this is an even more fundamental flaw with accounting for other numbers in the MMR system. MMR is meant to measure probability to win. By adding other statistics to it, it now becomes "weight1 * Probability to win + weight2 * probability of having most healing + weight3 * probability of having most kills etc etc"...

    The reason people want such a silly change is simply for personal satisfaction; they feel as if they are better than their MMR shows and want to be "rewarded". If they're better then they should win and their MMR will go up. If they keep on getting most kills/healing/whatever, but not winning, then they should probably rethink what they're doing, not be rewarded, since it's obviously not working.

    Calculate the extra change people would see in their MMR due to this system, and at the end of the week / month see how it would change the MMR distribution .... [make it] close to the "correct" value
    Okay, here's another problem. What's the correct value? Their current MMR? I can't think of a more correct value to use.. it's not like valve magically knows how good a player is beyond the current MMR (which is already their best guess). If you scale this mythical system so that it's close to the current MMR in levels, you just suddenly give an artificial number boost to everyone (1000 -> 1003, 1500 -> 1503) but without really changing anything else. That's completely pointless. The only thing that would be different is there would be an exploitable system for people to slowly discover.
    Last edited by Eskimo; 12-24-2013, 03:14 PM.

    Comment


    • #32
      Originally posted by DarkLite View Post
      This would only be true if Valve publicly released a list of things that give you bonuses. The MMR you win / lose is already a mystery to us - we have no idea how the number is calculated. We know it's probably to do with the MMRs of the other players in the game, but that's it. It would be similarly opaque if it gave bonuses of some sort - it'd be very hard to tell if there's an extra + 3 or 5 points or what it might come from.
      Most of the points you raised were answered by Eskimo, and I wholeheartedly agree with him. About this specific part, it has been shown time and time again that people will do whatever it takes to eke out an "unfair" advantage. Best example is diretide; how on earth would people figure out hundreds of bugs?? Overgrowth cancels rosh's invulnerability.. who actually went to test out every single spell under every single circumstance?!?!?! Same thing goes for wraith night, even though the rewards aren't nowhere near the valve of PBRs... Even the original incarnation of the mute system got figured out by someone messing around with it...

      Comment


      • #33
        23-9-9 Full build, great start and lh mvp of the game. Lost the game and... LOST 24 ELO wwuuuuuuuuuuuut! so if i play 100% and the team throw the game i will lost ELO (Ranking)? no matterswhat?
        Pd: basic english hope your guys can understandme and match id: 439077176
        PDD: i got a afk and a quiter. this ranked game's dont have logic at all

        someone can explain?

        Comment


        • #34
          Originally posted by Eskimo View Post
          Your example is bad in its own right. For example, I'd say that any game Juggernaut is in, he does more healing than anyone else with almost no exceptions. Witch doctor / necrophos likewise gets a massive boost. Favouring heroes is obviously not a fair way to judge skill. Even if you limit it to only items there are other problems. It would then just arbitrarily reward whoever got the mek at that point.
          That's an interesting point, I hadn't considered those issues. I guess if a combination of factors was considered rather than one isolated factor it'd would lessen the impact, though it's fairly hard to judge how much of an issue something like that would be without testing it in some way.

          Also, you say "Let's make it a small number of points". What's the point then? The original goal of such a system is to get people to their true MMR faster, but such a small amount would hardly be fast. Either it's large enough to be a bad idea, or it's small enough to be so trivial it doesn't matter, and thus still a bad idea.
          Individually it's a small amount - but if you consistently play well it has an effect. If you win five games at 25+5, that's equivalent to winning six games at 25+0. Keeping it small makes it harder for rare events (like the horrible "oh god oh god what is happening why can't i stop feeding please help me" games I'm sure we've all experienced) to have much of an effect, while ensuring that consistent performance is rewarded.

          However, your example brings up an even bigger problem: If you account for other factors in MMR, you now change what it means. Right now MMR (basically ELO) is purely a determination of your ability to win. Nothing else. If you add more factors, MMR becomes a determination of a combination of how much you win and how much you fit those factors. To continue with your example: Let's take two equally skilled players of MMR 1000. One of them typically plays Crystal Maiden while the other typically plays Witch Doctor. They both sometimes get mek (say half the time), but even in games where WD doesn't get mek, he tops the healing chart. His MMR moves towards 1003 because he's the top healer always while the CM is only the top player when she gets a mek and isn't matched with WD/Juggs, so she moves to about 1001. Is this fair? Is his "true" MMR really 1003, not 1000? Obviously not in the scenario I set up. They both still have a completely equal chance to win, yet one has a higher MMR. A world where MMR does not exactly map to "chance to win" is a bad world - the point of matchmaking is not to have "equal chances to either lose and heal a lot or win and heal less" on each team.

          I didn't verbalize this in my last post, but I think this is an even more fundamental flaw with accounting for other numbers in the MMR system. MMR is meant to measure probability to win. By adding other statistics to it, it now becomes "weight1 * Probability to win + weight2 * probability of having most healing + weight3 * probability of having most kills etc etc"...

          The reason people want such a silly change is simply for personal satisfaction; they feel as if they are better than their MMR shows and want to be "rewarded". If they're better then they should win and their MMR will go up. If they keep on getting most kills/healing/whatever, but not winning, then they should probably rethink what they're doing, not be rewarded, since it's obviously not working.
          Hm, interesting point. I'd argue that that players who individually play well (which is ultimately what extra stats would track in some way) and win games as a team are more likely to win than players who win games but do poorly individually - that people who do well tend to win games (though not necessarily the other way around). Do you think that's not the case?

          Okay, here's another problem. What's the correct value? Their current MMR? I can't think of a more correct value to use.. it's not like valve magically knows how good a player is beyond the current MMR (which is already their best guess). If you scale this mythical system so that it's close to the current MMR in levels, you just suddenly give an artificial number boost to everyone (1000 -> 1003, 1500 -> 1503) but without really changing anything else. That's completely pointless. The only thing that would be different is there would be an exploitable system for people to slowly discover.
          If you felt it was important to weight things, the "correct" value would be one that skewed the shape of the distribution as little as possible while maximising the change in individual ratings relative to each other, and without negatively affecting the quality of matches. If it moves people up or down the rankings while improving game quality and keeps the shape the same, then it's doing its job well.

          Originally posted by Heh_ View Post
          Most of the points you raised were answered by Eskimo, and I wholeheartedly agree with him. About this specific part, it has been shown time and time again that people will do whatever it takes to eke out an "unfair" advantage. Best example is diretide; how on earth would people figure out hundreds of bugs?? Overgrowth cancels rosh's invulnerability.. who actually went to test out every single spell under every single circumstance?!?!?! Same thing goes for wraith night, even though the rewards aren't nowhere near the valve of PBRs... Even the original incarnation of the mute system got figured out by someone messing around with it...
          I think it's very misleading and unhelpful to compare MMR to how in-game spells work. There's a huge difference between the two situations: you have an expected outcome with spells. It's obvious when something is different or wrong. The change in your MMR after a game has no expected outcome aside from "up" or "down". And the mute system was relatively simple and easy to test (partially on account of the fact that it was so laughably broken). It was also back before smurfs had to complete tutorials. In order to figure out how this system works, you'd first have to find how the MMR change is affected by who you played against / with using a set of ten accounts of known MMR (all of which need > 150 games and their calibration matches completed), checking at the same time if game length etc. affect it. You'd then need to start experimenting with different strategies, accounting for the effect of different rankings, and compare the results over and over again. Finally, you can make a guess at what they're tracking and how important everything is. I'd be amazed if anyone even bothered to do the first half of that Herculean task, particularly when nobody's bothered to try and reverse-engineer the matchmaking algorithms for any similar games. Never mind the second half where you've got a vast number of uncontrolled variables to deal with. I genuinely don't think there'd be a risk of someone figuring out what's inside the black box if Valve put this in.
          Please help keep the forums clean by reporting rude or abusive posts!
          Got a problem? PM me or another moderator!
          Forum Rules

          Comment


          • #35
            Originally posted by DarkLite View Post
            Individually it's a small amount - but if you consistently play well it has an effect. If you win five games at 25+5, that's equivalent to winning six games at 25+0. Keeping it small makes it harder for rare events (like the horrible "oh god oh god what is happening why can't i stop feeding please help me" games I'm sure we've all experienced) to have much of an effect, while ensuring that consistent performance is rewarded
            Consistent performance is rewarded. If you consistently play well it has the effect of you winning, and thus going up in MMR. If your "oh god I'm feeding" games are truly rare, you won't go down that much and your "consistent" good play will ensure you stay where you belong.

            Hm, interesting point. I'd argue that that players who individually play well (which is ultimately what extra stats would track in some way) and win games as a team are more likely to win than players who win games but do poorly individually - that people who do well tend to win games (though not necessarily the other way around). Do you think that's not the case?
            Of course that's the case. That emphasizes even more why having an additional set of weights is worthless - if you're good you win games. I think that "individually play well" is a misleading statement. There is no such thing in dota. It's a team game. Someone is only playing well if they improve their team's chance of winning. "Individual skill" should only matter insomuch as it increases that person's team's chance of winning. It doesn't matter if, say, I can flawlessly land hooks as pudge if I only ever do it when my team isn't there to help so I don't actually secure kills or other advantages.

            If you felt it was important to weight things, the "correct" value would be one that skewed the shape of the distribution as little as possible while maximising the change in individual ratings relative to each other, and without negatively affecting the quality of matches. If it moves people up or down the rankings while improving game quality and keeps the shape the same, then it's doing its job well.
            I STRONGLY disagree with this. So you want it to keep the same shape of distribution overall. That's fine. But you want to maximize individual differences? I can't think of a logical reason for that... that worsens everything. Weighting differences that cause before evenly-matched players to be as far apart as possible is nonsensical. In fact, you'd maximize that by picking a metric that is consistent per player, but completely random overall. For example whether a person is using an AMD or Intel processor would result in the same overall shape (it averages out) but much higher relative differences. Anything that both measures skill (as the current MMR does) and keeps the same overall shape should not result in high relative differences.

            As for game quality, good luck measuring that.


            I genuinely don't think there'd be a risk of someone figuring out what's inside the black box if Valve put this in.
            There shouldn't be any risk at all because there shouldn't be any harm in people knowing. Have you ever heard of "Security by obscurity"? It's commonly held to be bad, and the reason is that if your security is good, then it should be unbreakable even if everyone knows how it works. This is the basis of modern cryptography.

            Likewise, MMR should not be a black box. Everyone should be able to know how it works without it being broken or exploitable. In fact, right now we know it's a modified ELO (valve said so in a recent blog post) that tries to measure roughly your probability to win a match. Knowing this, I can't exploit it beyond trying to win. That's a lot better than saying "Yeah, let's make it complicated and hope noone can figure it out".
            Last edited by Eskimo; 12-24-2013, 04:09 PM.

            Comment

            Working...
            X