Announcement

Collapse

Forum Rules

  • No flaming or derogatory remarks, directly or through insinuation.
  • No discussion, sharing or referencing illegal software such as hacks, keygen, cracks and pirated software.
  • No offensive contents, including but not limited to, racism, gore or pornography.
  • No excessive spam/meme, i.e. copious one liners in a short period of time, typing with all caps or posting meme responses (text/image).
  • No trolling, including but not limited to, flame incitation, user provocation or false information distribution.
  • No link spamming or signature advertisements for content not specific to Dota 2.
  • No Dota 2 key requests, sell, trade etc.
  • You may not create multiple accounts for any purpose, including ban evasion, unless expressly permitted by a moderator.

  • Please search before posting. One thread per issue. Do not create another thread if there is an existing one already.
  • Before posting anything, make sure you check out all sticky threads (e.g., this). Do not create new threads about closed ones.
  • It is extremely important that you post in correct forum section.

  • Balance discussion only in Misc.
  • All art related (such as hero model) feedbacks go to Art Feedback Forum.
  • All matchmaking feedback should go here: Matchmaking Feedback
  • All report/low priority issues should go here: Commend/Report/Ban Feedback
  • No specific workshop item feedback. These should go to workshop page of that item.
  • When posting in non-bugs section (such as this), use [Bugs], [Discussion] or [Suggestion] prefix in your thread name.



In case you object some action by a moderator, please contact him directly through PM and explain your concerns politely. If you are still unable to resolve the issue, contact an administrator. Do not drag these issues in public.



All rules are meant to augment common sense, please use them when not conflicted with aforementioned policies.
See more
See less

dota2_nn: Extremely WIP deep supervised learning full-takeover bots

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Oh woops sorry, I'm on Linux so I already had zlib... didn't think that it wouldn't be there on Windows :P

    Definitely motivation to figure out luastatic and get it all bundled up though.
    dota2_nn - deep supervised learning bots

    Comment


    • #17
      Originally posted by axetion View Post
      I intend to make the server just a prebuilt statically-linked binary (so no Torch or dependencies needed) once I figure out how this works: https://github.com/ers35/luastatic. (I'd much rather not have to use the server at all and just use Torch in Dota 2 directly though...)


      They actually have a docker image so I could do that, however you need multiple steam accounts to be able to process the amount of replays they/Dotabuff get.



      I think restricting the hero pool would be a good idea for starting out, especially considering right now heroes like Doom and Rubick that steal spells screw up the corpus builder anyways.... :PP




      Ooh, this looks nice. Will look into.



      I have more things to add on my checklist (status modifiers and number of nearby summoned units are the biggest ones), but the limiting factor is really retrieving them from the replays. Very undocumented spooky magic going on there.



      I think you might have just missed it: here's where coordinates are scaled for instance. I am using rectified linear units though so scaling matters a little less.



      Absolutely -- pull requests welcome too! Once I get a little more organized I'll make a GH project board that I can add tasks and issues too so that people can see progress on things.

      Yeah, SecondPass is nasty (understandbly though, it's the meat and potatoes of it). I'll split it up into SecondPass and ConstructFeatureVector when I get a chance.
      Like I say, hopefully someone will do a video of this working, since it'll be interesting to see this in action (at least for me).

      Comment


      • #18
        I checked out the code. It indeed is very cool.

        I'm gonna tell you what I told Lenlrx when he wanted to use machine learning for bots: I don't think it can be done (i.e. the results won't be good) at least not with the current algorithms. The reason I was saying this is because either you will have to use a huge feature vector to cover the things that the algorithm should consider, in which case the learning process becomes impractically long, or you have to use a small one in which case the end results won't be good. Eventually, he deviated to what I was thinking and I deviated to what he was thinking . Right now I think this can be done, but you have to be careful with what you want to do. Do I think you can write any machine learning algorithm that learns dota and plays it (which is almost what you are trying to do) ? No! Can machine learning be used to improve hard coded bots? Yes (at least I think so)! I think you have to use a hybrid system that relies on hard coded scripts for basic tasks (similar to think functions of modes, but smaller) and uses machine learning for your high-level behavior (similar to what desires are doing right now for default bots, but more advanced). This is why I was trying to push ChrisC to make basic tasks modular so people focus more on high-level things.

        I strongly suggest that you should switch to a hybrid system before getting bad results from what you are currently doing and getting coldhearted (unless you are just experimenting or/and do not have high expectations).

        BTW another thing I think ML can be used for is creating tools that can be used in hard coded scripts, like predicting enemy's movement patterns or how much damage/heal/disable a hero can deal/provide given its level in a teamfight again n heroes etc.

        Comment


        • #19
          Originally posted by Platinum_dota2 View Post
          ! I think you have to use a hybrid system that relies on hard coded scripts for basic tasks (similar to think functions of modes, but smaller) and uses machine learning for your high-level behavior (similar to what desires are doing right now for default bots, but more advanced). This is why I was trying to push ChrisC to make basic tasks modular so people focus more on high-level things.
          I had the same conclusion and that was the reason for me starting: https://github.com/Nostrademous/Dota2-WebAI

          The readme.md summarizes it.

          Comment


          • #20
            Originally posted by Platinum_dota2 View Post
            I checked out the code. It indeed is very cool.

            I'm gonna tell you what I told Lenlrx when he wanted to use machine learning for bots: I don't think it can be done (i.e. the results won't be good) at least not with the current algorithms. The reason I was saying this is because either you will have to use a huge feature vector to cover the things that the algorithm should consider, in which case the learning process becomes impractically long, or you have to use a small one in which case the end results won't be good. Eventually, he deviated to what I was thinking and I deviated to what he was thinking . Right now I think this can be done, but you have to be careful with what you want to do. Do I think you can write any machine learning algorithm that learns dota and plays it (which is almost what you are trying to do) ? No! Can machine learning be used to improve hard coded bots? Yes (at least I think so)! I think you have to use a hybrid system that relies on hard coded scripts for basic tasks (similar to think functions of modes, but smaller) and uses machine learning for your high-level behavior (similar to what desires are doing right now for default bots, but more advanced). This is why I was trying to push ChrisC to make basic tasks modular so people focus more on high-level things.

            I strongly suggest that you should switch to a hybrid system before getting bad results from what you are currently doing and getting coldhearted (unless you are just experimenting or/and do not have high expectations).

            BTW another thing I think ML can be used for is creating tools that can be used in hard coded scripts, like predicting enemy's movement patterns or how much damage/heal/disable a hero can deal/provide given its level in a teamfight again n heroes etc.
            yeah Im going to be surprised if it works. to be honest, a self-driving car is a piece of piss in comparison to playing dota. but the bigger the challenge, the more fun to try

            I think though even with a hybrid system you dont necessarily need scripting. You do machine-learning on all the different modes, to learn how to effectively lane, how to effectively teamfight etc.
            then you have a higher level machine learning applied to just the decision system, choosing which mode to engage in.

            I always thought the biggest hurdles initially were going to be translating replay variables into inputs/outputs that the bots can train on....and this project has already made a really solid start on that.

            So im very optimistic
            https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
            https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

            Comment


            • #21
              For replay downloading:
              You can get the replay salt from the Opendota API - https://docs.opendota.com/#tag/replays
              With this you should be able to get pretty much any public match in ~ the last 2 weeks

              Comment


              • #22
                I actually used Opendota to get some replays but the problem is that downloading replays is ratelimited by Valve.
                dota2_nn - deep supervised learning bots

                Comment


                • #23
                  ^ add to that there is a 500 limit per IP address, even if you have multiple steams.

                  This may be something I havent seen in the training lua. dont understand it properly yet.

                  but you're taking current items as inputs, you're assigning them an integer.

                  wont the neural net try and assign some relative meaning to their ids.

                  example:
                  iron branch id : 80
                  sheepstick id : 79
                  bottle id: 3

                  the net will think 'Sheepstick is much more similar to iron branch, than it is to bottle', because it has a closer id. Which is nonsense.

                  is this already handled?

                  if its not, I 'believe' the correct thing to do is to transform items into binary-inputs. so rather than 6 fields (one for each current item).
                  You would need ~100 fields. [HasBranch, HasGhostScepter, HasBlademail] ...etc. although as can have multiples maybe it needs to be [NumBranches, NumBlademails, NumNullTalismans....]


                  If I'm correct (unsure), this is an example of how the number of features/inputs is going to explode upwards as it gets 'cleverer'.

                  It's kind of funny that neural nets claim to mirror how humans think/learn. But I think dota is complex enough, its an interesting example of how
                  humans will essentially 'learn' whilst ignoring most of the features, then over time they essentially add more and more features/inputs to their mental model of the game to make decisions on, as they understand the game better.

                  Jamming literally every input into a neural-net before doing any training, doesnt match how people learn dota.
                  https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
                  https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

                  Comment


                  • #24
                    also just to note. im on ubuntu

                    even with it installed needed to give ZLIB_LIBDIR instructions to luarocks, otherwise it couldnt find it itself. not sure if both variables are necessary or just one.
                    Code:
                    luarocks install pegasus ZLIB_DIR=/usr ZLIB_LIBDIR=/usr/lib/x86_64-linux-gnu
                    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
                    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

                    Comment


                    • #25
                      Im going to put all my questions here rather than pm or something in-case others have same.

                      from trainer.lua

                      CreateContainer is defined with only 3 parameters, however called like this. Is the 3 just a leftover from hardcoding stuff?
                      Code:
                      local move = CreateContainer(input_len, output_len, math.floor((input_len + output_len) / 2), 3)
                      If Im understanding correctly, have 3 hidden layers, each one have (inputs + outputs)/ 2 number of nodes.
                      I always struggle to know how people choose net structure. Is there reason behind current setup, or just something to get project working and testing?

                      I think you just forgot lua 1-indexing here `move_label_weights[i + 1] = (label[0] or label[1]) / move_total` I believe you are trying to index label, rather than search for an integer key
                      Last edited by TheP1anoDentist; 06-03-2017, 06:18 AM.
                      https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
                      https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

                      Comment


                      • #26
                        So I find it hard to understand exactly what the trainer is doing. It is hard to keep track of what data is what, because it is a bit dependent on knowing which index is which.

                        what are your thoughts on having the golang write to file a more structured file. i.e json.

                        that would let you do results = [{"inputs": {"items": [1, 3, 4], "positions": []}, "outputs": []}]

                        then your trainer unpacks this into raw arrays for torch.
                        you obviously still have to deal with raw arrays at some point, but I think converting data to them as late as possible makes adding new inputs/outputs easier.

                        label weights is just the matrix of weights connecting all the nodes between input/hidden/output layers right?
                        but I dont know what class_weights are. dont think ive ever used them before. Ive never had multiple outputs, do they become important then. if so can you explain their function here?

                        A small mspaint style diagram of the net structure would probably be quite helpful

                        training_err is the error the network claims it has. but you've split your data into training data and test data, and the validation_err describes how much trained model differs from test data?

                        edit: are items inputs or outputs?
                        Last edited by TheP1anoDentist; 06-03-2017, 07:43 AM.
                        https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
                        https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

                        Comment


                        • #27
                          Originally posted by TheP1anoDentist View Post
                          if its not I 'believe' the correct thing to do is to transform items into binary-inputs. so rather than 6 fields (one for each current item).
                          You would need ~100 fields. [HasBranch, HasGhostScepter, HasBlademail] ...etc. although as can have multiples maybe it needs to be [NumBranches, NumBlademails, NumNullTalismans....]
                          .
                          Yes, this is how it works (one-to-many). The reason why the corpus builder outputs the IDs (which are essentially the indexes you set to 1) is because at the time it doesn't know how many items there are so it has no idea how many fields to put. Instead the trainer preprocesses this for it when loading the examples.

                          Originally posted by TheP1anoDentist View Post
                          I think you just forgot lua 1-indexing here `move_label_weights[i + 1] = (label[0] or label[1]) / move_total` I believe you are trying to index label, rather than search for an integer key
                          `label` is not an array, it's a map mapping classes to the number of times that class occurred. Depending on the class the "ignored/none" class may either be `0` or `1`.


                          The trainer used to have more comments and was a lot cleaner, but in the past couple days I've really changed how it works so now it's kind of nasty. I will clean it up and add more comments explaining all of this (especially in ParseMoveBatch).
                          Last edited by axetion; 06-03-2017, 08:14 AM.
                          dota2_nn - deep supervised learning bots

                          Comment


                          • #28
                            Originally posted by TheP1anoDentist View Post
                            training_err is the error the network claims it has. but you've split your data into training data and test data, and the validation_err describes how much trained model differs from test data?
                            In essence. The training loss is used to drive the actual changes the network makes, while the validation error is used to make sure that the network isn't overfitting the training data by having a set of data the network never looks at when optimizing.

                            Originally posted by TheP1anoDentist View Post
                            edit: are items inputs or outputs?
                            Inputs are items, outputs are *active* items (items that the corpus builder has observed being used)
                            dota2_nn - deep supervised learning bots

                            Comment


                            • #29
                              Originally posted by TheP1anoDentist View Post
                              CreateContainer is defined with only 3 parameters, however called like this. Is the 3 just a leftover from hardcoding stuff?
                              Code:
                              local move = CreateContainer(input_len, output_len, math.floor((input_len + output_len) / 2), 3)
                              Good catch. Thank you.
                              dota2_nn - deep supervised learning bots

                              Comment


                              • #30
                                Originally posted by TheP1anoDentist View Post

                                It's kind of funny that neural nets claim to mirror how humans think/learn. But I think dota is complex enough, its an interesting example of how
                                humans will essentially 'learn' whilst ignoring most of the features, then over time they essentially add more and more features/inputs to their mental model of the game to make decisions on, as they understand the game better.
                                This is something that neural networks can do too, with feature selection There aren't a lot of good fast algorithms for it though....
                                dota2_nn - deep supervised learning bots

                                Comment

                                Working...
                                X