Announcement

Collapse

Forum Rules

  • No flaming or derogatory remarks, directly or through insinuation.
  • No discussion, sharing or referencing illegal software such as hacks, keygen, cracks and pirated software.
  • No offensive contents, including but not limited to, racism, gore or pornography.
  • No excessive spam/meme, i.e. copious one liners in a short period of time, typing with all caps or posting meme responses (text/image).
  • No trolling, including but not limited to, flame incitation, user provocation or false information distribution.
  • No link spamming or signature advertisements for content not specific to Dota 2.
  • No Dota 2 key requests, sell, trade etc.
  • You may not create multiple accounts for any purpose, including ban evasion, unless expressly permitted by a moderator.

  • Please search before posting. One thread per issue. Do not create another thread if there is an existing one already.
  • Before posting anything, make sure you check out all sticky threads (e.g., this). Do not create new threads about closed ones.
  • It is extremely important that you post in correct forum section.

  • Balance discussion only in Misc.
  • All art related (such as hero model) feedbacks go to Art Feedback Forum.
  • All matchmaking feedback should go here: Matchmaking Feedback
  • All report/low priority issues should go here: Commend/Report/Ban Feedback
  • No specific workshop item feedback. These should go to workshop page of that item.
  • When posting in non-bugs section (such as this), use [Bugs], [Discussion] or [Suggestion] prefix in your thread name.



In case you object some action by a moderator, please contact him directly through PM and explain your concerns politely. If you are still unable to resolve the issue, contact an administrator. Do not drag these issues in public.



All rules are meant to augment common sense, please use them when not conflicted with aforementioned policies.
See more
See less

dota2_nn: Extremely WIP deep supervised learning full-takeover bots

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • dota2_nn: Extremely WIP deep supervised learning full-takeover bots

    https://github.com/axetion/dota2_nn

    This is the start of a crude deep learning bot using Torch I've been working on. Unlike most of the previous ML bots here, this is purely supervised learning, which means that a set of examples is constructed ahead of time for the models to learn from as opposed to slowly learning in-game (unsupervised). These examples are made by extracting basic information on actions made by the top 3 players on the winning team (currently measured by kills, I intend to use fantasy Dota points instead as soon as I figure out how to extract them) from replays using Dotabuff's excellent Manta replay parser (written in Go) along with some nasty data munging of the bowels of the Source engine.

    Currently installation is not very streamlined (I intend to package up the server that handles the Torch requests into a single executable as soon as the bots are ready for people to actually play against):

    - Install Torch
    - Install dependencies for the server:

    Code:
    luarocks install luajson
    luarocks install uuid
    
    luarocks install https://raw.githubusercontent.com/benglard/waffle/master/waffle-scm-1.rockspec
    - Extract repo to <steam path>/steamapps/common/dota 2 beta/game/dota/scripts/vscripts/
    - Start the server with
    Code:
    th <steam path>/steamapps/common/dota\ 2\ beta/game/dota/scripts/vscripts/server/server.lua
    before starting a match

    I haven't included any pre-trained models with this, since currently only moving and attacking is working (no purchasing items yet) and I don't have the replays to make good models (I tried with 40 replays, which sounds like a lot until you realize that that accounts to 1-2 matches per hero at most which is very little to generalize from) -- the best way people can contribute is by getting me more replays, since you can only download 100 replays per 24 hours. To train your own models:

    - Get Go
    - Clone repo (if you're doing it from the command line, be sure to use the --recursive option)
    - If you're on Windows, run set "GOPATH=%cd%". Otherwise (OS X, Linux) run export GOPATH=`pwd`
    - Build the corpora from the replays:
    Code:
    go run corpus_builder.go <replay files...>
    - Train models:
    Code:
    th trainer.lua
    - Move ability_data.lua to <steam path>/steamapps/common/dota 2 beta/game/dota/scripts/vscripts/bots
    - Move the data folder to <steam path>/steamapps/common/dota 2 beta/game/dota/scripts/vscripts/server

    This is still very much a WIP, but I wanted to get it out a little early to demonstrate the concept and get some feedback
    Last edited by axetion; 01-21-2018, 07:18 PM.
    dota2_nn - deep supervised learning bots

  • #2
    Ill try and run it today. Looks cool.

    Would be nice to see a sample data file. So I can see what the inputs are...and how determine output actions.

    i.e. I imagine you can get attacks from combatlog damage to other things.
    you could use spell usage in a similar way if thats the case, but working out where to target aoe spells would be do-able but not trivial.

    I was considering just starting out doing it with a 1v1 mid bot for simplicity, but one thing thats a big part of 1v1 is drawing creep aggro and I couldnt figure an easy way to extract that information from dota2 replay data.
    I think you could possibly track creep positions, see that theyve moved towards a hero despite having other enemy creeps in range...inferring they've been aggroed. but again its not simple

    I imagine there's going to be so many inputs for 'playing dota in general', will need a lot of nodes, and so you'll literally need >1million games to not overfit

    "you can only download 100 replays per 24 hours" dotabuff and opendota must get around this somehow. although they may have to use thousands of steam accounts to do it
    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

    Comment


    • #3
      I dont know what corpus_builder.go is :/

      EDIT: found it in your nn_trainer.

      edit: my issue was trying to use source1 replay. working now
      Last edited by TheP1anoDentist; 06-02-2017, 02:11 AM.
      https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
      https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

      Comment


      • #4
        Sounds hard to install. I hope someone uploads a video of this in action though.

        Comment


        • #5
          Yes as I had assumed quite limited inputs.

          but its still really impressive and a great starting point.

          GetTargetIndex() is awesome. exactly the thing I was trying to find in clarity (the java parser) and couldnt spot.
          the bots can probably learn aggro themselves with your setup. they attackmove hero then move command before attack finishes

          goroutines are pretty fun to play around with. you could use them to do replays in parallel for generating the training data

          Im still a NN noob. but some things:

          I dont see any feature scaling. does torch handle the feature scaling automatically from the inputs, have I missed where you've done it...does it need adding?
          Last edited by TheP1anoDentist; 06-02-2017, 03:19 AM.
          https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
          https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

          Comment


          • #6
            I think the biggest issue to start with is getting enough replays.

            Going to be hard to tell if models are working well, or failing. if never have enough results to train on.

            Might be worth going to opendota discord channel and maybe ask if its possible to replicate their setup.

            what you could do, is plan to train 5/10 heroes only to start with. just find matches with them through the web api, and only parse their replays

            I could run the corpus_build.go on a couple of my servers and have the results be downloadable. could get 200 per day.

            In general want to be appending your example files rather than overwriting when you re-run with new replays, or at least have the option to do so.
            either that or have a separate file for each match, and the torch lua stitches them together

            I think 2_moveexamples, 3_moveexamples is kind of confusing naming. dire/radiant would be much clearer. I know 2/3 comes from their name in the protobuff..but dont need the examples files named like that.

            confused me for a bit.

            (if you want to get 100 replays fairly quickly, whilst filtering to make sure a certain hero is in those games, can use the --downloadOnly option for a tool I have https://github.com/ThePianoDentist/dotaHighlightHelper
            only works on pro-games..but that shouldnt matter
            )
            Last edited by TheP1anoDentist; 06-02-2017, 05:44 AM.
            https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
            https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

            Comment


            • #7
              I would rather contribute to this, than do my own NN stuff from scratch.

              When you are ready for contributions, if you have a rough plan of the multiple things that need improving/changing. It would be good to make github issues so I dont go and work on stuff you've already improved.

              Would you be ok with people creating github issues for things they would like to see changed (even if just for refactoring purposes....i.e the MAMMOTH secondPass function )
              https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
              https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

              Comment


              • #8
                What has been your result thus far?

                Comment


                • #9
                  Originally posted by DreadedGhoul575 View Post
                  Sounds hard to install. I hope someone uploads a video of this in action though.
                  I intend to make the server just a prebuilt statically-linked binary (so no Torch or dependencies needed) once I figure out how this works: https://github.com/ers35/luastatic. (I'd much rather not have to use the server at all and just use Torch in Dota 2 directly though...)

                  Originally posted by TheP1anoDentist View Post
                  I think the biggest issue to start with is getting enough replays.

                  Going to be hard to tell if models are working well, or failing. if never have enough results to train on.

                  Might be worth going to opendota discord channel and maybe ask if its possible to replicate their setup.
                  They actually have a docker image so I could do that, however you need multiple steam accounts to be able to process the amount of replays they/Dotabuff get.

                  Originally posted by TheP1anoDentist View Post
                  what you could do, is plan to train 5/10 heroes only to start with. just find matches with them through the web api, and only parse their replays
                  I think restricting the hero pool would be a good idea for starting out, especially considering right now heroes like Doom and Rubick that steal spells screw up the corpus builder anyways.... :PP


                  Originally posted by TheP1anoDentist View Post
                  (if you want to get 100 replays fairly quickly, whilst filtering to make sure a certain hero is in those games, can use the --downloadOnly option for a tool I have https://github.com/ThePianoDentist/dotaHighlightHelper
                  only works on pro-games..but that shouldnt matter
                  )
                  Ooh, this looks nice. Will look into.

                  Originally posted by TheP1anoDentist View Post
                  Yes as I had assumed quite limited inputs.

                  but its still really impressive and a great starting point.
                  I have more things to add on my checklist (status modifiers and number of nearby summoned units are the biggest ones), but the limiting factor is really retrieving them from the replays. Very undocumented spooky magic going on there.

                  Originally posted by TheP1anoDentist View Post
                  I dont see any feature scaling. does torch handle the feature scaling automatically from the inputs, have I missed where you've done it...does it need adding?
                  I think you might have just missed it: here's where coordinates are scaled for instance. I am using rectified linear units though so scaling matters a little less.

                  Originally posted by TheP1anoDentist View Post

                  Would you be ok with people creating github issues for things they would like to see changed (even if just for refactoring purposes....i.e the MAMMOTH secondPass function )
                  Absolutely -- pull requests welcome too! Once I get a little more organized I'll make a GH project board that I can add tasks and issues too so that people can see progress on things.

                  Yeah, SecondPass is nasty (understandbly though, it's the meat and potatoes of it). I'll split it up into SecondPass and ConstructFeatureVector when I get a chance.
                  dota2_nn - deep supervised learning bots

                  Comment


                  • #10
                    Originally posted by TheP1anoDentist View Post
                    GetTargetIndex() is awesome. exactly the thing I was trying to find in clarity (the java parser) and couldnt spot.
                    Here's the full proto: https://github.com/dotabuff/manta/bl...ges.proto#L562

                    To be honest I consider myself lucky that I found it :P
                    dota2_nn - deep supervised learning bots

                    Comment


                    • #11
                      Originally posted by nostrademous View Post
                      What has been your result thus far?
                      The models trained with 40 random replays (which again amounts to only 1 replay per hero usually) moved around (spotted them standing in ward spots and following the creep front) and knew when to run away but didn't really use abilities. I've weighted the training towards learning abilities appropriately so I'm pretty sure I need more examples for it to infer when to use them. (Another possibility is that it might just be thrown off by the fact that doesn't have any items.)
                      Last edited by axetion; 06-02-2017, 06:30 AM.
                      dota2_nn - deep supervised learning bots

                      Comment


                      • #12
                        A better installation approach would be really useful. I just tried to install it (per directions) and ended up getting a critical error when doing the "luarocks install json".

                        It's kind of strange though, b/c I install torch (Windows 10) per their instructions (with Anaconda3) first in order to get luarocks in the first place, but the message I got is:
                        Code:
                        json 1.0-0 depends on torch >= 7.0 (not installed)
                        So it's probably a PATH env issue.

                        Comment


                        • #13
                          Originally posted by axetion View Post
                          (Another possibility is that it might just be thrown off by the fact that doesn't have any items.)
                          Easy way to test this... don't gut the item_purchase_generic

                          Comment


                          • #14
                            Originally posted by nostrademous View Post
                            A better installation approach would be really useful. I just tried to install it (per directions) and ended up getting a critical error when doing the "luarocks install json".

                            It's kind of strange though, b/c I install torch (Windows 10) per their instructions (with Anaconda3) first in order to get luarocks in the first place, but the message I got is:
                            Code:
                            json 1.0-0 depends on torch >= 7.0 (not installed)
                            So it's probably a PATH env issue.
                            Actually that was my fault -- I gave the wrong package name. The correct one is luajson and I've updated the post.

                            (Although I would recommend checking to see if the luarocks you're using corresponds to Torch's luajit not whatever other Lua install you might have)
                            Last edited by axetion; 06-02-2017, 07:10 AM.
                            dota2_nn - deep supervised learning bots

                            Comment


                            • #15
                              Originally posted by axetion View Post
                              Actually that was my fault -- I gave the wrong package name. The correct one is luajson and I've updated the post.

                              (Although I would recommend checking to see if the luarocks you're using corresponds to Torch's luajit not whatever other Lua install you might have)
                              I'm running it out of the torch\install directory in an Anaconda shell.

                              installing luajson works no problem now.

                              installing pegasus throws an error (seems lzlib dependency fails):
                              Code:
                              (C:\Users\nostr\Anaconda3) C:\torch\distro-master\install>luarocks install pegasus
                              Installing https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/pegasus-0.9.2-1.src.rock
                              Missing dependencies for pegasus 0.9.2-1:
                                 lzlib >= 0.4.1.53-1 (not installed)
                              
                              pegasus 0.9.2-1 depends on lzlib >= 0.4.1.53-1 (not installed)
                              Installing https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/lzlib-0.4.1.53-1.src.rock
                              
                              Error: Failed installing dependency: https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/lzlib-0.4.1.53-1.src.rock - Could not find header file for ZLIB
                                No file zlib.h in c:/external/include
                              You may have to install ZLIB in your system and/or pass ZLIB_DIR or ZLIB_INCDIR to the luarocks command.
                              Example: luarocks install lzlib ZLIB_DIR=/usr/local
                              Digging some more, looks like lzlib expects you to pull down the zlib source and build it first and just point the luarocks include location at it (not sure why it doesn't just do it for you...)
                              Last edited by nostrademous; 06-02-2017, 07:55 AM.

                              Comment

                              Working...
                              X