Page 1 of 5 1 2 3 ... LastLast
Results 1 to 10 of 45

Thread: dota2_nn: Extremely WIP deep supervised learning full-takeover bots

  1. #1
    Basic Member axetion's Avatar
    Join Date
    Jan 2017
    Posts
    38

    dota2_nn: Extremely WIP deep supervised learning full-takeover bots

    https://github.com/axetion/dota2_nn
    https://github.com/axetion/dota2_nn_trainer

    This is the start of a crude deep learning bot using Torch I've been working on. Unlike most of the previous ML bots here, this is purely supervised learning, which means that a set of examples is constructed ahead of time for the models to learn from as opposed to slowly learning in-game (unsupervised). These examples are made by extracting basic information on actions made by the top 3 players on the winning team (currently measured by kills, I intend to use fantasy Dota points instead as soon as I figure out how to extract them) from replays using Dotabuff's excellent Manta replay parser (written in Go) along with some nasty data munging of the bowels of the Source engine.

    Currently installation is not very streamlined (I intend to package up the server that handles the Torch requests into a single executable as soon as the bots are ready for people to actually play against):

    - Install Torch
    - Install dependencies for the server:

    Code:
    luarocks install luajson
    luarocks install uuid
    
    luarocks install https://raw.githubusercontent.com/benglard/waffle/master/waffle-scm-1.rockspec
    - Extract repo to <steam path>/steamapps/common/dota 2 beta/game/dota/scripts/vscripts/
    - Start the server with
    Code:
    th <steam path>/steamapps/common/dota\ 2\ beta/game/dota/scripts/vscripts/server/server.lua
    before starting a match

    I haven't included any pre-trained models with this, since currently only moving and attacking is working (no purchasing items yet) and I don't have the replays to make good models (I tried with 40 replays, which sounds like a lot until you realize that that accounts to 1-2 matches per hero at most which is very little to generalize from) -- the best way people can contribute is by getting me more replays, since you can only download 100 replays per 24 hours. To train your own models:

    - Get Go
    - Get Manta:
    Code:
    go get github.com/dotabuff/manta
    - Build the corpora from the replays:
    Code:
    go run corpus_builder.go <replay files...>
    - Train models:
    Code:
    th trainer.lua
    - Move ability_data.lua to <steam path>/steamapps/common/dota 2 beta/game/dota/scripts/vscripts/bots
    - Move the data folder to <steam path>/steamapps/common/dota 2 beta/game/dota/scripts/vscripts/server

    This is still very much a WIP, but I wanted to get it out a little early to demonstrate the concept and get some feedback
    Last edited by axetion; 06-15-2017 at 09:48 AM.

  2. #2
    Ill try and run it today. Looks cool.

    Would be nice to see a sample data file. So I can see what the inputs are...and how determine output actions.

    i.e. I imagine you can get attacks from combatlog damage to other things.
    you could use spell usage in a similar way if thats the case, but working out where to target aoe spells would be do-able but not trivial.

    I was considering just starting out doing it with a 1v1 mid bot for simplicity, but one thing thats a big part of 1v1 is drawing creep aggro and I couldnt figure an easy way to extract that information from dota2 replay data.
    I think you could possibly track creep positions, see that theyve moved towards a hero despite having other enemy creeps in range...inferring they've been aggroed. but again its not simple

    I imagine there's going to be so many inputs for 'playing dota in general', will need a lot of nodes, and so you'll literally need >1million games to not overfit

    "you can only download 100 replays per 24 hours" dotabuff and opendota must get around this somehow. although they may have to use thousands of steam accounts to do it
    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

  3. #3
    I dont know what corpus_builder.go is :/

    EDIT: found it in your nn_trainer.

    edit: my issue was trying to use source1 replay. working now
    Last edited by TheP1anoDentist; 06-02-2017 at 02:11 AM.
    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

  4. #4
    Sounds hard to install. I hope someone uploads a video of this in action though.

  5. #5
    Yes as I had assumed quite limited inputs.

    but its still really impressive and a great starting point.

    GetTargetIndex() is awesome. exactly the thing I was trying to find in clarity (the java parser) and couldnt spot.
    the bots can probably learn aggro themselves with your setup. they attackmove hero then move command before attack finishes

    goroutines are pretty fun to play around with. you could use them to do replays in parallel for generating the training data

    Im still a NN noob. but some things:

    I dont see any feature scaling. does torch handle the feature scaling automatically from the inputs, have I missed where you've done it...does it need adding?
    Last edited by TheP1anoDentist; 06-02-2017 at 03:19 AM.
    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

  6. #6
    I think the biggest issue to start with is getting enough replays.

    Going to be hard to tell if models are working well, or failing. if never have enough results to train on.

    Might be worth going to opendota discord channel and maybe ask if its possible to replicate their setup.

    what you could do, is plan to train 5/10 heroes only to start with. just find matches with them through the web api, and only parse their replays

    I could run the corpus_build.go on a couple of my servers and have the results be downloadable. could get 200 per day.

    In general want to be appending your example files rather than overwriting when you re-run with new replays, or at least have the option to do so.
    either that or have a separate file for each match, and the torch lua stitches them together

    I think 2_moveexamples, 3_moveexamples is kind of confusing naming. dire/radiant would be much clearer. I know 2/3 comes from their name in the protobuff..but dont need the examples files named like that.

    confused me for a bit.

    (if you want to get 100 replays fairly quickly, whilst filtering to make sure a certain hero is in those games, can use the --downloadOnly option for a tool I have https://github.com/ThePianoDentist/dotaHighlightHelper
    only works on pro-games..but that shouldnt matter
    )
    Last edited by TheP1anoDentist; 06-02-2017 at 05:44 AM.
    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

  7. #7
    I would rather contribute to this, than do my own NN stuff from scratch.

    When you are ready for contributions, if you have a rough plan of the multiple things that need improving/changing. It would be good to make github issues so I dont go and work on stuff you've already improved.

    Would you be ok with people creating github issues for things they would like to see changed (even if just for refactoring purposes....i.e the MAMMOTH secondPass function )
    https://github.com/ThePianoDentist/t...dentistdotabot Lina bot which pulls small camp when 'laning' (Aim to work on pulling and stacking bots initially)
    https://github.com/ThePianoDentist/dotabots-ml-tools Parsing data from bot games

  8. #8
    Basic Member
    Join Date
    Dec 2016
    Posts
    644
    What has been your result thus far?

  9. #9
    Basic Member axetion's Avatar
    Join Date
    Jan 2017
    Posts
    38
    Quote Originally Posted by DreadedGhoul575 View Post
    Sounds hard to install. I hope someone uploads a video of this in action though.
    I intend to make the server just a prebuilt statically-linked binary (so no Torch or dependencies needed) once I figure out how this works: https://github.com/ers35/luastatic. (I'd much rather not have to use the server at all and just use Torch in Dota 2 directly though...)

    Quote Originally Posted by TheP1anoDentist View Post
    I think the biggest issue to start with is getting enough replays.

    Going to be hard to tell if models are working well, or failing. if never have enough results to train on.

    Might be worth going to opendota discord channel and maybe ask if its possible to replicate their setup.
    They actually have a docker image so I could do that, however you need multiple steam accounts to be able to process the amount of replays they/Dotabuff get.

    Quote Originally Posted by TheP1anoDentist View Post
    what you could do, is plan to train 5/10 heroes only to start with. just find matches with them through the web api, and only parse their replays
    I think restricting the hero pool would be a good idea for starting out, especially considering right now heroes like Doom and Rubick that steal spells screw up the corpus builder anyways.... :PP


    Quote Originally Posted by TheP1anoDentist View Post
    (if you want to get 100 replays fairly quickly, whilst filtering to make sure a certain hero is in those games, can use the --downloadOnly option for a tool I have https://github.com/ThePianoDentist/dotaHighlightHelper
    only works on pro-games..but that shouldnt matter
    )
    Ooh, this looks nice. Will look into.

    Quote Originally Posted by TheP1anoDentist View Post
    Yes as I had assumed quite limited inputs.

    but its still really impressive and a great starting point.
    I have more things to add on my checklist (status modifiers and number of nearby summoned units are the biggest ones), but the limiting factor is really retrieving them from the replays. Very undocumented spooky magic going on there.

    Quote Originally Posted by TheP1anoDentist View Post
    I dont see any feature scaling. does torch handle the feature scaling automatically from the inputs, have I missed where you've done it...does it need adding?
    I think you might have just missed it: here's where coordinates are scaled for instance. I am using rectified linear units though so scaling matters a little less.

    Quote Originally Posted by TheP1anoDentist View Post

    Would you be ok with people creating github issues for things they would like to see changed (even if just for refactoring purposes....i.e the MAMMOTH secondPass function )
    Absolutely -- pull requests welcome too! Once I get a little more organized I'll make a GH project board that I can add tasks and issues too so that people can see progress on things.

    Yeah, SecondPass is nasty (understandbly though, it's the meat and potatoes of it). I'll split it up into SecondPass and ConstructFeatureVector when I get a chance.

  10. #10
    Basic Member axetion's Avatar
    Join Date
    Jan 2017
    Posts
    38
    Quote Originally Posted by TheP1anoDentist View Post
    GetTargetIndex() is awesome. exactly the thing I was trying to find in clarity (the java parser) and couldnt spot.
    Here's the full proto: https://github.com/dotabuff/manta/bl...ges.proto#L562

    To be honest I consider myself lucky that I found it :P

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •