Announcement

Collapse

Forum Rules

  • No flaming or derogatory remarks, directly or through insinuation.
  • No discussion, sharing or referencing illegal software such as hacks, keygen, cracks and pirated software.
  • No offensive contents, including but not limited to, racism, gore or pornography.
  • No excessive spam/meme, i.e. copious one liners in a short period of time, typing with all caps or posting meme responses (text/image).
  • No trolling, including but not limited to, flame incitation, user provocation or false information distribution.
  • No link spamming or signature advertisements for content not specific to Dota 2.
  • No Dota 2 key requests, sell, trade etc.
  • You may not create multiple accounts for any purpose, including ban evasion, unless expressly permitted by a moderator.

  • Please search before posting. One thread per issue. Do not create another thread if there is an existing one already.
  • Before posting anything, make sure you check out all sticky threads (e.g., this). Do not create new threads about closed ones.
  • It is extremely important that you post in correct forum section.

  • Balance discussion only in Misc.
  • All art related (such as hero model) feedbacks go to Art Feedback Forum.
  • All matchmaking feedback should go here: Matchmaking Feedback
  • All report/low priority issues should go here: Commend/Report/Ban Feedback
  • No specific workshop item feedback. These should go to workshop page of that item.
  • When posting in non-bugs section (such as this), use [Bugs], [Discussion] or [Suggestion] prefix in your thread name.



In case you object some action by a moderator, please contact him directly through PM and explain your concerns politely. If you are still unable to resolve the issue, contact an administrator. Do not drag these issues in public.



All rules are meant to augment common sense, please use them when not conflicted with aforementioned policies.
See more
See less

dota2_nn: Extremely WIP deep supervised learning full-takeover bots

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • axetion
    replied
    Worth also noting that I've merged the dota2_nn and dota2_nn_trainer repos into one repo

    Leave a comment:


  • axetion
    replied
    After a long delay, I've finally pushed out the refactor. I haven't banished all of the spaghetti code from the trainer yet, but both the actual bot code and the corpus builder are now nice and modular. The corpus builder outputs a JSON example format in place of the ugly CSV one as well as using the new Manta version which is much faster. There's still some bugs to quash but this should be much easier to work with going forward.

    Leave a comment:


  • axetion
    replied
    You definitely don't need my permission to fork this :P I recommend waiting a bit until I clean up the code before trying to mess with it (the bot code is also slightly broken currently because of some API changes)

    Leave a comment:


  • senz
    replied
    hey, very impressed with the work youve done so far. I want to get into DOTA bot modding myself. If you need help id love to join your efforts, or if you prefer working alone i wanna ask for your permission to fork your codebase.

    Leave a comment:


  • axetion
    replied
    Forgot to mention that to go with the hero selection, I'm having the corpus builder make different models for different farming positions: that is, there will be two nets for, say, position 4 abaddon (support) and position 3 abaddon (offlaner). I think this will solve the "which lane will the bots take" problem -- rather than learning whatever the most common lane is or having to learn a bunch of features to tell which lane it should take at the beginning, it'll learn different things for two or three of the most common farming positions and the hero selection algorithm will pick the model it wants to fit the role it needs.

    The corpus builder determines the farming position by assigning positions 1-3 to the top 3 in last hits based on their starting lane, then assigning position 4 to whoever has the higher net worth of the remaining two.

    The main downside for this is now the training data is getting spread even more thin across different nets for different teams and different farming positions. I'd like to find a way to abuse the symmetrical nature of the Dota map and eliminate the need for separate models for different teams, but it's not as simple as reflecting the coordinates across the river (jungles are shaped differently for one). Thoughts on this would be appreciated.
    Last edited by axetion; 08-14-2017, 09:41 AM.

    Leave a comment:


  • axetion
    replied
    Since this has been taking longer than I had anticipated (and also we're seeing a lot of hype around ML in Dota 2 with the OpenAI mid bot) I figured a status update was in order for anyone interested.

    I'm still working on this, and I currently have a couple things I'm toying with (some finished already, others not):

    - Per TheP1anoDentist's suggestion I've ditched the ugly CSV format for the data and have instead made a JSON format. While it isn't a whole lot more human readable, it's surprisingly easy for the trainer to read and should be rather flexible as the models change.

    - I'm currently creating models for purchasing items and leveling abilities, which was already in the bot code but didn't have any code to train them.

    - I'm significantly improving the code quality across pretty much everything, breaking it up into multiple files and what not. The current code is rather ugly.

    - I'm making a better all pick (and potentially captain's mode) hero selection algorithm that tries to fill farming positions and pick heroes that counter what the enemy has and synergize with what was already picked. It's not anything statistical/ML but it should be a lot better.

    - I wanted to turn the server into a Valve Server Plugin in C++ (https://developer.valvesoftware.com/wiki/Server_plugins), but it looks like Dota 2 doesn't load those. Metamod: Source currently hooks into the GameDLL which doesn't seem to work with listen servers (at least not without -insecure). If anyone knows a way to pull this off let me know

    Leave a comment:


  • TheP1anoDentist
    replied
    Originally posted by axetion View Post
    Last I checked -8288 -> 8288 was what GetWorldBounds() returned. That might have changed.
    kk thanks

    Leave a comment:


  • axetion
    replied
    Last I checked -8288 -> 8288 was what GetWorldBounds() returned. That might have changed.

    Leave a comment:


  • TheP1anoDentist
    replied
    for your positions where did you get -8288.0 -> 8288.0 from for X, Y?

    I thought it was -8192 -> +8192 as I thought there were 128 grid cells and each cell was length 128 coords.

    so 128x128 = 16384

    also from this function in open-dota, and from checking m_cellX, m_cellY stuff.
    I have a funny feeling that minimum m_cellX for the map is 64, and maximum is 192. (same for y)
    Not sure if you need to take that into account? (can probably check by seeing if get a float32(cellX)*CELL_SIZE > 2*Max_X)

    export const gameCoordToUV = (x, y) => ({
    x: Number(x) - 64,
    y: 127 - (Number(y) - 64),
    });


    it's just really confusing in general and there's basically 0 proper documentation on it


    edit: someone just linked me this which might be helpful http://devilesk.com/blog/creating-a-dota-map-image
    Last edited by TheP1anoDentist; 06-25-2017, 04:42 AM.

    Leave a comment:


  • axetion
    replied
    Been a little distracted by the Battle Pass, but here's a small status update for those who are interested:

    I've switched the server from Pegasus to Waffle, which is cleaner and doesn't need zlib (should simplify install quite a bit)

    I've also been experimenting with getting Travis to build binaries of everything automatically (I started with the trainer since it has less dependencies), but it doesn't quite work yet. Once it does however you can simply download a binary package for your platform from here: https://github.com/axetion/dota2_nn_trainer/releases

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post
    do you have any tips on doing dimensional analysis on tables in lua?
    I print out #table, #table[1], #table[1][1] until I get an index error, but it feels really clunky and there should be a better way.
    I doubt pure Lua has anything better (Lua is notoriously minimalist). But one of the reasons I love Torch's tensors is that they make it really easy to manage multi-dimensional data. If you have a tensor all you have to do is:

    Code:
    tensor:size()
    and it returns a 1D tensor containing the sizes like so:

    Code:
    th> tensor
     1  2  3
     4  5  6
    [torch.DoubleTensor of size 2x3]
    
                                                                          [0.0001s]
    th> tensor:size()
     2
     3
    [torch.LongStorage of size 2]
    
                                                                          [0.0002s]
    th>
    You can also do :size(<num>) to get a specific dimension.

    Originally posted by TheP1anoDentist View Post
    I only notice it because I use intellij IDE with lua plugins, which flag things like that up for me. I highly recommend
    Hm. I currently use VS Code, but the Lua extension only really checks syntax.
    Last edited by axetion; 06-04-2017, 09:23 AM.

    Leave a comment:


  • TheP1anoDentist
    replied
    Originally posted by axetion View Post
    Thanks. I can't seem to get used to that (annoyed me when I first learned Octave too)...
    I only notice it because I use intellij IDE with lua plugins, which flag things like that up for me. I highly recommend

    do you have any tips on doing dimensional analysis on tables in lua?
    I print out #table, #table[1], #table[1][1] until I get an index error, but it feels really clunky and there should be a better way.

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post
    in your actual bot part you might have some actual accidental index 0-ing unit:GetNearbyNeutralCreeps(1600)[0], unit:GetNearbyHeroes(1600, true)[0] etc
    Thanks. I can't seem to get used to that (annoyed me when I first learned Octave too)...

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post
    I'm still not sure what move_class_weights and move_label_weights are doing.
    are these weights the same type of weights that live in the hidden layers
    do they get updated during backpropagation like normal-weights when loss is called?
    No, they're not part of the network and they don't change. In essence, they're supposed to weight the loss function so that less-common classes (or labels even) matter more. Otherwise the network would always output the most common class to "minimize" the error.
    Last edited by axetion; 06-03-2017, 11:18 AM.

    Leave a comment:


  • TheP1anoDentist
    replied
    thanks for all the answers so far. I've learnt an awful lot today
    I think im finally starting to understand how it works now

    the 'target' also gets unrolled into 0-1 probabilities for each target type in the output vector. same for ability and item

    so AbilityUsed is a LABEL.... Shuriken Toss..or whatever its integer equivalent is, is the CLASS

    I'm still not sure what move_class_weights and move_label_weights are doing.
    are these weights the same type of weights that live in the hidden layers
    do they get updated during backpropagation like normal-weights when loss is called?

    or are they an extra layer you've added...between final hidden and output...after output? to handle it being multi-label, multi-class model.

    or are they just the final hidden layer, but you have essentially overridden the default randomizing of weights, with values based on how frequent those classes appear. leading to faster learning?

    basically I have no idea what nodes these weights are actually connections between

    in your actual bot part you might have some actual accidental index 0-ing unit:GetNearbyNeutralCreeps(1600)[0], unit:GetNearbyHeroes(1600, true)[0] etc

    Leave a comment:

Working...
X