Announcement

Collapse

Forum Rules

  • No flaming or derogatory remarks, directly or through insinuation.
  • No discussion, sharing or referencing illegal software such as hacks, keygen, cracks and pirated software.
  • No offensive contents, including but not limited to, racism, gore or pornography.
  • No excessive spam/meme, i.e. copious one liners in a short period of time, typing with all caps or posting meme responses (text/image).
  • No trolling, including but not limited to, flame incitation, user provocation or false information distribution.
  • No link spamming or signature advertisements for content not specific to Dota 2.
  • No Dota 2 key requests, sell, trade etc.
  • You may not create multiple accounts for any purpose, including ban evasion, unless expressly permitted by a moderator.

  • Please search before posting. One thread per issue. Do not create another thread if there is an existing one already.
  • Before posting anything, make sure you check out all sticky threads (e.g., this). Do not create new threads about closed ones.
  • It is extremely important that you post in correct forum section.

  • Balance discussion only in Misc.
  • All art related (such as hero model) feedbacks go to Art Feedback Forum.
  • All matchmaking feedback should go here: Matchmaking Feedback
  • All report/low priority issues should go here: Commend/Report/Ban Feedback
  • No specific workshop item feedback. These should go to workshop page of that item.
  • When posting in non-bugs section (such as this), use [Bugs], [Discussion] or [Suggestion] prefix in your thread name.



In case you object some action by a moderator, please contact him directly through PM and explain your concerns politely. If you are still unable to resolve the issue, contact an administrator. Do not drag these issues in public.



All rules are meant to augment common sense, please use them when not conflicted with aforementioned policies.
See more
See less

dota2_nn: Extremely WIP deep supervised learning full-takeover bots

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • axetion
    replied
    Originally posted by Platinum_dota2 View Post
    The reason I was saying this is because either you will have to use a huge feature vector to cover the things that the algorithm should consider, in which case the learning process becomes impractically long, or you have to use a small one in which case the end results won't be good. .
    I agree with what you're saying, but I went into this knowing full well the former is what would probably end up happening. I don't consider the current features I'm using complete by any stretch, and I'm willing to sink long periods of compute time into getting good models. So we'll see how it pans out

    FWIW the bot is already something of a hybrid. Things like managing the courier and basic laning are too micro for the neural network to bother with.

    Leave a comment:


  • axetion
    replied
    Improving the readability of the examples is something I'd like to do but not sure how much of a priority it should be. In the mean time you can find out what the item IDs are by doing this in a REPL:

    Code:
    require "ability_data"
    ability_data.items.npc_dota_hero_whatever[<team>][<id>]
    ability_data houses all the IDs used in classification (abilities, items, active abilities and active items)

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post

    It's kind of funny that neural nets claim to mirror how humans think/learn. But I think dota is complex enough, its an interesting example of how
    humans will essentially 'learn' whilst ignoring most of the features, then over time they essentially add more and more features/inputs to their mental model of the game to make decisions on, as they understand the game better.
    This is something that neural networks can do too, with feature selection There aren't a lot of good fast algorithms for it though....

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post
    CreateContainer is defined with only 3 parameters, however called like this. Is the 3 just a leftover from hardcoding stuff?
    Code:
    local move = CreateContainer(input_len, output_len, math.floor((input_len + output_len) / 2), 3)
    Good catch. Thank you.

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post
    training_err is the error the network claims it has. but you've split your data into training data and test data, and the validation_err describes how much trained model differs from test data?
    In essence. The training loss is used to drive the actual changes the network makes, while the validation error is used to make sure that the network isn't overfitting the training data by having a set of data the network never looks at when optimizing.

    Originally posted by TheP1anoDentist View Post
    edit: are items inputs or outputs?
    Inputs are items, outputs are *active* items (items that the corpus builder has observed being used)

    Leave a comment:


  • axetion
    replied
    Originally posted by TheP1anoDentist View Post
    if its not I 'believe' the correct thing to do is to transform items into binary-inputs. so rather than 6 fields (one for each current item).
    You would need ~100 fields. [HasBranch, HasGhostScepter, HasBlademail] ...etc. although as can have multiples maybe it needs to be [NumBranches, NumBlademails, NumNullTalismans....]
    .
    Yes, this is how it works (one-to-many). The reason why the corpus builder outputs the IDs (which are essentially the indexes you set to 1) is because at the time it doesn't know how many items there are so it has no idea how many fields to put. Instead the trainer preprocesses this for it when loading the examples.

    Originally posted by TheP1anoDentist View Post
    I think you just forgot lua 1-indexing here `move_label_weights[i + 1] = (label[0] or label[1]) / move_total` I believe you are trying to index label, rather than search for an integer key
    `label` is not an array, it's a map mapping classes to the number of times that class occurred. Depending on the class the "ignored/none" class may either be `0` or `1`.


    The trainer used to have more comments and was a lot cleaner, but in the past couple days I've really changed how it works so now it's kind of nasty. I will clean it up and add more comments explaining all of this (especially in ParseMoveBatch).
    Last edited by axetion; 06-03-2017, 09:14 AM.

    Leave a comment:


  • TheP1anoDentist
    replied
    So I find it hard to understand exactly what the trainer is doing. It is hard to keep track of what data is what, because it is a bit dependent on knowing which index is which.

    what are your thoughts on having the golang write to file a more structured file. i.e json.

    that would let you do results = [{"inputs": {"items": [1, 3, 4], "positions": []}, "outputs": []}]

    then your trainer unpacks this into raw arrays for torch.
    you obviously still have to deal with raw arrays at some point, but I think converting data to them as late as possible makes adding new inputs/outputs easier.

    label weights is just the matrix of weights connecting all the nodes between input/hidden/output layers right?
    but I dont know what class_weights are. dont think ive ever used them before. Ive never had multiple outputs, do they become important then. if so can you explain their function here?

    A small mspaint style diagram of the net structure would probably be quite helpful

    training_err is the error the network claims it has. but you've split your data into training data and test data, and the validation_err describes how much trained model differs from test data?

    edit: are items inputs or outputs?
    Last edited by TheP1anoDentist; 06-03-2017, 08:43 AM.

    Leave a comment:


  • TheP1anoDentist
    replied
    Im going to put all my questions here rather than pm or something in-case others have same.

    from trainer.lua

    CreateContainer is defined with only 3 parameters, however called like this. Is the 3 just a leftover from hardcoding stuff?
    Code:
    local move = CreateContainer(input_len, output_len, math.floor((input_len + output_len) / 2), 3)
    If Im understanding correctly, have 3 hidden layers, each one have (inputs + outputs)/ 2 number of nodes.
    I always struggle to know how people choose net structure. Is there reason behind current setup, or just something to get project working and testing?

    I think you just forgot lua 1-indexing here `move_label_weights[i + 1] = (label[0] or label[1]) / move_total` I believe you are trying to index label, rather than search for an integer key
    Last edited by TheP1anoDentist; 06-03-2017, 07:18 AM.

    Leave a comment:


  • TheP1anoDentist
    replied
    also just to note. im on ubuntu

    even with it installed needed to give ZLIB_LIBDIR instructions to luarocks, otherwise it couldnt find it itself. not sure if both variables are necessary or just one.
    Code:
    luarocks install pegasus ZLIB_DIR=/usr ZLIB_LIBDIR=/usr/lib/x86_64-linux-gnu

    Leave a comment:


  • TheP1anoDentist
    replied
    ^ add to that there is a 500 limit per IP address, even if you have multiple steams.

    This may be something I havent seen in the training lua. dont understand it properly yet.

    but you're taking current items as inputs, you're assigning them an integer.

    wont the neural net try and assign some relative meaning to their ids.

    example:
    iron branch id : 80
    sheepstick id : 79
    bottle id: 3

    the net will think 'Sheepstick is much more similar to iron branch, than it is to bottle', because it has a closer id. Which is nonsense.

    is this already handled?

    if its not, I 'believe' the correct thing to do is to transform items into binary-inputs. so rather than 6 fields (one for each current item).
    You would need ~100 fields. [HasBranch, HasGhostScepter, HasBlademail] ...etc. although as can have multiples maybe it needs to be [NumBranches, NumBlademails, NumNullTalismans....]


    If I'm correct (unsure), this is an example of how the number of features/inputs is going to explode upwards as it gets 'cleverer'.

    It's kind of funny that neural nets claim to mirror how humans think/learn. But I think dota is complex enough, its an interesting example of how
    humans will essentially 'learn' whilst ignoring most of the features, then over time they essentially add more and more features/inputs to their mental model of the game to make decisions on, as they understand the game better.

    Jamming literally every input into a neural-net before doing any training, doesnt match how people learn dota.

    Leave a comment:


  • axetion
    replied
    I actually used Opendota to get some replays but the problem is that downloading replays is ratelimited by Valve.

    Leave a comment:


  • Lord_Zera
    replied
    For replay downloading:
    You can get the replay salt from the Opendota API - https://docs.opendota.com/#tag/replays
    With this you should be able to get pretty much any public match in ~ the last 2 weeks

    Leave a comment:


  • TheP1anoDentist
    replied
    Originally posted by Platinum_dota2 View Post
    I checked out the code. It indeed is very cool.

    I'm gonna tell you what I told Lenlrx when he wanted to use machine learning for bots: I don't think it can be done (i.e. the results won't be good) at least not with the current algorithms. The reason I was saying this is because either you will have to use a huge feature vector to cover the things that the algorithm should consider, in which case the learning process becomes impractically long, or you have to use a small one in which case the end results won't be good. Eventually, he deviated to what I was thinking and I deviated to what he was thinking . Right now I think this can be done, but you have to be careful with what you want to do. Do I think you can write any machine learning algorithm that learns dota and plays it (which is almost what you are trying to do) ? No! Can machine learning be used to improve hard coded bots? Yes (at least I think so)! I think you have to use a hybrid system that relies on hard coded scripts for basic tasks (similar to think functions of modes, but smaller) and uses machine learning for your high-level behavior (similar to what desires are doing right now for default bots, but more advanced). This is why I was trying to push ChrisC to make basic tasks modular so people focus more on high-level things.

    I strongly suggest that you should switch to a hybrid system before getting bad results from what you are currently doing and getting coldhearted (unless you are just experimenting or/and do not have high expectations).

    BTW another thing I think ML can be used for is creating tools that can be used in hard coded scripts, like predicting enemy's movement patterns or how much damage/heal/disable a hero can deal/provide given its level in a teamfight again n heroes etc.
    yeah Im going to be surprised if it works. to be honest, a self-driving car is a piece of piss in comparison to playing dota. but the bigger the challenge, the more fun to try

    I think though even with a hybrid system you dont necessarily need scripting. You do machine-learning on all the different modes, to learn how to effectively lane, how to effectively teamfight etc.
    then you have a higher level machine learning applied to just the decision system, choosing which mode to engage in.

    I always thought the biggest hurdles initially were going to be translating replay variables into inputs/outputs that the bots can train on....and this project has already made a really solid start on that.

    So im very optimistic

    Leave a comment:


  • nostrademous
    replied
    Originally posted by Platinum_dota2 View Post
    ! I think you have to use a hybrid system that relies on hard coded scripts for basic tasks (similar to think functions of modes, but smaller) and uses machine learning for your high-level behavior (similar to what desires are doing right now for default bots, but more advanced). This is why I was trying to push ChrisC to make basic tasks modular so people focus more on high-level things.
    I had the same conclusion and that was the reason for me starting: https://github.com/Nostrademous/Dota2-WebAI

    The readme.md summarizes it.

    Leave a comment:


  • Platinum_dota2
    replied
    I checked out the code. It indeed is very cool.

    I'm gonna tell you what I told Lenlrx when he wanted to use machine learning for bots: I don't think it can be done (i.e. the results won't be good) at least not with the current algorithms. The reason I was saying this is because either you will have to use a huge feature vector to cover the things that the algorithm should consider, in which case the learning process becomes impractically long, or you have to use a small one in which case the end results won't be good. Eventually, he deviated to what I was thinking and I deviated to what he was thinking . Right now I think this can be done, but you have to be careful with what you want to do. Do I think you can write any machine learning algorithm that learns dota and plays it (which is almost what you are trying to do) ? No! Can machine learning be used to improve hard coded bots? Yes (at least I think so)! I think you have to use a hybrid system that relies on hard coded scripts for basic tasks (similar to think functions of modes, but smaller) and uses machine learning for your high-level behavior (similar to what desires are doing right now for default bots, but more advanced). This is why I was trying to push ChrisC to make basic tasks modular so people focus more on high-level things.

    I strongly suggest that you should switch to a hybrid system before getting bad results from what you are currently doing and getting coldhearted (unless you are just experimenting or/and do not have high expectations).

    BTW another thing I think ML can be used for is creating tools that can be used in hard coded scripts, like predicting enemy's movement patterns or how much damage/heal/disable a hero can deal/provide given its level in a teamfight again n heroes etc.

    Leave a comment:

Working...
X