Page 2 of 3 FirstFirst 1 2 3 LastLast
Results 11 to 20 of 25

Thread: Creep Block Test Scenario for Episodic Reinforcement Learning

  1. #11
    Basic Member
    Join Date
    Dec 2016
    Posts
    76
    I dont think it is a good start for ai to learn blocking creeps since it is an advance skill for human.
    nobody learn dota with blocking creep as their first lesson.
    I am training my ai with the 1st objective: go to the lane and get some XP, don't get hit by creeps and towers.
    2st objective:get some last hit.
    https://github.com/lenLRX/Dota2_DPPO_bots ----My ML bot work in progress

  2. #12
    My computer is busy running the simulations at the moment. Heres a snippet of how it does so far: http://recordit.co/WoeJ9EyTuP
    Update: Lol it seems to have learnt some drunk back and forth movement at the t2 tower while waiting for the creep wave to arrive

    Once I can get a good estimate of the model required to solve these sort of problems I will be trying to extend it to 1v1 scenarios.

    Interestingly because I can asynchronously gather experience, and train my network, it is entirely possible to crowdsource the bot through POST and GET requests. If I can show good progress, I am hopeful that I can get at least a few people onboard.
    Last edited by BeyondGodlikeBot; 08-19-2017 at 06:07 AM.

  3. #13
    Does anyone know of any Lua libraries which can do sampling from multivariate normal distributions?? I couldn't find anything via google searches.

    Alternatively has anyone successfully been using Torch inside Dota 2 Lua?? I am just hesitant to go down the this route as it seems it is a rather complicated setup on a windows machine (not even entirely sure if Dota 2 Lua can call Torch stuff due to the whole virtual machine thing? not entirely clear)

    The reason I need to sample from multivariate normal distributions is because I am now investigating continuous policies instead of discrete ones. The idea is based on what other researchers have done for continuous action space environments where a model will output parameters for multivariate distributions from which an action will be sampled.
    In my case the distribution will be 2D from which I can sample (x,y) which is the offset my hero will move

  4. #14
    Basic Member
    Join Date
    Dec 2016
    Posts
    76
    Quote Originally Posted by BeyondGodlikeBot View Post
    Does anyone know of any Lua libraries which can do sampling from multivariate normal distributions?? I couldn't find anything via google searches.

    Alternatively has anyone successfully been using Torch inside Dota 2 Lua?? I am just hesitant to go down the this route as it seems it is a rather complicated setup on a windows machine (not even entirely sure if Dota 2 Lua can call Torch stuff due to the whole virtual machine thing? not entirely clear)

    The reason I need to sample from multivariate normal distributions is because I am now investigating continuous policies instead of discrete ones. The idea is based on what other researchers have done for continuous action space environments where a model will output parameters for multivariate distributions from which an action will be sampled.
    In my case the distribution will be 2D from which I can sample (x,y) which is the offset my hero will move
    i think pytorch is better choice lol, since you can't upload any NN lib to workshop. i think we should finish our bot on python side first than rewrite the network in pure lua. the performence of pure lua code should not be a problem.
    https://github.com/lenLRX/Dota2_DPPO_bots ----My ML bot work in progress

  5. #15
    How is the creep blocking learning progress going (I want to know how fast a NN can learn a simple technique)? In your last reply, it seemed like it's getting better!

  6. #16
    Basic Member axetion's Avatar
    Join Date
    Jan 2017
    Posts
    40
    Quote Originally Posted by BeyondGodlikeBot View Post
    Does anyone know of any Lua libraries which can do sampling from multivariate normal distributions?? I couldn't find anything via google searches.

    Alternatively has anyone successfully been using Torch inside Dota 2 Lua?? I am just hesitant to go down the this route as it seems it is a rather complicated setup on a windows machine (not even entirely sure if Dota 2 Lua can call Torch stuff due to the whole virtual machine thing? not entirely clear)

    The reason I need to sample from multivariate normal distributions is because I am now investigating continuous policies instead of discrete ones. The idea is based on what other researchers have done for continuous action space environments where a model will output parameters for multivariate distributions from which an action will be sampled.
    In my case the distribution will be 2D from which I can sample (x,y) which is the offset my hero will move
    My bots use Torch (see signature), but despite my best efforts you can't load Torch directly from the Dota 2 VM (without modifying the VM that is) because it relies heavily on native C extensions.
    Last edited by axetion; 08-23-2017 at 08:30 AM.

  7. #17
    Basic Member axetion's Avatar
    Join Date
    Jan 2017
    Posts
    40
    Quote Originally Posted by lenlrx View Post
    i think pytorch is better choice lol, since you can't upload any NN lib to workshop. i think we should finish our bot on python side first than rewrite the network in pure lua. the performence of pure lua code should not be a problem.
    Funny you should phrase it like that: LuaJIT is light years faster than CPython will ever be. I'd recommend just using pure Lua :P

  8. #18
    Ok guys. I have successfully trained my bot with a continuous policy


    I have provided an addon in my git which has the pre-trained network embedded inside. You don't need any webservice or anything to run that addon. It will just repeat the scenario over and over and you can see my bot at work.
    If you want to actually tinker with my model and do the training yourself, my addon and webservice stuff is also there.

    I have edited the first post in this thread to reflect what I have done.
    As I am going on holidays from today, I won't be able to take the next steps to start to extend this to 1v1 until I come back ~3 weeks later.
    Last edited by BeyondGodlikeBot; 08-24-2017 at 03:48 AM.

  9. #19
    Basic Member
    Join Date
    Dec 2016
    Posts
    731
    Nice work!

    I'm going to pull your code and see if I can extend it to a generic hero model by providing the hero movement speed, hero turn rate, and hero bounding box size as inputs and randomly selecting a hero model from all heroes when resetting an episode. It should take longer to learn, but the ultimate result should be a hero-agnostic creep blocking system (I hope).

    PS - anyone successfully install tensorflow on windows using Python 3.6? Supposedly as of release 1.2 (they are on 1.3 now) they support Python 3.6 but when I run
    Code:
    C:\Users\nostr>pip3.6 install --upgrade tensorflow
    Collecting tensorflow
      Could not find a version that satisfies the requirement tensorflow (from versions: )
    No matching distribution found for tensorflow

  10. #20
    Basic Member
    Join Date
    Dec 2016
    Posts
    46
    Quote Originally Posted by BeyondGodlikeBot View Post
    Ok guys. I have successfully trained my bot with a continuous policy


    I have provided an addon in my git which has the pre-trained network embedded inside. You don't need any webservice or anything to run that addon. It will just repeat the scenario over and over and you can see my bot at work.
    If you want to actually tinker with my model and do the training yourself, my addon and webservice stuff is also there.

    I have edited the first post in this thread to reflect what I have done.
    As I am going on holidays from today, I won't be able to take the next steps to start to extend this to 1v1 until I come back ~3 weeks later.
    Congratulations, it looks good

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •