Page 3 of 4 FirstFirst 1 2 3 4 LastLast
Results 21 to 30 of 39

Thread: pydota2

  1. #21
    Basic Member aveyo's Avatar
    Join Date
    Aug 2012
    Location
    EU West
    Posts
    2,927
    That's amazing progress in itself!

    As for the dumb thing, can't you salvage any bit of code from Starcraft AI? Does it not feature the same nav grid - based "steering" like Dota does?

    And would not dota_gridnav_show / dota_bot_debug_grid + dota_bot_debug_grid_cycle (Bad Guys Avoidance / Passability etc.) help a lot with the movement states? That is, if it's usable by your cold-hearted, non-human-like agent ;p

  2. #22
    Basic Member
    Join Date
    Dec 2016
    Posts
    732
    Okay, I did some test. It does work although I had to tweak the reward_decay (aka discount factor) and the learning rate to be more appropriate to the problem at heart. With discount factor of 0 (meaning we only care about the immediate reward, not the long term reward - aka - we care about just the last action, not a sequence of actions) and a learning rate of 0.5 (this determines to what extend "new" information about optimal state->action over-rides "old" information) we get to the middle in about 1 min of learning across the 5 agents.

    Now, I still believe we have some possible bugs (which might be related to bugs in the "facing" provided by the game client not being accurate) and perhaps the fact that I'm NOT including X,Y,Z coordinate information might be making my system confused. Who knows.... I'm still learning.

  3. #23
    Basic Member
    Join Date
    Dec 2016
    Posts
    732
    Also, regarding the grid navigation. Sure... it would help and be useful, but it would explode my state space. Because now for every state which would be (X,Y,Z cell block on map) I would have possibly the 12 actions I can take. Rather than just 360 states I currently have (1 for each degree of a heading since I convert to an Integer), I would have X number of "cell" on the grid of some "shape" (not sure if the grid used by Dota2 is a square, a hexagon, an octagon, a dodecagon, etc.). Not sure how big a "cell" is either in terms of units.

    Dota world is approx 14,000 units wide and 14,000 units tall. If I were to use say "400 units" as the standard diameter of a cell I would have 1,225 states (14,000/400 * 14,000/400).

  4. #24
    Basic Member aveyo's Avatar
    Join Date
    Aug 2012
    Location
    EU West
    Posts
    2,927
    But the grid is always centred on your screen view - your immediate surroundings, taking account of fog of war & etc. you should not care about the whole map?

    Damn, just looked at the minigames - movetobeacon & etc. and I only see the pre-compiled maps, where are the ai agents? wtf not very helpful, it looks more like a setback.

  5. #25
    Basic Member
    Join Date
    Dec 2016
    Posts
    732
    Is there a way to dump that grid to a JSON file with X,Y,Z coordinates, bIsPassable, bHasTrees, info?

  6. #26
    Basic Member aveyo's Avatar
    Join Date
    Aug 2012
    Location
    EU West
    Posts
    2,927
    ChrisC must help, as nav_ stuff is broken and I don't see any other way of exporting it. No idea what format would dota.nav be in, but looking at dota.gnv after decompiling the map, it looks parse-able

  7. #27
    Basic Member
    Join Date
    Dec 2016
    Posts
    732
    I mean, I can do it myself by hand at some granularity and then train the pathing ML algorithm on my map. Say 200 unit per square so.. 70 x 70 grid. Then label all non-allowed edge transitions and rest are allowed per 8 directional exits.

  8. #28
    Basic Member aveyo's Avatar
    Join Date
    Aug 2012
    Location
    EU West
    Posts
    2,927
    It should have been provided in a parse-able format, maybe even in the protobuff stuff, since it's predefined anyway for each map. Re-creating the grid, waypoints, slicing it and etc. is a massive waste of dev time and cpu cycles that would have to be repeated for every map, instead of having access to the built-in grid with all the goodies in real time and just cherry-pick states while making use of default path-finding optimizations.

    Blizzard made tons of adjustments on their part to ease up the integration, they've translated most of their "visual" layers. It should be easier for Valve to make adjustments for ml since it's already data-driven. But until then, you're pretty much a one man show, doing everything from the ground up, and I don't see many of us script kiddies following in your footsteps atm. pySC2 being so incomplete also does not help.

    For now I'm just looking through the older, but more complete broodwar bots projects (even found a python cross-over:cybw - not very scavenge-able, it's more of a study material). Lots of extensions and tools (bwem is heavily used - that's the kind of info that the Dota bot api should provide for headless ml), lots of opensource bots available to learn some tricks and algorithms even if most are in C.

  9. #29
    Basic Member
    Join Date
    Dec 2016
    Posts
    76
    On the C++ side:
    There is a high probability that we will need a Dota2 simulator/emulator like lenlrx created to simply be able to train faster in a headless & parallelized manner. To do this, we would circumvent the HTTP POST method to instead talk to the simulator and then simulate the environment step() forward.

    I am glad to help.
    But my simulator can support very limit features of Dota2, since time is limited.
    So hard part is to define flexible low level API or middle layer.
    And CI is required.
    https://github.com/lenLRX/Dota2_DPPO_bots ----My ML bot work in progress

  10. #30
    Basic Member
    Join Date
    Dec 2016
    Posts
    732
    @aveyo - pySC2 being incomplete is because they do not intend on sharing their actual RL agent code (claiming that it is too hardware specific to their own implementations of TPU (Tensor Processing Units) and thus would not be useful to anyone). I'm going to do a quick test later today and see what happens when I encode generalized X,Y coordinates (meaning I call a "cell" a 200 x 200 unit box) into the movement agent and see how it performs. Seems like I'm looking at about 156,000 states in my full model.. which isn't that much honestly since I don't have to exhaustively visit them all to illustrate the principle or have a working "Radiant" movement bot (as it will never need to visit 3/4th of the map to achieve it's current mission). Now that I'm providing X,Y coordinates the bots can figure out "allowed" pathing based on rewards acquired for each cell and motion taken based on direction attempted.

    @lenlrx - I would love to have your help and I understand that "simulating" a complex environment can be a lossy and incomplete process. I'm wondering how we can best benefit from abstracted simulation for various model training. For example - to simulate a "last-hitting & denying" model we don't need to really simulate much outside of a basic concept of a creep, basic movement and basic attack/damage rules.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •