Announcement

Collapse

Forum Rules

  • No flaming or derogatory remarks, directly or through insinuation.
  • No discussion, sharing or referencing illegal software such as hacks, keygen, cracks and pirated software.
  • No offensive contents, including but not limited to, racism, gore or pornography.
  • No excessive spam/meme, i.e. copious one liners in a short period of time, typing with all caps or posting meme responses (text/image).
  • No trolling, including but not limited to, flame incitation, user provocation or false information distribution.
  • No link spamming or signature advertisements for content not specific to Dota 2.
  • No Dota 2 key requests, sell, trade etc.
  • You may not create multiple accounts for any purpose, including ban evasion, unless expressly permitted by a moderator.

  • Please search before posting. One thread per issue. Do not create another thread if there is an existing one already.
  • Before posting anything, make sure you check out all sticky threads (e.g., this). Do not create new threads about closed ones.
  • It is extremely important that you post in correct forum section.

  • Balance discussion only in Misc.
  • All art related (such as hero model) feedbacks go to Art Feedback Forum.
  • All matchmaking feedback should go here: Matchmaking Feedback
  • All report/low priority issues should go here: Commend/Report/Ban Feedback
  • No specific workshop item feedback. These should go to workshop page of that item.
  • When posting in non-bugs section (such as this), use [Bugs], [Discussion] or [Suggestion] prefix in your thread name.



In case you object some action by a moderator, please contact him directly through PM and explain your concerns politely. If you are still unable to resolve the issue, contact an administrator. Do not drag these issues in public.



All rules are meant to augment common sense, please use them when not conflicted with aforementioned policies.
See more
See less

Dota 2 Demo File Format

Collapse
This is a sticky topic.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by onethirtyfive View Post
    So when Valve released their 'demoinfo' tool last year they provided with it a protocol specification. Here's an example of what we get for free, in the form of a parsed replay. (Warning: HUGE)

    https://gist.github.com/onethirtyfiv.../gistfile1.txt

    The contents of this file are similar to what Valve's own demoinfo.exe dumps, because the protocol definitions are the same. If you look at it, you'll see a lot of "classes" defined at the beginning, followed by a list of packet messages. These packet messages in turn contain "service messages", the types of which I've listed for each packet. (ex. "packet entities")

    The problem is, the "packet entities" messages contain a "data" field with no specification. So the task is to find the relationship between those tables at the beginning of the replay and the contents of these "packet entities" datablobs.

    papa_pointnet might have some information, if he's feeling generous?

    The parser I'm hacking together is publicly available at https://github.com/onethirtyfive/skadi
    Hmm... okay. I've been trying to get location data from what is available in the current parser using the locations the parser gives and some logic. By logic, I mean if a hero picks up the Aegis or attacks a tower, I now know that hero's location. You can then bootstrap that info using the combat log. If that hero interacts with another hero in the combat log in some non-global way, within a certain a certain time frame (I've been using 2.5 seconds) the other hero's location is now known (not precisely but the general area).

    The current parser also gives a bunch of locations from particle messages. For example, all teleport start and end locations are given and like 90% of the locations where heroes die are given. Depending on the match I can get a decent amount of data points on hero locations. In the two 30-35 minute games I've gotten the full data from there were 2,500 location points for the first game and 4,000 location points for the 2nd one. Unfortunately, the location points tend to clump up around certain heroes and game times so the location points I've found are not as useful as I'd hoped. Here's a crappy applet I came up with to illustrate what I've got so far with location data: http://www.datdota.com/s_recap.php?q=131163610

    Anyway, if any of that is useful to the people trying to find the complete location data and I can assist in anyway feel free to ask me any questions or pm me. Getting complete location data would be awesome for heat maps, map control analysis, vision maps, etc.
    datdota.com -- Dota 2 Stats for the Professional Scene

    Comment


    • Anyone been have trouble pulling the match ID and the league ID from the parser recently? For the past few days, I keep getting league ID: 0 and match id: 0 instead of the actual league and match ids.
      datdota.com -- Dota 2 Stats for the Professional Scene

      Comment


      • Ok i'm kinda new to this replay thing... i was wondering if there was a "ending result" thing in the current replay stuff..
        simply put
        1. analyze a replay file to get the stats (winner, kill, deaths.. etc..)
        but when i look at the output from the demoinfo.exe it seems like i have to manually count them.. is this how we are supposed to do this? or am i missing out on something?

        Comment


        • Has anyone found a way to track which items are given out as tournament prizes (like "spectator X has received an item for witnessing ...")? I see some scammers and possible scammers trying to sell items based on them allegedly being tournament drops. I'd like to start tracking what items drops in tournament games in order to hopefully cut down on this but am not sure if it's possible.

          Originally posted by thinkong View Post
          Ok i'm kinda new to this replay thing... i was wondering if there was a "ending result" thing in the current replay stuff..
          simply put
          1. analyze a replay file to get the stats (winner, kill, deaths.. etc..)
          but when i look at the output from the demoinfo.exe it seems like i have to manually count them.. is this how we are supposed to do this? or am i missing out on something?
          I would start by reading through this entire thread and checking out these two links:
          http://www.cyborgmatt.com/2013/01/do...-parser-bruno/
          http://dev.dota2.com/showthread.php?...=parsing+parse
          datdota.com -- Dota 2 Stats for the Professional Scene

          Comment


          • Originally posted by Razumov View Post
            I'd like to start tracking what items drops in tournament games in order to hopefully cut down on this but am not sure if it's possible.
            For sure you can know who droped something but I'm not sure you can have the item_id somewhere. You need to take a look at GameEventList.dota_tournament_item_event.

            Comment


            • Originally posted by papa_pointnet View Post
              For sure you can know who droped something but I'm not sure you can have the item_id somewhere. You need to take a look at GameEventList.dota_tournament_item_event.
              Thanks, I'll check it out. I know there are messages for when someone gets an item and the user name who receives an item in a message like this:

              dota_tournament_item_event eventid:172
              winner_name: HellChicken
              event_type: 2

              And this list may be useful (or it might just be the items the players themselves have), though I'm not sure how to decipher it:

              #3 EconItems flags:0x1 (49 Items) 1654 bytes
              #0 '388385800' (48 bytes)
              #1 '388385803' (48 bytes)
              #2 '388385798' (48 bytes)
              #3 '388385799' (48 bytes)
              #4 '363311967' (52 bytes)
              #5 'd330' (0 bytes)
              #6 'd327' (0 bytes)
              #7 'd326' (0 bytes)
              #8 'd329' (0 bytes)
              #9 'd328' (0 bytes)
              #10 '141248623' (40 bytes)
              #11 '374501101' (41 bytes)
              #12 'd9' (0 bytes)
              #13 '374522906' (41 bytes)
              #14 '374501103' (41 bytes)
              #15 '142529018' (46 bytes)
              #16 'd375' (0 bytes)
              #17 'd376' (0 bytes)
              #18 'd377' (0 bytes)
              #19 '379298401' (51 bytes)
              #20 'd107' (0 bytes)
              #21 '372296465' (51 bytes)
              #22 '372296458' (51 bytes)
              #23 '372296462' (51 bytes)
              #24 '372296474' (51 bytes)
              #25 '247784029' (50 bytes)
              #26 '247784015' (50 bytes)
              #27 'd396' (0 bytes)
              #28 'd398' (0 bytes)
              #29 'd397' (0 bytes)
              #30 'd399' (0 bytes)
              #31 'd26' (0 bytes)
              #32 'd84' (0 bytes)
              #33 'd82' (0 bytes)
              #34 'd83' (0 bytes)
              #35 '373547484' (49 bytes)
              #36 '373547031' (49 bytes)
              #37 '329764984' (48 bytes)
              #38 '323506890' (52 bytes)
              #39 '337085459' (42 bytes)
              #40 '226629212' (46 bytes)
              #41 'd203' (0 bytes)
              #42 'd205' (0 bytes)
              #43 'd204' (0 bytes)
              #44 'd419' (0 bytes)
              #45 'd108' (0 bytes)
              #46 'd109' (0 bytes)
              #47 'd110' (0 bytes)
              #48 'd219' (0 bytes)
              datdota.com -- Dota 2 Stats for the Professional Scene

              Comment


              • I may as well cross post my thread in here since seeing it fall of the front page makes me sad : )

                I finished my packet entity replay parsing tool! It's super rough and terrible but hopefully y'all will get some use out of it anyways.

                As a test to see if it works I ended up plotting the positions of deaths from SL2 Na`Vi vs mouz game 1. I wanted to do a few more but I'm super tired of this project, I'd like to see talented people make awesome stuff with this.



                Circles are mouz dying and squares are Na`Vi dying. Green corresponds to early in the game and red to later in the game.

                Comment


                • Well done, I just got it compiling on Ubuntu, I can send a pull request. What does the executable actually generate though?

                  Comment


                  • Originally posted by pork View Post
                    I may as well cross post my thread in here since seeing it fall of the front page makes me sad : )

                    I finished my packet entity replay parsing tool! It's super rough and terrible but hopefully y'all will get some use out of it anyways.

                    As a test to see if it works I ended up plotting the positions of deaths from SL2 Na`Vi vs mouz game 1. I wanted to do a few more but I'm super tired of this project, I'd like to see talented people make awesome stuff with this.
                    ...
                    Circles are mouz dying and squares are Na`Vi dying. Green corresponds to early in the game and red to later in the game.
                    This is really awesome! Great job, I know how much work was put into getting this to work! Going to spend some time playing around with it tonight. Thank you!
                    Last edited by mugsy__; 05-14-2013, 07:36 PM.

                    Comment


                    • Originally posted by hackcasual View Post
                      Well done, I just got it compiling on Ubuntu, I can send a pull request. What does the executable actually generate though?
                      Oh sweet! I figured that there were a billion platform specific things I accidentally wrote. I don't understand Github but pull requests sound great.

                      So since I just wanted results I was lazy and everything it prints comes from: https://github.com/dschleck/edith/bl...ding_visitor.h . This is the code I used to get data for that picture. It's really shitty design, but that object gets created here: https://github.com/dschleck/edith/bl...c/main.cpp#L26 . You can search for "visitor->" and see where it's called.

                      Originally posted by mugsy__ View Post
                      This is really awesome! Great job, I know how much work was put into getting this to work! Going to spend some time playing around with it tonight. Thank you!
                      Thanks! Let me know if you find anything that looks off, I'm still suspicious of everything but I seemed to be getting correct data.
                      Last edited by pork; 05-14-2013, 10:17 PM.

                      Comment


                      • Originally posted by pork View Post
                        Oh sweet! I figured that there were a billion platform specific things I accidentally wrote. I don't understand Github but pull requests sound great.

                        So since I just wanted results I was lazy and everything it prints comes from: https://github.com/dschleck/edith/bl...ding_visitor.h . This is the code I used to get data for that picture. It's really shitty design, but that object gets created here: https://github.com/dschleck/edith/bl...c/main.cpp#L26 . You can search for "visitor->" and see where it's called.
                        No worries dude, only needed to add a couple of cmake flags, include <algorithm> in one file, and be explicit with the 128 bit conversions in properties. I figured out the issue, you're masking off the hero id's with 0x3ff, but I found the ids in my game where above that, and used 0x7ff. If that makes sense let me know.

                        After that I think I'll try to get python visitors working. Really well designed interface.

                        Comment


                        • Originally posted by hackcasual View Post
                          No worries dude, only needed to add a couple of cmake flags, include <algorithm> in one file, and be explicit with the 128 bit conversions in properties. I figured out the issue, you're masking off the hero id's with 0x3ff, but I found the ids in my game where above that, and used 0x7ff. If that makes sense let me know.

                          After that I think I'll try to get python visitors working. Really well designed interface.
                          Aha, good call on 0x7FF. That corresponds to MAX_EDICTS (see https://developer.valvesoftware.com/wiki/Entity_limit ). Please send me a pull request for all of that!

                          Comment


                          • @pork

                            Good job, man!

                            Comment


                            • I'm pulling a blank, but does anyone know how to get the DTI sendtable summary from the game client? I feel like (with DTI enabled) it used to just appear when the game loaded, but I can't get it to print now.

                              Comment


                              • Originally posted by onethirtyfive View Post
                                I'm pulling a blank, but does anyone know how to get the DTI sendtable summary from the game client? I feel like (with DTI enabled) it used to just appear when the game loaded, but I can't get it to print now.
                                Don't you just have -dti as a launch option and then it automatically saves files in steam/steamapps/common/dota 2 beta/dota 2 or whatever? Maybe you can use dti_flush too.

                                Comment

                                Working...
                                X