Register
Results 1 to 6 of 6
  1. #1

    [Coding] Parsing the AH using the New Armory/Scripting Questions

    Hi,

    I run my own little parser and AH script -- nothing of the scale of TUJ, but I was wondering if you would be willing to share some of the tips and tricks you've come across for parsing the AH.

    1. Do you know how to switch characters using the new armory? (It seems like it calls a POST request to http://us.battle.net/wow/en/pref/character but when I try to do that, I get a 404) If you still use the old JSON interface and haven't run into this issue, that's fair.

    2. How long does it take you to scan the entire AH for a given realm? It takes me about 10 minutes, and I've read posts mentioning sub-minute "parse times" but I'm not sure what you're referring to by that.

    Thank you -- I love the Undermine Journal and look forward to seeing it grow.

  2. #2
    Maybe I can help: I run my own scans on my EU realm atm: eu.wowarmory and it takes about 2-3min but I am not properly using keep-alive and not multithreading (i.e. it is as of now: login, and as long as there are more pages: fetch page, parse-json, execute sql insert)

    And afaik the new armory does not output JSON/XML for browsing, has that changes? do you really parse that ugly html?

  3. #3
    I'm on Mal'Ganis/Horde, one of the largest AHs -- 60,000 auctions isn't uncommon. I get 200 at at time.

    I'm using JSON for everything, but there's no JSON method to see active auctions listed by the logged in character, so I figured I'd try the XHTML -- it isn't too bad to parse using XPath, but the problem is it seems to use whatever character I log into the armory (manually) with and not what I'm logged in normally.

  4. #4
    Athkatla's Avatar
    Posts
    88
    Reputation
    5
    Tagged in
    0 Posts
    Add to this user's reputation
    If I recall correctly from the dev blog, TUJ uses the XML methods on wowarmory rather than the JSON - I believe XML gets compressed and JSON doesn't, so bandwidth usage on the scanner machines is lower.

  5. #5
    For me JSON is easier to parse and I don't have bandwidth problems at the moment. But one should really just use whatever is easier to parse for him/her.

  6. #6
    Erorus's Avatar
    Posts
    1,404
    Reputation
    114
    Tagged in
    677 Posts
    Add to this user's reputation
    I'm not running anything on the battle.net armory, so I don't know offhand how to change characters there. I still use the old wowarmory.com feeds. I've never had to "switch characters" using wowarmory.com; every request gets sent with the character, faction and server, so I never bothered.

    Takes about 3-4 minutes to scan a realm (which means gather all the info from the Armory into a file) and now under 5 seconds to parse it (update old auctions and insert new auctions into the database).

    I've always used XML for The Undermine Journal because wowarmory.com can gzip the result. It won't gzip JSON. The gzipped XML is about 25% the size of comparable JSON. I'm also more comfortable with parsing XML, so it's win/win.

 

 

Similar Threads

  1. A couple of TSM questions
    By Dane in forum General TradeSkillMaster Discussion
    Replies: 2
    Last Post: March 10th, 2011, 11:51 PM
  2. some crafting questions
    By Duco in forum General TradeSkillMaster Discussion
    Replies: 6
    Last Post: March 10th, 2011, 01:06 PM
  3. Mobile Armory - Remote Auction House
    By Allaphon in forum Archive (Auction House)
    Replies: 28
    Last Post: November 16th, 2010, 06:16 PM
  4. Mobile Armory - Remote Auction House Review
    By Allaphon in forum Archive (Addons and Macros)
    Replies: 10
    Last Post: November 12th, 2010, 02:02 PM
  5. Addon Questions
    By Moravec in forum Archive (Addons and Macros)
    Replies: 5
    Last Post: October 1st, 2010, 09:12 AM