Register
  • Calling All Bloggers - Help TUJ!

    As you may know, The Undermine Journal has closed due to changes in Blizzard's remote AH infrastructure. TUJ is one of the most powerful tools I've ever seen as an entrepreneur, and it saddens me that such a great site is closing down. Here's a quote from Erorus, the developer, explaining the problems TUJ is facing:

    Quote Originally Posted by Erorus
    Over the past 48 hours, the wowarmory.com site has not been cooperative. There are more error pages than usual, so many that my scan completion rate is around 60% when just last week I had a 98% completion rate. I have one idea that may help, but that's only if the problem is where I think it is on Blizzard's end, and it might not be. Still, I'll probably tinker with the crawler a little bit tomorrow to see if it helps.

    Then I notice that Blizzard's shiny new battle.net auction interface is up. "Woo, maybe I can use that." Doesn't look likely. It's no longer presented in XML, so I would have to use horrid regular expressions against the html to pull data out. Ugly, but still workable. However, that whole "search beyond 200 results" thing I've been using doesn't appear to work there. Search beyond 200 results and it just returns none. Trying to narrow down by category won't help, since it takes so much longer, and even if you go down to the item level, it's possible that there are more than 200 auctions for a given item.

    I went so far as to look at the packets going between the mobile armory app on my android phone and their servers, hoping it's using XML and HTTP somewhere. No such luck; looks like proprietary protocols that would take me a very long time to reverse-engineer.

    So, someday soon, wowarmory.com will no longer present its current interface to the auction house, since Blizz wants people to use the shiny new battle.net one they've developed. And I won't be able to pull auction data after that. I've emailed them a few months ago and haven't heard squat. They probably don't know this exists. And I have no good way to get in contact with them.

    Um, panic? If you want The Undermine Journal to stick around, you might want to get Blizzard's attention and have them contact me, or start hoping they'll have some bot-friendly interface for some reason.

    All this, just in time for me to buy the last accounts I need for full US realm support... fantastic.
    Further details can be found in this thread. Erorus has attempted to contact Blizzard about the problems but was unsuccessful. What we need to do is band together, and collectively ask Blizzard to clarify their position. I'm kindly asking all bloggers out there to help me and get the word out to the public. I've started a thread on the public forums found here. Keep in mind, this thread is not a petition, as these types of threads are generally closed quickly.

    So go ahead and voice your opinion in the general forum thread. Who knows, maybe we can bring back TUJ!

    Sterling beaming out!
    Comments 15 Comments
    1. jchurch5566's Avatar
      jchurch5566 -
      Hi guys,

      This is sad news. I discovered The Undermine Journal (TUJ) about 4-5 weeks ago. It very quickly became my #1 goto stop for pricing. I am not sure how we can help, but I would very much like to see TUJ alive and well again.
    1. Stede's Avatar
      Stede -
      Got my latest blog post out there on this. To be honest, I was content to stay out of it until I saw the thread-crapping going on. I'm all-in now, Sterling - thanks for leading the charge here.

      Lots of flames in that thread...
    1. calianna's Avatar
      calianna -
      Lots of common morons in that thread. The earlier response, to the effect of "I don't know what this is so it's stupid", was pretty much spot on.
    1. Aeg's Avatar
      Aeg -
      best thing we can do is respond to the morons with good answers which will keep that thread active. Especially with the holidays we do not want that falling off the first page. Also posts with multiple pages I would assume has a better chance at being looked at by a dev.
    1. Zerohour's Avatar
      Zerohour -
      It's thread responses like that that make me wish they had gone through with the Real ID Reality idea. Pathetic.

      Anyhow, while I have hope for it I don't have my hopes up. They may be working on the accessibility of the Armory but who knows. With Blizzard they tend to prioritize their projects like anyone, and to pull resources off of things in motion to satisfy a smaller percentage of the player base may not be in the cards. I was looking forward to when he was going to make subscription based features because I used that site more than WoWhead at one point. Such a sad day for entrepreneurs everywhere.
    1. drpachinko's Avatar
      drpachinko -
      I posted some technical jargon that's less than true but hopefully they will recognize that being SEO optimized and easily crawl-able will bring revenue passively via stumblers. etc...

      I'm going to wait a few hours my next post from a different account will be about the psychology of quiting wow and how these sub sites re attract people to playing.

      Some what weak arguments but will hopefully give the thread more credibility.
    1. calianna's Avatar
      calianna -
      Quote Originally Posted by Aeg View Post
      best thing we can do is respond to the morons with good answers which will keep that thread active. Especially with the holidays we do not want that falling off the first page. Also posts with multiple pages I would assume has a better chance at being looked at by a dev.
      I think Aeg just pegged something here. Having the morons posting on that thread is not a bad thing at all. They are just providing free bumps and making sure that even more people see it. We continue to provide positive feedback, they are more than welcome to provide their stupidity.
    1. calianna's Avatar
      calianna -
      I just noticed the thread got moved to the Website Bug Reports forum.
    1. usiris's Avatar
      usiris -
      Im much more active on the wow forums, and will be pretty active in the above thread, but im not that great at writing, so if i say something that you guys dont agree with or is wrong, please correct me

      thank you
    1. Undine's Avatar
      Undine -
      You want to get a lot of support in that thread, offer free fortune cards on your server /2 to all characters who post in that thread and bump the UMJ with positive replies. It'd be pretty easy to verify if a character posted or not, you could also weed out abusers by setting an eligibility of chars level 10+ . The popularity of these cards can benefit the AH community far greater than just making a quick buck on something that will be mediocre in value in due time.

      Just my thoughts.
    1. Aeg's Avatar
      Aeg -
      Awesome idea Undine. I plan on having a bunch of cookies/cards for the holiday weekend. I will give some out for free for this.
    1. Bigjimm's Avatar
      Bigjimm -
      Well, P3P is pitching in. http://phase3profit.net/2010/12/plea...rmine-journal/
    1. Sterling's Avatar
      Sterling -
      Update: Erorus suspects that the high error % returned is due to the volume of requests per account, meaning he would have to grab 1 account per realm in order to have 100% success rate. I'll just quote him instead of making myself sound technically inept:

      "This evening I noticed something interesting. All 16 realms were now using Account 10 instead of 1-6, and they were getting errors back from the armory once again! Not so many as to stop scans from completing, but while some scans had 0% errors, about half of the scans were getting errors from the armory for 20%-40% of requests. Nothing else changed in the setup.

      What this means is that I probably wasn't throttled by requesting IP, connections or anything like that.. but I was probably throttled by the number of simultaneous requests on a given account!

      I can only speculate why multiple characters on the same account being scanned at the same time would cause problems. I mean, logically, logins wouldn't be shared by multiple people, so 99% of players wouldn't run into issues. But technically, is the server caching results per realm, and then getting bothered when one account requests a completely different page on a completely different realm every time? Or maybe it's an artificial limitation to block crawlers across multiple realms?

      Whatever the reason, it seems that buying more crawler VPSes (for more requesting IPs) wouldn't have solved our error problems. It looks like I shouldn't fill up WoW accounts with 50 realms each, and I'd have to buy more WoW accounts at $14/mo to avoid errors. Yeah right.

      Let's hope that the future battle.net xml data exports can return the whole auction house at once, so I won't have to spend so much scanning time per account. "


      Crossfingered.
    1. Thomus's Avatar
      Thomus -
      Quote Originally Posted by Sterling View Post
      Update: Erorus suspects that the high error % returned is due to the volume of requests per account, meaning he would have to grab 1 account per realm in order to have 100% success rate. I'll just quote him instead of making myself sound technically inept:

      "Whatever the reason, it seems that buying more crawler VPSes (for more requesting IPs) wouldn't have solved our error problems. It looks like I shouldn't fill up WoW accounts with 50 realms each, and I'd have to buy more WoW accounts at $14/mo to avoid errors. Yeah right."

      "Let's hope that the future battle.net xml data exports can return the whole auction house at once, so I won't have to spend so much scanning time per account. "
      Crossfingered.
      If an XML fix cannot be found, I may be willing to convert one of my accounts to an info gatherer, or donate to your account cost.
    1. Mageshadow's Avatar
      Mageshadow -
      Making some progress at least! Best of luck Erorus