Register
Results 1 to 4 of 4
  1. #1

    Undermine Journal API issue via Web Application

    Hi folks,

    I built a cross faction/server arbitrage page (yep, I know there's one coming, but it was fun and I like to have complete control)...

    It works wonderfully if I grab a Market Data XML URL, paste it into the download window of a browser to manually download it, repost the XML file to a hosting service and paste THAT URL into my Arbitrage web app. The apps loads the data into a SQL database and lets me analyze it.

    However, if I paste the Market Data XML URL I get from UJ straight into my web app, the download retrieves no data. One can see Status Code 200/OK, but the file content shows Connection Failure.

    Is there any logic on the UJ side to block automated data requests, even while enforcing a 5 DL/day limit? I use ColdFusion and it even has a user agent property I can set to emulate a browser.

    Pardon the formatting, but here is the data structure I receive:


    struct
    Charset [empty string]
    ErrorDetail [empty string]
    Filecontent Connection Failure
    Header HTTP/1.1 200 OK Connection: close Expires: Tue, 19 Apr 2011 20:47:17 +0000 Date: Tue, 19 Apr 2011 15:59:21 GMT Server: lighttpd/1.4.26 X-Powered-By: PHP/5.3.4 Vary: Accept-Encoding ETag: 459-1303209282 Content-Type: text/xml
    Mimetype text/xmL

    Response Header:
    Connection close
    Content-Type text/xml
    Date Tue, 19 Apr 2011 15:59:21 GMT
    ETag 459-1303209282
    Expires Tue, 19 Apr 2011 20:47:17 +0000
    Explanation OK
    Http_Version HTTP/1.1
    Server lighttpd/1.4.26
    Status_Code 200
    Vary Accept-Encoding
    X-Powered-By PHP/5.3.4

    Statuscode 200 OK
    Text YES

  2. #2
    Erorus's Avatar
    Posts
    1,404
    Reputation
    114
    Tagged in
    677 Posts
    Add to this user's reputation
    Quote Originally Posted by grayforge View Post
    Hi folks,

    I built a cross faction/server arbitrage page (yep, I know there's one coming, but it was fun and I like to have complete control)...
    No problem there.

    Quote Originally Posted by grayforge View Post
    It works wonderfully if I grab a Market Data XML URL, paste it into the download window of a browser to manually download it, repost the XML file to a hosting service and paste THAT URL into my Arbitrage web app. The apps loads the data into a SQL database and lets me analyze it.

    However, if I paste the Market Data XML URL I get from UJ straight into my web app, the download retrieves no data. One can see Status Code 200/OK, but the file content shows Connection Failure.

    Is there any logic on the UJ side to block automated data requests, even while enforcing a 5 DL/day limit? I use ColdFusion and it even has a user agent property I can set to emulate a browser.
    There is no restriction in place to block automated requests (and actually that would be kinda missing the point if that were the case..). For your request that you sent at 2011-04-19 15:59:21, I see that TUJ returned 261748 bytes after 3.986 seconds. Perhaps you have a short timeout on the HTTP GET?

    Also, at 261k, that response is probably gzipped. It's supposed to return gzipped content only when requested with a "Accept-Encoding: gzip" request header. Maybe you're getting gzipped data and then not unzipping it?
    Last edited by Erorus; April 19th, 2011 at 12:54 PM.

  3. #3
    Thanks for the quick response!

    Very interesting. I just zipped a previously downloaded version of that XML doc. Sure enough, the file size is very close to 261k.

    I'll look at the call to see if there's a way I can force a non zipped version and to extend the timeout.

    Thanks!

  4. #4
    Erorus, that was it! Thank you so much for the debugging info from your end.

    It seems ColdFusion was not able to decompress the content, so claimed a connection failure. I added a couple parameters to the request header to ask politely that the server not compress the data and lo and behold, it works. :-)

    Thanks again!

 

 

Similar Threads

  1. Replacement for The Undermine Journal
    By guberry in forum Archive (Auction House)
    Replies: 1
    Last Post: January 19th, 2011, 11:48 AM