Register
Results 1 to 6 of 6
  1. #1

    [Addon Suggestion] Average/StdDev Price, etc.

    I saw that addon that's been developed and I am ecstatic to see where this is going. I love The Undermine Journal and use it almost daily. Of course, the most laborious part of this process is having to switch programs and (re)load the various items to gather more information about the market I'm looking into. The prospect of having this information in-game at my fingertips is mind boggling! I do however have one suggestion to throw in, and that would be to add more relevant information to the tooltip. Right now it tells me the market price, items available, and how many auctions compose that availability. I'd like to see more over-time information; especially in regards to market price.

    Including Average price and possibly even the Confidence Interval would be much more useful than market price, in my opinion. There are a multitude of methods to find out what the lowest price of an item is right this second in-game, but far fewer to tell me what the average has been over the past two weeks and how the the current price compares to this. Even the standard deviation would give me an idea of how erratic the price is. Of course, I doubt you're going to put a graph right there in my tooltip. ;)

    TL;DR

    What it comes down to is I would like to see the Average item price added to the tooltips, if nothing else. I would personally enjoy seeing the confidence interval and standard deviation included as well. Change since yesterday for major bonus points!
    Last edited by DarthNerdus; February 23rd, 2011 at 01:54 PM. Reason: Should have included suggestion the title.

  2. #2
    Erorus's Avatar
    Posts
    1,404
    Reputation
    114
    Tagged in
    677 Posts
    Add to this user's reputation
    Avg/StdDev makes sense. It would be useful to see in the tooltip, I agree.

    Calculating Avg/StdDev for all items at once is very intensive. Basically that would be asking to examine every auction in that house for the past two weeks to generate those statistics. I can do it on the single item pages b/c I'm looking at all those auctions for that item anyway to generate the graphs. Current market price across the whole house is easy (what's in the addon today) since that's cached and it doesn't rely on past data.

    I may have improved capabilities when the SSD comes in and gets installed tomorrow. I can also work on slightly fuzzier statistics, like calculating the market value for each item for the entire day, then doing avg/stddev on 14 days' worth of cached single values instead of looking over 336 hours.

    Then I come back to how this addon doesn't play nice with TradeSkillMaster, and since I'm no Lua guru, I'm not particularly inclined to figure out why. So any modification to the addon is probably a back-burner sort of thing.

    In any case, thanks for the feedback, and I'll keep it in mind for the next version.

  3. #3
    I can see how gathering such data would be an issue with your current model. Personally, I would be fine seeing the market value only being updated once a day or so. In my opinion the use of this addon is you update it daily/bi-daily/weekly, it will never replace the actual site, and updating it constantly throughout the day (which is what having the current market price would push you to do) would be counter-intuitive. Updating the addon daily so that I have consistent (even if slightly fuzzy) average price and StdDev/etc. information available in the tooltip would be exactly what I'm looking for. Of course, if this can't happen then I'll just have to keep my insane number of TUJ tabs open in Chrome.

    Note: I didn't even realize that you calculated everything via hourly increments over the past two weeks. That's insane accuracy! That makes me appreciate the data you provide even more!

  4. #4
    jam
    jam's Avatar

    Computing STDDEV without reading all history

    Quote Originally Posted by Erorus View Post
    Avg/StdDev makes sense. It would be useful to see in the tooltip, I agree.

    Calculating Avg/StdDev for all items at once is very intensive. Basically that would be asking to examine every auction in that house for the past two weeks to generate those statistics. I can do it on the single item pages b/c I'm looking at all those auctions for that item anyway to generate the graphs. Current market price across the whole house is easy (what's in the addon today) since that's cached and it doesn't rely on past data.

    ...
    Did you know that you can compute stddev using a running total, rather than across the active data? Specifically, you have to track sum(X) and sum(X^2) and of course N. The normal equation for stddev is:

    stddev = sqrt(sum((x-avg)^2) / N-1)

    However, through some nice math properties, you can get

    stddev = sqrt(1/N sum(X^2) - (avg)^2)

    You can see this http://en.wikipedia.org/wiki/Standard_deviation
    The math solving it is here: http://en.wikipedia.org/wiki/Computa...r_the_variance
    (though you have to add a sqrt to go from Variance to Stddev).

    Anyway, that means that when you poll for new items, you log the value, and then add value^2 to an accumulator. When you delete columns, you just delete value^2 from the accumulator. Then when you want to display, you don't have to compute the sum of squares on the fly.

    Only thing you have to watch out for is sum(X^2) overflowing, but if you use a 64-bit int (BIGINT) I doubt that will ever be a problem.

    If you were using postgresql, you could even have an ON DELETE and ON INSERT triggers to keep the accumulators accurate without having to do it in your code. I don't know if newer MySQL supports those triggers.

    Anyway, it is a nice trick that turns a O(N) computation into O(1) operations.

  5. #5
    jam's Avatar
    Posts
    1
    Reputation
    0
    Tagged in
    0 Posts
    Add to this user's reputation
    Just posting that the above is me, I posted it before I signed up.

  6. #6
    Erorus's Avatar
    Posts
    1,404
    Reputation
    114
    Tagged in
    677 Posts
    Add to this user's reputation
    The 2nd and 3rd paragraphs of my recent devblog post have some thoughts on expanding the addon:

    I've been thinking more about the unannounced addon. The way it's set up right now, it's best used for rare-quality items and items that you don't normally come across, and even then it isn't as good as it could be. Knowing what the market price was at the time of import may not be very accurate; it'd be more useful to have an average and standard deviation along with it. This is difficult to compute at any decent speed for every item on every AH. I tried some different algorithms yesterday and decided to store the market price at 12:00am for every item at the end of each day, and save that value going back 1 week. Once I get a week's worth of day-end prices, there's some reasonable avg/stddev stuff I can do with that, at least better than a single snapshot.

    But even then, I'm not comfortable with the lua side of that addon. I'm not at all fluent in lua, and don't have the time, skill or patience to debug any addon issues. I heard that this addon doesn't work well alongside TradeSkillMaster, which pretty much makes this a non-starter. I saw that TSM has some sort of API that would allow for 3rd party auction databases. I'm not familiar with TSM and wouldn't know best how to work with and test it, but if someone wants to write a TSM module that uses TUJ prices, I could get behind that. Tell me what values TSM needs for each item (the fewer the better) to give accurate results, and I can try to squeeze them out of the database in some fashion. It would have to use an import mechanism similar to what the TUJ addon is doing now (copypaste from the website) but I can include different data. Anyway, think about it. If you're interested in making an addon like that work, and you've written WoW addons before so you know what you're talking about, start a dialogue with me via the contact form.
    So I think for these suggestions to take root, someone will have to volunteer to get down and dirty with TradeSkillMaster and some Lua.

 

 

Similar Threads

  1. Suggestion: Price information for items in a Wowhead search
    By Cuprum in forum Official Forum of The Undermine Journal
    Replies: 3
    Last Post: January 21st, 2012, 12:14 AM
  2. Replies: 13
    Last Post: September 26th, 2011, 07:31 AM
  3. How many Millionaires per average server?
    By DoubleO in forum Archive (Auction House)
    Replies: 16
    Last Post: July 20th, 2011, 11:43 AM
  4. [ Addon/API] Suggestion for automatic data updates
    By Amandria in forum Official Forum of The Undermine Journal
    Replies: 2
    Last Post: February 24th, 2011, 09:55 AM