Thread: Consortium Total Goblin System

Hey folks.

This was originally going to be called the Consortium Notification System, and it was supposed to have been completed a few weeks ago. Hopefully you'll see why I decided to change the name AND why it's taken a bit longer to develop. As this is an initial version, I'm sure there will be lots of bugs and features that need to be implemented. Please check out the "known issues" and "planned features" after the initial guide for more information.

Overview of the CTGS:
When development initially began of an alternative TUJ-inspired notification system, I was met with several quick hurdles. The biggest and most crippling was the fact that my personal shared hosting plan didn't offer significant enough resources to process more than 8-12 realms at a time. A far cry from the nearly 500 US and EU realms, I scrapped the idea altogether for a few days thinking it couldn't be done. In a separate discussion, I was reminded of WAMP, and things took a turn for the better again. Once I realized the potential of having an external database on the same machine as your WoW/addons/spreadsheets, etc, I knew this could be much, much bigger than just a notification system. So here we are.

Pros:
Flexible - Your data for Your realm the way You want it. Also, the only storage limitation is the capacity of your own local machine, so there is plenty of room for expansion here.
Integration - The development platform offers many options for getting data in and out of your existing resources, including in-game addon data.
Accessible - Run the application on your terms. Coding is completely open-source, so you're free to add/remove or modify anything that doesn't meet your needs.

Cons:
WAMP server requires a not-insignificant amount of default space (somewhere in the realm of 250MB)
Does require the machine it is installed on to be ON and also actively connected to the internet in order to be most useful.
Updates and Changes will require manual patching and upgrading. I will try and make this as painless as possible for everyone, and hopefully you're all at least "used" to this process given the nature of WoW addons, but still a drawback compared to a 100% "online" application.

Alright, let's get to the good stuff

Current Version:
version 1.0.2
link: ctgs_v1_0_2.zip

Installation Guide
(NOTE: this installation process HAS NOT been extensively tested on multiple platforms and OS'es, so please be sure to include as much information about your system in any bug reports as possible)

1. Download WampServer
For more information about, and to download the WAMP files, visit Install PHP 5 Apache MySQL on Windows : WampServer.
*** Important - this application's install procedure was specifically designed under default wampserver conditions. While their are many alternative options (LAMP, XAMPP, MAMP, etc.), I'm 99% certain the application WILL NOT install correctly under any of those, so if you are absolutely unable to run WampServer, please PM so I can work with your situation.

2. Install WampServer
Follow the default installation settings. When you get to the following screen, ignore your impulse to throw custom values in here. We'll change this later, but since there are additional settings we need to adjust, it make sense to do them all at once. If you do anything at all, remove both values altogether and attempt to proceed with empty fields.



3. Download Sendmail
One of the issues I first ran into with WAMP was the fact that the default PHP mail() function doesn't work. This was obviously a huge problem for a notification system, but I found a suitable workaround. NOTE: It is my understanding that XAMPP actually comes included with Sendmail, but since I had already gone to all the trouble of finding and configuring the WAMP/Sendmail solution, I stubbornly stuck with it.

Sendmail Link: sendmail

4. Unzip and Install the Sendmail files.
Copy the entire "Sendmail" folder to your Wamp's /bin folder. Default should be C:\wamp\bin\. The resulting sendmail.exe file should end up being located at C:\wamp\bin\sendmail\sendmail.exe - if it isn't, you'll likely run into problems.

5. PHP Settings
Launch WampServer. When wampserver is online, the icon should be green in the system tray, as such:



I did experience some conflict when using both Skype and wampserver simultaneously. I was able to remedy this by changing my Skype settings to ensure Skype no longer communicated via port 80. See below.



Next, we need to enable a couple of important PHP extensions. Click on the wampserver system tray icon, then > PHP > PHP extensions.



The 2 you'll need to enable are CURL and OPENSSL.





You'll also need to restart the server in order for the extensions to be activated. Click the "restart all services option" as seen below.




6. php.ini
There are actually 2 php.ini files that will need to be configured here. The first one affects browser based settings, and the second one affects the php command-line program, which we will be using with task scheduler/CRON jobs, etc. in order to get our program to actually "do stuff". My recommendation would be to edit the browser based php.ini file first and then copy it and replace the command-line php.ini file second.

The main php.ini file can be easily accessed through the wampserver tray interface, as such:



Once inside the php.ini file, we need to find our mail function settings. Best bet would be to do a search (CTRL+F in notepad) for "smtp". This is the section we're looking to get to and modify:



Clear any previous values from the following fields:
-SMTP
-smtp_port
-sendmail_from

Add the path to our sendmail.exe where it says sendmail_path = . The default will be "C:\wamp\bin\sendmail\sendmail.exe -t" Be sure to include the -t parameter and MAKE SURE you remove any semicolon from the beginning of this line. Semicolon denotes a non-executed comment in our .ini file and will cause our mail() function not to work. IE. Pretty big deal .

Save your php.ini file.

Now we need to change the second one. Under default installation, the first php.ini file can be found under \wamp\bin\apache\Apache2.*.*\bin\. Copy the php.ini file located there. The default location for our second php.ini file is \wamp\bin\php\php5.*.*\ - overwrite this php.ini file with the first (edited) one. Feel free to make a .bak version first, if you like.

Click "restart all services" again to activate our changes.

7. Download the application
link: ctgs_v1_0_2.zip

Unzip and copy the "cgts" folder to the \wamp\www\ folder. You can easily access this folder by clicking the www directory option from within the wampserver tray options.



Now, click on the "Localhost" option, again under our system tray options. This should open up your default web browser and take you to a page that looks something like this:



The "ctgs" folder show show up under "Your Projects". If not, your folder didn't get copied to the right place.

NOTE: Much of the application's interface uses jQuery from this point on. As such, you will need to ensure that javascript is enabled in your browser for this to work. Making a the application fully-web accessible is on my list of things to tackle, but it admittedly is nowhere near the top, so if this is a deal-breaker for you, send me a PM and we can talk about your options.

8. Installing the Application
Click on the "ctgs" folder. Then click on "install.php".

Welcome to the Consortium Total Goblin System!

The system is built to support up to 5 realms from either US or EU region, however, the install script only accepts 1 to begin with. So, choose the region and realm for the primary realm you'd like to use for your data. Also, enter the email address where you'd like to RECEIVE notifications.

Mailserver Settings: our wampserver/sendmail setup requires a mailserver in order to actually send email. As I mention in the install, I highly recommend using gmail, but many mailservers will work. FYI - I tried to get this setup under my godaddy-hosted account, but was unsuccessful. Gmail works fine . If you decide to use gmail (which you should), you will need to enable POP access from within your email account settings. See below.



The SMTP details that gmail uses are explained here: https://mail.google.com/support/bin/...y?answer=13287, but I've also noted them in the install script. If your mailserver requires authentication (as gmail does) be sure to toggle that option and then submit your username and password as well.

Upon clicking install, you will be redirected to a confirmation page and all of your initial databases and tables will be created and your configuration settings will be saved. You should also note that the install script automatically updates your sendmail.ini file with your SMTP settings. If you need to check/change those settings, the default path would be C:\wamp\bin\sendmail\sendmail.ini. The script should also send you an email to the address you specified. If you do not receive this email (allow an appropriate delay before panicking), it pretty much also means that you won't receive any of your notification emails, so we should really get to the bottom of that before continuing on. Finally, click the "Launch" button to get to the CTGS app.

9. Running CTGS - Home
I've envisioned this page as a launching point for other elements of the application. Currently, it doesn't do much.

9b. Running CTGS - Notifications
Should be pretty self explanatory. Click "create notification" to get started. NOTE: There might be a slight delay when checking item names, since it fetches it live via the API, but app will return "Item not found" if you enter an invalid item ID, so just be patient.

There is currently no confirmation when deleting a notification, so use with caution.

9c. Running CTGS - Database
Currently not functional. I believe DB options will end up being a focal point of the CTGS, so I've given it it's own page. Expected features will likely include import/export, StatsDB settings, such as time period, fields, etc. Stay tuned, as this will likely be one of the areas that I update first.

9d. Running CTGS - Settings
ATM - all it does is display your settings, and none of the update buttons actually do anything. I'll fix this soon enough, but I thought it would be better to get an initial release ready to go, since I can easily walk someone through all of these settings if you run into any installation/configuration issues.

10 - Getting it to do something.
Most versions of Windows, especially modern ones, include some form of task scheduling application. This is what we'll be using to get the CTGS to actually do stuff. Included in the download is an executable batch file called ctgs.bat, which calls the php.exe command line and passes it the parameter of our fetch_data.php script. If your installation has used non-default file paths, this will need to be changed as well. PM me for help.

Anyway - setup a scheduled task for the ctgs.bat. You can specify any frequency you like, since the script will automatically check and compare last modified values. My recommendation would be to at least run the script every 30 mins so that you can ensure you never miss an API dump, but obviously the more frequently you run it, the more recent your notifications are guaranteed to be.

Check out Gimp's Post on setting up a scheduled task in Windows

NOTE ABOUT THE TASK SCHEDULER: setting your trigger as a specific time + repeat might be a better option than on startup + repeat.

Final Notes
Please see the next 2 posts in this thread for Known Issues and Planned Features. If you have any technical issues with the application or come across any bugs, feel free to either reply to this thread, PM me or send me an email at kathroman@breakthebankwow.com.

Current Version:
version 1.0.2
link: ctgs_v1_0_2.zip

Known Issues

-Currently only supports 1 realm (although fields are displayed for 5)
-Settings page does not actually update settings
-"Under" quantity notifications will not trigger if there are 0 in AH scan
-Code is inefficient and messy
-Database page does nothing

Hopefully those are all of them. If you come across anything else, please let me know.

Planned Features

This is a list of things I have thought about and/or am working on including. If you would like to add something to the list, or would like to voice support (likely bumping it up the list), let me know below.

In no particular order:

-"Activate" settings page
-Add functionality to the Database Page (CSV export would likely be basic feature)
-Create "stats" DB - for fields such as Market Value, Mean, STDDev, etc. (most of the code for this is already available from another project @Stede and I have been working on)
-Spruce up the visual styling
-Go through all of the code and make it more efficient
-Add a "plugins" feature. This would be to support additional modules and customization (sky's the limit)
-Import/export feature for notifications to facilitate integration with other programs, such as TUJ and in-game addon data
-TSM integration. TSM is one of the best tools currently available to goblins. I forsee CTGS acting as a sort of bridge between your internal and external WoW resources. IE. getting addon data into your external spreadsheets or websites and vice versa.
-"Online" web application version, for people to install on private servers to avoid some of the inherent limitations of the system.
-Additional notification types
-Probably a few more that I can't think of ATM

Let me know what you'd like to see out of this.

Live!

Let the flood of help requests begin . Hopefully not, but I'm anticipating the worst, since the system is blatantly untested.

WOOOT!

Good job man!
It took a while to setup and get going but I think I got it right. Busy populating my notification list now. Needing to enter item ID's is a pain though - figured out if you just hover over the item link on your TUJ notification list the item ID shows at the bottom of your page at the end of the URL link.

Suggestions so far:

• Might want to add that for people wanting to come back to the Consortium Total Goblin System after closing it, they need to go localhost and then click on CTGS and then click on Home.php.
• Would be great if we could create custom categories for our notifications (like rare weapons, enchanting formulas, sunwell boe profession recipes, low ore prices, etc.
• Way to edit notification conditions without having to delete and re-add.
• I'm not too familiar with the Windows Task Scheduler system and I'm sure others aren't either. Maybe you could create a quick guide for that too.
• Item tooltips
• Clickable item links either to Wowhead or to TUJ.
• Perhaps we could create a pre-compiled list (we have tons of these lists on the forums) of recommended notifications that users can simply check active and enter their own price/quantity. Instead of needing each person to add those items manually.
• Configure notification emails to include current lowest AH price for the item. Such as: "Notification met for: Haunted Memento on eu-silvermoon-a. Price is under 10000 g 0 s 0 c (200g 0s 0c)"

Originally Posted by Consortium IRC Channel

<Sapu_> oh wow

I see. The DB was not populated because... it's never populated . Anyway, I got errors when trying to fetch AH data. Fixed by adding a region parameter to ParseAH - the function was complaining about undefined region. I guess you have enabled global variables, and I don't.

It's now triggering the notifications and sending mails (after installing a mailer).

Now adding notifications .

Solution:

Edit inc/fetch_data.php, change line 42 :

PHP Code:

$horde = ParseAH( "Horde", $realm1, $region1 ); 


And also change line 128 to :

PHP Code:

$alliance = ParseAH( "Alliance", $realm1, $region1 ); 


Edit inc/functions.php, change line 127 to :

PHP Code:

function ParseAH( $faction, $realm, $region ) 


This should fix it.

If you want to test without waiting for BattleNet update, you can edit the lastmod table, and change the timestamp from whatever it is to 0. I think you should have phpmyadmin installed with WAMP.