I confess: I’ve been very lax with development on evedb. The biggest reason why is because too many features have been creeping their way into the application before other ones are completed. I have no project management going on here and I’ve let the project get away from me.
The moral is: track large projects and stick to a fixed roadmap. I must organise this project. I think I’ll give Trac a look.
Greetings dear readers.
The CCP guy (Garthagk) behind the Eve Online API we all use really has a hard task. As the API becomes more and more popular the stress on the webservers hosting the API and the database servers increases. Bossmen and bosswomen may cut the API from us to save resources (that would suck). It's up to us, the community, to do what we can to lessen the load on CCP's servers.
Garthagk provided us with a cacheduntil (Reve::API.cached_until in Reve). Up unil now Reve didn't privde any mechanism for really using this. Sure, you could use just the cached_until if it's just one character. But one of my hopes is that Reve will be used by operators of multi-user sites (like my vapourware evedb.info) that will aggregate many users' data. Cached_until isn't enough.
Enter the protected method Reve::API#compute_hash. (As of revision 22.)
After each API call Reve will compute a hash to represent that request. This will be available through the last_hash. This will tell you what the hash is after the call. But to respect the cached_until it's polite to check the current time against the "Last Call's" cached_until value. The last call will have a hash and that hash will be the key to the cached_until. Each API call now takes a hash for its parameters intead of the old style comma style. Treat the hash as a of transaction ID.
For a simple example let's get the alliances list:
For this simple call nothing has changed except the Reve library presents more info for the user. Store cached_until and last_hash in a database.
For the new parameter to get the hash:
A complex example:
Get the hash:
With luck this post is clear for Reve's users and all you guys will be kind to the API so we have it in the future.
Today I finally managed to migrate my Evedb.info SVN repository from an internal box (boron) to my repository at Dreamhost (Home to Reve, too). It is really simple to do: Just copy the local repository files from Subversion to the new repository base. This means the conf, dav, db, hooks and locks diretories.
The problem comes when one revision is copied (Call it revision 10) and is checked out to another machine (phosphorus), development is done, checked in (Dreamhost's Revision 11) and stupidly, development is also done on the local copy (boron) and checked in (Local revision 11). And just like that the migration away from a local file:// type.
It's a nightmare to sort but I've finally got it sorted and protected with an off-site source and SSH key file.
The moral is: Plan ahead, don't be stupid, and never make commits when you're really tired.
This is good timing. I need to know about this stuff tomorrow to help a colleague out. Rails Polymorphic Associations were a bit mysterious to me and so I didn't use them when I first started with Rails but the time has come to put them to work.
First, the scenario:
A Player and a Corporation both may have many Wallet Balance Entries and a Wallet Balance Entry belongs to either a Player or a Corporation.
The Models are this:
And (just the Player model)
The relevant bits of the WalletBalanceEntry schema look like this:
Note that wallet_balance_owner_type and wallet_balance_owner_id match the name that the Player model specifies with :as => :wallet_balance_owner. The Corporation model will have the exact same line and it will work! Should Alliances come to have many wallet balance entries the Alliance model will have this same line as well.
[ Quick edit: Yeah, I screwed up the title and that botched the permalink and now I can't change it. ]
On evedb.info I have Market categories set up as a tree (acts_as_tree) which means when drilling down the categories: "Ammunition & Charges -> Frequency Crystals -> Advanced Beam Laser Crystals", for example I would like to create breadcrumbs to get back to those parent categories.
Before I got Enumerable#inject working I was mangling with all sorts of recursion. It wasn't pretty.
Then along came the day when I wanted to rewrite it to make sense!
This bit of voodoo works brilliantly.
What it does, for those who aren't familiar with Enumerable#inject is:
For each element in the ancestors of the passed MarketCategory do:
And then just return the result of the block, the final link_str.
The Enumerable#inject method is really cool!
This whole bulk updating system I did a while ago works well, except it's very slow with a single mongrel. That problem could be solved with a simple proxy in front of many mongrels and a threaded updater application. For mass schema changes, for now, the easiest method is to dump from the CCP into my schema. I may be crazy but I'm not that crazy.
Things are coming to a close withthe RESTification of evedb. Next action once the RESTification is done is to get the site a proper design! I've got a sort of mental road map for evedb and so far I'm at what equates to "version 0.3". Long way to go but it's a fun ride. I think I need a vapourware release...
DRY CRUD!
That is, of course, "Don't repeat yourself".
With my evedb.info site I have to essentially rsync two databases whilst mangling schema. The process will take FOREVER (Mainly because I'm not multithreading this process) but it's automated.
I kept having to write CRUD methods in all of my controllers to deal with updating evedb.info as I talked about in this linked post and they were ALL the same. So being the lazy programmer I am I put them all in the ApplicationController:
Here's the update method
By abusing Rails's constantize I can mangle the controller's name ("corporations") into the model it's handling: Corporation and treat it klass as if it was actually spelled Corporation. Personally I think that is very cool.
object_diff is a method I wrote to find the differences between two ActiveRecord::Base derrived objects. Loop through attributes hash and note things that change. changes is those attributes that have changed.
The only part I haven't been able to DRY out is the index method which always returns YAML for my scripts but only the IDs of the objects.
But I don't think I can DRY it out becase Corporation YAML finder actually selects another column that my script uses. Oh well.
Aside from having my hair cut today I wrote a Ruby library for the Eve Online API. I also wrote the above page to act as the ad-hoc home for the library until something more formal can be arranged.
It's still early stages (and I forgot to remove my old API key from reve.rb!) but it seems to work. Have a look and post some feedback!
Following up to this post of mine. The schema can be found here and the full thing (including Schema) can be found here.
In my quest to RESTify Evedb.info I keep finding more reasons to fawn ove REST (and CRUD): With the most recent Eve Online Database Dump for Revelations I have a need to migrate the data to my own schema. I also need to treat the database dump as a "Master" copy of what should appear in my site; stale data must be removed from my database.
I like to automate things. Prior to the RESTification efforts I would pump out raw SQL files and import them into Postgres in a batch. The SQL took about 45 minutes to import and was often held up due to foreign key issues.
Anyways, with REST I can funnel all of that through my controllers which have that functionality anyways (albeit heavily secured so normal users can't use this CUD API). I can have the same standard method for updating my version of CCP's data.
CCP has a lot of extra data in its Database dump (read: I don't use every column from every table). When I felt the need to import one of those I had to mess about with SQL files. Evedb.info was not using migrations! The SQL files were too big to be under source control. And a better solution was needed.
Enter: Evedb::Updater.
I wrote a class today to use the libcurl libraries for Ruby since the native Ruby Net::HTTP libraries were giving me trouble with the encoding of POST data. The general structure of the class I wrote is the following:
) for the specific model. This results in a list of things to create, update and destroy
It was a fun bit of code to write! The highlight is probably this bit:
The hash is generated by
The result of the to_curl method is to make the POST data that how we rails folk like it! (agent[name]=foo).
One downside to this is that some models need to query the Rails database (not the CCP one) for information to use. I might be better off modifying those specific models to use virtual attributes and some clever use of before_create!
Powered by WordPress