CrudVision - Lisa Seelye

Following up JungleDisk vs Amanda

Lisa Seelye — Tue, 02 Sep 2008 21:01:33 +0000

Following up to Backing up to Amazon S3 using Amanda (June 28, 2008):

I’ve been having issues with both Amanda+S3 as well as JungleDisk. I’ll outline these here.

JungleDisk Problems

JungleDisk sometimes destroys all of my data on S3 in the bucket JungleDisk uses and then on the next backup re-uploads all of the data! This is clearly a problem. Luckily I do not have more than a gigabyte of data thats backed up with JungleDisk. If I did this bug (or feature?) would be very expensive.
JungleDisk doesn’t smartly handle moves. I’d like to be able to move things around on my local filesystem and have JungleDisk notice this and move them. Moving them over WebDAV isn’t feasible.
JungleDisk scans individual files and doesn’t combine a whole bunch of them into one tarball. This gets very expensive! I wish they’d tar it all up first like Amanda.

Amanda+S3 Problems

Sometimes the Amanda S3 device module has problems talking to S3. The only fix I’ve found so far is to destroy the bucket, remove it from the tapelist, readd the bucket, reidentify the bucket to Amanda and run amflush. This is clearly not good as it’s just as bad as Jungledisk destroying everything. I haven’t figured out why this happens yet.

Both products are still good and I’ll continue to use them. I’m considering using Amanda on my laptop, however, but this could cause problems in cases where it isn’t connected to the network at backup time.

Playing with Git

Lisa Seelye — Thu, 28 Aug 2008 17:05:11 +0000

I’ve begun playing with git over at github by importing Reve’s source.

Eventually I will do away with subversion and trac and move to github for Reve completely. Trac is not very good and often locks up.

As I’m still new with git and github I’ll have to keep contributions to a minimum since I don’t know exactly what I’m doing.

Reve is going to MIT (license)

Lisa Seelye — Thu, 14 Aug 2008 20:52:59 +0000

There’s a new revision of Reve that has just been uploaded to Rubyforge: Revision 99.

This includes a major change - Reve is now licensed under a proper license: the MIT License. The terms of the license are now distributed with the package and are.

There is also a fix for the character_id that came back from a market order request.

Why a new license now? Simply because I think it’s important to realise that it’s unlikely that anyone will use Reve to create some billion dollar enterprise that I could have got in on. Yeah, not everyone can be Delicious, Flickr or Facebook. ha ha.

So go forth, ye multitudes and fix, hax, and use. Ruby and Rails both use the MIT license so now Reve should be at home.

Eve Online Empyrean Age 1.0 Database Dump for PostgreSQL

Lisa Seelye — Wed, 30 Jul 2008 18:29:16 +0000

Today CCP announced the release of their MS SQL dump. Following in step with past conversions here is a Postgres version. Tested with 8.0.15 but should work on everything.

This dump is odd: CCP is using integers where they mean to use booleans. So in those cases remember 1 is true, 0 is false. Good luck.

Edit: As a note the Eve Online Database Viewer is updated with this dump.

Edit 2 (July 31, 2008): I nearly forgot to give props to bunjiboys for providing the .sql files from which the above dump is derrived.

Fixtures in Rails Suck - this makes them suck less

Lisa Seelye — Wed, 30 Jul 2008 01:31:20 +0000

If you've got a boat load of legacy fixtures laying around (and who doesn't?) it can be a pain now that Rails handles the ID of objects so you don't have to.

On the train back from holiday I wrote a snippet of code to migrate them:

PLAIN TEXT

RUBY:

def Util.real_yml(klass,col = 'name')
y = YAML.load_file(File.join(RAILS_ROOT,'test','fixtures',"#{klass.to_s.tableize}.yml"))
real_names = y.inject([]) { |names,(key,val)| names <[key][col] if y[key][col] ;names }
real_objs = klass.find :all, :conditions => [ "#{col} IN (?)",real_names ], :order => "#{col} asc"
yml = ""
real_objs.each do |real_obj|
yml += "#{real_obj.name.downcase.gsub(' ','_').gsub('-','')}:\n"
real_obj.attributes.each do |k,v|
if k =~ /_id$/ && real_obj.respond_to?(k.gsub(/_id$/,'').to_sym) && ! v.nil?
n_klass = k.gsub(/_id$/,'').camelize.constantize.find(v) rescue nil
key = k.gsub(/_id$/,'')
val = n_klass.name.downcase.gsub(' ','_').gsub('-','')
yml += " #{key}: #{val}\n"
else
yml += " #{k}: #{v}\n" unless k.to_s == 'id'
end
end
end
yml
end

The use is:

PLAIN TEXT

RUBY:

require 'util'
print Util.real_yml(Corporation)

Copy/paste into corporations.yml

So far it's working great!

The decline of Rails-Core mail list

Lisa Seelye — Tue, 29 Jul 2008 15:35:38 +0000

Lately the rails-core mail list has declined in quality. Spammers have moved in and operate unfettered. Reporting offenders to Google does nothing.

In fact, in order to isolate my personal mail from spam and to keep dspam from getting very confused I've moved the rails-core list to a google mail account. It's a shame but I'm more interested in keeping my other mail free from spam and the dspam quarantine free from ham. Isolation of rails-core until Google can get the spammers off their lists will accomplish the goal.

Reve Supports Eve’s Factional Warfare API

Lisa Seelye — Sat, 12 Jul 2008 20:04:21 +0000

Here's a quick announcement to let people know that Reve (0.0.96) now supports Factional Warfare.

The cool new things are:

Faction-wide war stats
System Occupancy Data
List of the top characters and corporations and factions.
Personal and Corporate war stats

I know the documentation may not be the best, but I'll do my best to improve it for the next release.

Grab the new gem (0.0.96) and give it a whirl!

Backing up to Amazon S3 using Amanda

Lisa Seelye — Sat, 28 Jun 2008 16:18:44 +0000

Recently a friend of mine (Sarah) purchased a dedicated server at a hosting company and moved her data to it. Obviously she was in need of a backup solution. She chose Amanda 2.6.0 and to use S3 as her "tape" choice.

Seeing how well it worked for her I asked her to show me how it worked and now my server is also using Amanda to backup to S3. Recovery works with amrecover.

I have quite a bit of data to back up (4.5GB in /home, for example) and with a home DSL connection it takes a long time. However with a bigger pipe using S3 and Amanda would be extremely viable.

I'm very happy with the solution and will likely use it across all of my servers from now on.

For my laptop I'm still using JungleDisk, which seems to work fine. 2.0 is a very good improvement.

New Reve Gem, release 91

Lisa Seelye — Tue, 10 Jun 2008 22:03:12 +0000

Just a quick note. There's a new Reve gem out as of June 7th. This one's release notes are:

Typo fix in Reve::Classes::MapKill - shipJills -> shipKills
Resolve trac ticket 3 (lowercase field names in postfields method)
Update documentation to make 'charid' more consistantly 'characterid'
Resolve trac ticket 2 (Reve does not use @charid, but takes it as an argument in initialize)
Be strict about currentTime's casing.

The code changes can be seen At the Reve Trac.

Mini Match is Alive!

Lisa Seelye — Tue, 03 Jun 2008 11:14:49 +0000

Yesterday we launched Mini Match, an application me and my colleagues from work wrote for Cartoon Network.

Late last year we opened a beta that was, unfortunately, short lived. Friday (May 30, 2008) we opened the system up on a much improved codebase for a few hours and had all systems "green". Based on the positive success from Friday we opened it up yesterday.

It opened slowly at first with just a small advertisement on the Cartoon Network Games' Page, and then a larger one on the same page and then we made the Cartoon Network home page with a small advertisement again. Today, I reckon, a larger advertisement will be put on the front page and we'll really start to see traffic!

Some details on the application:

Flash/Flex/AS/Whatever front-end GUI (really, it's one of them)
Java-based persistance server
Rails-based funnel into the database with a bit of logic.

The Rails part is RESTful (for the most part) and is the "glue" of the application, to quote someone from IRC.

Today should be a fun day!