If you've got a boat load of legacy fixtures laying around (and who doesn't?) it can be a pain now that Rails handles the ID of objects so you don't have to.
On the train back from holiday I wrote a snippet of code to migrate them:
RUBY:
-
def Util.real_yml(klass,col = 'name')
-
y = YAML.load_file(File.join(RAILS_ROOT,'test','fixtures',"#{klass.to_s.tableize}.yml"))
-
real_names = y.inject([]) { |names,(key,val)| names <<y[key][col] if y[key][col] ;names }
-
real_objs = klass.find :all, :conditions => [ "#{col} IN (?)",real_names ], :order => "#{col} asc"
-
yml = ""
-
real_objs.each do |real_obj|
-
yml += "#{real_obj.name.downcase.gsub(' ','_').gsub('-','')}:\n"
-
real_obj.attributes.each do |k,v|
-
if k =~ /_id$/ && real_obj.respond_to?(k.gsub(/_id$/,'').to_sym) && ! v.nil?
-
n_klass = k.gsub(/_id$/,'').camelize.constantize.find(v) rescue nil
-
key = k.gsub(/_id$/,'')
-
val = n_klass.name.downcase.gsub(' ','_').gsub('-','')
-
yml += " #{key}: #{val}\n"
-
else
-
yml += " #{k}: #{v}\n" unless k.to_s == 'id'
-
end
-
end
-
end
-
yml
-
end
The use is:
RUBY:
-
require 'util'
-
print Util.real_yml(Corporation)
Copy/paste into corporations.yml
So far it's working great!
First the problem:
I'm writing a load test framework at work and I need to consume JSON webservice and pass on the output to another request. But I don't always know what the data is that I need to pass on but I do know the basic "path" ("hpath" -- "hash path") to get to the data as the JSON data is uniform.
Now the method:
RUBY:
-
# Picks out data from a (JSON) decoded hash based on the @passon hash,
-
# which looks like this:
-
# { "id" => "packet.products[0].attributes.id",
-
# "quantity" => "packet.products[0].attributes.quantity"
-
# }
-
# The "id" and "quantity" are the new keys for the return data;
-
# packet.products[0].attributes.id will look at the value of the id key
-
# in the attributes hash in the 0th element of the products array in the
-
# packet hash.
-
def pick_out_passon(hash_data)
-
return {} unless @passon
-
returnhash = {}
-
nh = hash_data.dup
-
@passon.each do |newkey,part_str|
-
parts = part_str.split(".").reverse
-
while (part = parts.pop) do
-
m = part.match(/\[(\d+)\]/)
-
index = nil
-
if m
-
index = m[1].to_i
-
part.gsub!(/\[#{m[0]}\]/,'')
-
end
-
nh = nh.values_at(part).first
-
nh = nh.at(index) if index
-
end
-
returnhash.merge!({ newkey=> nh})
-
nh = hash_data.dup
-
end
-
returnhash
-
end
What's it do? It does magic!
I'll step through it...
Looks for @passon instance variable and doesn't do anything useful unless it exists.
Duplicate the input hash because the process done is destructive to it and we may need to reuse it.
For each new hash key and "hpath" pair from @passon split up the hpath into its parts and reverse it so Array#pop will work in the while loop to get the next first part to try.
Since each part of the hpath can examine an Array by index it has to be checked for and the index removed from the part (and saved).
Next, investigate the copy of the hash_data, nh by the computed key; if the value in the hash was an Array use the index to get the desired value. Then compute the next part!
Once we're out of parts stuff the new key and data into the returnhash and keep going til there's no more @passon pairs.
And thus some fun code was written.
In reference to the post about My First Patch to the Rails trac I mentioned in the edit of the post that I emailed the maintainer. There's been no response, unfortunately.
So I'll post the patch here so it'll get a bit more visibility and hopefully Oracle users can find it helpful in speeding up rake db:migrate on Oracle systems with a lot of indicies.
Patch to speed up Rails’s OracleAdapter#indexes method
This bit of code is the Soundex algorithm based on some C written by a friend of mine (Hi Jen!). She writes really compact C (read: hard to read) and it took me a long time to understand what the code she wrote did.
So here is the annotated version of the code (correlates to the Rule numbers on Wikipedia).
RUBY:
-
class String
-
@@sem = [ 0,1,2,3,0,1,2,0,0,2,2,4,5,5,0,1,2,6,2,3,0,1,0,2,0,2 ] # Letter codes, Rule 3
-
-
def to_soundex
-
s = self.upcase
-
chars = s[1 .. s.size - 1].split(//) # Split the word into characters to encode
-
ret = s[0,1] # Rule 1
-
chars.each_index do |i|
-
c = chars[i]
-
next if ( c <'A' or c> 'Z' ) # Not a letter
-
next if i>0 and c == chars[i - 1] # Rule 4
-
d = @@sem[ c[0] - 'A'[0] ] # Encode it
-
ret += d.to_s unless d == 0 # Rule 2
-
end
-
ret += "0000" # Rule 5
-
ret[0..3] # Rule 5
-
end
-
end
It was fun to write and it works well.
Greetings dear readers.
The CCP guy (Garthagk) behind the Eve Online API we all use really has a hard task. As the API becomes more and more popular the stress on the webservers hosting the API and the database servers increases. Bossmen and bosswomen may cut the API from us to save resources (that would suck). It's up to us, the community, to do what we can to lessen the load on CCP's servers.
Garthagk provided us with a cacheduntil (Reve::API.cached_until in Reve). Up unil now Reve didn't privde any mechanism for really using this. Sure, you could use just the cached_until if it's just one character. But one of my hopes is that Reve will be used by operators of multi-user sites (like my vapourware evedb.info) that will aggregate many users' data. Cached_until isn't enough.
Enter the protected method Reve::API#compute_hash. (As of revision 22.)
After each API call Reve will compute a hash to represent that request. This will be available through the last_hash. This will tell you what the hash is after the call. But to respect the cached_until it's polite to check the current time against the "Last Call's" cached_until value. The last call will have a hash and that hash will be the key to the cached_until. Each API call now takes a hash for its parameters intead of the old style comma style. Treat the hash as a of transaction ID.
For a simple example let's get the alliances list:
RUBY:
-
require 'reve'
-
api = Reve::API.new
-
alliances = api.alliances
-
p api.last_hash # => 'eve/Alliances.xml.aspx'
-
api.cached_until
For this simple call nothing has changed except the Reve library presents more info for the user. Store cached_until and last_hash in a database.
For the new parameter to get the hash:
RUBY:
-
require 'reve'
-
api = Reve::API.new
-
a_hash = api.alliances :just_hash => true
-
p a_hash # => 'eve/Alliances.xml.aspx'
A complex example:
RUBY:
-
require 'reve'
-
api = Reve::API.new('userid','apikey')
-
sheet = api.character_sheet :characterid => 0123456789
Get the hash:
RUBY:
-
require 'reve'
-
api = Reve::API.new('userid','apikey')
-
h_sheet = api.character_sheet :characterid => 0123456789, :just_hash => true
With luck this post is clear for Reve's users and all you guys will be kind to the API so we have it in the future.
This is good timing. I need to know about this stuff tomorrow to help a colleague out. Rails Polymorphic Associations were a bit mysterious to me and so I didn't use them when I first started with Rails but the time has come to put them to work.
First, the scenario:
A Player and a Corporation both may have many Wallet Balance Entries and a Wallet Balance Entry belongs to either a Player or a Corporation.
The Models are this:
RUBY:
-
# app/models/wallet_balance_entry.rb
-
class WalletBalanceEntry <ActiveRecord::Base
-
belongs_to :wallet_balance_owner, :polymorphic => true
-
end
And (just the Player model)
RUBY:
-
# app/models/player.rb
-
class Player <ActiveRecord::Base
-
has_many :wallet_balance_entries, :as => :wallet_balance_owner
-
end
The relevant bits of the WalletBalanceEntry schema look like this:
RUBY:
-
create_table "wallet_balance_entries", :force => true do |t|
-
t.string "wallet_balance_owner_type"
-
t.integer "wallet_balance_owner_id"
Note that wallet_balance_owner_type and wallet_balance_owner_id match the name that the Player model specifies with :as => :wallet_balance_owner. The Corporation model will have the exact same line and it will work! Should Alliances come to have many wallet balance entries the Alliance model will have this same line as well.
In my evedb.info efforts I need a quick way to expire old market orders. The CSV has a duration column that is an integer number of days. Postgres allows its users to typecast between various data types in SQL so there doesn't need to be a SELECT duration FROM ... ; (in code) to_delete = Time.now + duration.days ; DELETE FROM ... WHERE duration < to_delete;
It just so happens that the form for casting an integer to do math with a timestamp is a bit funky. An integer may not be directly added to a timestamp. The magic is:
SQL:
-
DELETE
-
FROM foo
-
WHERE now()> (created_on + CAST( (duration || ' days') AS INTERVAL));
The || ' days' I don't understand just yet but it works!
[ Quick edit: Yeah, I screwed up the title and that botched the permalink and now I can't change it. ]
On evedb.info I have Market categories set up as a tree (acts_as_tree) which means when drilling down the categories: "Ammunition & Charges -> Frequency Crystals -> Advanced Beam Laser Crystals", for example I would like to create breadcrumbs to get back to those parent categories.
Before I got Enumerable#inject working I was mangling with all sorts of recursion. It wasn't pretty.
Then along came the day when I wanted to rewrite it to make sense!
RUBY:
-
def marketHierarchy(cat)
-
cat.ancestors.reverse.inject("") { |link_str,a_cat| link_str += link_to(h(a_cat),market_path(a_cat)) + " -> " }
-
end
This bit of voodoo works brilliantly.
What it does, for those who aren't familiar with Enumerable#inject is:
For each element in the ancestors of the passed MarketCategory do:
- Do the block, but first initialize link_str to ""
- Append a link to link_str and ->
- After the link is appended pass set link_str for the next element in the ancestors list to the result of the block (link_str)
And then just return the result of the block, the final link_str.
The Enumerable#inject method is really cool!
DRY CRUD!
That is, of course, "Don't repeat yourself".
With my evedb.info site I have to essentially rsync two databases whilst mangling schema. The process will take FOREVER (Mainly because I'm not multithreading this process) but it's automated.
I kept having to write CRUD methods in all of my controllers to deal with updating evedb.info as I talked about in this linked post and they were ALL the same. So being the lazy programmer I am I put them all in the ApplicationController:
Here's the update method
RUBY:
-
def update
-
singular = params[:controller].singularize
-
camel = singular.camelize
-
klass = camel.constantize
-
@temp = klass.new params[singular]
-
@real = klass.find params[:id]
-
changes = object_diff(@real,@temp)
-
logger.info "Changes: #{changes.inspect}"
-
if changes.keys.size == 0
-
render :nothing => true, :status => :ok
-
return
-
end
-
if @real.update_attributes(changes)
-
render :nothing => true, :status => :ok
-
else
-
render :json => @real.errors.to_json, :status => 400
-
end
-
end
By abusing Rails's constantize I can mangle the controller's name ("corporations") into the model it's handling: Corporation and treat it klass as if it was actually spelled Corporation. Personally I think that is very cool.
object_diff is a method I wrote to find the differences between two ActiveRecord::Base derrived objects. Loop through attributes hash and note things that change. changes is those attributes that have changed.
The only part I haven't been able to DRY out is the index method which always returns YAML for my scripts but only the IDs of the objects.
RUBY:
-
if params[:format] != 'yaml'
-
@corporations = Corporation.find normal finder
-
end
-
respond_to do |format|
-
format.html
-
format.yaml { render :text => Corporation.find(:all, :select => 'id', :order => 'id').to_yaml }
-
end
But I don't think I can DRY it out becase Corporation YAML finder actually selects another column that my script uses. Oh well.
At work yesterday I found out that by default ActiveSupport::JSON.unquote_hash_key_identifiers is true. This means that a JSON string looks like:
{attributes: {name: "0-3VW8 I", coordinate_id: "6229", system_id: "30000995", id: "40063299", igbtype: "7"}}
Flash JSON decoders (and probably others) expect the above attributes word to be doublequoted. This is achieved (after much Googling) by setting ActiveSupport::JSON.unquote_hash_key_identifiers=false.
But it doesn't stop there. In the above JSON output the IDs are quoted, that means they're encoded as strings and not integers. This is against the spec.
This means that
RUBY:
-
ActiveSupport::JSON.unquote_hash_key_identifiers = false
-
planet = Planet.find :first
-
print planet.to_json # => {"attributes": {name: "0-3VW8 I", coordinate_id: "6229", system_id: "30000995", id: "40063299", igbtype: "7"}}
The integers should not be double quoted.
With the help of lifo from #rails-contrib the problem seems to have been narrowed down to instance_values/instance_variable_get borking things up. From my poking around (n.b., I'm not all that experienced with the source for Rails itself! I may be wrong) the issue seems to lay within vendor/rails/activesupport/lib/active_support/json/encoders/object.rb. This bit of code seems to work for AR::Base derived models:
RUBY:
-
class ActiveRecord::Base
-
def to_json
-
ret = ActiveSupport::JSON.unquote_hash_key_identifiers ? "{attributes: " : '{"attributes": '
-
ret + self.attributes.to_json + '}'
-
end
-
end
Edit (10 July 2007): It seems that this isn't exactly a drop-in replacement fix: If a model has associations loaded (Foo.find :first, :include => :bars) the above JSON-fixing snippet will not work because it looks at attributes only and not the loaded associations. I'll have to work around that.
