CrudVision - Lisa Seelye

July 30, 2008

Fixtures in Rails Suck - this makes them suck less

Filed under: open source, rails, snippet, testing — Lisa Seelye @ 1:31 am

If you've got a boat load of legacy fixtures laying around (and who doesn't?) it can be a pain now that Rails handles the ID of objects so you don't have to.

On the train back from holiday I wrote a snippet of code to migrate them:

RUBY:
  1. def Util.real_yml(klass,col = 'name')
  2.     y = YAML.load_file(File.join(RAILS_ROOT,'test','fixtures',"#{klass.to_s.tableize}.yml"))
  3.     real_names = y.inject([]) { |names,(key,val)| names <<y[key][col] if y[key][col] ;names  }
  4.     real_objs = klass.find :all, :conditions => [ "#{col} IN (?)",real_names ], :order => "#{col} asc"
  5.     yml = ""
  6.     real_objs.each do |real_obj|
  7.       yml += "#{real_obj.name.downcase.gsub(' ','_').gsub('-','')}:\n"
  8.       real_obj.attributes.each do |k,v|
  9.         if k =~ /_id$/ && real_obj.respond_to?(k.gsub(/_id$/,'').to_sym) && ! v.nil?
  10.           n_klass = k.gsub(/_id$/,'').camelize.constantize.find(v) rescue nil
  11.           key = k.gsub(/_id$/,'')
  12.           val = n_klass.name.downcase.gsub(' ','_').gsub('-','')
  13.           yml += "  #{key}: #{val}\n"
  14.         else
  15.           yml += "  #{k}: #{v}\n" unless k.to_s == 'id'
  16.         end
  17.       end
  18.     end
  19.     yml
  20.   end

The use is:

RUBY:
  1. require 'util'
  2. print Util.real_yml(Corporation)

Copy/paste into corporations.yml

So far it's working great!

March 12, 2008

Reve Release 86

Filed under: api, eve online, ruby, testing — Lisa Seelye @ 12:09 am

Another Reve release, 86. This time I'm fixing a bug with the strict XML parsing not picking up cachedUntil. Tests and test XML have been fixed. Check trac for specific changes.

February 17, 2008

First Reve Trac Ticket

Filed under: api, bugs, eve online, open source, rails, reve, ruby, subversion, testing, trac — Lisa Seelye @ 11:58 am

Had my first bug report today about the to_i method I made on the String class breaking Rails migrations because leading zeroes weren't being treated nicely in my method: "001".to_i # => 1 and fails the test 1.to_s == "001" and so a String was being returned fromto_i. Crap.

I don't think that I need the method anyways (I'm going to spend part of today to make sure with more tests!) so I've removed it and tagged Release 80 and uploaded the gem to RubyForge. It should be available to gem update or gem install shortly.

This release also has a larger test coverage and some other unnecessary methods were pruned. Check it out!

February 9, 2008

An evil snippet for hash traversal

Filed under: json, rails, ruby, snippet, testing, work — Lisa Seelye @ 12:28 pm

First the problem:

I'm writing a load test framework at work and I need to consume JSON webservice and pass on the output to another request. But I don't always know what the data is that I need to pass on but I do know the basic "path" ("hpath" -- "hash path") to get to the data as the JSON data is uniform.

Now the method:

RUBY:
  1. # Picks out data from a (JSON) decoded hash based on the @passon hash,
  2. # which looks like this:
  3. # { "id" => "packet.products[0].attributes.id",
  4. #   "quantity" => "packet.products[0].attributes.quantity"
  5. # }
  6. # The "id" and "quantity" are the new keys for the return data;
  7. # packet.products[0].attributes.id will look at the value of the id key
  8. # in the attributes hash in the 0th element of the products array in the
  9. # packet hash.
  10. def pick_out_passon(hash_data)
  11.   return {} unless @passon
  12.   returnhash = {}
  13.   nh = hash_data.dup
  14.   @passon.each do |newkey,part_str|
  15.     parts = part_str.split(".").reverse
  16.     while (part = parts.pop) do
  17.       m = part.match(/\[(\d+)\]/)
  18.       index = nil
  19.       if m
  20.         index = m[1].to_i
  21.         part.gsub!(/\[#{m[0]}\]/,'')
  22.       end
  23.       nh = nh.values_at(part).first
  24.       nh = nh.at(index) if index
  25.     end
  26.     returnhash.merge!({ newkey=> nh})
  27.     nh = hash_data.dup
  28.   end
  29.   returnhash
  30. end

What's it do? It does magic!

I'll step through it...

Looks for @passon instance variable and doesn't do anything useful unless it exists.

Duplicate the input hash because the process done is destructive to it and we may need to reuse it.

For each new hash key and "hpath" pair from @passon split up the hpath into its parts and reverse it so Array#pop will work in the while loop to get the next first part to try.

Since each part of the hpath can examine an Array by index it has to be checked for and the index removed from the part (and saved).

Next, investigate the copy of the hash_data, nh by the computed key; if the value in the hash was an Array use the index to get the desired value. Then compute the next part!

Once we're out of parts stuff the new key and data into the returnhash and keep going til there's no more @passon pairs.

And thus some fun code was written.

December 19, 2007

Reve revision 39 (Eve API Version 2 support)

Filed under: api, eve online, reve, ruby, subversion, testing — Lisa Seelye @ 9:00 am

It's coming. All I have to do is zip the release up and update the RDoc documentation.

This fully supports the API Version 2. Next tasks on Reve include setting up a Trac as well as refactoring a lot of the methods (corporate and personal methods are almost always identical but wrapped in different classes). I also want to add the ability to save and feed the library your own XML.

Look for Reve revision 39 before the 23rd of December.

October 3, 2007

So Rails 2.0

Filed under: REST, critique, oracle, rails, testing — Lisa Seelye @ 12:16 am

It's coming.

The new REST helpers are most appreciated.

Moving the proprietary adapters to a separate repository without any tests makes it impossible to modify the adapters. That irritates me a lot. How are we supposed to test patches that improve thing?

September 19, 2007

The Final Day of RailsConfEurope07

Filed under: RailsConfEurope, RailsConfEurope07, germany, railsconf, selenium, testing, watir — Lisa Seelye @ 10:32 pm

This was the final day.

Cyndi Mitchell's talk about "Bringing Ruby to the Enterprise" was innovative (Wish I could get a copy of her slides!) and really good. I reckon she could have talked for longer.

The "best practices" lecture afterwards (Michael "Koz" Koziarski and Marcel Molina Jr.) was less impressive. It seemed to advocate refactoring an entire application into a completely bottom-up approach. Most of the (contrived) examples were bad ones and readability was not helped at all by the code changes. It's in everyone's best interests to develop their own sense of "best practice" and stick to it. Be ready to re-evaluate and update code if a new technique comes along.

The pair advocated controllers with no more than 7 actions of 7 lines each. It is not a bad thing to have many lines of code in your controller if it's relevant to the action. Having a switch statement to handle various cases of input (especially for polymorphic models) is good. It doesn't help to have a Car controller, a FourWheeledCar controller, a ThreeWheelCar controller, a DiselFourWheelCar controller and so on. They all subclass Car and they should all be in the common controller. When lines should be trimmed is when the same programmatic steps are being repeated: special formatting of return data, emptying a shopping cart (when there is more than one way to empty the cart!) and so on.

I also abstract controller code into libraries when there's a task that I need to do over and over in the console to help other guys on my devteam do their job. This also helps with creating modular application code for multiple clients.

The bottom line is: Who cares as long as everything is tested to meaning, you can understand your code and it won't take someone else who has to maintain the code a month to figure out what it does.

Okay lecture guys, but most if it seemed to be creating work for work's sake, especially the proxy class...

Unfortunately the Ruby on Rails Security lecture by Heiko Webers was...dull. The material Heiko covered seemed to be basic web security. Perhaps it was meant for newcomers?

After that things got interesting. Selenium and Watir!

Selenium was up first in Browser-based Testing by Till Vollmer. This is a REALLY cool utility and I'm sure we'll (at work) take a good hard look at it and see if it meets our needs. In any event it's cool technology nontheless. Kind of like integration testing on steroids. Very cool.

Afterwards Continuous Validation by Dane Avilla was up. His lecture touched on the same topic as Till's but it was from another approach angle: validating HTML, CSS, JavaScript and using a distributed technique to remote control browsers (with Watir) to click through a form. I imagine this could be as complex as Till's demo with a little work. How cool is it to have a farm of all browsers set up to be run after every commit through rake test and report any failures in tests across browsers!

After the Watir one I called it a day.

Railsconf has been very interesting this year. It's been my first time and I may come back, perhaps for the American version, though since Germany is a little intimidating for someone who speaks no German! But we'll see what next September has in store.

September 17, 2007

Refactoring Rails

Filed under: RailsConfEurope, RailsConfEurope07, rails, testing — Lisa Seelye @ 11:44 am

I must say that Trotter Cashion's presentation on refactoring wasn't totally new material. At work we have at least one project that desperately needs some refactor/rewrite love. I'm hoping to convince bossmen that it's a good idea to rework its source. ;)

One of the things he mentioned is that not refactoring is like accruing debt. It made sense immediately. I've said before that it's easier and more fun to work with a codebase that is well written and easy to read. Not refactoring (or rewriting when necessary) is the way to not give yourself a good codebase. No one can, from the start, architect the perfect application, code, or methods. Cruft begins to collect when the coder is under time constraints and begins to hinder work because cruft on top of cruft means a lot of searching to find out what the code is doing before anything can be done!

The next session is the "Flex on Rails" tutorial and that should be painful and evil.

August 24, 2007

Capistrano - an easier way?

Filed under: capistrano, deployment, rails, testing, work — Lisa Seelye @ 11:41 pm

I'll admit it. I haven't gotten around to learning Capistrano 2 yet so many or all of these issues may be solved.

One of our clients at work has a very bizarre deployment environment. To be brief (without getting sacked) it involves svn replication, shared NFS directories and a dedicated deploy box. They also have this setup twice, once for production and once for a "staging" environment.

Since they copy our SVN tagged release to their own SVN repository it means the Capistrano script doesn't actually do the mirroring (yet?); that is done by the client. I wrote a helper bash script that will take the tag, environment (production / "staging") and a capistrano command to select the right capistrano script, set the right environment (although now that I think about it this is redundant now) and run the command.

But there has to be an easier way to "One Click" this with Capistrano 1.x. It is my task to find a recipe to do it!

The problems are, due to the convoluted nature of the deploy environment I may have to abuse roles or invent my own or hack Capistrano to add qualifications to roles (similar to :primary): The dedicated deploy box is the box to which the application is deployed (acts like an app server!) but it doesn't/shouldn't run any mongrels. In a sense it's the primary app server since it has the authoritative source that's shared to other app servers.

But it isn't! It's a nightmare! It needs to be better done. I may try to one-shot this: sh deploy.sh SomeTag production deploy_first or deploy_upgrade.

August 14, 2007

Conquered Oracle

Filed under: oracle, rails, testing, work — Lisa Seelye @ 10:30 am

Yesterday at work I deployed the 'Oracle Project' to the customer's staging server. The only hiccup was (I think) a problem with the renumbering of migrations that was coming from two branch merges and a merge into trunk. I apparently misnumbered one and a critical migration wasn't run. Oops.

I've noticed that at the end of rake db:migrate that Rails rebuilds the db/scema.rb file (nothing new there) except with Oracle each table takes about 45 seconds to process! We've got about 35 models! That's almost 25 minutes! What gives?

Anyways, Oracle is conquered and it is my bitch.

Older Posts »

Powered by WordPress