So it’s been a long time since I’ve posted anything so I thought I’d give an update so that there is something here.
I love my job to death! I work with awesome people and have a lot of cool stuff to do. I’m also climbing three times a week, which doesn’t leave me with a lot of time to code or be at home. When I am home I’m working on a STOMP library for the Arduino Ethernet Shield.
The STOMP library is going to be used in part of a home automation system my partner and I are working on. I say “working on” because she’s just as busy as I am and doesn’t have much time to work on all of the projects she’s got on the go.
So I’m still around and just really busy!
So I’ve been very lax in posting. I’m very busy lately with rock climbing and work and other social things.
I have so many things to talk about like the awesomeness of OS X Server, MCX, LDAP and other nifty things. I haven’t been doing much Ruby development lately since I have other priorities. Reve is moved over to github.
I will no longer be updating svn here at crudvision.com. I want to move Reve to git (at github). I won’t do it if I can fall back to svn, so I’m jumping ship.
Moving to github will also mean I can dump dreamhost as a host and move crudvision.com to my colocated machine to save money each month.
I’m sorry for the complete void in posting. I’ve been very busy with life, work and rock climbing. Yes, that’s right, rock climbing.
In early March I started a new job in Toronto for a gaming company (I’m not really allowed to talk about it much, unfortunately). I’m a sysadmin there and I love it. It’s much better than developing the same web applications over and over. I’m almost ready to resume programming for a hobby now that I don’t do it all day every day for pay.
And the rock climbing. My partner and I started climbing recently. We’ve noticed an unusual number of IT professionals that climb! It’s cool and odd at the same time.
I hope to post more, but the real goal of the post is to say that I’m ditching subversion, trac and intend to use github as the only source of documentation for Reve. (I just hope I can get traffic from github to this blog to pad my ego.)
Yesterday we launched Mini Match, an application me and my colleagues from work wrote for Cartoon Network.
Late last year we opened a beta that was, unfortunately, short lived. Friday (May 30, 2008) we opened the system up on a much improved codebase for a few hours and had all systems “green”. Based on the positive success from Friday we opened it up yesterday.
It opened slowly at first with just a small advertisement on the Cartoon Network Games’ Page, and then a larger one on the same page and then we made the Cartoon Network home page with a small advertisement again. Today, I reckon, a larger advertisement will be put on the front page and we’ll really start to see traffic!
Some details on the application:
- Flash/Flex/AS/Whatever front-end GUI (really, it’s one of them)
- Java-based persistance server
- Rails-based funnel into the database with a bit of logic.
The Rails part is RESTful (for the most part) and is the “glue” of the application, to quote someone from IRC.
Today should be a fun day!
This coming Monday I’m flying off to Atlanta, Georgia for the week. Not keen on going as Atlanta is like descending into the armpit of summer. I’m hoping that the weather co-operates and I don’t melt. 
I’m flying down there for work, we’re beta launching a product for a customer (that’s based in Atlanta – go figure) and they (the customer) have requested a contingent of us from work be on site for the release. So it’s me and two others from the UK.
We’re down on Monday then flying back home on Friday evening – I’m hoping to be in bed by midnight (landing 22:05 – 30 minute taxi home, getting bags and immigration and such). A SysAdmin that works for our customer suggested Zuma sushi bar to me for sushi. I like sushi a lot and hope to stop in for some glorious sushi.
I received an email to my work account today that started with “Dear Sirs…”. The email was sent, presumably, to a large number of customers at once about a networking issue by a UK hosting company.
I am female and at the time the salutation bugged me. In normal society one doesn’t generally greet a member of one gender with the salutations of the other. When I received the email my brain put together discussions from LinuxChix mail lists and other recent discussions on women in IT.
I took offence at the salutation “Sirs” because it implies that there are no women working in IT which is clearly not the case.
Anyways, I sent an email in reply to the email (which went to a support@ email) saying that I didn’t appreciate being addressed as a “Sir”. Less than 25 minutes later I received an email from the managing director of saying he wrote the message and thought the salutation “Sirs” was …was an acceptable greeting if the gender of the other party is unknown. I’m not sure I agree but I appreciated the feedback anyways and replied to him thanking him for taking the time to reply. Hopefully he will do as his email says and take my feedback on board and change the way his company addresses emails to women.
My good deed for the day is done.
First the problem:
I'm writing a load test framework at work and I need to consume JSON webservice and pass on the output to another request. But I don't always know what the data is that I need to pass on but I do know the basic "path" ("hpath" -- "hash path") to get to the data as the JSON data is uniform.
Now the method:
RUBY:
-
# Picks out data from a (JSON) decoded hash based on the @passon hash,
-
# which looks like this:
-
# { "id" => "packet.products[0].attributes.id",
-
# "quantity" => "packet.products[0].attributes.quantity"
-
# }
-
# The "id" and "quantity" are the new keys for the return data;
-
# packet.products[0].attributes.id will look at the value of the id key
-
# in the attributes hash in the 0th element of the products array in the
-
# packet hash.
-
def pick_out_passon(hash_data)
-
return {} unless @passon
-
returnhash = {}
-
nh = hash_data.dup
-
@passon.each do |newkey,part_str|
-
parts = part_str.split(".").reverse
-
while (part = parts.pop) do
-
m = part.match(/\[(\d+)\]/)
-
index = nil
-
if m
-
index = m[1].to_i
-
part.gsub!(/\[#{m[0]}\]/,'')
-
end
-
nh = nh.values_at(part).first
-
nh = nh.at(index) if index
-
end
-
returnhash.merge!({ newkey=> nh})
-
nh = hash_data.dup
-
end
-
returnhash
-
end
What's it do? It does magic!
I'll step through it...
Looks for @passon instance variable and doesn't do anything useful unless it exists.
Duplicate the input hash because the process done is destructive to it and we may need to reuse it.
For each new hash key and "hpath" pair from @passon split up the hpath into its parts and reverse it so Array#pop will work in the while loop to get the next first part to try.
Since each part of the hpath can examine an Array by index it has to be checked for and the index removed from the part (and saved).
Next, investigate the copy of the hash_data, nh by the computed key; if the value in the hash was an Array use the index to get the desired value. Then compute the next part!
Once we're out of parts stuff the new key and data into the returnhash and keep going til there's no more @passon pairs.
And thus some fun code was written.
With Rails there's a useful set of callback methods that can be used within model classes. They include after_create, before_validation and the one I want to talk about after_initialize.
A lot of cool things can be done with after_initialize, especially if one uses the database as a means to store meta-data about a model. Use the after_initialize method to transform your models with Ruby after they're fethced from the database. Consider the following User model definition:
RUBY:
-
class User <ActiveRecord::Base
-
has_many :memberships,
rder =>
"memberships.group_id, memberships.expires_at desc", :group =>
"group_id"
-
end
It's pretty clear that with the :group => "group_id" that I want a unique set of Membership objects. That works well when I've already got a User and do user.memberships but not at all when I do User.find(:first, :include => :memberships); likely because of the way the join is set up and grouping on that may not be possible.
I still wanted to use eager loading so I thought that it would be an OK sacrifice to fetch all the Membership objects even if I wanted to pare it down after I initialize the User object (with eager loading). I defined:
RUBY:
-
class User <ActiveRecord::Base
-
-
def after_initialize
-
do stuff with @memberships instance varible
-
end
-
end
Except it doesn't work. Quickly I found the following from active_record/base.rb (click link for source): right where it does the after_initialize callback! But it wasn't working for me. I needed to dig further.
Further research indicated that before the creation of the @memberships instance variable that would hold the collection of Membership objects the after_initialize callback was being fired off in the User model. Using after_initialize would not work due to the design of ActiveRecord. I'm still searching for an elegant way to work the way I want. I'll post again when I find it!
I've submitted my first patch to the Rails trac today to speed up the OracleAdapter#indexes method (used in migrations). A client at work uses Oracle and in the production environment this method was taking upwards of 45 seconds to run per table!One of their DBAs gave me the SQL there, so I can't take the credit, I just submitted it.
I hope it's accepted. Incidentally, it works and has saved me a lot of time with migrations.
Edit: Yeah, turns out that since turning these adapters into gems patches for them shouldn't be done through Rails trac. How irritating, but whatever. I've sent the patch do the current maintainer for review.
One part of code optimisation that is relatively easy is to cache the results of expensive operations. It's really simple. CACHE.set. But it isn't so simple. There are a seemingly endless possible ways the cache can be used improperly or otherwise fail. Handling these problems is the real work of caching.
The last checkout (around rev 250?) I did of Cache_fu (the acts_as_cached rewrite) it was pretty immature, chiefly in our need: An easy way to do arbitrary key names. n.b., I'm aware that there is a way but it seemed against the flow of this codebase. I use it, still, for the magic :all keyname.
But I need more, to store created JSON (To create what results in 100KB of JSON is a lot of objects and a lot of create time!) with a dynamic key in several distinct situations.
That round of caching improved API calls from 3 req/s to 300 req/s.
For the application most of the data can be cached but it needs to be done smartly: memcache server failures need to be handled (I suspect they are not, currently), server layout needs to be better (One daemon on each app server is not good, long term, IMHO), expiration of caches on data changes across the entire farm need to be smarter, deciding how long to cache data needs to be done.
CACHE.set and CACHE.get is really misleading; there's a lot of things to consider!
I relish these challenges and look forward to solving them.
