It turns out that there was a bug with the Amanda S3 device: it had issues with buckets that had a lot of keys (files) in it. Due to the way Amanda works it makes one gig pieces of data on the holding disk and then the S3 device chunks them up smaller and puts them on the S3 device. When the tape/bucket was due to be resused the files in it had to be removed. Fetching the key list (file list) that could be rm’d had a buffer overflow.

I found this out with the awesome help of djmitche (and Freenode #amanda) who wrote a patch for me. I’m pretty sure the patch made its way into amanda trunk. In Gentoo the patch is in app-backup/amanda-2.6.0_p2-r1.

Since applying the patch I’ve had no problems with Amanda at all and it’s run perfectly for me.

The /dev/hd[egik] device nodes were owned by root:disk and 660. However, the hddtemp binary was trying to do some restricted ioctl operations. Even though the binary was setuid and my non-root user was in the disk group I couldn’t access the disk to get the temperatures.

I found out that there is a hddtemp daemon which can query the disks for my unpriveleged user.

The daemon listens on a TCP port and provides output like this:

|/dev/hde|ST3160811AS|39|C||/dev/hdg|ST3160811AS|40|C||/dev/hdi|ST3160811AS|38|C||/dev/hdk|ST3160815AS|39|C|

Should be easy to modify my disk_temperature.rb to connect to that port and parse the data.

Now my temperature graphs will have new data (as soon as the change is made).