Better living through xgrid

Please note that all blog posts before 8 April 2007 were automatically imported from LiveJournal.  To see the comments and any LiveJournal-specific extras such as polls and user icons, please find the source posting at http://brianenigma.livejournal.com/2006/10/

After some trial and error, I learned a few things about the OS X xgrid distributed computing technology:

1) xgridctl and xgrid are pretty powerful command line tools and really take care of all of the heavy-lifting of distributing an application. You say “xgrid -job submit /bin/echo ‘hello world'” and it’ll make sure the program and arguments get out to another machine and the results get back to you. Presumably, the application would need to be a universal binary if there is a mix of PPC and Intel boxes on the network. Supposedly, there’s even a Linux xgrid client, but I have no idea how (or if) that would handle these OS X binaries. It would be useful for distributing shell scripts, though, and you can be pretty creative with those.

2) When specifying a controller on each of your participating clients, do it by IP address and not machine name because only numeric IP addresses and DNS addresses will resolve–not that this is mentioned in any sort of documentation. Local machine names don’t resolve, even though they do with all other services/commands. It doesn’t even seem to consult /etc/hosts.

3) If using VisualHub for distributed video transcoding, your distributed clients can’t be using password security. Their authentication method must be set to “none.”

4) Make sure your source video files are on the hard drive when you send out a batch of transcoding jobs. If you’re trying to convert a DVD full of TV episodes from mpg/avi/whatever to iPod h.264, the CD ends up thrashing around too much, becoming a giant bottleneck. Also, gigabit ethernet is similarly helpful.

And that, my friends, is what I learned today. I’m not sure what, how, or why, but I plan on using xgrid in some future project now that I see how easy it is.

Posted in: Code Software

Leave a Reply

Your email address will not be published. Required fields are marked *