Spent the past several days brushing up on elegant ways to do various kinds of operations in shell scripts. Learned quite a few new things. Under the pressure of the interview, I reverted back to the quick and dirty ways of every day writing of shell scripts. Now that the pressure is off, I’m also thinking of the mistakes I made. Doing a pre-increment when I really should have used a post-increment. The kind of mistake I wouldn’t make except when I’m under the gun and second guessing everything I do.
We’ll see how things go. If it ends here, at least it ends. I can move on and start looking for other work. Tomorrow the free ice cream will once again flow.
I’m not sure if you realized this, but you accidentally used the words “elegant” and “shell script” in the same sentence. I’ve done a good bit a shell scripting, mostly bash, and none of it is what I would call elegant. If you want elegant, use Ruby.
Did you get any of the off the wall questions during your interview? I recently got asked how I would describe color to someone who was blind since birth, and how many dump trucks would it take to move Mt.Fuji? (while interviewing for an IT position)
I would be curious to know what they asked you to script, as it may help me in my planning/preparation for future interviews.
I would consider myself an average scripter but having to worry about pre vs post incrementing seems like they were really challenging you.
It was a phone interview, only on shell scripting.
Was that an SA position or a job doing shell scripting only? A job doing shell scripting only sounds like it would have a high suicide rate.
If you’re administering clusters, pretty much everything is a script. You also want the computers doing as much of the work for you as you can, so you automate recurring tasks. Though, in this case, shell is just a vehicle for testing me. Python is fast becoming the language of choice for scripting, and Python rocks.
“Python rocks” Ummm…. nevermind.
Writing scripts from scratch to manage clusters is certainly better than doing it manually, but that is certainly no longer state of the art.
So Google isn’t state of the art? Because that’s pretty much how they are doing it.
Yes, and the roll-out of G+ was flawless wasn’t it? Oh wait!
Whenever I hear things like “this is what Google uses” (or substitute some other high profile company), my first question is “oh really! are you using Google’s budget too?”
That’s not to say that scripting doesn’t work, and in some places it is unavoidable. But the trend in the industry is to use more specialized tools that require a lot less development and QA time.
I would also think that Google, being as large as they are, uses a mixture of things. Older installations likely use the older methods, whereas the newer installations and services likely get the newer toys. That’s fairly typical. So a blanket statement about Google or somebody else is probably not appropriate. That being said, I know plenty of people who work for Google… I’ll inquire.
Best of luck on the job hunt.
Just saying I’ve never worked in any distributed HPC environment where I wasn’t doing a lot of scripting. Even if it’s something like adjusting the default settings on a disk mount in /etc/fstab… if you have to do it on 500 machines, there’s not really getting around doing a script, even if it’s a ten liner. I never had the time or the budget to seek out off the shelf solutions for every problem I ran across. Sometimes taking an hour or so to script out a solution to a problem is the most time and money efficient answer.
Google’s infrastructure is very specialized for their own applications, and everything is engineered to run on cheap commodity hardware. I don’t think something on their scale could be easily re-impelemented with off the shelf solutions. But nor is everything they do suitable for every company. They have white papers published for most of their infrastructure, which gives you a pretty good sense of how everything is put together.
“Even if it’s something like adjusting the default settings on a disk mount in /etc/fstab…” That’s the exact sorta thing you want a distributed host configuration tool to do for you instead of writing a generic script.
And what’s the licensing going to run on that spread across 1000 nodes? :)
They are open source, but you are more than welcome to pay me $10/node. :) Look into Chef and Puppet. There are a couple of others too.
Some OS vendors and independents do have commercial offerings, and app-server vendors also have equivalents taylored to their products.
I’m a python dev and startup guy so I have to do sysadminy things regularly. Andy is right that distributed bash scripts are going away in favor of other tools, but the trend is quite recent; it’s only picked up steam in the last year or so.
* Configuration Mangement (syncing your /etc files, crontabs, and installed packages): Puppet & Chef are the current kings. We just finished converting our bash bootstrapping scripts to puppet and it is quite nice.
* Scripting: when you need to tell a server to do something _right now_ there’s Fabric. It’s basically a thin python wrapper around ssh that lets you script things like deployments. We used to do a weekly midnight deployment with all-hands (and usually some downtime) but after getting fabric up and running we deploy several times a day no hassle.
The tools have come from startups where devs do everything in the stack from customer support through sysadmin (“devops” is the new buzzword). Even with all those new tools we still have a part time sysadmin consultant because he has specialized knowledge; we ask him to recommend a configuration instead of benchmarking it ourselves because he’s either benchmarked it already or can do it quicker. He also deals with colos & cloud providers regularly so he can get issues resolved faster than we could ourselves.
Hope that helps,
-Jack