Original URL: https://www.theregister.com/2013/11/08/something_for_the_weekend_batch_process_nightmare/
How my batch process nightmare was solved by a Wombat
Script me up, baby… ooh how was it for you?
Posted in Software, 8th November 2013 12:16 GMT
Something for the Weekend, Sir? “I enjoy these night shifts but what I’d really like is a six-day week with a three-day weekend.”
Today's pocket sociologist – let's just call him "Seb" to protect his anonymity – might be onto something here. I'd popped into the office on my way home after a long day buzzing from one client premises to another, only to remain chained to my laptop sitting opposite Seb for a further six hours straight, being obliged to beat to death a relatively small task that had for the past two weeks been rejecting all attempts at completion.
Oh lordie, these late night coding sessions do strange things to your hair
Source: Fatal Attraction, Paramount Pictures
And beating it to death is exactly how the process felt. The first week was The Evil Dead, the second Fatal Attraction. No sooner had I wiped the earth from my hands or slumped against the sink, the vicious bastard would thrust a rotting limb from its shallow grave or spring bodily out of the bath.
Under the circumstances, having worked about 15 days in a row, a three-day weekend sounds positively spanky. If only I can nail this fucker, get some kip and fight through Friday.
I think I have another three hours’ work ahead of me, which would mean heading home in the morning daylight again. And then Seb explains that the batch program we use to generate workflow metadata from the raw files has been upgraded.
He shows me what to do, and I can already see that my remaining three-hour manual slog is about to evaporate and er, recondense (damn those metaphors) into a 10-minute automated jive.
I won’t bore you with the details, which you understand is my way of saying I don’t really know how it works, but this batch program began as a script that fit easily on one screen and has since evolved into a pretty clever beast.
Indeed, this workflow manager batch program has even been given an animal’s name – let’s just call it "Wombat" to protect its anonymity – and it is saving staff hours of work every night. Yet it was developed in-house by one dude as a side-project to his proper job. I hear he bashes off another line of the batch code whenever his manager isn’t looking.
This causes me to ruminate. Don’t worry, I’ll clear it up later. But can you remember your first successful exercise in programming?
BBC Micro coding masterclass anyone? Ah, it takes you back – more on YouTube here
I mean programming in the broadest sense: not some compiled C application but simply your first tidy loop in BBC BASIC, your "director’s cut" of AUTOEXEC.BAT or the first time you wrote a snippet of code to persuade that clunky Commodore PET at school to send a variable string to a dot matrix printer. It could just be your first recorded macro.
For me, it was macros. The whole programming language thing passed me by, I’m afraid. Besides, there’s only so much teenage nerdy blandness a neighbour can inflict upon you with his efforts with a ZX Spectrum before you want to kill him. About 37 seconds is my limit.
I was equally hopeless at school, preferring to write confident essays about on/off gates rather than do any practical work. Procrastination is my game, which is why I became a journalist.
Then I got a job, was thrust in front of a Qubié clone and told to teach myself something called Lotus 1-2-3. Three months later, I was the macromaster of Camden and civilisations fell before the awesome power of my slash-file-retrieve. Entire tens of kilobytes – yes, even a megabyte once – of data danced at my fingertips as my megamacros processed monthly management accounts faster than I could swap out the 5.25-inch floppies.
Later, I found myself working on a Lotus customer magazine in which we printed clever "how-tos" on what you could do with 1-2-3 and Symphony with the help of macros. It began with useful stuff such as automated budget-vs-actuals comparisons but after a year of this kind of thing, it all got rather silly.
I remember a daft macro that made column AA extremely wide and jumped to the next row when the cell was full, turning Lotus 1-2-3 into a word processor for nutjobs. We were about two issues away from running workthroughs explaining how to use 1-2-3 as a pair of socks before the mag was canned.
What has always struck me is the immense personal satisfaction you can experience when you successfully run your own macro, batch file or script that saves you, ooh, all of three minutes – despite the fact that you spend the best part of the previous three days trying to get it to work. OK, clearly that’s just me, but you get the point.
Compare this to everyone’s wild dismissal of Big Boy programming. Microsoft Office? Arse fodder. Latest Ubuntu? Nob cheese.
But that 69p app for my phone that makes farting noises? Wow, that’s just the bomb, kids!
To be fair, there’s a good argument that big programming projects are destined for failure. As the IT Crowd observed about MS Vista during a bomb scare…
As for Wombat, that cuddly critter made a changed man of me. I went from staring a nightmare scenario in the face at 2.30am to walking out the door with a spring in my step at 2.45. I love you Wombat, I want to have your little Wombat babies… roasted might be nice.
Now for that three-day weekend of theatre seats, hot-stone massages and take-away curries. Wish me well for my birthday and see you here next week. ®
Alistair Dabbs is a freelance technology tart, juggling IT journalism, editorial training and digital publishing. Looking back, he can hardly believe he could have scripted a nativity play, let alone management accounts for a publishing company six months after beginning his first job. Amazingly enough, IDG Communications lives on regardless.