We call this "folk software" at my workplace.

The multi-national company I work for had a chunk of government-regulator-critical data running on an "unapproved" Access database.

Fortunately it was very well written by someone formerly from a company with a big blue logo - but once they left, the IT dept. didn't want anything to do with it because *they* hadn't written it (aka made one that DIDN'T work but officially) themselves.

I seem to recall a lot of high-level argy-bargy, but the result was we kept the database because they weren't able to replace / reproduce it :-p

Elsewhere, I cobbled together some small utils in QBasic (!) that sifted data sets and kept our heads above water for two months because a planned "merger" of work from disparate systems wasn't planned, implemented, working, or able to do what was needed (and still isn't to a large extent <cough>).

Congrats on your first article for El Reg Aubry.

I haven't experienced much of this, as I've worked mostly in embedded systems - there's thankfully not much scope for people to knock up a microcontroller system in a lunchtime.

I have, however, cursed the day Visual Basic was born on several occasions, having been subjected to various half-baked internal solutions over the years.

I've got nothing against making programming easier in principle though. The commercial packages I am forced to endure on a daily basis are bad enough. In many cases you'd be hard pressed to tell the difference between them and something built by a history teacher in his spare time, unless you look at the price tag.

There is a difference between FOSS and Folkware, Most of the time if a piece of free software is created for some use it will only be used heavily if it's technically good, this is where having the many eyes outside of a single organisation can weed out all the bad projects.

You never hear about the poor quality FOSS projects simply because they die quickly. It's like a rather extreme version of software evolution with stuffy technical IT staff acting as it's reaper.

So in the "library database" scene, what would really happen if the folksware writer showed up in the early stages of the grass-roots rollout stage and asked to hand off his code? The IT department would (and should) flat out refuse, firstly because they didn't write it, and secondly because they were not funded to support it. Probably also at that point the true need and value of the solution had not yet been established with the IT community. They would not "get it" at that early stage.

Home-grown scripts/tools/etc are *incredibly* useful, and in earlier days I made way more than my share of them, but the naiive creators don't realize that writing the code is a drop in the bucket of the overall life-cycle costs. The expensive part is in maintaining it. Its like buying someone a puppy. You have to feed it forever. It is even pretty common to have users write great stuff but then bail on the upkeep since it becomes such a drag to their real work.

I wish I knew of a better solution to this as there are truly lots of good ideas but the IT or development departments are usually already short-staffed to deliver the big things.

I am a former "folkware" author. Fresh out of university I signed up for a temping agency, who sent me to a relatively recently privatised business (not giving away much there, I know) to do office admin duties.

They had an utter decrepit old Access database system that involved some poor bod inputting around data five times over whenever they wanted to add a new record (it was all based on *one* table). After a few weeks I couldn't take it any more, and over the course of a few months I pulled the thing apart and remade it as an actual relational database with a .NET front-end application that actually made sense of what was going on. Of course, I was learning it all as I went along, so the backend code was a hideous, hideous mess.

I almost feel guilty, but I want to stand up for folkware programmers out there who are doing a job no-one in the IT department could give a crap about. I tried to involve the IT team at every turn, but they barely never replied to my e-mails, and when they did it was to decline whatever request I made. My (non-IT) manager was, of course, happy to take the credit for the improved department efficiency, so I became a software developing office temp, paid £6 an hour.

So yes, some bugger in my old company's IT department is, or soon will be, pulling his hair out over this uncommented mess of code an entire department has been using for a year and now relies on. Perhaps once he finds my name in there he'll remember me e-mailing him, asking if I can have Visual Studio Express installed, or if there's room on the SQL Server for another database- and perhaps he'll question why he would just deny these requests, and not at least ask what the hell I was trying to do.

I somehow doubt it he'll remember me, though.

(posted as AC because, well, you never know...)

Everywhere I've worked, there's been "Folkware". The only way to stop it is to engage with the users and find out what they actually want. On current form, it's never going to go away.

Surely you jest...

The current project I'm working on dates back to the 60's (so I'm told, and I believe it!). It is a giant group of Fortran programs (remember those) that have been pieced together step by step to produce an income generating system (yes, money is made on it!). Now nice people like me are brought in to add features to it, or otherwise alter the code, and while I pride myself in being a reasonably competent programmer, I wonder how this code ever got off the ground. Comments are few and far between, and most of the code (including some recent additions in C) look like they have been written by a Basic (the language) programmer (who learned on a C=64). In any event I persist as I know I will have "full employment" for quite a while.

The only saving grace is that there is some sort of version control, and we can always point a finger at the person who "broke it".

Life goes on.

Advice to budding programmers: Comment for yourself, in 6 months you won't have a clue what you were doing, and the comments will be your OWN salvation!

I worked at a museum, and it took all the support I could manage to get a computer (over the opposition of a pointy-haired boss).

Buying software? "What? You want MORE?"

IT department? I was the only one who could do anything besides word processing.

So I wrote a dBase III program to let me keep track of what we had, and later ported it to Microsoft Access. I'm retired now, but they still have to haul me in as an independent contractor to teach the new curators how to use the thing.

But I tell ya - nowhere in there was there a chance for me to actually BUY software, nor did we have a real IT department. It wasn't ideal, but it got done. If I hadn't done it that way, it would still all be pieces of paper covered with handwriting, and a picture (formerly) stuck on with rubber cement.

So give me an IT department, and I'll be glad to let them do the work. As it is, the program I wrote in 1985 and ported about 2000 is still doing the job.

"A project under any coding philosophy is only as strong as its weakest development team member and mashups allow everybody to be a member."

And that is as valid a good reason to ensure that any super programme is coded by a development team of one .....with a clear and abiding philosophy.

The future of software has to be user generated applications. As the technical garbage that is today's code becomes open, standardized, and easier to use it will eliminate many of the "specialists" required to create applications today. Removing people is the best way to reduce IT costs and making it possible for people to create their own apps is the best way to do this.

It may be a pain during this transition process, but sooner or later coders are not going to be nearly as important as they are today. This is a good thing.

Solomon,

I would disagree and suggest that specialist coders will become the new Masters of the Universe as they recode the System to ensure their Reward whilst also attending to the Systems needs. And they will do this be presenting a Fait a Complis to Systems Administration which they would recognise as being capable of Crashing the System Catastrophically, should the Code go Wwwild. Ye Olde Danegeld Solution favoured by the more Intelligent Pioneer/XXXXPlorer.

Having such a coder on the home team ....and at whatever cost .... is not a cost, it is an inward investment to counter any possible discovery of the vulnerability should it be deemed too onerous to change. Although to maintain such a vulnerability would clearly be the dumb ass option, rather than fixing it so that it disappears.

After All, it is only Paper Money you'd be resting with him/her rather than resting it in some Skull and Bones numbskull vault for plonkers to play with.

When IntelAIgently Designed to Evolve, IT could even claim to be AI Prequel to Kickstart Moribund Economies and who would want to keep that Information under wraps whenever it is out in the Wwwild? That would only, quite rightly, Lead to a very Public Disaster rather than Build on a very Private Solution for the Future.

"Removing people is the best way to reduce IT costs and making it possible for people to create their own apps is the best way to do this."

But it's simply not going to happen. Parametrization - yes. Creating apps - no. The last 30 years have conclusively shown that the complexity of the application does not go away if a new development tool comes along. Trained people may be able to develop faster but untrained ones will still be unable to develop.

Remember those ads in the 80's mags? "Mr. [mgt guy] has gone Code Free with his new [some tool]. He swears he will never go back."

Mr. [mgt guy] was part of the problem.

My (former) employer had a large IT department. Unfortunately, setting development priorities was a giant political exercise where the winner was the manager with the most clout. (We had a regionalized structure with 20+ semi-autonomous sub offices. The heads of these offices were in constant competition with one another to climb up the ladder into central management. Backstabbing was perpetual and he who got the largest proportion of his pet ideas implemented won.)

The net result was a system that often did not deliver what the grunts at the working level really wanted.

Furthermore, there was no ongoing process to prune away unnecessary complications, hence the entire system gradually became extremely involuted, convoluted, over-elaborated, and fine grained. After some 25 years of adding innumerable bells and whistles, the overall system had become so unwieldy that it had to be replaced by a wholly new system written from scratch.

Moral: involvement of an IT department is no guarantee that those at the working level will get the systems they need.

As for folk software, one insidious tendency is that supposed one-off subsystems (typically, complex analytical reports) cobbled together from bits and pieces, like Frankenstein's monster, show a tendency to come to life. Being jury rigged, they are of course wholly unsuitable for production use and upgrading them becomes a major issue.

Moral: When someone says "oh, we only need it this once", don't believe them. They'll be back next year saying "where is that annual analysis you did for us last year?"

The IT departments that I've worked for over the last couple of decades have invariably reminded of "Mordac the Preventor" from Dilbert. The author here seems to have been an unfortunate proponent of that philosophy.

A sorry combination of "we aren't going to support it because we didn't write it" with a hefty dose of "we aren't going to write it because we don't think you need it even thought you're asking for it", and a dash of "we'll tell you what you want, how dare you tell us how you do your job".

Yes, jury-rigged apps are dangerous. However, the flip side of the coin is that if people are USING them, perhaps IT should get off its arse and make proper applications with the same functionality actually available. Instead, it was invariably a case for the business end of "better to beg forgiveness than ask permission". With the expected results.

So we have a system where the people on the front line of both IT and the business all know what is really needed, but it has to go through so many layers of management that by the time IT gets a hold of it, it bears no resemblance to what was required in the first place.

We had tremendous success at one workplace with a "semi-official skunk works project office". People on the business end would come up with nifty ideas for apps, someone in IT would hack it together PROPERLY (ie: designed, documented, tested, etc.) but in a minimal-overhead environment with high user involvement, and the user (or small group) would go off quite happy. If the app became popular it was then already in the IT system and it could then more easily be made more robust (having been designed right in the first place) for more extensive deployment. If it died only a few hours of someones time was wasted. The business case was that we were writing prototypes for new apps, and the users were happily testing them.

This system worked very well.

Unfortunately, that system died when the IT dept was outsourced to a large concern with IT "managers" who held their positions more due to their arse kissing abilities than their ability to actually get things done. Suddenly the people in the business couldn't get what they needed, and the company quickly became stupefyingly obtuse in its use of technology.

The team disbanded very quickly, and we went our separate ways. Pity really.

The Moral and XXXXPerience of the Story Being, yeah, right, Skunk Works work.

However, such is their Stealth Requirement to get IT done Properly, they invariably work "Innovatively" ....... and as for the Money requirement, that also, should the Establishment be too immersed in maintaining their own importance rather than ensuring AI Future Importance, it too will be provided "magically".

And if the Powers that Be are not involved in that exercise before they are compromised, they aint gonna play any part in any lead role in any Stealthy Programming and ITs Programs for they would have proved themselves to be Unfit for Purpose ........ Such is the Enigma and the Essence of the IT Mirror on the Hellerish Catch 22.

cc Thames House/PO Box 1300/Sadville Reality Spooks in AI Virtual World not of their own Choosing?

And reading between those lines of Steganographic Code, would IT render an AIdDelivery Invoice still waiting to be Paid and Creditted to Account? An Open Invitation to Supposed Intelligents Serving to Prove themselves Worthy of Colossal ARGonautics....... Virtual Reality for Real?

amfM Astute NEUKlearer HyperRadioProActivity would tell you IT42B, most definitely and most definitively so.

And the Alien because things can be Real Spooky whenever you want them to be so....... but that would only be by Choice for and in an Artificial Design.

my favourite user app bummer is some genius hardcoding the server name or IP address, compiling the code and then losing the source. Hurrah! Trying fixing that in 5 years time -

"but i said we'd support all these apps - can you not just write a script to scan for all the hardcoded IP addresses in the code & change them....?"

"Well see it's COMPILED code - so we'd need to write a script that would magically locate the old CD with the source on it down the back of a filing cabinet in a skip somewhere; stick it in the CD-ROM drive & then scan that....."

Did once hear about a wonderful chap who wrote some very specialized stuff for the MOD & documented it all in Welsh, hence ensuring his support contract going forwards,,,,

Not just one, but *two* comments from AMFM... I feel honoured.

@yeah, right: Actually, we would have welcomed it with open arms if it came with a letter of recommendation from the librarian (not just the guy who wrote it). At the time, there was NO library software available for anything less than a full-blown university. *Anything* capable of doing the job at hand on a 486 would have been accepted. As for your own experience with the outsourcing, it doesn't surprise me - most outsourcing companies are interested only in delivering what has been contracted... not making it maintainable by others.

As for commenting, when I became the man in charge of overseeing the QA mechanism (I didn't duck fast enough at the meeting... OK, so I was dozing) I quickly wrote a script which would strip out everything in a source-code file *except* the comments. If I could not figure out what the code was doing based on the comments alone (note: *what*, not *how*) without having to trawl through the code, then the file was sent back. YMMV, of course, but it worked for us.

My last soliloquy will be about legacy systems. A little off-topic, but with the same flavour; I recently had to do a gap analysis on a legacy system with views to replacing and archiving it. Ye gods. It was a mess, even with all the documentation available. Having seen the spaghetti coding made possible by the current trends of OO development (I have nothing against the OO philosophy, just some people's implementation of it), I feel sorry for the poor sods whose job it will be to do the gap-analysis on *those* systems. With luck, I'll have retired long before this happens.

Bad memories of the '90s here in Canberra, when everyone was writing their own MS Access databases. Yes, it's so much easier and quicker for a user to hack something up in access or excel ... until you find that your organisation has come to utterly depend on some hunk of cruft.

It is as true as it ever was: if something's worth doing, it's worth doing well.

My own personal self destructive, undocumented, and unmanageable entrails of "software" had already slimed their way from NTL to Motorola, Barclays and several others who really should have known better before I even turned 18! It wasn't until I returned to education that I eventually learned what a mess I had left behind. Don't doubt that somewhere a few GOTOs of mine are still running the show :)

This sounds like a lot of "not invented here" griping.

People usually don't take time out from their day jobs to write software unless they really need it.

And users don't have a monopoly on writing/procuring difficult-to-maintain software.

Yeah, it may not be the best written software, but if it saves them time, it probably has a net positive effect on the business.

I'd guess that the real reason IT/Management don't like "folk software" is that it makes them look incompetent. If you're so disconnected from the coal face that you don't notice an application becoming ubiquitously deployed and used in your organisation, you've got big problems.

The best response to this stuff is to move the authors to IT, where they probably belong (with a bit of mentoring). Integrating power users into IT is a good way to improve IT's understanding of the business.

John

Mashups worry me - it is like building a tower block on sand. You have no guarantee that the various data sources will remain unchanged and available for any length of time at all.

Even when microsoft changes the macro structure of word and excel - again - you have the chance to still run office 97, or whatever. If the provider of your mapping, or ethographics, or images decides to re-organise it's just tough.

Information crucial to the business should be managed by the business. If that management includes appointing and paying for IT staff and software, it has to be done. If it isn't, some alternative has to be provided for and funded.

McDonalds clean the floors and tables in their stores. They don't put the ovens on wheels and move around looking for areas recently cleaned by someone else. Data is the same - you need to look after it.

It's not that IT doesn't want to help, it's that users don't understand the impact it has on the infrastructure, cost, and managers that won't green light or fund any development.

Even if the users create a system that works (rare), it is frequently not backed up, secure, documented or scalable/adaptable to changes in infrastructure. It may also be taking up valuable resources - what happens when it does housekeeping at 3am and the backup/the program fail because one is taking all available resource?

Managers need to listen and ok the IT department to carry out the work. IT will quite sensibly refuse to create systems if all it will do is create management hassle for something that 'isn't their job'.

The bottom line is that if IT aren't allowed to help directly, the users should ask their managers. If managers won't help, then don't write it yourself, as IT will simply junk the project due to the management restrictions.

The only way you'll get IT to write software they're not permitted to write, is by demonstrating that the effort will reduce IT's support load. Whether it reduces the user's workload is not relevant here. Understand that IT do not exist primarily to help users - they exist to maintain the infrastructure and follow their assigned job role. In a sensible organisation the job role will be 'do whatever necessary to the infrastructure to make everyone's job more efficient' but often there's a balancing act with limited hardware, software and manpower resources.

If someone ignores the above and is still stupid enough to create their own system, then remember the golden rules of development : 1) use something the IT department use and 2) Don't use something that has licensing implications. Using a obscure app that is 50 quid a seat will lead to the user app to be instantly binned and banned.

"McDonalds clean the floors and tables in their stores. They don't put the ovens on wheels and move around looking for areas recently cleaned by someone else. Data is the same - you need to look after it." .... Posted Monday 21st January 2008 10:05 GMT

Hmmm?

And Data is just like a menu, Robert E A Harvey. Guard the nuggets upon which the business is built but offer tasty temptresses to BetaTest for Future Foreign success. But also bear in mind that the Master Chef will not be satisfied serving the same successful/popular dish, no matter how much it may be worth as there is always something new to be invented and pushed/pimped.

Binary Manipulation of Code Presents an Artificially Real/Digitally Enhanced Picture of the Future which IT and Media "nurture" akin to Genetically Modified DNA Seed which Science grows, except with Binary Code the Picture is Manmade without the vagaries/wild card of Nature?

Although a Crazy Computer Scientist may be hell bent on Destruction rather than Construction.

Eventually, the author was found, questions were answered and a compromise reached: the teacher was pulled from his teaching duties and moved to head office for the maintenance of the library software.

The teacher was now folded into the IT team, which meant the maintenance of the software could be properly managed and documented should he ever leave or once he retired. It did mean, though, this one teacher never taught again.

*******

So they took a willing volunteer with a successful idea and entwined him in a place he should have been most use? And this is not better than them hiring another teacher to replace him?

I would have thought the difference in having the lowest paid IT staff in the business eked out with someone with sharp end experience of what the educational needs are would be no bad thing.

Or are IT staff employed in schools paid what they are worth in some other places?

Because mashups occur ontop of a stable system (assuming IT department has done a job here!) the risk of damage by folkware is negated. Corruption of data and overuse of resources is restricted. So from there on in it is all productive win.

If IT dept can't produce suitable interfaces it's a tooling or imagination problem.

So has the new technologies created a change in systematic outcomes since the authors bad old days? I for one can see it has.