I've often wondered what the software development world would be like if we used prototypes. Experimental hacking seems like a cut-down version of this concept.
But what if we actually built things that we already agree in advance to throw away? Maybe even start from two or three different plausible designs, and push on each one for a while until one seems to be winning? Then rewrite it, learning from the other contenders and from the prototype itself, all before shipping.
The obvious answer is that it would take too long. But I'm not so sure. They say designing software takes too long, also, but when I spend a few weeks designing, the implementation ends up going smoothly and hitting the target. Usually the features that are designed thoroughly at the start of the release hit the target, and other "quick" features that are added in later, sans design, take longer than the designed features and end up pushing the release out. Any overruns or missteps in the implementation of the well-designed features seem insignificant in comparison to things that skimped on design work.
Dr. Winston Royce came to this conclusion over 40 years ago. Unfortunately people read his paper and developed the Waterfall methodology instead. See Step 3:
This is a well-discussed concept already - see the concept of "Spikes" in Agile development, or to a lesser extent, the idea of "Tracer Bullets" from The Pragmatic Programmer.
Of course, what's done in practice tends to differ, unfortunately.
I like Fred George's developer anarchy in this regard - especially the concept of micro-web-services. they are the equivalent of a unix command - do one thing (well) and join up with MQ. if you make a service small enough you can be confident of putting it up as a prototype and rewrite it next weekend in node.js
I always did this, it's just how I code. Trick is to keep them short and fixed duration under a day, preferably under half. a) that's where all the value is (longer and you're just going down the rabbit hole), and b) gives you time to quickly write production code based on the spike/prototype.
It could be argued that rapid prototyping is modern software development.
In the late 80's and early 90's, rapid prototyping was very fashionable in the 'anti-waterfall' software engineering schools of thought. The idea was to use dynamic languages with good development tools (various Lisps, Tcl/Tk, Smalltalk) to rough out the idea, which would then be rewritten in 'real' languages.
Of course this tactic failed successfully! We shipped the prototype. Nowadays, it would seem absurd to develop a web application (for example) written in a 'real' language like C++ or PL/1.
I'm doing this at the moment. We have a gnarly ball of legacy code that is burning out devs. It actively resists any attempt to disentangle its parts and after years of attempts at refactoring it's gnarliness is barely reduced. I'm of the opinion that the actual problem it is solving is not nearly as complex as the code. We've extracted a smaller problem that contains all the hardest parts and are solving it with 2-3 different approaches. A completely naive implementation of the core functionality took only a few days and is already faster than the original.
The trouble at the moment is convincing the management that a) this is productive work and b) it really is time to (incrementally) rewrite this code - it's beyond saving. From their point of view it looks like we want to scrap something that sort-of works most of time and do all the work from scratch. It's difficult to convey just how much psychic damage the current code is causing to someone who isn't buried in it day-to-day.
problem is any prototype that provides say 75% of the functionality will end up being pushed to production as soon as someone above developer level gets wind of it.
I throw away prototypes pretty often. Spending a short time thinking through a problem by experimenting with it is more in line with my kind of thinking than writing detailed design documents or UML diagrams.
But what if we actually built things that we already agree in advance to throw away? Maybe even start from two or three different plausible designs, and push on each one for a while until one seems to be winning? Then rewrite it, learning from the other contenders and from the prototype itself, all before shipping.
The obvious answer is that it would take too long. But I'm not so sure. They say designing software takes too long, also, but when I spend a few weeks designing, the implementation ends up going smoothly and hitting the target. Usually the features that are designed thoroughly at the start of the release hit the target, and other "quick" features that are added in later, sans design, take longer than the designed features and end up pushing the release out. Any overruns or missteps in the implementation of the well-designed features seem insignificant in comparison to things that skimped on design work.