The Art of Software Development
/As you may - or may not - know, in my other life I develop software for a living. It is not something I studied in school nor is it exactly where I intended to go with my life. I just found out that I was good at it, I enjoyed it and before I had the chance to course-correct my life, it became my career. Having been in this particular profession for my entire adult life, I think I am somewhat qualified to pontificate on the discipline as a whole. Before you non-geeks run screaming, I want to tell you up front that this column is more about the esoteric side of software development. I promise not to digress into a length discussion on ternary operators or anything else that might cause you to reach for the nearest sharp object.
You may have picked up from the previous paragraph that I have been doing this a long time. I have been actively writing code and designing software since 1978 and with a very brief detour into business contingency planning (and comic book shop management), it is all I have done. I am telling you this for a reason, and the reason is this: hardly anyone in the business understands how to approach development. That difficulty stems from the misconception that software development is a science when in fact it is more art.
Let's first define science before we figure out why development is not it. I hope you agree with the following assessment: science is the systemic knowledge of a part of the physical or conceptual world that is gained through observation and experimentation on that world. Our attempt to understand our universe and how we fit in it defines science and through that attempt we have arrived at the quantification of an approach to understanding that we have called the scientific method. Briefly, any time we can encapsulate an inquiry into a subject using verifiable and measurable evidence and yield a new understanding, correct a misconception or validate a hypothesis - we consider that encapsulation an addition to the body of scientific methods. And therein lies the problem.
Software development is about the application of knowledge to solve a defined problem or to achieve a particular goal. It is not about the definition of the problem but about the solution. While we would use the scientific method to help us define the problem, we apply what we know about the problem through code. Let's take a real-world example of how this works. I work on the web site of a large retailer and my job is to create the system which allows customers to actually buy merchandise from us. In order to create this system, we create hypotheses about how a customer might make a purchase and then build a system to support one of those hypotheses - say, credit card purchase. We decide that in order to prevent online fraud, each order will be accepted but later validated by a customer service person via a phone call. The system is built and deployed, and immediately we observe that customers are annoyed with the process. We realize that we have made a mistake, revise our fraud hypothesis and change our code to reflect the updated knowledge.
We can illustrate the point using a different example. In medicine we observe the effects of a disease on a patient and decide on a course of treatment. We prescribe medication, observe how that affects the patient and change our initial hypothesis based on those observations. The process is iterative and continues until we achieve balance - either the patient is healed or is able to maintain on his/her own - or the patient dies. I do not mean to make light of how doctors do their job, but the point is this: for them, the scientific method never actually ends until the problem is solved. In software development, the definition of the problem is the end of the method.
The subtle distinction here is that we use a scientific approach to define the problem but I suggest that is where the science ends. We define software development as the process of instructing a computer to perform a series of tasks through a computer language that bridge our physical world with its' digital one. The key word in that definition is language. This is what turns software development from science to art.
Just as in medicine when the outcome of an investigation is the production of a paper which details the problem and the knowledge gained from the experience, the outcome of the investigation of our problem is no less writing albeit in a different medium. The problem is that most people see a computer as an inherently "science-y thing" because they don't understand how it works or how to talk to it. Because of that misconception, they mistakenly think that any solution involving a computer involves some science beyond their ken when in fact it is actually learning how to effectively communicate with someone who doesn't speak your native language.
At the end of the day, language - any language - is made up of components like words and punctuation. How to place the words and punctuation together is the syntax of the language. These are things which can be learned, although frankly these days it seems to be something of a dying art. Learning how to use a language, however, does not make your communication effective. Take a practical example of this - Douglas Adams. Yes, I am going to talk about Douglas Adams again. Read anything he has written and you will see someone who not only knows the mechanics of the language, but has also mastered how to use it to effect. He can make you laugh when he tells you that learning to fly without the benefit of an airplane is simply nothing more than throwing yourself at the ground and missing it. He can make you empathize with the God of Rain who hates it yet has no idea who he is or why rain follows him around. And he can make you wonder how a bowl of petunias appeared 30,000 feet above a planet and why the only thought it had was "oh no, not again".
Take any of Douglas Adams' work and run it through Google Translate in French, and then give that to a native French speaker, and their impression of the work will be somewhat different than yours. This is because the translation program understands the syntax and the words, but does not understand how to put them together in a compelling way that touches the French speaker the same way the original did the English speaker. Science is the ability to pull together the words and punctuation following the rules of the language while art is the ability to do that in a way which effectively conveys the message behind the words and punctuation.
The best software developers I have known are the ones who are able to think in code in much the same way a native English speaker would think in French to communicate with someone in Paris. It is more than that, though - it is not just being able to put together the sentence that asks for directions to the train station, it is being able to have a conversation with the person and understand the subtle nuances of the interchange. And that is how the best software developers think about and approach their code - as an almost verbal exchange of conversation with the computer, understanding how to speak to it in a way which compels the device to do more than just var x = myList.Find(cc => cc.IsCustomerAnnoyed == true); - he knows that the computer will understand the request and do it, but that the computer will not like it when it finds a customer who isn't annoyed. And he will tailor the conversation so that the computer knows not to freak out when the customer is not annoyed, but rather to do something else.
Can people learn effective communication? Maybe a little. You can learn by example, but without understanding why the example works the best expected result is that you are able to mimic the original construct. Those who are effective with language learn early how to adapt and improvise communication and will make the experience different based on the audience, their mood, how the message is being relayed and all of the other variables involved in delivering the message. Effective software development is not different in any way. An artist knows his audience (the operating system and the computer language), the mood (this code will handle when the computer encounters something which it doesn't like), the medium (hmmm...I need to do this differently because it is a web page and not a program running on a smart phone), the cultural differences (my audience is running .NET 4.5, so I can address this bit more effectively like this) and a whole host of other parameters.
This is not to say that an artist doesn't make mistakes. Critics and the audience will judge how effective the work is and will behave accordingly. For the software developer, the critic is the tester and quality assurance group much like an editor critiques the work of the author. The computer does not behave as desired when it does not like the code delivered in much the same way that an audience might not buy a less than stellar work. And herein lies the base problem: we have few Douglas Adams in our field and way too many Sydney Sheldons. We have too many competent but unimaginative writers and not enough visionaries. And I think that comes from the perception that communication with a computer is difficult at best. We are not taught computer languages in a composition fashion as much as we are in a strictly mechanical way. Syntax over style.
I am not sure there is an easy solution to the problem. Advances in natural language processing by computers have made some of our interactions much more straight-forward, but even with that we still are pre-conditioned to address the computer in a decidedly non-natural way. Look at the way people interact with their phone: clicking icons and buttons work because it is basically a recipe to follow - click this button to get the computer to identify the song playing in the restaurant. Activate a voice-based interface, however, and the situation changes drastically. The average person will fumble over most words trying to think of the right - and likely, the most precise - way to ask a question in order to get the desired response.
Until we see interaction as a function of language, though, we will continue to fear computers and treat them as magical boxes - and I will continue to find it difficult to hire my next Douglas Adams. Until we see computer interaction as an art, we will continue to treat computers as science. And through that, we will continue to fear and fail.
