A journal to record my notes and ideas related to software development and computing

Wednesday, March 4, 2009

Moral dilemma

I have found myself in the unfortunate position of being asked to dumb-down the work that I do so it is understandable to an "average" programmer (whatever that means). I understand where the request is coming from, ie, there is a fear that if it's not dumbed down, there will be a difficulty in finding someone to understand and maintain the work in the future. However, for me to comply with the request would mean to knowingly write code that is less correct, contain more duplication (both in terms of the effort and the resulting code), and have more known potential traps for future maintenance - something that the "average" programmer would tell you is a "bad thing".

How did I find myself in this position?

In the past, I would try to adhere to the principles of DRY, but on a number of occasions, I would get to the point where in order to avoid repetition, I had to bend the language that I used in ways it wasn't designed for. The resulting code would end up being verbose and very difficult to understand after time, and I still wasn't achieving the re-use that I thought should be possible. In the end, I learned roughly where some of those boundaries of readability lay, and resigned myself to the belief that the duplication was a fact of life.1

Then, about a year ago, I was lucky enough to spend a bit of time learning about Functional Programming and Strong, Static Type Systems with someone who is a self confessed fundamentalist functional programmer. Over many months, I was able to change the way I thought about programming. My progress was very slow to begin with, and I found that a lot of the discussions and reading I was doing on the subject were out of reach of my understanding. However, through continual encouragement, and when I finally started doing the excercises and writing small programs in this new way, things started to sink in and make sense. I didn't change my way of thinking in a single Eureka! moment, but more in a series of small "A ha, now I think I get it" moments.

Now I can write far more concise code using very powerful abstractions, with the added benefit that a large class of potential bugs don't even get past the compiler.

What am I going to do about it?

Back to my dilemma. Obviously, if I want to do something about it, I need to present my argument in a convincing way. I don't have a way of showing the whole picture in a single, simple, succinct way. In fact, I'm very doubtful that such a way exists. There was a lot of unlearning I had to do before I could really begin to see the benefits of functional programming.

I had planned to write up my experiences along the way while I was learning, but I didn't think I could do any better than what other people have already put out there; people who have come to the same conclusions and made the same observations that I have. Having been forced back into the position of writing imperative code though has given me a good reason to get my thoughts down in writing.

So, with that in mind, I'm planning to write up some examples of situations I've come across, where I have relented and written repetitive, less correct, or more error prone code in order to fulfill the contradictory requirements that lead to my dilemma. Using these examples, I hope to build a case for using languages that support functional programming and strong, static type systems even if these languages aren't popular or in the mainstream.

From the personal experience of someone who heavily bought into TDD, OOAD, Design Patterns, etc, I can predict that some of the examples will look wrong, or completely opposite to what is regarded as "best practice" in the mainstream. I will try to identify those areas and drill down into them when I get a chance. In the meantime, you should at least be prepared to question your current knowledge.

This is not going to be another burrito tutorial, and I can almost guarantee that anyone who reads only what I plan to write will not come away with an understanding of Functional Programming. The aim is to show some of the possibilities and maybe encourage people to look further into the context that makes it possible.


I haven't really thought of a logical sequence, so things may seem out of order. However, I'll try to come back and provide cross-links to follow-ups and related entries.

Also, although there's not many existing entries in this blog, I'll use the tag whyfp (as in, Why Functional Programming Matters) to mark each entry that is related to my dilemma.

1 Interestingly, I recently read a blog post that discusses duplication/redundancy, by someone who seems to have come to similar conclusions that I came to. If I understand correctly, they refer to this as incidental redundancy, and conclude that you can't, and shouldn't, do anything about it. However, I now know that some redundancy that I previously thought was incidental, is in fact related and if you discover or know the abstractions and use a language that allows the abstractions to be easily expressed, the redundancy is avoidable.


Chris Ammerman said...

As the author of the blog post in your footnote, allow me to clarify that what I mean by "incidental redundancy" is redundancy that arises from the similar form that different problems may take, but for which it is not useful to make a generalized abstraction. For example, light waves and ocean waves follow some very similar rules. But you wouldn't want to take a general approach that shields you from the specifics of each, because it actually hinders you in most cases from drawing useful conclusions. When working with light waves, you CARE that they are light waves, and not some other kind of wave. So don't worry that other waves have similarities. Work against the light waves, not the general waves.

I think the readability and verbosity problems you describe are more likely "language smells": hints that you are reaching the edges of the abstractive power of the language in which you are working. Meta-programming solutions, such as code that relies on reflection in C#, are examples of this. Reflection is a way of doing meta-programming in a static language. Code that uses reflection is often quite nasty stuff, unintuitive and hard to read. The details of the framework are too "noisy", and work against the goal of elegance in a solution that uses the framework. A path to more elegant meta-programming solutions is to use a language that has dynamic/duck typing (even if it's just VB.NET). You'll be amazed how much less noise there is.

Kristian Domagala said...

@Chris, thanks for taking the time to clarify your point. I don't have much of a physics background, so unfortunately the waves example you gave was a bit lost on me.

I fully intend to expand on my assertion that, when you know the abstractions, some redundancy that looks incidental to begin with, can indeed be avoided. The example I have in mind covers the similarities between the Null Object "pattern", exception handling and list processing. Previously I don't think I would have made any connection between these seemingly disparate aspects of programming, but now that I know of the common abstraction over them, I have no need to repeat myself in a language that supports the abstraction.

Regarding your comment on readability and verbosity, I certainly agree that they are caused by limits of the language I am currently required to work with. However, I disagree that meta-programming in dynamically typed languages is the answer to my problems; it may solve some problems, but IMO, it comes at a huge cost.

This is actually the subject of the long overdue entry I'm currently working on in this series, so if you can wait a few more days, you can read more on my experiences with dynamically typed languages and why I think the cost far outweighs the benefits of using them.

Kristian Domagala said...

The next entry I referred to in my previous comment can be seen here.

About Me

Computer programmer with and interest in music, and a passion for brewing beer, which I'm working at developing into a career!