Programming Problems in Disguise

by Ville Laurikari on Saturday, September 12, 2009

Post image for Programming Problems in Disguise

Have you ever written a parser for a configuration file?  Or a data driven interpreter which walks through data, and dispatches appropriate actions?  I’ve written too many to count, and more than half of them were a waste of time. They were a waste of time because they were programming problems in disguise.

A classical disguised programming problem is makefiles. Traditional tools, such as make, treat building programs as a configuration problem.  For make, you create a configuration file to define what programs to build, what commands to run to generate each target, and so on.  For a simple project, makefiles work fine.  Sometimes, though, your build process is a bit more complicated.

Maybe you have to do a slightly different thing on different platforms.  Perhaps you’d like to avoid listing source code files, instead using a naming convention to indicate which files to build.  Or maybe you find that the makefiles have a lot of repetition which could be eliminated if you just could define some kind of an abstraction to reuse that piece of logic everywhere you need it. As a sidenote, there are some enlightened build tools which correctly treat building programs as a programming problem.  SCons is one of these tools, and I encourage you to check it out.

The appropriate solution to a programming problem is straight up code.  Instead of a configuration file parser or a data driven interpreter, write a library. Instead of a configuration file or input for your interpreter, you write normal code using your library.  You get to enjoy the power and flexibility of a real programming language, and you don’t need to implement any of it yourself.

Occasionally, you really do need to provide a closed interface with some kind of mini language, because that just the way it is.  Most regex APIs, for example, compile strings into regex objects.  But there is no reason why you cannot also provide an API to programmatically build regex objects without having to go through the stupid strings.

Here are some telltale signs to detect a disguised programming problem:

  • You have to generate configuration files to work around limitations.
  • The configuration file format has grown abstraction capabilities like functions, macros, and reusable variables.
  • The configuration allows embedding code.
  • In general, any features which smell like a programming language: loops, inheritance, namespaces, types, and so on.

So the next time you find yourself writing a configuration file parser or interpreter, note that you are writing that program in a perfectly good programming language.  Why not use it?

Related posts:

  1. The Essence of Lambda
  2. The Programming High
  3. Overriding System Functions for Fun and Profit
  4. Heuristics for Programming Language Design
  5. I Have Seen the Future, and It is Copy-On-Write

If you liked this, click here to receive new posts in a reader.
You should also follow me on Twitter here.

Comments on this entry are closed.


hackerboss September 12, 2009 at 21:20

Writing a configuration file parser or interpreter? It could be a programming problem in disguise:

This comment was originally posted on Twitter

terry September 14, 2009 at 20:12

Excellent post! I have a project not written by me where all the code is using a script parser to read scripts to run from an XML file and it is just too limiting. I had reservations about it from the beginning but now I’m convinced that all the “coding in XML” was a grave error and should be removed and replaced with a proper API immediately!!! More developers need to understand that XML is *not* a programming or scripting language. *Sigh*

Ville Laurikari September 14, 2009 at 21:11

I’m definitely with you on “coding in XML”. XML is not fit for human consumption. Sometimes, although rarely, creating your own domain specific language could be the right thing to do. Basing it on XML is never the right thing to do.

Julen Parra September 15, 2009 at 19:56

Well, one reason that comes to mind is that configuration files can be changed in-site, probably even giving instructions by phone, without having to recompile the whole thing and send it to the customer, who by your usual luck is in a ship with no Internet connection at all for the next two months.

Ville Laurikari September 15, 2009 at 20:32

That’s a good point. Interpreters typically have the capability to read code in without separately compiling it. Python, Ruby, Lua, PHP, and most Lisp dialects come to mind. The best choice of course varies with the language and your configuration needs.

Usually, a configuration file is just a configuration file and is best left at that. But sometimes, it’s actually a program, and you really should be using a programming language instead of inventing your own.

Previous post:

Next post: