TDD and Test-Driven Learning with Functional Programming

In a previous post, I alluded to using Test-Driven Learning (TDL) techniques to decompose a compact Fibonacci function from Programming Clojure:

def fibo((map first (iterate (fn[a b] [b (+ a b)]) [0 1]))

These learning tests are in Shubox - check it out.

A few colleagues and I recently discussed whether or not traditional (aka object-oriented) TDD techniques could get you to the fibo implementation. Aaron Bedra from ThinkRelevance was there and brought up some interesting points about things we need to bear in mind as we approach TDD in a functional language:

1. Are we hanging on to our OO-specific ideas?
2. Do we really understand the data abstractions in functional programming (FP)?
3. Does TDD create the best feedback loop for FP?

Here are my observations:

1. Are we hanging on to our OO-specific ideas? This is the crux of moving from one paradigm to another. You can write fibo recursively the way you might in Java, but you’ll hit a wall and either perform horribly at best or blow the stack at worst [http://en.literateprograms.org/Fibonacci_numbers_(Java)]. Stu does a good job of exploring the consequences of writing this kind of bad Clojure in the book, so I won’t belabor the point here. Suffice it to say, beware of approaching the problem with the wrong paradigm.

2. Do we really understand the data abstractions in functional programming? TDD will not get you to lazy sequences if you don’t already know them. You may discover the need for lazy sequences when you write a test that blows the stack, but you’re going to need reference material to find the solution. This is where I think TDL can help: By coding up some learning tests with which you can drill, you can explore the nuances of functional idioms until you’ve mastered them. You can then employ these techniques intuitively when needed.

1. Does TDD create the best feedback loop for FP? There’s no substitute for the REPL in quickly experimenting, but it’s no replacement for building up a suite of learning tests. Whatever you’ve coded up in the REPL is gone when you close your session. Learning tests, just like unit tests, can be composed in suites that you can review and practice with later. The REPL makes a great companion to a learning test or TDD session, however, for experiments with one-liners.

Given the need for tests, how do you gradually get to the solution in an FP TDD session? Sam Newman recently blogged about struggling with test-driven Clojure:

"My first instinct is to start decomposing functions, passing in stubs to the functions under test. But this just feels like I’m trying to shoehorn IOC-type patterns into a functional program. But what am I left with – testing large combinations of functions? That feels wrong too."

I think it depends on what you mean by “large”, Sam. If the functions are core library functions such as the one in fibo, I’d be comfortable having a test such as:

  (deftest fibo-is-an-infinite-sequence
       (are [description x expected] (= x expected)
         "first seven members" (take 7 (fibo)) '(0 1 1 2 3 5 8 )
         "5th member of the sequence" (nth (fibo) 5) 5
         "last 5 numbers of the ten thousandth fib number" (rem (nth (fibo) 10000) 100000) 66875))

Jay Fields will remind me to use one assertion per test, but I usually only listen to Jay when he’s talking about food, prosecco, or scotch. If my base functions were bigger one’s that I’d written, I’d write separate unit tests for them. Those tests would also probably have more than one assertion, though. Sorry, Jay. ;-)