Subscribe to RSS feed

splitbrain.org - electronic brain surgery since 2001

Compiling is not a very Useful Test

Guest post1) by Jürgen Geuter

Compiling by XKCD

When those that like statically typed languages (like C++, Java or C#) and those who prefer dynamically typed languages (like Python, PHP or Ruby) meet, two very different worlds collide. But let's for a second stop enjoying the fight and laughing at the bloody noses and look at one of the “arguments” that is being thrown around.

(Just think on someone “enterprisy” saying the following:)

“Dynamic languages2) are OK for prototyping but when you want to write big software they don't work cause you never know what kind of object you have. Statically typed languages are better because the compiler checks that for you.”

Now this argument looks like it makes a lot of sense: The compiler enforces that kind of correctness. If your Java method only accepts instances of the class Klass and you try passing it an instance of NotKlass the compiler will puke all over your terminal. Bad coder, go back and fix your code! Problem solved…

But what does this test actually tell us? It tells us just that we have only put the right types into functions, that we implemented all the weird interfaces and whatever else the language requires (raising only predefined exceptions for example). There is no doubt that that is a test but I dispute the actual usefulness of it.

What do we actually care about in code? Correctness, we want our code to do exactly what we intended it to do without side-effects that might break stuff (cause in general, breaking stuff is bad™). Having the kind of correctness a statically typed compiler can find only brings us a few baby-steps closer to what we want: Yes, it can check that the code is syntactically correct and that the types somehow fit. It says absolutely nothing about whether the code does what we want it to do.

If we want to check whether our code works most people will probably answer this with the same two words: Unit. Tests. You define tests that actually test the behavior of the code, if your tests run through, your code will do exactly what you want it to do (assuming that your test code does actually test what you need tested). This is true for both camps: The Java or C# crowd loves unit tests just as much as the Ruby or Python people. Why? Because unit tests check what we actually care about: That our code does what we want it to do.

When I look back at when I had to learn Java at university, I can still remember how it worked: Write code, run javac, fix the compiler errors somehow, javac, try fixing it again. Rinse and repeat. We didn't use unit tests, we just tried getting something to work. The compiler was helpful in pointing out that Java needed a ”;” at the end of each line, but it never helped with logic problems. Later I had to grade some student's Java code and I got stuff that might have compiled but that was utter nonsense in any meaning of the word.

Now this is not about saying that dynamic typing is superior. Dynamic and static typing are two different paradigms that appeal to different kinds of people, they match different ways of thinking: When we try to write code, we build a model of the things we are implementing. In a statically typed language you have all kinds of rules that try to coerce you into writing something that “makes sense”. The rules limit what you can do but they are intended to help you avoid doing something stupid.

For many people it doesn't work like that. I personally can't work effectively in statically typed languages because I need some freedom to have my ideas and creativity (wow, big words there ;)) unfold. The rules and limitations don't help me get stuff done, they hinder me. Others don't like the kind of freedom dynamic languages give, they feel that stricter rules allow them to model more clearly and give them a feeling of safety. Both views are completely valid but what I want you to get out of this post is that the kind of test the compiler does for you isn't really a useful one from a semantic perspective.

If you prefer the strictness of static typing, use it but don't complain when even simple tasks become an annoying chore. If you need the freedom of dynamic typing, use it but don't complain when you shot your own foot. Twice. But don't let people tell you that only static typing gives you enough “structure” or “safety” to do big jobs - statements like that are full of shit.

Whatever type of language you write your code in, you have to write automated tests. No software will create them for you magically, no compiler, no library, no magic box. You have to specify the requirements and model them in a machine readable way (and that is all that writing tests is!).

The language wars are over, use what you like best. And write unit tests.


About the author: Jürgen Geuter writes thoughtful posts about all kinds of stuff at his blog and you can follow him on identi.ca. He hates the authorship fetish and thus probably couldn't care less about this paragraph ;-)


The text of this post is released under the Creative Commons Attribution-Share Alike License.

Tags:
programming,
guestpost,
compiling,
testing
Similar posts:
1) What to do if you don't have anything to write about in your blog? Find someone who does and let him write a guest post :-)
2) usually those guys use the derogatory term “scripting languages”
Posted on Wednesday, September the 9th 2009 (5 months ago).

Comments?

1
Good point about correctness vs. completeness. It's a huge pain of using these weakly typed languages like Java and C. The compiler catches a lot, but certainly not enough to depend on it as a correctness statement about your program.  Anything less than completely strictly typed languages require unit testing to illustrate completeness and correctness.  Strongly typed languages (haskell) can use the type system to show correctness, but not completion.  They still need to unit test stuff to prove their function does everything that is needed.  (There are some cool libraries that actually do automatic unit testing based on the types you have, so it's easier to do unit testing)

Dynamically typed languages, obviously, need lots of unit testing to show they aren't violating functional contracts as the language does nothing to help you, and nothing to hold you back either.
2009-09-09 21:22:45
2
I am inclined to agree with you; most compiler errors are rubbish and few help in finding semantic problems. One thing formal calling conventions and the like help with is delegating responsibility; If your class implements Cloneable, you should provide a clone method and it should work (obviously, this does not apply to Java as most things you might call sensible). If your method expects a MyProfile instance it should goddamn handle my object of that type.

Anyway, Haskell is completely different stuff. The golden rule says “If your code compiles, it works”, and it‘s really true.
2009-09-09 21:33:17
3
There's quote attributed to Dijkstra:
"Testing shows the presence, not the absence of bugs"
http://en.wikiquote.org/wiki/E … r_Dijkstra

Typing statically proves the absence of a certain class of errors. It saves you the trouble of having to write tests for that class of errors. If you are working on a big application this really can be useful. The downside of typing is that you may have to write type annotations, and that it may restrict programs that are actually harmless.

However, you shouldn't take your experience with Java, C# or C++ as your only reference point for static typing. Languages such as Ocaml and Haskell have a far better type system that is both more expressive (so more programs can be accepted) and allows you to omit most type annotations.
2009-09-09 21:55:28
Bruno
4
One novelty in some IDEs is that you don't even need to compile to check the type safety; the IDE checks it for you (along with other syntactic chekcs) while you're writing, and highlights incorrect statements.

I feel it saves quite a bit of time - let the tools take care of childish errors, typos etc, and yourself concentrate on the real problems. Of course you still need to do the unit tests, but the earlier you catch something, the better, IMO.
2009-09-10 06:11:22
Joonas Pulakka
5
I gotta admit that my experience with OCaml and Haskell is really very limited, do you Haskell/OCaml people use Unit tests? Or is a different way of testing the dominant paradigma in those communities?
2009-09-10 08:51:52
6
I completely agree with the post.
To answer the question about Haskell (can't speak about OCaml, don't know it too well). There are two parts playing into this.
First the type system in Haskell does a little bit more. For example it does tell you and even finds out on its own if your code has side effects. This can go as far as checking logic errors, as for example seen here: http://blog.moertel.com/articl … gs-problem
Furthermore most of the "typical" Haskell program is pure, in the sense that it has no side effects. As with math this allows for checkin all functions individually. Without having to worry that a strange order of calling those functions will lead to different results.  
And the third part is most Haskell programmers will do extremely thorough testing of their stuff. There are two tools often used for this HUnit a Unit test framework and QuickCheck which is almost completely automatic fuzzing. Due to the strong type system QuickCheck can generate even complex data for fuzzing on it's own without you having to code mock objects or something equivalent.
Especially the last idea is very powerful instead of thinking up test cases like: if I input 0 this peace of code should give me 5 back and most certainly forgetting some. You write properties like: if I call reverse on a list l twice the result is identical to l.
This concept of property checking is making it's way to the jvm (not Java) in form of ScalaCheck for the programminglanguage Scala.
As this is very long for a comment already I stop here and promise a post on my blog when I find the time.
2009-09-10 10:15:54
7
True. Just giving the IDEs more hints how they can help you more precisely is a positive aspect not mentioned here. I think for guy like me who did notreally ever program something bigger than "hello world" since three years ago, the comprehensive aid of a sophisticated IDE is very important - and those love to know what they can show you if you press CTRL + Space.

But maybe in ten years I could tell you something else. Well. ;-)
2009-09-16 07:45:24
8
For me the argument between dynamic vs. static typing isn't about code correctness, it's about the fact that static typing allows smart tools to statically analyze a program and provide information to the programmer (auto-completion, "intellisense"), allow automated re-factoring, or intelligently generate documentation. All of which is impossible or very hard to do in an dynamic language since the type information isn't known until run-time.

I often find that people that just use a text editor for development prefer dynamic languages and those that use an IDE like Eclipse of Visual Studio prefer static languages. For the former group, type declarations are just wasted keystrokes and running the compiler an annoying step just to get their program to run. For the latter group, the IDE takes care of all that AND provides additional features such as continuous recompilation so errors are caught immediately and the aforementioned ones like auto-completion, inline documentation, and automated refactoring.

I think the abundance of posts that reduce the dynamic vs. static typing argument to a matter of correctness are missing the real reasons people prefer either.
2009-09-20 20:02:51
dbb
CAPTCHA

No HTML allowed. URLs will be linked with nofollow attribute. Whitespace is preserved.