TopTal, Codility, and Automated Skill Testing

Just to get this out of the way, I failed the TopTal[^] application.

How did I even find TopTal? In a Code Project newsletter!  I’d never even heard of them before.

TopTal’s primary screening process is to use Codility[^] to see how good your skills are.

Now, the 90 minute timed test at Codility asked me to solve three problems:

  1. the point in which in an array, the count of X from the left != count of X from the right.
  2. given some bit encoding scheme, convert N to -N with the least number of bits.
  3. the optimum number of moves a chess knight must make to get from (0, 0) to (m, n)

Regarding the last, someone on Code Project had posted about the idiocy of that last question a few months back.

In fact, they all are extremely poor tests of skill. Each (particularly #2 and #3) probably involve a simple trick to figure it out, and if you don’t know the trick (like me) you’re spending a lot of time just thinking about the problem. At least I was.

How exactly does solving an arbitrary algorithm test coding skills? How does it demonstrate good OOP practices, or DB architecture skills, or an understanding of things like Linq, or really much else other than “getting the trick?”

After working hard at #1, writing assertions, commenting the code, testing edge cases, optimizing the algorithm for O(1) performance, I realized I had spent an hour on one stupid problem. That left 30 minutes for the remaining two. Riiiight.

Now, Codility says something like “don’t worry if you don’t complete all three tests, just show your best work, even if you only complete one test.”

And then the test results are amusing. The requirements don’t state what to do with incorrect inputs into the “solution” method, and they clearly can’t handle exceptions being thrown — I noticed my score in their example test dropped dramatically when I used exceptions.

Now to TopTal. I got an automated email rejecting me because my score was too low. It was pretty obvious from the email that no one had even bothered to look at my code!  Relying on automated test results, not even looking at the quality of code, those are not ways to gain points with me.

Sadly, this sort of testing methods is probably going to be used more and more.

Now, oddly, I don’t particularly have a problem with the concept behind Codility (except of course that it doesn’t really test skills) but I definitely have a problem with how TopTal handled it – I would have expected, at a minimum, a follow up discussion with a technical person.


6 thoughts on “TopTal, Codility, and Automated Skill Testing

  1. I went through the same “interview” process and, also, failed it. It felt more like a final exam at a university than part of a job interview for professional software developers. I’m not sure how demonstrating that you can solve those kinds of puzzles qualifies you as being the “top talent” at anything that matters to most companies.

  2. I am a software developper with thousands of hours on a telecomuting plaforrm.
    Before that , +10 years in the IT with usually great references letter from my manager.
    So i can claim i am good.

    But i did fail those codiliyy test, and i actually cant tell why for some of them since i have
    no feedback. I also had to solve this pawn chess board test.

    Maybe its a good IQ test but its a bad coder test.Coding means implementing a specification,
    and telling how the pawn moves i like asking a coder in the automtive industry how the car engine
    shoud be designed. Its just not its job.

    People using those tests in their recruitment process should be reminded that latest methodologies
    are agile meaning business expert are involved in the dev process on a daily basis to help him understand
    the concept.

    After all i am not suprised toptal HR uses canned tests instead of face to face interviews because i
    freqently had to deal with HR that did not really understand who we are , what we are doing.
    Giving more importance to a canned test rather than a cv, reference letter and objective 5 stars rating
    and thousans of hours worked is just a proof of incompetence.

    What else?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s