I am beginner in algorithms. Last year I participated in Google Code Jam. One of the major issues I faced during the competition was my code was working fine on my test cases, but when I submitted on a large number of test cases, I failed to pass them causing some logical error.

So, my question is basically how can I trace logical errors in my code in such situation where it fails to match some test cases?

  • did you try using a debugger? Or a log file? Or both? – DXM Jan 10 '14 at 17:11
  • 1
    When I have a 1000 test cases and one of them fails, my test runner tells me: Test <test name here> failed. Assert failure results:... actual results:.... If you know what test is failing, disable all other tests and put that one in debugger. Where is the problem in doing that? – DXM Jan 10 '14 at 17:16
  • 3
    @DXM, the problem is google code jam has an automated tester that tests 1000s of cases and doesn't tell you which individual test failed. – Karl Bielefeldt Jan 10 '14 at 17:23
  • 2
    ah, so when you say "you've got test cases", they are not yours, they are Google's. In that case, the best we can do is a) suggest a debugger; b) log files; c) wish you good luck. There's clearly some edge condition that you didn't consider, but without anyone knowing your algorithm and your implementation, not sure what else can be suggested. – DXM Jan 10 '14 at 17:28
  • 2
    @JeffO: yeah, in "real" development, we'd just ship that code and let customers sort it out :) – DXM Jan 10 '14 at 19:15

There's no easy way around it. You have to look at the test case inputs one by one for situations you didn't think about, work out manually the correct answer, and see if your algorithm agrees.

For example, one thing that bit me on a recent Google Code Jam question I tried for practice was not allocating a big enough numeric type to hold the result. I was using a 32-bit int and needed at least 64 bits. Looking at the difference between the long and short test inputs, I noticed how much larger the input values were, like they were purposely trying to see if I would overflow.

That's part of the contest. They want to see how well you anticipate the edge cases. Otherwise, they would tell you which test case specifically failed.

Another thing that can help is writing your own test cases, so you can see for yourself which one failed before you submit. Try really large numbers, really small ones, different combinations of positive and negative, etc. Try to push the boundaries of the problem to make sure you cover all the angles. The trick is to make your own test cases simple enough to manually calculate while also complex enough to be a thorough test. That just comes with practice.

| improve this answer | |
  • 1
    +1. Not the exact answer I was looking for but understood there is no magic that can automatically tell when code is failing. – user88873 Jan 10 '14 at 17:49

Although it's not a panacea, there may be some code inspection tools for your language that can help find errant behavior.

For example, Pex is a white-box testing tool for .NET applications that looks for code paths and creates a series unit tests using data that exercise each code path. You can try it on the web at Pex For Fun. And I just discovered Code Digger, a lightweight version of Pex built as a Visual Studio extension.

When I've run it against my code, I've had some success discovering edge cases I'd failed to account for. I found the exercise made me a better unit test author.

| improve this answer | |
  • 1
    I'm a big Pex fan, it really does show you that you need to code to meet the failure conditions, which far outnumber the passing ones. – James Snell Jan 11 '14 at 15:36

Logical Issues can be solved only logically, you test cases may pass since they are specific data set designed and written by you

The best way to make sure your code does not fail in the logical sense is to take paper and pencil, and write down each logical step ( with data set in mind) as you go through the loops and the logical conditions

for example as you come across a loop think if the input is a 0 how will it react, if the input is a 100 how will the loop react, writing down helps you get clarity and actually cuts down the time you would spent in fixing bugs.

And as most coders will testify this system get embedded in you head as you code and more and more , then you don't need to write stuff down, you will automatically think about logical consequence of your code when you write code.

| improve this answer | |

It's always a good idea before you begin code to first write a clear data description, evaluating the full scope of what can be in the data. The once you have that it's much easier to get your code to work the first time, and to have it work on the data it will be presented with in the future.

You can also write a program to inspect and validate the incoming data. This is a good step to make sure the data is what you think it is.

| improve this answer | |