0

Hello I will try to explain what is my actual understanding of tests and why I have problems to grasp it's utility.

Even if I try my hardest to understand the concept of TTD, unit test, integration test, etc. From the most articles that I read it explain HOW to do it (with a trivial example or a framework) and sometimes how the tests saved them hours of debug in certain situation. And I have two problems with that :

First : the method

Let's use a basic example for the explaination of how to do a unit test :

You want to test that your add function works as expected so in your test file you write a mock of your function with an expected output lets say something like that :

import add from ./calculate
function myTestingFunction() {
    add(3, 5)
    expectedOutput(8)
}

Okay so you explain me how to do it but not why... and even with that, I have multiple questions :

  • You're testing a function with another function ... so why did you stop there and test the function you just created ?
  • It's code, it has algorithm so how to be sure it is correctly written ?
  • And why stop with the expected output ? Why not testing that the input are of the correct type, because if I give characters to my add function it will fail, no ?

And every article is basically like that. So with that questions in mind I cannot find the motivation to even try this methods : it's just don't seems logic.

Second : Tests are time and money saving

Let's say now I want to test the Divide function the same way and write and expected output of 3 for the input 9 and 3.

Okay so I write the same function than explained before. But if the user tries to divide by 0 a serious bug will logically occur for him ... this test doesn't covers this case and I'm writing test to ensure my code works well (or at least give the correct outputs).

In real life I now see two situation :

  • I had explained to my client that I will take more time to code (so he will pay me more) but as a result he will be sure that in final his functions are bullet proof. And here I'm saying to him :

"Well ... sorry I didn't think about this use case"

"So I pay you more because you said that the tests you've written will ensure a bug free app... but it's not ? And now I have to pay you to debug it ?"

(so no time and money save and a bad relation with my client)

  • I had coded the functionnality as explained and understood by me and my client. The new use case occur that we were'nt aware of. 'Kay no problem : I pull a branch to bug fix, fix the bug, push the hotfix, document the special use case if needed. And now in the future if I encounter an equivalent problematic I will have the algorithm already prepared because I faced the issue (time and money save for me and my client and a better relation overall)

I don't know if I'm clear about my state of mind but I've discussed these elements with many of my colleagues (some even with more years of practice or a better school degree than me) and they had the same problem than me about test : even if we really want to understand this topic and implement a good way of working if needed, as of now i'ts seems like some sort of intelectual pride.

UPDATE

Thank you all for your answers and comments, I learned a lot of terminology that will help my future research in order to clear my mind.

I also want to mark a new point :

How to make a relevant test

It seems you assume I know how to make a good test set to prove a point of my code or to validate my understanding of a function before pushing it to prod or anything.

It's not the case : even if I know how to do it regarding syntax to use or framework to implement let's say. I didn't find any resources to learn what is considered a "good" test set or a "right" way of doing it. That's why I don't know where do I stop the scenario of testing or how many expecting outputs it's need, if it is a question of quantity of output as well or if can just narrow to the not expected behavior maybe :

unexpectedOutput(!= 8) if my output differ from 8 my test fail

I didn't find resources to this very root knowledge about test and it seems to be a complete different job than the web developper that I am.

The same way I can configure let's say a CI/CD pipeline and implement several tools around it to automate it. But I'm not a devops, there are surely errors of optimisations I made or another tool more dedicated to the one I used but I didn't learn about it, etc.

5
  • Regardind input validation take a look at this answer.
    – blunova
    Dec 31 '21 at 14:53
  • 1
    You do (or at least should) test your tests. In TDD you run them when you're expecting them to fail, and check that they fail for the reasons you're expecting (and give useful feedback when they do) - hence red, green, refactor. And once they're passing you're constantly rerunning them, so you'll find out if something changes and get to decide whether the test or the implementation needs changing as a result. There are also methods of testing (like mutation and fuzz testing) that can find the problems you forgot to check for.
    – jonrsharpe
    Dec 31 '21 at 16:37
  • 6
    Your arguments here and in comments below all appear to assume that the objective of testing is 100% certainty that your code is correct. This is a strawman argument: it is an unrealistic goal, and is not the reason people test. The reason is that in practice we do catch errors in test. You do test your code in some way before delivering it, don't you? You don't just write so many lines of code and hand it over without actually trying it to see if it works, right?
    – David K
    Dec 31 '21 at 16:46
  • @jonrsharpe: And mutation testing is actually testing tests as well, or more precisely, testing test coverage. (Every mutation should lead to at least one failing test, indicating 100% coverage, but only a maximum small number of tests, indicating tightly focused tests.) Jan 1 at 10:18
  • @DavidK I know my vision might be wrong (that' why I'm making a post here trying to figure where I'm wrong) but in the comments and answer as well I see another flaw : You did all assume that I know how to make a relevant test set. And it is part of my point, I don't know that either and I cannot find a way to correct that because everywhere I look writers assume that. Of course I do monkey test on my apps before handling it but you know that better than me surely : I test it as a developper who made it, so basically it's hard to do it the wrong way cause I know how it was made.
    – Quentin O
    Jan 3 at 19:08
12

Testing saves time and money largely because we're not as good at abstract thinking as we think we are.

Personally, I have a PhD in computer science, am a team lead and have worked on maybe 100 projects of more than trivial size. I pride myself that I understand making the computer do desired things better than the average, and the metrics confirm this. But as soon as a system grows beyond a surprisingly small size, I'm no longer able to predict its behaviour well enough to avoid errors, and specifying it in advance through explicit tests becomes a net time saver..

I suspect that the trouble with most advocacy for TDD is that they use small bits of code so that they can be read quickly and understood easily. But that is exactly the kind of code that needs it least.

If you allow me to brag a second time, I'm reasonably certain that I can slam out getters/setters for a data class and maybe a simple wrapper for arithmetic on numbers without errors. But anything above that becomes hard because virtually always, different parts of a specification interact with each other. Assume that you have just four dimensions along which a task may vary, and each allows three alternatives. You now have 81 possible combinations of circumstances, and that is way too many to keep in your head. Many of them will be harmless; some will be unexpected; some will expose gaps in the spec (what does it mean to divide by zero? To raise a negative number to a fractional power? To arrive at a number that is smaller than your possible resolution, but definitely different from zero?); some will force you to rethink large parts of the solution, or even the problem.

This seems to be a surprisingly constant phenomenon. I've worked with and trained a lot of programmers; I have never, ever, met one who understood things as well as they initially thought. That is why fixating expected outcomes in a machine-verifiable way is a good idea, and that's basically what testing does. It verifies not only that you have a clear grasp of what is supposed to happen in a situation, but also that you've reached that goal, and that you don't backslide from that achievement through later changes.

There is a famous quotation by Donald Knuth saying something like, "Beware of this code; I have only proved it correct, not actually tested it". My attitude is that unless you are smarter than Donald Knuth, you need tests for any nontrivial code base, and it has served me well.

4
  • Even if I understand your point (tests are made for larger scale projects) the problem remain the same : I am human, I will make errors even when writting tests, I had misunderstood the purpose of a function and will do it again as you stated rightfully and so I will code my first tests wrongfully the same way, even if I finally grasp the basics of testing I will make errors in the future (false positive test, gaps in the spec, etc.) How would it be a net if I'm the one coding it in the first place and I'm not good at abstraction ?
    – Quentin O
    Dec 31 '21 at 15:30
  • 1
    Tests test one issue at a time, at a time when your mind is not preoccupied with three other aspects of the code base or details of the implementation you're about to write. It is much, much more likely that you get one thing at a time right than multiple things; in fact more than twice as likely. That is why "doubling" your work (by writing both tests and implementations) is worth doing. Dec 31 '21 at 16:07
  • 1
    @QuentinORDENER: there are two types of errors in your code: the ones where you have understood the purpose of a function correctly (but the function still does not do what it should), and the ones where you did not understand the purpose correctly. The first ones are the ones you can catch directly by writing a unit test for the function. The latter ones can be caught when you test the function in a larger context (maybe an integration test), or when someone else (who knows the requirements better than you) tests the function or the system where the function is used.
    – Doc Brown
    Dec 31 '21 at 19:32
  • ... so in all situations, there are tests involved, just on different levels and by different people.
    – Doc Brown
    Dec 31 '21 at 19:34
12

Testing is not necessary; it is inevitable

Your software will get tested. If not by you, then by your customers. You are not required to test, but it is kind to your customers.

The earlier the test, the cheaper the fix

When a customer finds a bug, he has to spend time figuring out if he is doing something wrong, look for workarounds on the internet, then log the bug with your company. Your company then has to assign an engineer, work the bug into the product roadmap, and create a release. All that is really expensive, and takes a long time. So it would be better to find the bug yourself.

If your find your bug during QA, the QA engineer has to determine the steps to reproduce, report the bug, triage it, enter it into the system, and it has to go through your SDLC. That is cheaper than a customer bug but still a lot of work.

If your developer finds the bug before QA, he just fixes it. It is far far less expensive to do it then.

Testing the tests

As you have figured out, the green light you see on the testing console does not tell you if the code is correct, since the test itself could be wrong. But that is not what the green light is supposed to tell you-- it has no way of knowing what "correct" means. Instead, what the green light does tell you is that the test agrees with the code. And that means it can tell you, at any given moment, what parts of the system have changed.

Why is that valuable? I don't think it's because it helps you write the first draft of the code. Indeed, it doubles the work. But it does help you with the second, third, and fourth drafts, and so on, throughout the product's life cycle. It helps because it tells you when you've accidentally broken something, meaning you can work faster and with more confidence without compromising on quality. Also, it makes it easier to let other developers work on your code base, since you can review the test results when approving their pull request.

Note: There are some purists who will tell you that you should write the tests first. That would seem to imply that the tests are meant to help you with that first draft of code, and they do, but that is because of the way guides the engineer's thought process and forces them to think about requirements first. At that point the benefit comes from the process, not the test itself. And that is good too.

1
  • I totally read that first line in Agent Smith voice. Jan 4 at 11:29
7

First : the method

[...]

Okay so you explain me how to do it but not why... and even with that, I have multiple questions :

  • You're testing a function with another function ... so why did you stop there and test the function you just created ?
  • It's code, it has algorithm so how to be sure it is correctly written ?

The idea with writing tests is that the test code is simple enough that you can read the code and immediately see if it does what it is supposed to do. A typical test function is just a linear sequence of statements with as most complex feature a loop.

On the other hand, the code being tested is typically way more complex and hard to fully understand for the average programmer.

  • And why stop with the expected output ? Why not testing that the input are of the correct type, because if I give characters to my add function it will fail, no ?

You don't stop with a test for the expected output. The more interesting tests are for corner cases in the inputs and unexpected inputs. For any non-trivial function, you should expect to have a multitude of test cases.

The problems with most examples used in blogs and tutorials is that they are so simple that the functions you are testing are (almost) as simple as the test code itself and can be verified for correctness without an extensive test suite.

I had explained to my client that I will take more time to code (so he will pay me more) but as a result he will be sure that in final his functions are bullet proof.

Tests don't make your code bullet proof. They provide a proof of the situations you have thought of for proving that your production code works. Without tests, how does your client know that your code does what was requested and that it is robust against most incorrect usage scenarios?

3

I had coded the functionality as explained and understood by me and my client. The new use case occur that we weren't aware of. 'Kay no problem

If you're lucky it's not a problem. If you're not lucky, it is a problem, and someone has experienced financial loss, time lost, or for some applications serious injury or death. I do think every software developer should read a bit of RISKS and its enormous catalogue of disasters, large and small. Not too much or you'll never want to write software again.

Some companies, including very prestigious ones, can afford to "test in production" because they have very good systems for spotting problems and rolling back the effects. Most companies have to determine that something works to an adequate standard before shipping it.

A good document on high-standard testing is that of SQLite:

It is (relatively) easy to build an SQL database engine that behaves correctly on well-formed inputs on a fully functional computer. It is more difficult to build a system that responds sanely to invalid inputs and continues to function following system malfunctions. The anomaly tests are designed to verify the latter behavior.

The scope of possible inputs, outputs, and adverse circumstances is huge. It is often difficult to reason about.

Back to your question:

But if the user tries to divide by 0 a serious bug will logically occur for him ... this test doesn't covers this case

Well, yes. That means your test suite is inadequate and you need more tests. This is where the question of test coverage arises; automated tools can tell you how much of your code is actually run during the tests and thereby suggesting extra tests to add. Experienced human testers are good at guessing what sort of inputs are likely to cause problems. Zero, negative numbers, giant numbers, long strings, invisible characters, weird bits of UNICODE, and so on.

4
  • So I can fail to write my tests and if I need more test there is more possibility to had errors in them. Even if I use automated tools => they are coded by humans as well and they can make errors too. So I'll had third party library or software to an already complicated system with no guarantee that this will help me in the end. I see your point for the experienced testers, but the problem here is I don't see the point of becoming one in the first place. It doesn't seems to had value. Worse, as strongly as I want to have a clean/secure code I don't see why hading code for testing will help.
    – Quentin O
    Dec 31 '21 at 15:52
  • 4
    @QuentinORDENER Stop demanding perfection. You're unlikely to ever be 100% sure that your code doesn't have bugs. Writing tests can help make you more confident than you would be without them.
    – bdsl
    Dec 31 '21 at 17:13
  • 7
    Your arguments here are equivalent to “seatbelts don’t save every life, so cars don’t need them.“
    – Eric G
    Dec 31 '21 at 18:37
  • I'm not saying my arguments are valid, it's just suiting my actual vision of testing. And deep inside I know I'm wrong (thats why I take the risk to ask guidance to change my vision about it) but it's hard to trick your brain unlearn what he tought was acquired. But I assure you every comments and answers works a little in that way.
    – Quentin O
    Jan 3 at 13:51
3

Tests of all sorts don’t guarantee much of anything. They are risk mitigation. As a human, you will make mistakes. Tests exist to catch those mistakes. Yes, the odds of making a mistake are low, but the odds of making two mistakes (one in the code and one in the test) multiply together. Two 1 in 20 chances becomes 1 in 400. If 1 in 400 is still too risky, add more tests.

Yes, you still need to do a good job writing the tests to cover input checking, bounds checking, etc. And even if the code is perfect, it might not solve the problem at hand. But catching mistakes early (while the code is still fresh in your mind, before it produces bad data, before other programmers depend on it) generally makes them cheaper to fix than without tests.

3
  • My problem is not only to write test (there is enough resource to learn a framework or the basic syntax) I also can't figure how to make a relevant test set. It seems a complete subject that I cannot find resources on. For me it's a complete job and formation that musn't be handled incorrectly. I want to grasp the basics in order to find what is lacking in my education but as of now I didn't found a good starting point.
    – Quentin O
    Jan 3 at 19:00
  • @QuentinO - sorry, I am having trouble understanding the problem. You mean something like softwareengineering.stackexchange.com/questions/750/… ? There are very many resources for coming up with test cases, finding boundary conditions, and generally writing a good test suite.
    – Telastyn
    Jan 3 at 20:44
  • Yes this is nearly my point and for example I never came across the term boundary condition or the terms I found in this topic you linked so I will lok upon it thanks
    – Quentin O
    Jan 3 at 21:07
2
  • You're testing a function with another function ... so why did you stop there and test the function you just created?

You should test both. But you don't need to write a third test case for that, you test both by running them together. When the code is all in place you expect to see the 'green' success result. If you write the test first you can check it fails as you expect by running it before you write the production code. If you write the production code first you can do a quick edit to it (or to the test) and run it to check it fails in the way you expect.

It may be enough to do checks like that manually as you're coding, but if you want you can automate them via a mutation testing tool, that will automatically make changes to your code and check that that causes your tests to fail, which proves that they are really checking what your code is like.

  • It's code, it has algorithm so how to be sure it is correctly written ?

You can't be completely sure, but you can be a lot surer than you could that the production code is correct if you don't test it. If you don't test it all you'd have very low confidence that it works. You can test it manually, which may give you good confidence if you do enough tests, but it can be tedious and you're unlikely to want to repeat every check every time you make a small change to the code.

The algorithm in your example should be something like:

result = add(3,5);
assertEquals(8, result);

This is very simple and much easier to read and understand than your own implementation of add would be. (Of course you wouldn't really create your own add function for integers since it's built into CPUs, but the same principle applies for more realistic examples - maybe you want to write a function that can add lengths. CPUs don't have this built in, but the test should be easier to read than the implementation:

result = length.of('12cm').add(length.of('5m'));
assertEquals(length.of('512cm'), result);
  • And why stop with the expected output ? Why not testing that the input are of the correct type, because if I give characters to my add function it will fail, no ?

When you write a test you want to be clear about what you're testing and what you're not testing. When you want to write and test a function the inputs are not created by that function you want to make sure that the function behaves as you intend it to. The function doesn't create it's own inputs so you can't check what they are as part of testing that particular function. But you can and should consider whether it's possible that someone could pass inputs of the wrong type, decide how your function should respond to that, and consider writing a test to check it does so.

If you want to test that that the inputs to your function are right you need to think about where those inputs are created. If there's another part of your code that creates inputs and calls your function you can write test for that. This could be in addition to the test for the function, or you could decide to test thing that calls it instead, and let the function you started with be tested indirectly.

4
  • I think the main problem I have is that when I read articles about testing, writers assume we understand what would be a good test or what could be a good failure test let's say, but it's not the case (for me at least) for the add function I can write a huge amount of failure tests for example and I think a few of them would be relevant (I can test the output is not a string, is not a boolean, is not a complete different number result from the one expected...) There is no resource to learn that particular topics and I can't find a term to search for.
    – Quentin O
    Jan 3 at 18:52
  • One option is to try TDD, alternating between writing a single test that checks one thing and the code that makes it pass. In that system you only write the test that you expect to fail, and then alter your production code to make it pass. You probably want to make the test assert something as specific as possible without overcomplicating it. So you don't need to assert that the output is 'not as string', you can just assert that it's the particular integer 4 (or whatever's correct in the particular case)
    – bdsl
    Jan 3 at 19:30
  • Then once you've got the code to return 4 correctly think about if there's anything else that's wrong with it. If so write at test to prove it's wrong and show what it should do instead, fix it and repeat the cycle. If not call it done.
    – bdsl
    Jan 3 at 19:31
  • Just to emphasise, you don't write tests for what the output should "not" be. You write tests that assert that it is what it should be.
    – bdsl
    Jan 3 at 19:42
2

The cost of writing software

Consider any new 'green field' project which starts from a clean slate; chances are that its developers probably had to produce MVP within a sensible timeframe (Perhaps a year or maybe two at most); many projects often start out with just one small (e.g. "two-pizza") team, or even smaller.

The problem is that this point in a software's overall lifespan is just MVP - software typically isn't just built and just left alone for years; indeed the term soft was originally used to imply that (unlike hardware), it should be easy to change and easily malleable, and stakeholders typically expect and demand it to be easily changed.

Whatever costs were incurred to reach MVP with an initial small team and limited set of simple requirements is often tiny compared to later costs as they accumulate over subsequent years as the product grows; requirements gain more complexity; a codebase has ever-growing moving parts with more and more developers contributing over time, and as the distant memories of the original requirements growing ever-fainter.

The cost of maintaining software

Consider software which reached MVP even 3 years ago; the original team may already have all left their jobs and moved elsewhere, but the stakeholders who paid for it likely still need the software and are also probably highly dependent upon its continued stability and need it to keep evolving and growing in scope.

New developers can't realistically know the history of the code nor its design, nor its original requirements, nor all the decisions or trade-offs which took place in its lifetime, nor all the "gotchas" and other quirks or hidden bits of tech debt that nobody ever had time to fix.

With knowledge of the software and all the decisions being lost (or buried in obscure wikis and other decaying documentation that nobody knows how to navigate) and new developers essentially inheriting legacy code, each time a change happens which impacts the existing code, developers should rightly be expected by stakeholders not to have broken that code, and to release something whose quality is equal to or better than the previous version.

If the original developers covered their code comprehensively in automated tests; the quality is easy to assure because those hundreds (maybe thousands) of tests should run in just a few minutes and highlight the regression errors before anyone even creates a Merge Request. The CI pipeline can immediately warn the current developers when some long-lost requirement in the tests somewhere has gotten broken and prevent the broken functionality being merged into the main branch.

If, on the other hand, the original code was written without tests; consider how many opportunities the intervening years might have created to break that original code and introduce regression issues which a manual tester might miss and could easily end up making it into production or be unnoticed for a long time.

The reality is that code-entropy can grow quickly -- a single change to a single line of code can create a bug; if developers are constantly changing tens/hundreds of lines of code or refactoring and adding/changing/removing logic linked to existing code, then the opportunities to break existing code grows exponentially.

The cost of testing software

while opportunities to break existing code only ever grows, a team's ability to detect those bugs by-hand diminishes; the knowledge gets lost and fades over time, people forget things, the huge increase in scope makes comprehensive manual testing increasingly expensive and less feasible as more code is added and the software deals with an ever-expanding list of increasingly complex requirements.

Testing is the only real protection any team has against existing code getting broken; and there are only two choices - either make it happen automatically, or pay someone to do it manually.

Writing automated tests incurs an up-front cost once; involving the original developer and at a point in time when everyone is fully aware of the requirements and expectations, so everyone has the knowledge fresh in their head.

Writing manual tests incurs repeated costs which grow over time; initially the original QA tester is likely to have been involved in the original discussion around the requirements, and can probably write an excellent set of test procedures, with a reasonable amount of time to pass QA.

Each time a system changes, a manual tester (maybe not the same manual tester) must dig out the same test procedures and try, as faithfully as humanly possible, to replicate all of those tests. If they are the original tester, that's probably going to be a reasonable amount of time -- but it's a violation of the DRY Principle (Don't Repeat Yourself)

(While DRY Principle refers to not replicating knowledge and replacing it with abstractions, an automated test could be considered an abstraction of a manual tester's process)

The cost of manually testing software

The problem of manual testing gets worse with time:

  • The violation of DRY leads to paying for all the time needed for manual testers to run the exact same set of time-consuming and error-prone manual tests over, and over, and over; for many months and years into the future.
  • If there's a rush to get a bugfix out the door, the tester doesn't have time to run all the tests, they'll just pick and choose a few at random and ignore 90% of the tests; that's a good way to let regression bugs into production.
  • If the manual tester isn't the original QA tester who wrote the plan, they need to try to acquire all the knowledge which the original QA tester had in their head to try to execute the plan faithfully and take all the same considerations/observations into account
  • The original tester perhaps assumed a bunch of things that future testers wouldn't know so future testers might interpret things differently because they're only human, and that could cause a lot of confusion, maybe having to go back to the business to question old requirements which haven't changed in years, possibly adding a lot of disruption/delay while the business tries to answer the question.
  • The system itself grows increasingly complex and manual tests need to be maintained and updated just as automated tests need to be maintained and updated; but the additional time to keep re-running and maintaining manual tests also gets worse because manual test plans tend to be written in a very tedious, repetitive way.

When software is not 'soft'

Check this question on SE.SE for an excellent example of what happens when software has grown so unwieldy that it becomes extremely hard for a developer to work with, and causes them to ask a question like this: I've inherited 200K lines of spaghetti code -- what now?

Consider the origin of this code; it probably didn't start out as 200k lines of spaghetti, and it probably worked perfectly for its intended use-case at the time (in fact, the original programmers may have even thought to themselves that it was just a one-off bit of code that would never change).

Code entropy is often the natural and unavoidable fate for nearly all legacy code which doesn't have strong automated test coverage; developers may often not even think their code is going to last very long before someone scraps it but are usually proven wrong.

Software development generally happens on a backdrop of real-world human issues such as time/budget constraints, changing requirements, new/inexperienced developers, mistakes in manual testing, and developers having no "rails" to guide them when they break something by accident.

The best way any software delivery team can provide great value-for-money to their stakeholders is to focus on building very high-quality code which tests itself and therefore protects itself from code-entropy and future regression issues.

Even if the original developers delivered bug-ridden code as part of MVP and some of the tests were bad or wrong, automated test coverage establishes a baseline for quality (just as is the case for static code analysis tools), so it also assures that quality can only get better even if the baseline wasn't so great to start with; the fact that it can only go in a positive direction means avoiding code-entropy and gives the code a maintainable future.

3
  • I think the main problem I have might that I'd never had the occasion to maintain a long term software/app or take a undocumented legacy code that I couldn't understand. As a junior web developper I only had short mission as an IT consultant and due to COVID, budget or other reason I didn't had the chance to stay on a long term mission to experiment your points. The code enthropy is something that light a new aspect of coding to me.
    – Quentin O
    Jan 3 at 16:34
  • Beside when you talk about QA and devops pipeline, to me it's a complete different subject that should be covered by a professionnal, not to be added onto a dev to reduce human ressource costs. Even if I like learning new things, there are concept that need to be learn properly in dedicated school that I didn't attend (like how to do a proper test thats cover properly the root subject or how to setup properly the good configuration of a pipeline)
    – Quentin O
    Jan 3 at 16:34
  • @QuentinO If I may, I'd like to challenge the notion of developers taking responsibility for automated testing and 'DevOps' as being about reducing human resource cost; it's quite the opposite. The reasoning is software craftmanship; a large topic (which may go by other names too) that prioritises software quality concerns over financial concerns, placing developer professionalism and discipline at the heart of software development, and seeking to avoid the endlessly-repeated mistakes of the past which led to code entropy and unmaintainable legacy systems. Jan 5 at 0:13
1

he will be sure that in final his functions are bullet proof

This is mostly where you're doing it wrong. Tests do not "ensure" that your functions are bullet-proof, and if you're telling your client that then you are digging yourself a very big hole. Tests add value only if the tests are of high quality.

I didn't think about this use case

And here we see the problem. You now had two chances to get this right, either when writing the code itself, or when designing your test cases. Tests do not remove the requirement on a developer to think.

1
  • Yes that's what a stated : I am the one who will right the test and as every humans I make errors. Even if a think of all the possible cases in advance I cannot be sure I will code it well to covers all this possibility. I see your point for the communication with the client, I just used a crude example in a way to summarize. Surely me and him will know that tests doesn't had direct value but he has to see an advantage for him if he allow the devs to take more time to code without seeing direct results.
    – Quentin O
    Dec 31 '21 at 15:42
0

Testing is needed because no one can fully understand the exact role of any -non-trivial sized- software.

Take as an example a C compiler like GCC or Clang. Understanding fully the C language (e.g. as specified in n1570) is non-trivial. Even with decades of C or C++ programming experience I cannot claim to know it fully. And many software (including GCC) have some parts of their C or C++ code which are machine generated (for example RefPerSys or Jacques Pitrat's CAIA system, described by an entire book: Artificial Beings, the conscience of a conscious machine).

Programs do fail.

In some cases, you can use a formal approach, e.g. with tools like Frama-C. Then you have to specify in some formal manner the behavior of your C program.

If that C (or C++) program is driving a neurosurgery robot or a robot on Mars, specifying it is a very difficult task. If it is just counting the number of vowels in some ASCII file, it is simpler (and bugs have less dramatic consequences).

Formal methods approaches reduce the number of bugs (or move them at a more abstract level). They don't remove all of them.

Read about Ariane flight v88. A rocket crash because of a software bug.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.