16
Your grammar probably has some rules for each token on how it can be produced (for example, that a { signifies a BLOCK_START token, or that a string-literal token is delimited by '"' characters). Start writing tests for those rules and verify that your lexer produces the correct token in each case.
Once you have a test for each single token, you can add ...
15
If you're writing the lexer yourself, this seems like an ideal case for test-driven development.
While “the number of combinations of tokens in a source file can be huge,” the number of branches in your source code is finite. The idea is that before you add a feature in your code—for instance an edge case to handle by the lexer—you start by writing the test ...
7
One alternative that others aren't mentioning, is to use a test generative approach—like QuickCheck from Haskell—to generate the edge cases from the grammar you've defined.
Now, once they're generated, you would then handwrite some additional conditions that you would expect to fail (e.g., assertRaises).
This would have the advantage of automatically ...
6
Two possible options.
sortTestMethodsUsing.
import unittest
class Test(unittest.TestCase):
def test_07_recover_file(self):
...
def test_08_recover_everything(self):
...
if __name__ == "__main__":
test_order = ["test_07_recover_file", "test_08_recover_everything"]
test_loader = unittest....
5
If the test fails, the software is broken. So fix the software.
It sounds like you haven't even figured out which operating system your software runs on, yet. Figure that out. If you are deploying the software on Linux then you must test it on Linux. If you are deploying the software on Windows then you must test it on Windows.
Then, tests run on the other ...
4
is higher-level testing a standard term in software engineering?
No, I don't think so. It is just a description, which distinguishes those tests from unit tests in the specific context of the book chapter where you found it.
does higher-level testing belong to the integration test as it pins down behavior for a set of classes?
To my understanding, it is ...
3
This answer is a frame challenge, because I feel like you're trying to have your cake and eat it too.
There are two ways people write tests in the real world: reasonably, and pedantically. These are just names I picked, they are not meant to imply one is better than the other, but they are very different beasts.
Reasonable testing strategies rely on only ...
2
I think this is an example of the XY problem. You are looking for an answer to the question 'Is there a better way to run sequential tests?' but you're using a framework that is built around running tests in whatever order they're discovered (i.e. the name).
It seems fundamental to your task in hand that:
steps are performed sequentially, iff the previous ...
2
I'd agree with your assessment. Black-box and white-box describe methods to create test cases, not select from existing test cases. Systematic and random testing describe methods to choose existing test cases for execution. However, I can see cases where attempting to review sets of test cases for systematic execution or placing into a pool for random ...
2
You want consumer-driven contract (CDC) testing. Basically, your customers demonstrate how they expect to use the API in a way so that you can re-run their tests before you deploy a change to your code.
(Beware: currently it is hard to search for "CDC tests" because you drown in covid-related results about genetic testing.)
1
There are many measures of test coverage... each requiring more test cases to achieve.
Statement Coverage shows that you have tested every statement (at least once)
Branch Coverage shows that you have tested every branch (imagine you've drawn a flowchart)
Decision Coverage shows that you have tested every decision (ie each true/false exit of a decision ...
Only top voted, non community-wiki answers of a minimum length are eligible
