> Is this an appropriate testing strategy?
No, because the your subdomain is a DSL (a kind of programming languangelanguage) and your compiler is part of an implementation detail for the us ecaseuse case that allows to automate actions/workflows in this domain using the dslDSL.
Since I do not know how your DSL looks like I assume that you have concepts like loop
, condition
, statement
, variable
using the example
for(int i=1;i =< 10;i++) {subtask();}
Using a bdd-gherkin like language you could write something like
as a automation user
i want to have a for loop with startvalue, endvalue, loopincrement
so that i can repeat subtasks several times.
given startvalue=1
and endvalue = 10
and loopinclrement = 1
when i execute for(int i=%startvalue%;i =< %endvalue %;i+=%loopinclrement%)
then the subtask should have been executet 10 times.
This is quite a lot of work to prove that your compiler works as expected.
> If not, what should we be doing differently?
> What organization strategy should we use for our tests?
I would create a big repository of examples for input with corresponding output.
The automated test would iterate through the examples and verify that the compiler output matches the expected output.
Example: if your invoice/order-related dsl compiles to java a repository entry would look like:
example: loop over orderentries
dsl-source: foreach orderitem in orders do calculateTaxes(orderitem)
expected errormessage: none
expected java output: for(OrderItemType orderitem : orders)
{calculateTaxes(orderitem);}
example: loop with syntax errors
dsl-source: foreach orderitem in orders
expected errormessage: missing "do"-keyword in line 1
expected java output: none
So instead of writing a lot of code to fit bdd you simply have to add examples of hardcoded input/output values.