1

A coworker of mine recently setup testing in a new project (a JS library) where a transform step hooks in to the babel config for Webpack in the production config.

For reference, this is the setting used with Jest: https://jestjs.io/docs/en/configuration.html#transform-object-string-string

The production build targets ES5, while our CI is on Node 10 and up. This means that, for all of our tests, the source code is getting transformed by all the unnecessary Babel transforms. Mind you, our source code is regular ES2016 Javascript, nothing too fancy. The only transform required might be the ES6 import syntax.

My gut reaction was that this was quite wasteful and unnecessarily couples the tests to the production build config. But my coworker's justification was that he wants to make sure that the tests run against the same artifacts that users will be using.

That makes a lot of sense to me, but I am not sure what the right answer is. What are the pros and cons of each approach? What are the dangers of running your tests against the production build transforms?

  • 1
    What kind of tests? Your question is tagged with unit-testing, but it's not clear from its content whether you intend the answer to only cover unit tests. – Vincent Savard Jan 23 at 18:23
  • That being said, yes, of course you want to test under the same settings that will run in production. This ensures that the compilation process will not alter the intended behaviour of your application. How can you guarantee that the test will also run in a production environment if you don't test it against the same conditions? – Vincent Savard Jan 23 at 18:53
2

QA versus development.

In a QA process it is important that you test the artifacts that you actually wan to deploy. This rules out possible sources of errors. You will want to perform QA with the fully transpiled code.

During development it is important that you get quick feedback and are able to quickly locate and debug any issues. If this speeds up the process or aids debuggability, you may want to run the tests using un-transpiled code.

Note that whether running some transformations is wasteful or not is largely irrelevant. A computer is doing that, and computers are good at boring, repetitive tasks. Such things are also easy to scale. Instead, focus on the value that these tests bring (e.g. how reliable the tests are in either case) and what the impact on the development experience will be.

Note that it may be possible to run the same tests both in a development mode that produces results more quickly but provides lower confidence, and in a release test mode that runs the tests against the release candidate artifacts. That would be the ideal setup that gives you the benefits of either approach.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.