Intuitively it seems appropriate that the development environment (and all test environments) be as close to the production build as possible. Are there any documented arguments in support of this intuition?
6
-
6Hm? Common sense not enough? (honest question, sometimes common sense is not enough, especially when talking with managers)– yannisAug 28, 2012 at 23:30
-
2What variances are people arguing for? There are situations where variances are appropriate-- dev for example might reasonably have less data in the database than prod does assuming that you don't need to do full-scale performance testing in dev.– Justin CaveAug 28, 2012 at 23:40
-
4"Common sense" is just shorthand for "unsubstantiated subjective opinion"– SynessoAug 29, 2012 at 3:46
-
It's not clear what you mean by 'documented argument'. Do you want a randomised controlled trial comparing some organisations that had dev like prod against some that didn't? How likely do you think such a thing is to exist? Note that there aren't any randomised controlled trials demonstrating that parachutes are better than nothing for preventing injury when jumping out of aeroplaces, and yet people still think they are. Common sense.– AakashMAug 29, 2012 at 9:26
-
1@AakashM: That's not merely common sense, it's demonstrated in the field and backed by scientific theory that has been tested (gravity, drag, terminal velocity). We don't need an RCT for every individual claim in order to prove that it holds water - it's sufficient that the claim is based on existing, reliable evidence and doesn't introduce new premises or logical fallacies. RCTs are usually best, but many legitimate scientific studies are performed simply by extensive sampling.– AaronaughtAug 30, 2012 at 0:29
|
Show 1 more comment
3 Answers
It's one of the founding principles of continuous delivery that your integration tests and manual tests need to run in a "production-like" environment in order to have any assurance of a stable release. The more production-like your testing and staging environments are, the more confident you can be, up to and including the point of daytime fire-and-forget releases.
That being said, your development environment does not need to be the same as production, and it definitely should not have production data - privacy leaks, ad-hoc updates, all sorts of problems there. Integration happens after your code leaves the development environment (specifically, in your CI environment, assuming you have one), and most teams don't run integration tests locally, so mirroring production in dev won't be that helpful since your code and unit tests are generally going to abstract away any environmental dependencies (assuming that you've designed them correctly).
It is, however, useful to use the same deployment scripts for both local/dev and test/staging/prod, because it adds another layer of testing to the deployment itself and helps you refine your process. But it doesn't need to be the same. It's not really cost-effective to buy an Oracle license for every single dev box, for example, so don't count on perfect consistency.
answered Aug 28, 2012 at 23:40
-
Thank you for this, Aaronaught. Do you have a reference for this (what do I call it?) being one of the founding principles of CD?– SynessoAug 29, 2012 at 5:11
-
I don't expect this answer will be highly upvoted or very popular as I know people love to develop on their platform of choice, but here it goes...
I once witnessed a developer spend 12 hours attempting to replicate a single dependencies installation on his Macbook Pro. If you are doing enterprise development you will have many dependencies. Often with inter-os dependencies someone other than the original author re-packages the content and releases it with a slightly modified code base.
I would be very wary of any developer attempting to develop long-term in an environment that was not production. Not only would the time taken for him to manage his own environment be largely wasted when automated processes already exist to setup the production environment he's risking adding features and adding dependencies unknowingly that may not work altogether.
In my organization we go so far as to have bulk unit-tests to know everytime a python package changes version. Sometimes it happens unintentionally and when it does we want to know. A good exmaple of this is PyMssql and this bug -> http://code.google.com/p/pymssql/issues/detail?id=98
Lastly, I can't understand these claiming to be "agile" development teams that require everyone to continually commit their code. "If it's not in Version Control it doesn't exist!" - and then they turn around and have a completely laxed attitude about the environments, modules, version, being developed in. How about "if it's not developed within production environments it doesn't exist" ??
-
-1 This does not make sense. What does it mean to "replicate a dependency"? Do you mean "replicate a bug caused by a dependency problem"? And why do you need tests do know about a version change? Does software magically update itself on your system?– sleskeAug 29, 2012 at 8:08
-
@sleske "Replicate" the behavior of a modules installation that is a dependency for a program. In python we call these eggs and often have C extensions associated with them. Dist-utils in python this works like a package manager and can resolve an applications dependencies for you. Aug 29, 2012 at 11:04
-
@sleske And yes unfortunately because of the complexity between package managers, and the other tools that are available to install shared C libraries, Python Extensions, etc it can be at times very difficult to guarantee a version of a program or shared library that is an open-source dependency has not changed. MongoDB is a good example. The underlying BSON implementation is a separate (C project) from the Driver itself. So the BSON implementation version can change or be altered independent of the Python (PyMongo) package itself. Aug 29, 2012 at 11:09
-
1Not really sure how package dependencies have anything to do with how similar the development environment is to production. You obviously have to test in an environment similar to production, and it should be done fairly continuously during the cycle, but that does not imply that your dev environment IS prod, nor does it imply that you would run end-to-end tests on every check-in (those kinds of tests are invariably too slow for that). I think you're confusing development and testing; if not, please clarify the important differences in your answer. Aug 29, 2012 at 22:50
-
1My primary means of testing locally - as should be every developer's primary means regardless of the type of software or target environment - is unit tests. I may "test" by running the application locally, but the results of that manual test are utterly irrelevant until the code has been integrated with the rest of the team's, so there's no advantage to having a dev environment that's very close to production. Obviously it should have similar components - but if you follow the "D" in SOLID and rely on abstractions, then environment differences are an issue of configuration, not code. Aug 30, 2012 at 0:20
More precisely the Unit Test Environment should be as similar to production as possible.
In many cases the development environment may not be on the same platform as application will run on. (Think Android where the development is done on PC/eclipse but the code runs on your phone).
The main reason for not doing so in real life boils down to the cost of hardware or software licenses. Its hard to justify a multi-site, high availability cluster of multi-processor machines just for development.
answered Aug 29, 2012 at 6:51
-
2This does not make sense. What is a "Unit Test Environment"? The very point of unit tests is that they do not need any specific environment - they are self-contained. Or did you mean the environment for integration tests?– sleskeAug 29, 2012 at 8:04
-
Some code is independent of its environment, but, any code that uses a library, contacts a database, reads configuration, touches the file system, or uses an API will have some external dependencies. Aug 30, 2012 at 8:19
-
1If a test runs code that uses a library, contacts a database etc., then it is by definition not a unit test (though it may still be an important test). Or maybe we are using a different definition of unit test... Maybe you could change the text to read "Integration test environment", that would make sense.– sleskeAug 30, 2012 at 10:41