I'm working with a very large project with tons of established tests that include some reasonably complicated environment setups and what not. I've recently run into a situation testing some eventually true conditions and stuff like that where I want the ability to run a specific test many many many times and figure out the approximate run time to help me figure out how I should be configuring the waiting times on those tests before I assume failure. I wanted to do this via ruby so I could easily dump the rspec results into json blocks for ease of interpretation and modifcation, as many of the tests dump TONS of text into the sysout stream, and I don't really want to parse it all just to get the bit at the end that reports results. Eventually I also want the ability to inject flags into the middles of tests to give me finer grain timing information about how long, for example, a specific part of a specific test runs on average.
So far my attempt looks like this.
#some setup code to get $runnerargs configs etc until $i >= $repetitions do $config = RSpec.configuration $json_formatter = RSpec::Core::Formatters::JsonFormatter.new($config.output) $reporter = RSpec::Core::Reporter.new($json_formatter) $config.instance_variable_set(:@reporter, $reporter) #some other stuff RSpec::Core::Runner.run($runnerargs) #get results in a json file and do cool things with them end
The problem I'm having is that it looks like ruby doesn't seem to refresh the environment from within a script by default when running rspecs. For example if I do a similar thing to this test in a shell script it works fine, but gives me really ugly output that I don't want to work with. but running things this way, the first test runs fine, but the next test crashes because environment variables used within the test spec have changed during the execution of the first test and they are not properly restarted.
I'm not sure if the fault is ruby or rspec. Anyways, anyone else encounter something similar? Is there a way to force ruby or rspec to completely close down the old environment and spin up a new one the way running from shell script does?