4

I have several algorithms that I would like to test against the same data sets to compare their results. I don't know how to design it so there is maximum readability and maximum efficiency.

I have considered creating a class for each algorithm, and giving it a copy of the data to work with, but it doesn't seem that that is the right answer:

  1. Each data set is fairly large (10,000 float numpy array), so I don't want to copy each one ~30 times.
  2. Many of the algorithms have similar pre-processing routines (thus repeating them for each algorithm seems wasteful)
  3. Some algorithms have nearly identical code, except a few parameters which are different.

At the same time, having one function call per algorithm also seems wrong: as per (2), many will call the same preprocessing functions, and then it becomes very difficult to tell who is calling who.

I want to be able to allow the user (which will be me) to easily call a variety of algorithms on the data, while keeping the code as clear as possible.

I just keep thinking I need the inverse of a class; where each objects of a class will have the same methods but different data, I need something where each member will have the same data but different methods.

  • Any non-trivial algorithm will take much longer than the copy, so simply copy it. – maaartinus Sep 10 '13 at 15:27
2

Basically you want to create the large data once and cache it between invocations of your different algorithms.

For this to work, the data you want to share should not be altered by the algorithms you run.

How about a caching factory and some dependency injection?

class DataFactory(object):

  def __init__(self, ...):
    self._raw_data = None  # will be lazy-loaded and cached
    self._preprocessed_with_foo = None  # ditto
    self._preprocessed_with_bar = None  # ditto

  @propery
  def raw_data(self):
    if not self._raw_data:
      self._raw_data = loadDataSomehow(...)
    return self._raw_data

  @propery
  def foo_data(self):
    if not self._preprocessed_with_foo:
      self._preprocessed_with_foo = fooTransform(self.raw_data)
    return self._preprocessed_with_foo

  # etc

class Algo1(object):

  def __init__(self, data_factory):
    self.data_factory = data_factory

  def calculateOne(self):
    return someAlgorithm(self.data_factory.raw_data)

  def calculateFooOne(self):
    return anotherAlgorithm(self.data_factory.foo_data)

class Algo2(...): ...  # by the same pattern

It works like this:

factory = DataFactory(...)  # nothing much happens

algo1 = Algo1(factory)  # dependency on factory is injected into algo1

print aglo1.calculateOne()  # factory loads raw data
print algo1.calculateFooOne() # factory reuses raw data, calculates foo data

algo2 = Algo2(factory)  # note: the same factory

print algo2.calculateTwo()  # assumedly reuses raw data
print algo2.calculateFooTwo()  # reuses both raw data and foo data
  • 1
    +1 for suggesting to place the code for providing the data in a separate class. However, IMHO the caching aspect may be secondary. The important thing that your solution provides a resusable way of delivering the data to the different algorithms. – Doc Brown Sep 10 '13 at 5:52
  • @DocBrown It seems there are two unrelated ideas here: a separate data class which can be processed and passed around, and caching the data. You point out the former. But a solution using caching could inherit the relevant injections for the preprocessed bits that it uses. However, I don't know where I would store the data so all the subclassed objects could get at it--a global dict? Then it becomes very similar to the seperate-class answer, except you don't have to pass in the factory. – ari Sep 10 '13 at 18:41
  • @ari: honestly, I don't understand which problem you still see which is not solved by this answer. You need a uniform way to provide access to the same data, including some reusable preprocessing methods - this is what the DataFactory provides. You need the data access globally? Provide global access to the factory. You hace measured (and not guessed) performance problems from reloading the data multiple times? Implement a cache inside your DataFactory. What is missing? – Doc Brown Sep 10 '13 at 20:30
1
> a = np.random.randn(10000)
> %timeit a.copy()
100000 loops, best of 3: 3.9 us per loop

So copying that 30 times takes an imperceptible amount of time. While I would like more information about what these algorithms are (and heartily recommend %timeit, along with the rest of ipython), I doubt that the copies will be the largest of your concerns.

While it is tempting to share initialization blocks, you already say you have most of the meaningful code separated out already. I would suggest writing each algorithm as straightforwardly as possible, above almost any other consideration.

  • It's good to know that copying takes so little time. The smallest difference between the algorithms (i.e. the smallest amount of additional computation for a modified algorithm) is probably an np.where call, which is significantly slower at 103 us per loop. – ari Sep 10 '13 at 18:52
  • The "preprocessing" (which may have been a poor word choice) code is a fundamental part of the algorithms--many algorithms need the gradient of the array, because that's their basis. That stuff takes a while to get and could be shared among many algorithms. Recalculating it would be a significant duplication of effort. Also, while I agree about the importance of making each algorithm readable, it's also important to make the differences between algorithms easily discernible, and the changes between two nearly identical algorithms might be needles in the haystack with your suggestion. – ari Sep 10 '13 at 19:01
0

you say you want to "test" different algorithms. If you want to test them to pick the "best" one, I would keep one class per object. At first, some of the code will be duplicated, but at the same time, modifying one algorithm won't have any impact on the other algorithms, and separating them at the end will only be easier as well as adding new algorithms to your test.

If certain algorithm share the same pre processing by design, there's no reason you could not use inheritance for that family of algorithms and cache the result of that preprocessing. It would slightly flaw the comparison between this family of algorithms and other completely unrelated algorithms (memory consumption, CPU cost of caching...).

Now, I say "best" and I don't know what it means. If it's just a matter of performances, an array copy is probably going to be much cheaper than running the algorithm on the whole dataset. And that's something you probably won't have to do once you've selected your best one.

Also, you say you don't want to copy the dataset 30 times. If you don't modify it in your algorithm (which you should not), you don't have to copy it in the first place.

0

Just to start off, one should never pre-optimize code; code which is easier to understand and more logical is preferable 98% of the time. But since you're bringing it up I suspect you are having problems with it.

Since you're doing a benchmark, you should have one:

class Benchmark(object):
    def __init__(self, data):
        self.data = data
    def __call__(self, algos):
        return (algo.test(self.data) for algo in algos)

benchmark = Benchmark(get_data())
benchmark([Algo1(), Algo2(), ..]

However, I realize upon reading the question again that you would prefer to have something more interactive.

When it comes to the common code between algorithms, you should have proper inheritance to sort all of them out (It's normally bad practice to copy code at all, so work against that by pushing as much code as possible high up in the tree). For the "some algorithms" being identical with changed parameters, one could inject via the constructor like this:

[Algo1(n_states=16), Algo1(n_states=8), Algo2(), ..]

If you want to make a new class out of it you could do

from functools import partial
Algo1a = partial(Algo1, n_states=16)

It's hard to give tips on the problem with common code without more knowledge on your particular algorithms.

For the particular problem where the preprocessing is common for almost all algorithms, I still don't have a straight answer: it depends a bit on your code. But each class should have the responsibility to choose how to do the preprocessing (Single responsibility principle), although a DataFactory is effective.

I wouldn’t ruin a logical code to gain speed by not recomputing the preprocessing, but then again I don't know your particular problem.

  • Regarding benchmarking: yes, eventually there will be a benchmark, but would that effect the organization of the algorithms? Also when I said "user" in the question, I meant the one preforming the benchmark. – ari Sep 10 '13 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.