I am looking to better understand best practices for handling large quantity of parameters. I am particularly interested in the types of parameters involved in machine learning code bases and considering the patterns in which they're manipulated. These machine learning parameters typically possess the following characteristics:
- Do not change: Once the configuration is specified, the code does not change the value.
- Name collisions: Two modules that share the same parameter (e.g.
lrfor learning rate).
- Defaults exist: Many times a specific run of the code only requires configuring a handful of parameters, the rest using defaults.
- Related parameters can be pervasive: For example, the
minibatch_sizemay be used by a large quantity of functions, including data preparation, gradient computation, etc.
I have leveraged three patterns in my experience:
Pattern A (Passing using Kwargs)
def do_b(c=3, **kwargs): b = kwargs['b'] + 1 a = kwargs['a'] + 2 # Raises KeyError: 'a' return b def do_a(a=5, c=4, **kwargs): b = do_b(**kwargs) a = a + 1 return a, b, c def main(): a = 1 b = 2 c = 3 do_a(a=1, b=2, c=3) if __name__ == "__main__": main()
This pattern is the most clunky in my experience for several reasons:
1. Unpacking a variable from kwargs in caller function make it unavailable in the called function
2. Where are the defaults set?: If the caller does not override the defaults specified in the function's parameters, you could be traversing a large code base trying to find where the default is set. Also, the default may be specified in several different functions, making it difficult to determine which default is used (e.g.
c in the example).
Pattern B (Passing via Global Object)
This pattern treats the configuration parameters more or less as global variables. The
Config class is instantiated only once and the object gets passed around from function to function.
class Config: def __init__(self): self.a = 1 self.b = 2 def do_b(opt): a = opt.a + 1 b = opt.b + 1 return b def do_a(opt): a = opt.a + 1 b = do_b(opt) return a, b def main(): opt = Config() do_a(opt) if __name__ == "__main__": main()
1. Enables easy access to any configuration parameter as long as the configuration object is within the namespace.
2. Enables methods to be leveraged for controlled updating of configuration (e.g. merging configurations).
Global variable concept and all of the cons that come along:
1. Any function can update the object, and therefore makes it difficult to trace down the bug
2. No argument/parameter checking at compilation time.
Pattern C (Closures)
I have also seen each module, and their respective configuration parameters, exposed at a high-level using closures via
lambda functions. See below for an example from ShangtongZhang/DeepRL:
class DDPGAgent: def __init__(self, config): self.network = config.network_fn() self.replay = config.replay_fn() # ...etc # DDPG def ddpg_continuous(**kwargs): config = Config() config.merge(kwargs) ... config.network_fn = lambda: DeterministicActorCriticNet( config.state_dim, config.action_dim, actor_body=FCBody(config.state_dim, (400, 300), gate=F.relu), critic_body=TwoLayerFCBodyWithAction( config.state_dim, config.action_dim, (400, 300), gate=F.relu), actor_opt_fn=lambda params: torch.optim.Adam(params, lr=1e-4), critic_opt_fn=lambda params: torch.optim.Adam(params, lr=1e-3)) config.replay_fn = lambda: Replay(memory_size=int(1e6), batch_size=64) ... run_steps(DDPGAgent(config))
And then the classes are instantiated by the
DDPGAgent. Unfortunately, this couples
DDPGAgent with the
lambda functions and makes testing very difficult.
Which of the three patterns would you chose and what are the pros and cons? Do you have any better options? Any guidance is greatly appreciated!