#Yes, Yes, and Yes

You have struck on all the ways for passing information around.

• Construct a container and refer to it everywhere
• Pass many, many arguments through the function signature
• Construct an Agent and have them solve retrieving the information based on a request.

There really isn't a best way.

What might be reasonable will depend on the platform, language, and architecture of the program.

1. Do not change: Once the configuration is specified, the code does not change the value.

... It is certainly possible to make memory read-only. However I don't think this is the goal you are actually looking for.

I think what you mean is that a given function/object respects the configuration information provided to it and acts properly.

That is achieved by writing code in the function that does just that. If it passes the value through to some collaborator, then that is what it does. If it performs some computation then it does that. This is shown to be working by using unit tests that show the behaviour is as expected from the various configurations.

If each piece respects the passed in configuration, then the sum of the pieces will respect the configuration.

2. Name collisions: Two modules that share the same parameter (e.g. lr for learning rate).

Name collision is easy enough to solve. The trick is to add a prefix, a suffix, or use an indexor.

A naive prefix/suffix would be to just add more letters to the name, eg: foo_lr or lr_foo. A more elegant approach is to nest the configuration. Instead of having a flat container with whatever key/value pairs, you instead have a tree. The path through the tree adds prefixs/suffixs to the variable name until its sufficiently nuanced, eg: foo.lr or lr.foo. The beauty of using a tree is that it can be decomposed into sub-trees and even smaller sub-trees as more and more specific functions are called.

Similarly you could use an indexor, by creating something like an array. So lr is actually lr[index] and index identifies the specific context. The beauty here is that lr and other variables such as w can be made to have the same index as the node they are associated with.

3. Defaults exist: Many times a specific run of the code only requires configuring a handful of parameters, the rest using defaults.

It usually a bad idea to default values. If you must it is a better idea to have a factory which accepts your small set of parameters and then fleshes them out into the full high detail configuration expected by the rest of the code.

That way you can create a different set of defaults by creating a different factory, and none of your already working code is affected.

4. Related parameters can be pervasive: For example, the minibatch_size may be used by a large quantity of functions, including data preparation, gradient computation, etc.

Again, What might be reasonable will depend on the platform, language, and architecture of the program.

• An Object Orientated architecture would co-locate this data with the set of functions that share it.
• A pure functional program would have the value passed in as an argument, it should not care how the argument is sourced.
• An Actor program would expect the information to be passed in by request, or be retrievable by request.

Your rough choices are:

• keep one/a few copies of the data and share a reference to it.
• copy the value around through different configuration objects/agents/arguments

Neither way is superior.

• The more shared the data is, the more shared its interface is, and therefore more likely to cause issues when the code is changed, but updating data is far easier.
• The less shared the data is the less likely that a change will ripple through the code base, but also the more difficult it is to change the data consistently, and the greater change of inconsistency.