The right answer depends on how big your application is and how you test it. If your application is big, modular enterprise application and you use unit-tests, then the only solution you really have is to go object-oriented way. In that case you create an interface IExceptionHandler, and all client classes consume that interface. You may then use some object-creation pattern like a factory method to create your exception handling objects that implement that interface, or handle it with Dependency injection. That way you make your production code decoupled from exception-handling mechanism and easily adjust or change exception-handling behaviour in future without touching the code in other classes.
If your application is small and mostly for personal use, then just implemet the exception-handling logic the way you feel is most convinient to you.
Hope this helps!