I think your premise is a little confused here, you speak of injecting a factory, but the factory pattern is a creational pattern whose purpose was to do a subset of what a dependency injection framework does, when DI frameworks weren't prevalent this pattern was useful for that reason. However if you have a DI framework, you no longer really need a factory as the DI framework can fulfill the purpose the factory would have fulfilled.
That said, let me explain a bit about dependency injection and how you would generally use it.
There's a variety of ways to do dependency injection, but the most common are constructor injection, property injection, and direct DIContainer. I'll speak regarding constructor injection as property injection is the wrong approach most of the time (the right approach some of the time), and DIContainer access is not preferable except when you absolutely cannot do either of the other approaches.
Constructor injection is where you have the interface for a dependency and a DIContainer (or factory) that knows the concrete implementation for that dependency, and wheresoever you need an object that depends on that interface, at construction time you hand the implementation from the factory to it.
i.e.
IDbConnectionProvider connProvider = DIContainer.Get<IDbConnectionProvider>();
IUserRepository userRepo = new UserRepository(connProvider);
User currentUser = userRepo.GetCurrentUser();
Many DI frameworks can simplify this significantly to where your DIContainer will inspect the constructor of UserRepository for interfaces it knows concrete implementations for, and will automatically hand those to it for you; this technique is frequently called Inversion of Control, though DI and IoC are both terms which get interchanged a lot and have vague(if any) differences.
Now if you're wondering how the overarching code accesses the DIContainer, well you can either have a static class for accessing it or what is more appropriate is that most DI frameworks allow you to new up a DIContainer, wherein it will actually just behave as a wrapper to an internal singleton dictionary for the types which it knows to be concrete for given interfaces.
That means, you can new up the DIContainer anywhere you want in the code and effectively get the same DIContainer that you had already configured to know your interface-to-concrete relationships. Usual means of hiding the DIContainer from parts of the code which shouldn't interact with it directly is to simply ensure only the necessary project(s) have a reference to the DI framework.