Despite your efforts, part of your question is still off-topic (recommendations attract opinionated answers that won't have lasting value to others). I'll attempt to answer the other part in a way which would make it valuable for other people.
There are a bunch of authentication mechanisms out there. You may want to check popular APIs such as Twilio, Google's APIs or Amazon web services to get an idea of how simple or how difficult it could be. Usually:
When the goal of the API is to be able to identify you as a consumer of that API (for instance for invoicing purposes), then you'll be provided with an identifier and a secret.
When the goal of the API is to give programmatic access to the data of a person, OAuth or OpenID will be used.
Aside those two mechanisms, there are more esoteric ones. For instance, all APIs I produce rely on client-side SSL certificates, which makes it possible to have a good level of security at nearly zero cost, the code being of extreme simplicity. While those mechanisms can present some benefits, I wouldn't advise you to use them unless you know the audience (i.e. the programmers who will be developing software which would use your API) are familiar with them. In general, this means sticking with ID/secret or OAuth/OpenID.
One of the benefits of using those common mechanisms is that most frameworks implement them for you. Never, ever try to implement your own authentication; unless you're a security expert, you'll get it wrong.
Some of those frameworks would handle the error handling for you, i.e. send a coherent response to the user who was unable to authenticate. When this is not the case, do not use redirects; instead, return an HTTP 401 or HTTP 403 depending on the case, possibly providing a error message. The message should be short and should not contain technical details; specifically, never include a stack trace.
When defining the interface of your API, remain technology-agnostic. The user shouldn't know nor care if you're using .NET Core or Ruby on Rails. This includes using proper terminology; for instance, the user shouldn't have to know what claims are.
As with any authentication, be prepared to spend a few hours implementing a solution which works, and months refining it. Edge cases are usually tricky and security is at stake. For instance, how do you protect the authentication part of your API against brute force attacks, while being sure your legitimate users won't be blocked if, at some point, they use a wrong configuration with a wrong secret key? Or, when the hashes of secret keys leak because you were hacked, how do you inform your users about this incident?
since this is an API I assume I need to worry about authentication
It depends on your business case. Some APIs are public, and so they have no authentication. In general, it's a good idea to put authentication even for an API serving non-personal data for free in order to protect yourself against abuse.