Not all tests are equal. Some have value, and some are a waste of time. Especially for user interface details, it's possible to get carried away and write automated tests where this isn't helpful.
How would that behaviour break? E.g. if you use a single template for all your pages, then all pages would have such a link unless that template is broken. But whoever changes that template (and whoever reviews that change) would see that they would be removing such a link. A test case may not have additional value here.
However, if you have many different pages without a common template, an automated test ensuring that you navigate to the login page from each to them may have more value: a change may have been made in one place without updating others.
If you don't write an automated test, how would breakage be detected? As mentioned, reviews are also part of a QA process. Manual testing can also be an option, especially for highly visible aspects. Depending on your taste to “move fast and break things”, relying on user feedback (or dog fooding) can be sufficient.
What is the cost of a defect that makes it into production, both for lost opportunities and the cost of fixing it? Especially if deployments are easy, it might be possible to fix bugs very quickly. A problem in production might not be catastrophic.
What is the cost of testing this behaviour? Creating a tests costs development time. In extreme cases, it can cost you by slowing down the test suite, and therefore reducing your ability to deploy quickly.
What is the purpose of this test? E.g. when testing for the presence of a link, this could range from a simple HTML-level test to browser automation that also tests aspects such as contrast levels, accessibility, localization, and click target sizes.
So these factors have to be weighed against each other. The cost of tests can vary wildly. In general, the cost of tests is low after setting up the necessary framework, and the value may be low to high. So it is usually a good idea to write tests for new functionality.
The test you propose – the presence of a specific link target in the DOM – is a fairly non-brittle test and may therefore be a good choice. Depending on your goals, this may be sufficient. It can perhaps even be tested without browser automation. But it may also be entirely insufficient, e.g. if you would rather check for accessibility issues. But those can be so hairy that other testing strategies besides automation might be better.
Personally, if the cost of a problem with login/logout functionality isn't very high, I'd rather not try to automate this as such functionality is likely to be covered through manual smoke tests, code reviews, and through dog fooding. However, you have to find the correct balance for your circumstances.
