Even if you are working continuously on a project, unless the project is tiny, you'll have to switch between features, some of which you haven't modified for months or years.
And if you work in a team, chances are you will constantly discover parts of the code base you haven't written in the first place.
So what makes it possible to reduce the time wasted rediscovering the project?
Refactoring. This is the most important technique which will allow you to spend less time asking yourself what was you thinking about when you wrote a piece of code two years ago.
It is not unusual, when developing a new feature, to try ideas, and to care less about architecture and design, simply because requirements may be unstable, and you may not be sure how the requirements should be implemented. However, once the feature is implemented, the work is not done. With the help of regression tests, come back and refactor the code. Regularly.
As explained by DocBrown, by being refactored regularly, your code will become self-explanatory, which has a huge benefit when coming back to the project in a few years: instead of constantly switching between code and documentation, you'll be able—ideally—to simply follow the code. If you also follow the five principles of SOLID, then it would become much easier to bring changes later: sometimes, you'll simply be able to create a new class for a new feature, without even touching to the existing code. In other cases, you'll need to modify the existing code, but you'll know that the changes will be limited to one or few classes, with little risk of creating regressions somewhere else.
Architecture and design. Do you have one? (It seems from your question that you do) If not, it will be difficult from the code to get a larger picture; thus, understanding will suffer.
Style. Coding standards matter because they make it easier to read the code. As explained by DocBrown in his excellent comment:
A class naming standard can help to find the correct class to change more easily. A naming convention for making a distinction between local and non-local variables makes it easier to understand the impact of a planned change. Moreover, exploring an existing project involves lots of code reading, so everything which makes the code easier to read will speed up the process.
Given that it is usually very easy to enforce a common coding standard by using an existent standard and enforcing it automatically in the pre-commit hook, there are no excuses not to use one.
Documentation. You don't need to write a one hundred pages document explaining everything in detail: it will be boring, and you won't enjoy reading it. However, a few diagrams would provide a tremendous help later. Documenting different choices is a good idea too: for instance, if you decided to write your own thing instead of using something which exists already, explain why you did so: maybe third-party solutions were incompatible with the system, or too slow, or had some severe limitations.
The list of terms. Quoting from your question:
As there is some jargon used, I need to read through the docs just to recall the terms being used.
In some projects with a lot of technical (or domain) terms, I was actually creating a list of terms. The goal is then to ensure that you update this list regularly, and that you use those terms consistently.
The list should be as affordable as possible. If it's not easy to read, you (and your pairs) won't use it. If it's difficult to modify, it will quickly become outdated. Note that it takes time and practice to get a format which will suit you.
Note that whatever you do, you will spend some time understanding the code two years later, even if it is the code you've written yourself. Look at open source projects: some are written by talented developers who do a great job, but even then you'll spend a few hours to a few days exploring the project before you can contribute to it. Your code is not different: you can't remember every detail for years, which means that when coming back to your project a few years later, you find yourself in the same situation as if the project was done by someone else. Moreover, your style may have changed; you've learned new tools, language features and APIs, which could only widen the gap between you today and you two years ago.
it took me a couple of weeks to get into the details of implementation
Depending on the project size and your meaning of details of implementation, spending a couple of weeks may not be exceptional; if it takes four hours to develop a feature for your customer, and you ask to pay for a couple of weeks and four hours, things may go wrong. Make sure you learn about your project only things you need to know in order to implement the feature.
For instance, if the product is an e-commerce website, and the feature is a change in the way PDF invoices are generated once a user bought something, there is only a tiny part of the project you need to understand. You don't have to know, for instance, how products are displayed on the website, or how cart works. Unless it's a legacy spaghetti codebase, your changes will only affect the code which deals with PDF generation of invoices, and won't propagate to other parts of the product.