Basically, if you merge unfinished branches into development branch for the CI (so everyone's code is as up to date as possible etc.), you're never really going to end up with a stable, ready to deploy version on your master, as there's always some unfinished work going on.
Well, yes. If you realize that having mud on the floor makes your house inappropriate for dinner parties, and you then specifically let people come into the house with dirty shoes because you want them to be inside as soon as possible, then the house is going to be in a perpetual state of being inappropriate for dinner parties.
How are you meant to create a deployment version if you keep updating your master with unfinished features?
Well, you can't. Not unless you want to deploy those unfinished features, which you obviously don't want.
The solution here is to exercise enough patience so that people can clean their shoes. Or, to drop the analogy, only merge completed, tested and reviewed work.
Continuous integration doesn't mean that you don't care about the stability of the thing you're integrating. It just means that you don't shelve things after they've been completed.
(so everyone's code is as up to date as possible etc.)
I don't quite understand why you're interested in merging unfinished work. If you already need to depend on those incomplete implementations, then you've subdivided your tasks poorly. If dev A can't work until they can depend on dev B's work, which is in progress and not stable yet, then you should've waited with giving dev A this task until dev B was finished.
In rare cases where dev A and B need to work simultaneously, A should simply work with B on the same feature branch (depending on the impact they could sub-branch it if necessary). In that sense, A and B's task is actually one big task, and thus one feature branch. What they do in that branch, and if they're able to work with each other's unfinished work, that's their business.
But in the meanwhile, your master/dev branch must stay clean at all times.
I read something about not merging unfinished features, and only merging master into the unfinished branches, but wouldn't that cause the very problem CI is trying to prevent?
CI naturally aligns itself with small tasks and short-lived feature branches. And you are correct that it's more ideal to work with small tasks here. But in reality, some big changes are just inevitable. Or, more correctly, further subdividing them would create more paperwork and branch juggling than it would help your development.
In these cases, the best you can do is have your feature developer frequently merge master into their feature branch. This significantly lowers the likelihood of having huge merge conflicts when you eventually try to merge the pull request into master.
Personally, I only merge master into my feature branch when my code isn't broken up. That doesn't mean finished, it does mean that it at the very least compiles, and ideally without any broken tests. This is just to ensure that in case I run into merge issues, I am able to quickly build/test my branch to see if I fixed the merge issues.
As in, if you have 2 big feature branches being worked on, before they get finished you have no changes to the master
Well, your master shouldn't be updated if there hasn't been a new feature that's been completed. You should only have changes when, well, a change is ready for master. "Unfinished" inherently means "not ready".
and then you have 2 huge pull requests from both features being complete
Big changes entail big pull requests. That's why we call them big changes. I'm not quite sure what your issue is here.
If it's about the size of the changes, then compartmentalize your development tasks more finely, so you get more but smaller tasks, and thus more but smaller pull requests.
If it's about the timing of it, i.e. everyone creates a pull request at the end of the sprint, that's sort of unavoidable; since you should only create a pull request when you're done and an ideally planned sprint means that people are finishing their tasks when near the end of the sprint.
How you resolve such a timing conflict depends on what you're able to do. Maybe avoid doing two invasive changes (i.e. those total makeover kind of change) at the same time if you can.
Or you can simply dedicate a final piece of your sprint purely to PR reviewing, testing, refactoring and merge conflict resolution.
One company I worked at used 3 week sprints, and the last 2 days were "non development" in the sense that they expected your PR to be up by the third Wednesday, and then the devs shifted gear into reviewing/fixing/merging/retrospective. In cases where there simply wasn't enough conflict to keep people occupied for two days, there's always some low prio bug that can be addressed, or small POC to work on, which is not essential to the sprint's completion if you can't complete it before the sprint end.
