The Stack Overflow podcast is back! Listen to an interview with our new CEO.
2 of 2 added 947 characters in body

I think you should have an unit test with the current behaviour and in the comments, add the right test and right behaviour. Example:

@Test
public void test() {
  // this is wrong, it should be fixed some time
  Assert.assertEquals(2, new Calculator().plus(2,2));
  // this is the expected behaviour, replace the above test when the fix is available
  // Assert.assertEquals(4, new Calculator().plus(2, 2));
}

This way, when the fix is available, the build will fail, noticing you the failed test. When you'll look at the test, you will know that you changed the behaviour and the test must be updated.

EDIT: As Captain Man said, in large projects, this will not get fixed anytime soon but for the documentation sake, the original answer is better than nothing.

A better way to do it is duplicating the current test, making the clone assert the right thing and @Ignore it with a message, e.g.

@Test
public void test() {
  Assert.assertEquals(2, new Calculator().plus(2,2));
}

@Ignore("fix me, Calculator is giving the wrong result, see ticket BUG-12345 and delete #test() when fixed")
@Test
public void fixMe() {
  Assert.assertEquals(4, new Calculator().plus(2, 2));
}

This comes with the convention in your team to reduce the number of @Ignored tests. The same way you'd be doing with introducing or changing the test to reflect the bug, except you don't fail the build if this is critical for your team, like OP said that the bugfix won't be included in the current release.