Incorporate assertion-based testing into your unit test suite for property-based testing of your algorithm. In addition to writing unit tests which check for specific output, write tests designed to fail by triggering assertion failures in the main code.
Many algorithms rely for their correctness proofs on maintaining certain properties throughout the stages of the algorithm. If you can sensibly check these properties by looking at the output of a function, unit testing alone is enough to test your properties. Otherwise, assertion-based testing lets you test that an implementation maintains a property every time the algorithm assumes it.
Assertion-based testing will expose algorithm flaws, coding bugs, and implementation failures due to issues such as numerical instability. Many languages have mechanisms strip assertions at compile time or prior to the code being interpreted so that when run in production mode the assertions do not incur a performance penalty. If your code passes unit tests but fails on a real-life case, you can turn the assertions back on as a debugging tool.