So it turns out that when conventional wisdom speaks, it often is correct. Who could have known.
One of the main lessons that I have learned through doing numerous programming projects is that you really do need to keep things simple. Identify exactly what problem you are trying to solve, and then solve that problem. Don't add cruft or clutter. Don't add features that you don't need.
I think that the hardest part of this realization is that sometimes you work really hard on a feature and then realize that it doesn't quite fit in your product, that your product would be more streamlined without it. But you worked so hard on it that you might as well keep it in there, right?
No. In most cases, you should tear it out. Which hurts, because you put a lot of time in to that feature.
However, when people use your software, they don't want every tool under the sun. They just want to be able to complete their task. It's better to have five easy-to-find and easy-to-use features than ten good features hidden amid hundreds of other features.
As with everything, there are exceptions to this rule. However, if you're building a software package or an API, people will almost definitely value a simple and clean interface above a system with tons of features. And that's before we even acknowledge that keeping it simple is easier.
If your code is getting too complex then you should step back, think it through, and probably cut something out. Simple code holds less errors.