It could be hard to get to a point where you really feel like the long-term is a possibility for the two of you, because you just have such different ideas of what's important in a relationship. For one of you, an intellectual match is really important -- but that's not likely what you find here, or at least you don't think yours is a meeting of the minds. You might even be creating a situation in your mind that doesn't exist in real life.
You convince yourself that your lover isn't as smart as you'd hope for, and there will always be something lacking in the way you two communicate. Instead of being calm and accepting about this, you more likely get mad about it. You could start baiting your lover, trying to spur them into an argument, patronizing them, belittling their intellect. That's no way to treat someone you're supposed to care about! But you both feel the tension and the friction when you try to talk things out. You just don't get where the other is coming from, and you probably are more focused on getting your own side of things explained than on hearing your partner and really trying to understand them. Needless to say, the relationship suffers as a result. For things to last, you need someone who supports you, who gets you, who allows you to speak your mind and who doesn't make you feel stupid or silly. That might be hard to find with this person.