You two might have gotten along really well for a time, but then realized that you have less in common than you thought -- or that there's some sort of disconnect in your communication. There's a good chance that you've both formed images of each other that aren't entirely accurate, and as a result, you haven't known each other as well as you should or could have, as the relationship progressed. On your part, your incorrect image of this person is based on the fact that they aren't entirely straightforward with you about who they are and what they're thinking or feeling.
This could be as minor and intangible as simply holding a part of themselves back due to insecurity or past experiences, but the effect is the same -- you don't get a full picture of them, and you could be surprised when certain personality traits start to come out of the blue. And the fact that you two got involved with each other before you really knew each other could mean that your expressions of affection are less than sincere, because you might not even be certain of your own heart! If you can both try to represent yourselves as honestly as you can -- even when you think the other person isn't going to like something you're about to say or reveal to them -- you'll stand a chance of moving past this difficult influence.