Things could work out for you two long-term, given enough time and motivation; you two definitely stand a chance of getting to the point where you understand each other's needs better and can make a conscious effort to fulfill them. Don't let that act as false hope, either. Making a good, solid emotional connection may be pretty difficult between the two of you.
You feel instinctively as if this could be a big-deal relationship, and it could; but you're going to have to work out some pretty heavy emotional issues before you can make a long-term commitment work. You both want to find a lover who nurtures you emotionally, who helps you shoulder some of the burdens of daily life, and who allows you to do the same for them. But there is a certain coldness or remoteness between the two of you that makes it hard to feel very united or well-supported. If either of you has issues already with insecurity or dependency -- if you need a lot of extra attention, affection and reassurance to make you feel secure in a love relationship, or if you tend to rely too heavily on your lover for support -- those issues could really be exacerbated by this relationship. It definitely presents you with a challenge, to learn to rely on yourself more and to learn to be more generous and open in reaching out to your lover and letting them know how you feel about them.