top two vs set

I was going to first post this hand without showing the results to see how you guys would have played it differently but to get the best feedback I decided to just tell you how the hand played out. This was at a Full Tilt 1/2 table 9 handed. I had been playing fast and agressive taking down many small pots until this hand came up:

I was on the cutoff with A Ks, it had folded around to me so I made it $7 to go. Button called and so did BB. Flop came out A K 6 rainbow. BB checked and I led out for $12 (pot $22). Button quickly called and BB folded. Turn came a 2, so once again I led out for $24 (pot $46). Button once again quickly called. River came a J which made a board with no flush draws. I led out again once more for $64 (pot $104). Villian quickly min raised to $128 without thinking at all. Now, I wanted to raise here because he rasied so quickly his bet really looked like 2 pair to me. AJ seemed most likely, I couldn't really put villian on Q 10, so the only other possible hands he could have were 22 or 66. I couldn't fold here, I had pot committed myself by taking the lead in the hand. Anyway, I decided to just call because there are more hands here that beat me than hands that I have him beat (both of us had about $100 more after the call). Villian flipped over 6 6 for a flopped set. Now, here is my question: What could I have done differently here? How would you have played this hand (assuming that whatever you did villian was going to smooth call you, or at least play to trap)? I feel like I lost the minimum here based on my style and how I play but still cannot think of any other situation where I could have not lost money here.