virginal difficulties

wow. last night was a whirlwind of a night. 12:15, CGT (Cute Gym Teacher) texts me. we have a back and forth and he asks if i want to come over. so i did, around 7. we were basically just hanging out, watching tv, listening to music. it was chill. then he asked me if i smoked and of course, i do. so we went to go get some weed, came back, and smoked it. in the midst of our high, we listened to r&b and got... physical. that sounded so gay. but it was pretty perfect... listening to 'anytime, anyplace' by janet jackson, super duper high and relaxed, and sensually kissing this dude. i'm thinking to myself... that hes thinking to himself... that i really am about to have sex with him! and then i think to myself, okay, how can i justify sleeping with him? i can't. its not like we are in a relationship, we aren't even old friends. he is just a coworker and i saw him for the first time a week ago. if this is how i lose my virginity, i'll be mad at myself. and this is what scares me... i really could have had sex with him. i mean, everything for sex was in order. a simple yes...and my virginity would be out the window. now i don't know what to do going forward. should i just leave it at that and not chill with him? should i just wait for him to contact me? should i tell him i'm waiting to be in a committed relationship to have sex? AHH i don't know! see, this is why i can't have a relaxed normal relationship! i'm not going to make any moves.
peace