How would you feel if you were me?

So my boyfriend has this neighbor who is in our grade too, but the girl is ALWAYS with my boyfriends little brother. I find this quite weird.. why does a 16 year old want to hang out with a 12 year old. But she does, i think she hangs out with his little brother to be around my boyfriend. It's not like my boyfriend tries to avoid her and not be around when she's around, because he does. He's gone to the pool, crabbing, the beach, and other places with her. When i try to talk to my boyfriend, all he says is "it's not my fault" and stuff. It bothers me, a lot. And he doesn't seem to try and change it because in his eyes, he's not doing anything wrong.. Not to mention, whenever i'm around another guy, or talk about them, he gets pissy with me. What do you think about this situation? Help, please.