Am I going too far by leaving my boyfriend over a "joke" about his brother harassing me?