Could be the damsel getting aggressive, they don't normally attack inverts so I'm guessing thats why you haven't lost any of them. What kind of damsel is it? Anything besides chromis' are very aggressive