The reason you had tons of bristle worms was because of detritus. Guess what bristle worms eat. You aer not doing yourself a favor by not having LR in your tank. I think we need to see what your water parameters are. Have you done any analyses? Water changes aren't going to do it by itself.