Question: Does anyone vaccuum out their fuge during a water change?

I mean, mine has caulerpa growing in it, and like a 2 inch bed of sand, ... I know it is collecting detritus, should I try toclean it out every once in a while?