yes, you can have too much crap
when you backflush your filter, can you capture that water in a drum? if so, after your done backflushing, drop an airstone in the drum and let it percolate for a day, then pull the airstone out.. when the water settles you can put the nutrient rich water either back in the fit, or use it to water plants outside the system, and collect the solids for a worm farm