That is "potting" material aka electrical insulation, to isolate and protect the circuit board within from vapors and moisture.
You could run it that way, but probably better to repair it. Brush out the loose material very carefully with a non-metallic brush, then you can pour some new potting material into the back of the switchbox. You can get some potting material fairly inexpensively from Amazon.
Here's a link:
www.amazon.com/WANJAO-Electronic-Grade-Potting-Epoxy/dp/B0FBWT5X96
Be sure to degrease the switchbox area to be poured. Use something like alcohol which is not as harsh, and will rapidly evaporate. You don't want to harn the delicate circuits. I'd pour some on a cloth then wipe carefully.
Lay the box out on a table or bench, circuit board side up ans on the level.
Mix the epoxy resin as per mfr's instructions and pour into the back of the switchbox, covering the circuit board and enough to roll up the sides of the switchbox case. Let it cure per instructions and you're ready to go!
HTH.........ed