Parts of my privet hedge have died this summer. Should I remove the dead branches or leave well alone and hope it recovers?

Parts of my privet hedge have died this summer ... The severe end of the scale, which I hope isn’t the case, might be honey fungus. There is very little that can be done if you have honey ...