You probably could, but the fence system needs it's own ground, probably at least 3 rods about 10 feet apart in fact. This will help provide an adequate ground during dry weather.
As SGADawg said ...
Put it on it's own good ground ... it is possible to introduce hum to house wiring if you use the same ground... the newer chargers are nowhere as bad as were some of the older ones...