It looks like you're new here. If you want to get involved, click one of these buttons!
I think this is the entire point of them. You (hypothetical person) have been enemied, presumably for good reasons. You are now not allowed to be [wherever]. If you could easily kill the guards and just go there anyway, then what's the point of the enemy mechanic at all? So it should be a challenge.
Now, you might also say right now, that there is no reason to go to the effort to kill the guards to go to [wherever] because there is no reward for doing so, which is also currently more or less correct. But, at least to me, that's separate from whether or not the guards function as intended.
I think the costs proposed by @Ilyos are more than fair, and certainly a good place to start.
As for raid mechanics, I'm extremely ambivalent. They can be fun sure, but I've also been on the receiving end when it was very much "Oh, hell. Not this again." over and over and over. I like that we are not currently 3 armed camps, with every member of the opposing factions enemied the minute they leave the tutorial. I sincerely hope that doesn't change no matter what mechanics are put in place.