This is from the Steam forum:
"can't remove my clanners who are guarding from their guarding duty. It says "npcs outside of town can only guard". Tried recruiting them to my party and then changing their duty, no dice."
I've encountered this glitch. The problem is that their HOUSING is outside the walls.
Now that there's more town configurations, when you move to the next/new town, the housing assignments don't change although the map changes. So now there's folk living outside the walls, and they're kinda stuck outside town in terms of certain game features. I figured it out, moved them into housing. But of course, it's still a glitch/problem even though there's a workaround. Most people wouldn't know where to look at first to figure out that they live outside the walls of town. :/
Someone on Steam answered the issue, here's the link to the Glitches thread for anyone curious: