“I was there to just do my job and they didn’t like that simply because I had a GOP lanyard on, even though I’m an unaffiliated voter,” said one poll challenger.