An elderly rebel says illegal attempts were made to silence him when he stopped construction of a "monstrosity" next to his home.