I will try to surprise you 
The presentation of the columns as voting is perhaps misleading because it leads to images of invariant representations that are shared and “agreed on.” I think the problem Jeff addresses is how the system could functon without needing those representations being passed around. It is an interesting angle to avoid that problem and I would like to see a neuroscientist review that aspect of the book.
The concept of voting is our projection of an explanation onto the dynamics. The “votes” are actually context at the inputs of cortical columns i.e. the votes are massively distributed, not centralised, and each “vote” is on a unique ballot - it is just our independent observation that interprets this to have a result “as if” the columns were voting. In a “real” voting system there is a common ballot i.e. a shared representation. The brilliance (IMO) of Jeff’s idea is to avoid the shared representations.
I think you are missing the critique - the problem is not whether Jeff is or isn’t a nice person. The problem is that his conception of ethics is antiquated. Like neuroscience, philosophy is a moving target, and if you are not engaged in reading the contemporary work then you are running with ideas the are probably many, many decades, if not centuries, old. Ask the average person in the street about neuroscience and you’ll probably get answers that are from the ealry 20th century (and probably much earlier!)