Mining Pools and 51% Attack

Am I understanding this correctly? Large mining pools may centralize hashing power within the network, but individual miners still vote by way of committing resources to the chain of their choice. Wouldn't this make the likelihood of a 51% attack, even if the pool represents a huge majority of hashing power, extremely unlikely?