What does invariance cost?

A representation that ignores nuisance variation can make downstream decisions easier. But invariance is also information loss: distinctions in the input are deliberately collapsed.

This suggests a tension worth measuring. When does discarding variation improve robustness, and when does it remove signal that a later task would have needed? Could the answer be expressed as a rate–distortion or information-bottleneck tradeoff?