Note: In thinking about this idea I stumbled on this paper by Tom Wheeler, Phil Verveer, and Gene Kimmelman, which shares some ideas I agree with, and goes into further detail that I found illuminating and inspires some of my above points.