In talking about tricky "when the law hits the network" questions, we often assume that non-cableco ISPs can't know all that much about what their subscribers are doing online. It would take so much computational effort to look at packets zipping by that the user experience would grind to a halt - people would take their business elsewhere.
It turns out that's just not true. Anderson's piece points out that there are vendors selling products that are designed to dig into a packet's payload and make educated guesses about what the packet is part of. And more than that - they can reconstitute webmail messages and chat sessions.
[S]ome of [these DPI products] can inspect and shape every single packet -- in real time -- for nearly a million simultaneous connections while handling 10-gigabit Ethernet speeds and above.
Patient visitors to this blog will remember that I've spent a lot of their time talking about CALEA. Well, these same vendors make CALEA compliance easy for ISPs, because they can just isolate all the traffic coming from a particular subscriber and forward it on (in response to adequate legal process, you hope) to law enforcement.
The vendors' argument on "traffic shaping" is that it's only fair - why should some bandwidth hogs get away with whatever they want to, when capacity is constrained? A response could be: why don't you provision more bandwidth, and then charge people for using more capacity?
The key point, the money quote, is here:
Where you come down on these questions may vary depending on where Deep Packet Inspection gear is deployed; many people have less problems with its use by last-mile ISPs who interact directly with consumers. Throttling P2P traffic to keep the network open for other uses might be fine, but the concern is magnified when such gear is rolled out by the backbone operators, like AT&T and Verizon...
Think about that for a second. We assume for purposes of the whole Net Neutrality debate in this country that competition is absent in the "last mile." What if there's no competition for backbone transport? What if the backbone providers think they can get away with private traffic shaping too? We'll have no way of knowing, and they'll be able (apparently) to watch the payload of every packet.