CryptoFilter: Privacy-Preserving Traffic Analysis of Weak Transport Layer Encryption at Internet Gateways

Transport-layer encryption is fundamental to protecting the basic rights of Internet users from tracking, surveillance, and machine-in-the-middle (MITM) attacks by adversaries along the network path. Despite decades of progress in applied cryptography and an increasing adoption of HTTPS, a substantial amount of network traffic continues to lack proper transport-layer encryption, threatening the privacy of end users and the integrity of the networks they inhabit. Researchers and network operators examining the issue, however, face a dilemma – in order to understand and mitigate these risks, they often have to examine the very information they aim to protect, such as sensitive user data exposed in plaintext. This tension raises ethical concerns, impedes research efforts, and limits our understanding of the threats posed by unencrypted traffic. In this paper, we present CryptoFilter, a system designed for scalable, protocol-agnostic, privacy-preserving analysis of network traffic with weak transport-layer encryption at network gateways. CryptoFilter incrementally reduces gateway-volume traffic (tens of Gbps) in real-time (scalable) to a manageable amount of actionable information, while preserving user privacy (raw packet payloads never leave memory) and protocol semantics (protocol agnostic). Over a month-long deployment at a regional ISP, we use CryptoFilter to reason about network traffic composition, finding that a concerning amount of traffic is unencrypted and not associated with web browsers. We perform privacy threat mining to attribute subsets of the bytes within traffic to potential surveillance (e.g., geolocation information) or infrastructure threats (e.g., plaintext javascript). Finally, we identify applications not previously analyzed and confirm through manual reverse engineering that they leak sensitive information or contain other privacy and security exposures.