Icefex: Protocol Format Extraction from IL-based Concolic Execution

• Fan Pan,
• Li-fa Wu,
• Zheng Hong,
• Hua-bo Li,
• Hai-Guang Lai,
• Chen-hui Zheng,

Vol. 7, No. 3, March 30, 2013
10.3837/tiis.2013.03.010, Download Paper (Free):

• protocol reverse engineering
• protocol format extraction
• semantic inference
• concolic execution
• Intermediate Language

Abstract

Protocol reverse engineering is useful for many security applications, including intelligent fuzzing, intrusion detection and fingerprint generation. Since manual reverse engineering is a time-consuming and tedious process, a number of automatic techniques have been proposed. However, the accuracy of these techniques is limited due to the complexity of binary instructions, and the derived formats have missed constraints that are critical for security applications. In this paper, we propose a new approach for protocol format extraction. Our approach reasons about only the evaluation behavior of a program on the input message from concolic execution, and enables field identification and constraint inference with high accuracy. Moreover, it performs binary analysis with low complexity by reducing modern instruction sets to BIL, a small, well-specified and architecture-independent language. We have implemented our approach into a system called Icefex and evaluated it over real-world implementations of DNS, eDonkey, FTP, HTTP and McAfee ePO protocols. Experimental results show that our approach is more accurate and effective at extracting protocol formats than other approaches.

Statistics

Cite this article