1.1 Scope and Limitation
1.2 Significance of the Study
Almost 20 years after the first publicized buffer overflow vulnerability and exploit used by Morris worm in November, 1988  then followed by Code Red worm in July 2001  and Slammer worm  in January 2003, the buffer overflow still one of the top vulnerability, proven to have a severe effect, which has been exploited successfully. Since then, the buffer overflow vulnerable exploited again and again which cover a wide range of computer application, library, operating system (kernel and embedded system as well) and networking.
There are many hardware and software based techniques and tools that have been proposed and developed to detect and/or protect from buffer overflow vulnerable. However this vulnerable still happens and based on the trend it look likes this problem will continue to happen.
It is publicly known that the buffer overflow happens when there is no bound checking on the used buffer in programs. In the plain source codes, this no bound checking is a normal for unsafe programming language that dominated by C and C++ for a set of the standard library functions. Unfortunately, C and C++ are the languages that most widely used for critical applications such as kernel, Operating System (OS), database engine and device driver.
A buffer is small and reusable temporary data storage during the program execution. Normally it is declared as sized array data type in C and C++ programs though it is not limited to the declared sized array because other unsafe standard C and C++ functions used for string and character manipulation such as strcat() and gets() also resembled similar characteristics. Without bound checking, input size that bigger than the declared size of the array will overwrite other adjacent data in memory and corrupting them. From the programmer point of view, when declaring a sized array in a program, we normally already expected or pre-calculated the maximum number of input. However, we cannot accurately determine the maximum number of input from user and other applications which will use the program. This input validation is more prevalent for applications that will call and share the program such as DLL.
With freely available tools and proof-of-concept (POC) codes, on the attacker side, without proper user input validation, it looks like it is easy to exploit the buffer overflow vulnerability. Furthermore, user input validation issue already exploited by other code injection exploit methods such as SQL injection.
In finding the root cause, it is quite obvious that buffer overflow vulnerable is not a simple thing for the average person to understand. It is not just a programming flaw that represents programmer’s competency to solve it. Much more knowledge such as processor architecture, memory layout and operation, library, how compiler works and low level programming in certain circumstances, must be fully understood in order to detect and protect from buffer overflow effectively.
1.1 Scope and Limitation
The scope of this for fun study is limited to the stack-based buffer overflow with local exploit. All the hardware and related software used in the controlled demonstration can be found in Chapter 3. Keep in mind that there are several buffer overflow variants such as heap based, integer, off-by-one and also a similar subset found in format string attack.
Based on the previous experiences, it is quite weird for the commercial grade software that has been distributed and used by million of the end users, for example, several months later found to be vulnerable to a buffer overflow problem. Although, if the best practices have been adopted in the application development cycle, it is obvious that the application which contains buffer overflow vulnerability even passed the alpha and beta phases 'silently'.
A proper understanding and better awareness on how and why the stack-based buffer overflow exists and happens will provide a very useful input for the more robust and effective detection and prevention mechanism in the design and implementation stages. It should be very beneficial and more productive for everyone mainly programmer to understand, aware and then enforce appropriate protection and defense means to contain, avoid or at least to minimize this problem. Furthermore, when analyzing the buffer overflow protections at many stages of computer system, this will enhance our current knowledge regarding the computer related security breaches at all level, from the processor execution environment, kernel and up to the web application. It is also can be a basis for understanding the new vulnerability and exploit variants. Take note that the experimental demonstration was done using the Fedora 9 virtual machine and hence the buffer overflow vulnerability and exploit ‘reaction’ on the virtual machine can also be observed.