The Technical Foundations of Hacking

This chapter is from the book

Certified Ethical Hacker Exam Prep

Learn More Buy

This chapter is from the book

This chapter is from the book 

Certified Ethical Hacker Exam Prep

Learn More Buy

Security and the Stack

To really understand many of the techniques and tools that hackers use, you need to understand how systems and devices communicate. Hackers understand this, and many think outside the box when planning an attack or developing a hacking tool. As an example, TCP uses flags to communicate, but what if a hacker sends TCP packets with no flags set? Sure, it breaks the rules of the protocol, but it might allow the attacker to illicit a response to help identify the server. As you can see, having the ability to know how a protocol, service, or application works and how it can be manipulated can be beneficial.

The OSI model and TCP/IP are discussed in the next sections. Pay careful attention to the function of each layer of the stack, and think about what role each layer plays in the communication process.

The OSI Model

Objective: Understand the Open Systems Interconnect (OSI) Model

Once upon a time, the world of network protocols was much like the Wild West. Everyone kind of did their own thing, and if there were trouble, there would be a shoot-out on Main Street. Trouble was, you never knew whether you were going to get hit by a stray bullet. Luckily, the IT equivalent of the sheriff came to town. This was the International Standards Organization (ISO). The ISO was convinced that there needed to be order and developed the Open Systems Interconnect (OSI) model in 1984. The model is designed to provide order by specifying a specific hierarchy in which each layer builds on the output of each adjacent layer. Although its role as sheriff was not widely accepted by all, the model is still used today as a guide to describe the operation of a networking environment.

There are seven layers of the OSI model: the Application, Presentation, Session, Transport, Network, Data Link, and Physical layers. The seven layers of the OSI model are shown in Figure 2.1, which overviews data moving between two systems up and down the stack, and described in the following list:

Figure 2.1 The OSI model.

• Application layer—Layer 7 is known as the Application layer. Recognized as the top layer of the OSI model, this layer serves as the window for application services. The Application layer is one that most users are familiar with as it is the home of email programs, FTP, Telnet, web browsers, and office productivity suites, as well as many other applications. It is also the home of many malicious programs such as viruses, worms, Trojan horse programs, and other virulent applications.
• Presentation layer—Layer 6 is known as the Presentation layer. The Presentation layer is responsible for taking data that has been passed up from lower levels and putting it into a format that Application layer programs can understand. These common formats include American Standard Code for Information Interchange (ASCII), Extended Binary-Coded Decimal Interchange Code (EBCDIC), and American National Standards Institute (ANSI). From a security standpoint, the most critical process handled at this layer is encryption and decryption. If properly implemented, this can help security data in transit.
• Session layer—Layer 5 is known as the Session layer. Its functionality is put to use when creating, controlling, or shutting down a TCP session. Items such as the TCP connection establishment and TCP connection occur here. Session-layer protocols include items such as Remote Procedure Call and SQLNet from Oracle. From a security standpoint, the Session layer is vulnerable to attacks such as session hijacking. A session hijack can occur when a legitimate user has his session stolen by a hacker. This will be discussed in detail in Chapter 7, "Sniffers, Session Hijacking, and Denial of Service ".
• Transport layer—Layer 4 is known as the Transport layer. The Transport layer ensures completeness by handling end-to-end error recovery and flow control. Transport-layer protocols include TCP, a connection-oriented protocol. TCP provides reliable communication through the use of handshaking, acknowledgments, error detection, and session teardown, as well as User Datagram Protocol (UDP), a connectionless protocol. UDP offers speed and low overhead as its primary advantage. Security concerns at the transport level include Synchronize (SYN) attacks, Denial of Service (DoS), and buffer overflows.
• Network layer—Layer 3 is known as the Network layer. This layer is concerned with logical addressing and routing. The Network layer is the home of the Internet Protocol (IP), which makes a best effort at delivery of datagrams from their source to their destination. Security concerns at the network level include route poisoning, DoS, spoofing, and fragmentation attacks. Fragmentation attacks occur when hackers manipulate datagram fragments to overlap in such a way to crash the victim’s computer. IPSec is a key security service that is available at this layer.
• Data Link layer—Layer 2 is known as the Data Link layer. The Data Link layer is responsible for formatting and organizing the data before sending it to the Physical layer. The Data Link layer organizes the data into frames. A frame is a logical structure in which data can be placed; it’s a packet on the wire. When a frame reaches the target device, the Data Link layer is responsible for stripping off the data frame and passing the data packet up to the Network layer. The Data Link layer is made up of two sub layers, including the logical link control layer (LLC) and the media access control layer (MAC). You might be familiar with the MAC layer, as it shares its name with the MAC addressing scheme. These 6-byte (48-bit) addresses are used to uniquely identify each device on the local network. A major security concern of the Data Link layer is the Address Resolution Protocol (ARP) process. ARP is used to resolve known Network layer addresses to unknown MAC addresses. ARP is a trusting protocol and, as such, can be used by hackers for APR poisoning, which can allow them access to traffic on switches they should not have.
• Physical layer—Layer 1 is known as the Physical layer. At Layer 1, bit-level communication takes place. The bits have no defined meaning on the wire, but the Physical layer defines how long each bit lasts and how it is transmitted and received. From a security standpoint, you must be concerned anytime a hacker can get physical access. By accessing a physical component of a computer network—such as a computer, switch, or cable—the attacker might be able to use a hardware or software packet sniffer to monitor traffic on that network. Sniffers enable attacks to capture and decode packets. If no encryption is being used, a great deal of sensitive information might be directly available to the hacker.

Anatomy of TCP/IP Protocols

Objectives: Have a basic knowledge of the Transmission Control Protocol/Internet Protocol (TCP/IP) and their functionality

Describe the basic TCP/IP frame structure

Four main protocols form the core of TCP/IP: the Internet Protocol (IP), the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and the Internet Control Message Protocol (ICMP). These protocols are essential components that must be supported by every device that communicates on a TCP/IP network. Each serves a distinct purpose and is worthy of further discussion. The four layers of the TCP/IP stack are shown in Figure 2.2. The figure lists the Application, Host-to-host, Internet, and Network Access layers and describes the function of each.

TCP/IP is the foundation of all modern networks. In many ways, you can say that TCP/IP has grown up along with the development of the Internet. Its history can be traced back to standards adopted by the U.S. government’s Department of Defense (DoD) in 1982. Originally, the TCP/IP model was developed as a flexible, fault tolerant set of protocols that were robust enough to avoid failure should one or more nodes go down. After all, the network was designed to these specifications to withstand a nuclear strike, which might destroy key routing nodes. The designers of this original network never envisioned the Internet we use today.

Because TCP/IP was designed to work in a trusted environment, many TCP/IP protocols are now considered insecure. As an example, Telnet is designed to mask the password on the user’s screen, as the designers didn’t want shoulder surfers stealing a password; however, the password is sent in clear text on the wire. Little concern was ever given to the fact that an untrustworthy party might have access to the wire and be able to sniff the clear text password. Most networks today run TCP/IPv4. Many security mechanisms in TCP/IPv4 are add-ons to the original protocol suite. As the layers are stacked one atop another, encapsulation takes place. Encapsulation is the technique of layering protocols in which one layer adds a header to the information from the layer above. An example of this can be seen in Figure 2.3. This screenshot from a sniffer program has UDP highlighted.

Figure 2.2 The TCP/IP stack.

Figure 2.3 Encapsulation.

Let’s take a look at each of the four layers of TCP/IP and discuss some of the security concerns lassociated with each layer and specific protocols. The four layers of TCP/IP include

1. The Application layer
2. The Host-to-host layer
3. The Internet layer
4. The Network access layer

The Application Layer

Objective: Describe application ports and how they are numbered

The Application layer sets at the top of the protocol stack. This layer is responsible for application support. Applications are typically mapped not by name, but by their corresponding port. Ports are placed into TCP and UDP packets so that the correct application can be passed to the required protocols below.

Although a particular service might have an assigned port, nothing specifies that services cannot listen on another port. A common example of this is Simple Mail Transfer Protocol (SMTP). The assigned port of this is 25. Your cable company might block port 25 in an attempt to keep you from running a mail server on your local computer; however, nothing prevents you from running your mail server on another local port. The primary reason services have assigned ports is so that a client can easily find that service on a remote host. As an example, FTP servers listen at port 21, and Hypertext Transfer Protocol (HTTP) servers listen at port 80. Client applications, such as a File Transfer Protocol (FTP) program or browser, use randomly assigned ports typically greater than 1023.

There are approximately 65,000 ports; they are divided into well-known ports (0–1023), registered ports (1024–49151), and dynamic ports (49152–65535). Although there are hundreds of ports and corresponding applications in practice, less than a hundred are in common use. The most common of these are shown in Table 2.1. These are some of the ports that a hacker would look for first on a victim’s computer systems.

Table 2.1 Common Ports and Protocols

Blocking these ports if they are not needed is a good idea, but it’s better to practice the principle of least privilege. The principle of least privilege means that you give an entity the least amount of access only to perform its job and nothing more. If a port is not being used, it should be closed. Remember that security is a never ending process; just because the port is closed today, doesn’t mean that it will be closed tomorrow. You will want to periodically test for open ports. Not all applications are created equally. Although some, such as SSH, are relatively secure, others, such as Telnet, are not. The following list discusses the operation and security issues of some of the common applications:

• File Transfer Protocol (FTP)—FTP is a TCP service and operates on ports 20 and 21. This application is used to move files from one computer to another. Port 20 is used for the data stream and transfers the data between the client and the server. Port 21 is the control stream and is used to pass commands between the client and the FTP server. Attacks on FTP target misconfigured directory permissions and compromised or sniffed clear-text passwords. FTP is one of the most commonly hacked services.
• Telnet—Telnet is a TCP service that operates on port 23. Telnet enables a client at one site to establish a session with a host at another site. The program passes the information typed at the client’s keyboard to the host computer system. Although Telnet can be configured to allow anonymous connections, it should be configured to require usernames and passwords. Unfortunately, even then, Telnet sends them in clear text. When a user is logged in, he or she can perform any allowed task. Applications, such as Secure Shell (SSH), should be considered as a replacement. SSH is a secure replacement for Telnet and does not pass cleartext username and passwords.
• Simple Mail Transfer Protocol (SMTP)—This application is a TCP service that operates on port 25. It is designed for the exchange of electronic mail between networked systems. Messages sent through SMTP have two parts: an address header and the message text. All types of computers can exchange messages with SMTP. Spoofing and spamming are two of the vulnerabilities associated with SMTP.
• Domain Name Service (DNS)—This application operates on port 53 and performs address translation. Although we sometimes realize the role DNS plays, it serves a critical function in that it converts fully qualified domain names (FQDNs) into a numeric IP address or IP addresses into FQDNs. If someone were to bring down DNS, the Internet would continue to function, but it would require that Internet users know the IP address of every site they want to visit. For all practical purposes, the Internet would not be useable without DNS.
• The DNS database consists of one or more zone files. Each zone is a collection of structured resource records. Common record types include the Start of Authority (SOA) record, A record, CNAME record, NS record, PTR record, and the MX record. There is only one SOA record in each zone database file. It describes the zone name space. The A record is the most common, as it contains IP addresses and names of specific hosts. The CNAME record is an alias. For example, the outlaw William H. Bonney went by the alias of Billy the Kid. The NS record lists the IP address of other name servers. An MX record is a mail exchange record. This record has the IP address of the server where email should be delivered. Hackers can target DNS servers with many types of attacks. One such attack is DNS cache poisoning. This type of attack sends fake entries to a DNS server to corrupt the information stored there. DNS can also be susceptible to DoS attacks and to unauthorized zone transfers. DNS uses UDP for DNS queries and TCP for zone transfers.
• Trivial File Transfer Protocol (TFTP)—TFTP operates on port 69. It is considered a down-and-dirty version of FTP as it uses UDP to cut down on overhead. It not only does so without the session management offered by TCP, but it also requires no authentication, which could pose a big security risk. It is used to transfer router configuration files and by cable companies to configure cable modems. TFTP is a favorite of hackers and has been used by programs, such as the Nimda worm, to move data without having to use input usernames or passwords.
• Hypertext Transfer Protocol (HTTP)—HTTP is a TCP service that operates on port 80. This is one of the most well-known applications. HTTP has helped make the Web the popular protocol it is today. The HTTP connection model is known as a stateless connection. HTTP uses a request response protocol in which a client sends a request and a server sends a response. Attacks that exploit HTTP can target the server, browser, or scripts that run on the browser. Code Red is an example of code that targeted a web server.
• Simple Network Management Protocol (SNMP)—SNMP is a UDP service and operates on ports 161 and 162. It was envisioned to be an efficient and inexpensive way to monitor networks. The SNMP protocol allows agents to gather information, including network statistics, and report back to their management stations. Most large corporations have implemented some type of SNMP management. Some of the security problems that plague SNMP are caused by the fact that community strings can be passed as clear text and that the default community strings (public/private) are well known. SNMP version 3 is the most current, and it offers encryption for more robust security.

The Host-to-Host Layer

Objectives: Describe the TCP packet structure

Know the TCP flags and their meaning

Understand how UDP differs from TCP

The host-to-host layer provides end-to-end delivery. Two primary protocols are located at the host-to-host layer, which includes Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).

Transmission Control Protocol (TCP)

TCP enables two hosts to establish a connection and exchange data reliably. To do this, TCP performs a three-step handshake before data is sent. During the data-transmission process, TCP guarantees delivery of data by using sequence and acknowledgment numbers. At the completion of the data-transmission process, TCP performs a four-step shutdown that gracefully concludes the session. The startup and shutdown sequences are shown in Figure 2.4.

Figure 2.4 TCP operation.

TCP has a fixed packet structure that is used to provide flow control, maintain reliable communication, and ensure that any missing data is resent. At the heart of TCP is a 1-byte flag field. Flags help control the TCP process. Common flags include synchronize (SYN), acknowledgement (ACK), push (PSH), and finish (FIN). Figure 2.5 details the TCP packet structure. TCP security issues include TCP sequence number attacks, session hijacking, and SYN flood attacks. Programs, such as Nmap, manipulate TCP flags to attempt to identify active hosts.

Figure 2.5 TCP packet structure.

The ports shown previously in Table 2.1 identify the source and target application, whereas the sequence and acknowledgement numbers are used to assemble packets into their proper order. The flags are used to manage TCP sessions—for example, the synchronize (SYN) and acknowledge (ACK) flags are used in the three-way handshaking, whereas the reset (RST) and finish (FIN) flags are used to tear down a connection. FIN is used during a normal four-step shutdown, whereas RST is used to signal the end of an abnormal session. The checksum is used to ensure that the data is correct, although an attacker can alter a TCP packet and the checksum to make it appear valid. Other flags include urgent (URG). If no flags are set at all, the flags can be referred to as Null, as none are set.

User Datagram Protocol (UDP)

UDP performs none of the handshaking processes that we see performed with TCP. Although that makes it considerably less reliable than TCP, it does offer the benefit of speed. It is ideally suited for data that requires fast delivery and is not sensitive to packet loss. UDP is used by services such as DHCP and DNS. UDP is easier to spoof by attackers than TCP as it does not use sequence and acknowledgement numbers. Figure 2.6 shows the packet structure of UDP.

Figure 2.6 UDP packet structure.

The Internet Layer

Objective: Describe how Internet Control Message Protocol (ICMP) functions and its purpose

The Internet layer contains two important protocols: Internet Protocol (IP) and Internet Control Messaging Protocol (ICMP). IP is a routable protocol whose function is to make a best effort at delivery. The IP header is shown in Figure 2.7. Spend a few minutes reviewing it to better understand each field’s purpose and structure. Complete details can be found in RFC 791. While reviewing the structure of UDP, TCP, and IP, packets might not be the most exciting part of security work. A basic understanding is desirable because many attacks are based on manipulation of the packets. For example, the total length field and fragmentation is tweaked in a ping of death attack.

Figure 2.7 IP header structure.

IP addresses are laid out in a dotted decimal notation format. IPv4 lays out addresses into a four decimal number format that is separated by decimal points. Each of these decimal numbers is one byte in length to allow numbers to range from 0–255. Table 2.2 shows IPv4 addresses and the number of available networks and hosts.

Table 2.2 Ipv4 Addressing

A number of addresses have also been reserved for private use. These addresses are non-routable and normally should not been seen on the Internet. Table 2.3 defines the private address ranges.

Table 2.3 Private Address Ranges

IP does more than just addressing. It can dictate a specific path by using strict or loose source routing, and IP is also responsible for datagram fragmentation. Fragmentation normally occurs when files must be split because of maximum transmission unit (MTU) size limitations.

If IP must send a datagram larger than allowed by the network access layer that it uses, the datagram must be divided into smaller packets. Not all network topologies can handle the same datagram size; therefore, fragmentation is an important function. As IP packets pass through routers, IP reads the acceptable size for the network access layer. If the existing datagram is too large, IP performs fragmentation and divides the datagram into two or more packets. Each packet is labeled with a length, an offset, and a more bit. The length specifies the total length of the fragment, the offset specifies the distance from the first byte of the original datagram, and the more bit is used to indicate if the fragment has more to follow or if it is the last in the series of fragments. An example is shown in Figure 2.8.

Figure 2.8 Fragmentation (3,600).

The first fragment has an offset of 0 and occupies bytes 0–999. The second fragment has an offset of 1,000 and occupies bytes 1,000–1,999. The third fragment has an offset of 2,000 and occupies bytes 2,000–2,999, and the final fragment has an offset 3,000 and occupies bytes 3,000–3,599. Whereas the first three fragments have the more bit set to 1, the final fragment has the more bit set to 0 because no more fragments follow. These concepts are important to understand how various attacks function. If you are not completely comfortable with these concepts, you might want to review a general TCP/IP network book. TCP/IP Illustrated by Richard Stevens is recommended.

To get a better idea of how fragmentation can be exploited by hackers, consider the following: Normally, these fragments follow the logical structured sequence as shown in Figure 2.8. Hackers can manipulate packets to cause them to overlap abnormally, as shown in Figure 2.9.

Figure 2.9 Overlapping fragment attack.

Hackers can also craft packets so that instead of overlapping, there will be gaps between various packets. These nonadjacent fragmented packets are similar to overlapping packets because they can crash or hang older operating systems that have not been patched.

One of the other protocols residing at the Internet layer is ICMP. Its purpose is to provide feedback used for diagnostics or to report logical errors. ICMP messages follow a basic format. The first byte of an ICMP header indicates the type of ICMP message. The following byte contains the code for each particular type of ICMP. The ICMP type generally defines the problem, whereas the code is provided to allow a specific reason of what the problem is. As an example, a Type 3, Code 3 ICMP means that there was a destination error and that the specific destination error is that the targeted port is unreachable. Eight of the most common ICMP types are shown in Table 2.4.

Table 2.4 ICMP Types and Codes

The most common ICMP type in Table 2.4 is the type 0 and 8, which is a ICMP ping request and reply. Although a ping is useful to determine if a host is up, it is also a useful tool for the attacker. The ping can be used to inform a hacker if a computer is online. Although the designers of ICMP envisioned a protocol that would be helpful and informative, hackers use ICMP to send the ping of death, craft Smurf DoS packets, query the timestamp of a system or its netmask, or even send ICMP type 5 packets to redirect traffic. A complete list of Type 3 codes are provided in Table 2.5.

Table 2.5 Type 3 Codes

Address Resolution Protocol (ARP) is the final protocol reviewed at the IP layer. ARP’s role in the world of networking is to resolve known IP addresses to unknown MAC addresses. ARP’s two-step resolution process is performed by first sending a broadcast message requesting the target’s physical address. If a device recognizes the address as its own, it issues an ARP reply containing its MAC address to the original sender. The MAC address is then placed in the ARP cache and used to address subsequent frames. You discover that hackers are interested in the ARP process as it can be manipulated to bypass the functionality of a switch. Because ARP was developed in a trusting world, bogus ARP responses are accepted as valid, which can allow attackers to redirect traffic on a switched network. Proxy ARPs can be used to extend a network and enable one device to communicate with a device on an adjunct node. ARP attacks play a role in a variety of man-in-the middle attacks, spoofing, and in-session hijack attacks.

The Network Access Layer

The network access layer is the bottom of the stack. This portion of the TCP/IP network model is responsible for the physical delivery of IP packets via frames. Ethernet is the most commonly used LAN frame type. Ethernet frames are addressed with MAC addresses that identify the source and destination device. MAC addresses are 6 bytes long and are unique to the Network Interface card (NIC) card in which they are burned. To get a better idea of what MAC addresses look like, review Figure 2.10, as it shows a packet with both the destination and source MAC addresses. Hackers can use a variety of programs to spoof MAC addresses. Spoofing MAC addresses can be a potential target to attackers attempting to bypass 802.11 wireless controls or when switches are used to control traffic by locking ports to specific MAC addresses.

MAC addresses can be either unicast, multicast, or broadcast. Although a destination MAC address can be any one of these three types, a frame will always originate from a unicast MAC address.

The three types of MAC addresses can be easily identified, as follows:

Figure 2.10 MAC addresses.