Not Just Another AWS Virtual Private Cloud (VPC) Tutorial

A VPC forms the backbone of most AWS application deployments. Yet it is a service that may not be as obvious to developers in terms of its power and capabilities. A VPC is not just a provider of key computing resources, but it is also a critical component for your deployment’s security. In this tutorial, we will cover VPC basics and key concepts like the Internet Gateway, Route Tables, Subnets, Peering Connections, and Security Groups.

VPC Overview

As the name suggests, a VPC is like a virtual data center with the following characteristics.

• It uses virtualized resources, such as a virtual load balancer. A virtualized resource is provided by one or more physical resources and often offers quite the same type of capabilities as the equivalent physical resource. For example, a virtual load balancer offers very similar capabilities like a physical load balancer. So, what’s the benefit? The benefit is better utilization of physical hardware resources. Virtualization makes it possible to have completely isolated virtual resources that are served by the same physical hardware infrastructure. Thus one set of such hardware can offer many virtualized resources leading to better utilization and overall cost optimization. This also helps the cloud providers in keeping their costs more manageable.
• A VPC is completely isolated from the other VPCs. This means even if you have two or more VPCs in the same AWS account, these will not be able to communicate with each other by default. Later in this tutorial, we will talk about the techniques to enable inter-VPC communication.
• It supports both public and private subnets. Think of a subnet as a slice of your network that serves a bunch of virtual devices. A public subnet can route to and from the Internet. Whereas, a private subnet is primarily meant only for the components within a VPC to talk to each other. Can you think of reasons why would you have these two types? As you can imagine, typical application deployments involve tiers. Certain tiers are meant to be public facing (such as the web tier). Whereas, certain tiers are meant to be not exposed to the Internet (such as the database tier). The public and private subnets cater to hosting such resources. This is an important point when designing your application. Network admins have known this for a long time. But, it may not be always as obvious to folks coming from a pure development background. Do not put all components in the same subnet. Follow this tiered model and put the components in the respective subnets only.
• A VPC uses an Internet Gateway to route traffic to/from the Internet.
• It uses Route Tables for both – routing traffic within the VPC and out of VPC.
• VPC provides an important entity called Security Group, which is a virtual firewall. It plays a critical role in granting access to your infrastructure resources. For example, if you are hosting a web tier on an EC2 instance, you can associate a security group with that instance that permits public access on port 80 and 443. Likewise, if you want to restrict access to the database port to be only from private subnets only, you can configure that via the security group rules.
• A VPC supports multi-zone deployment. An application’s deployment can span multiple AWS Availability Zones (AZs) for availability and resiliency purpose. A VPC can span multiple AZs to handle scenarios when one or more AZs go down, the other AZs can continue to serve the application infrastructure. This is an important aspect of enterprise readiness. You should always assume that failures can occur and design your application and the deployment to handle these as optimally as possible. Of course, keeping in mind other important aspects like the cost of having such an infrastructure. The good news in case of VPC is many such capabilities do not come at an additional cost.
• VPC also offers an interesting entity called Elastic IP. It is a public IP address that can be assigned to a resource, can be taken back and then be assigned to another resource without impacting the consumers. Where would you need this? Typically you would use an Elastic IP for scenarios where the consumers need a guaranteed address independent of the underlying resource. For example, you may have an EC2 instance that is hosting your application. If that instance is rebooted, it will get a different public IP. Or if that instance is terminated and you spin up another instance, it would get a different IP. This is where Elastic IP can be useful. You will assign an Elastic IP to such an instance. When the instance reboots, the Elastic IP does not change. If you recreate the instance, you can simply assign the Elastic IP to the newly created instance.
• What makes VPC even more interesting is that you can enable two VPCs to talk to each other using private IP addresses. And, these can be in different AWS accounts altogether. The communication between two VPCs is facilitated using a Peering Connection. We will elaborate on this a bit later. For now, think of a Peering Connection as a way to have two VPCs connect with each other.
• VPC also supports the following advanced capabilities.
  • Endpoints: A VPC Endpoint enables communicating with the supported AWS services without having to route the traffic via the Internet, such as using an Internet Gateway. This offers better performance and security.
  • Endpoint Services: A VPC Endpoint Service enables providing a service from a VPC via an AWS PrivateLink that can be consumed by others. The consumers will create a VPC Endpoint to connect to your service. This is useful for scenarios where you do not want to expose a service publicly, but only to specific consumers.
  • VPC also offers several options to connect it with your corporate data center, such as a site-to-site VPN, AWS Direct Connect and so on. In general, these capabilities are used to make the AWS VPC infrastructure an extension of your corporate network so that non-cloud components can also access AWS resources.

VPC Components Walk Through

VPC Information

A good way to familiarize with various VPC concepts is to walk through these in the VPC dashboard.

The dashboard shows the list of VPCs. It shows some key attributes for each VPC, such as its name, ID, status and the IP address block (CIDR), etc. AWS creates a default VPC automatically. While this VPC may be fine for some playing around and quick start activities, you should avoid/minimize using it for hosting your applications. Why? Because it is very generically designed and has a Security Group setup that provides very open access. While this may be helpful in the beginning, as you make progress in your application deployment you want to ensure that the VPC design caters to your application’s needs, both from deployment and security perspective. Also, you want to set up VPCs based on purpose. For example, a VPC to hold common infrastructure components, like database, an SSH jumper, etc and application VPCs to host the respective applications.

When you click on a VPC on the list, you can see more information about it.

• Whether this VPC uses DNS resolution and DNS hostnames.
• You can also see the default route table for the VPC.

Subnets

As we discussed earlier, a subnet is a slice of your network that hosts nodes and other components that require access to the VPC resources. For example, an EC2 instance running within the VPC gets it’s IP address from the subnet it is attached to. Likewise, you could have a lambda access VPC resources using an Elastic Network Interface (ENI). Here are some important points about subnets.

• A subnet belongs to an Availability Zone (AZ).
  • You can have multiple subnets in an AZ.
  • A VPC can have subnets that are in different AZs. For example, subnet-1 in us-east-1a, subnet-2 in us-east-1b and so on. In fact, this is a recommended practice to use multiple AZs in your deployment to for high availability purpose.
• A subnet has a private IP address range allocated. A private IP address is not routeable to/from the Internet.
• You cannot change the IP block once a subnet has been created. However, you can always add another subnet, such as when you start running out of IP addresses in the existing subnets.
• VPC supports both public and private subnets.
  • A public subnet permits communication to/from the Internet via the Internet Gateway. It is a perfect candidate for public-facing components like the web tier.
  • A private subnet does not permit inbound communication from the Internet. However, you can enable outbound communication to the Internet from a private subnet using a NAT Gateway. But, it is not a must. A private subnet is often used to host components that should not be publicly accessible and should only be accessed by application components, such as a database.
• You can see the route table that the subnet is associated with.
• A public subnet supports an interesting configuration whether you want to auto-assign public IPs. You can disable this as an extra security measure to ensure only components that are meant to be public facing are assigned a public IP. For example, with subnet level auto-assign public IP off, if you are setting up a web server on an EC2 instance in that subnet, you can choose to assign it a public IP during the instance creation time. The benefit of this approach is if you accidentally created an instance on this subnet that was not intended to be public facing, it won’t automatically be assigned a public IP. So, look at this as a useful security measure.

Route Tables

Routing is a critical functionality provided by VPC. It accomplishes this with the help of route tables. A route table contains information on how to route traffic to a given network destination. A destination is typically an IP range. But, it could also be a specific host. The screenshot here shows 2 routes.

1. The destination 172.31.0.0/16 is routed locally. That is, it is treated as local traffic.
2. The destination 0.0.0.0/0 is routed via the Internet Gateway associated with the VPC. 0.0.0.0/0 refers to any IP address.

An important thing to understand about route tables is that a more specific route always takes precedence over a generic one. For example, route#1 in our example will take precedence. Hence, the local traffic will not be routed via the Internet Gateway, which is precisely what we want.

Internet Gateway

As the name suggests – an Internet Gateway is the gateway to the Internet. It enables both – the communication to and from the Internet. In order to use an Internet Gateway you simply need to associate it with the VPC.

Understanding Peering Connections

A Peering Connection enables private IP-based communication between two VPCs. What’s the benefit? To understand the benefits better, let’s take a couple of scenarios.

1. An application in a VPC wants to talk to the database in a shared VPC: This is a fairly common scenario where expensive resources like an AWS RDS instance are shared across multiple applications each running within their respective VPC. Note that the RDS instance is shared and not the application database itself. If you are unfamiliar with RDS instance think of it as a database server. Since it is an expensive resource it is generally a good idea to share it. You can size it appropriately and also resize it on-demand! Such a shared RDS instance will be hosted in a common VPC to avoid application VPCs having to depend on each other unnecessarily. Let’s call it the “infra-vpc”. This is an important point when you start thinking of your deployment architecture. Host components in appropriate VPCs and avoid unnecessary VPC dependencies to keep it more clean and manageable. Now, when the application in an app VPC wants to talk to the database in the infra-vpc all that needs to happen is to enable communication between these two VPCs using private IPs. Why private IPs? Because you do not want to expose your database publicly and connect to it via the Internet. The private IP-based communication also often yields low latency thus minimizing the impact of any network delays. A VPC Peering Connection can be created to enable this private communication as shown in the screenshot.
2. An application in a VPC wants to talk to a service in another VPC. This scenario is possible in cases where you have micro-services based architecture and some of the micro-services are organized in individual VPCs. In such cases, if an application wants to talk to a micro-service residing in another VPC it could use VPC Peering and take advantage of the private IP-based communication.

Now that we understand some of the benefits of VPC Peering Connection, let’s understand what it is.

• It is an explicitly created connection that grants two VPCs permission to connect to each other using private IPs. The Peering Connection set up process involves issuing a request and granting of permissions.
• A Peering Connection can be created between VPCs within the same AWS account or totally different accounts.
• A Peering Connection can also be created between VPCs in different AWS regions!
• The two VPCs should not have overlapping IP ranges.
• A VPC can have multiple Peering Connections.
• As part of setting up a Peering Connection, the route tables need to be updated so that the destination traffic for the target VPC can be routed via the appropriate Peering Connection. For example, in the screenshot shown a Peering Connection is created between the c9-membership-dev-vpc and the infra-vpc so that the application running in the dev VPC can talk to the database in the infra-vpc. Hence, the routing table of the c9-membership-dev-vpc has to be updated to route traffic for the infra-vpc subnet via the Peering Connection and vice-versa.
  

Understanding Security Groups

A Security Group is a Virtual Firewall. It is an essential component of AWS deployments. Hence, it is worth taking time understanding how these work and setting these up correctly.

• A Security Group provides one or more allow rules. By default, all traffic is denied. Hence, the Security Groups only contain allow rules. For example, in the screenshot shown the infra-vpc-db-sg Security Group allows MySQL traffic on port 3306 from the IP range 10.0.0.0/16.
• The rules support several application layer protocols like HTTPS. However, when there is no specific match you can use TCP and UDP protocols.
• A Security Group allows specifying rules for both – inbound and outbound traffic. By default, all outbound traffic is allowed.
• The rules are stateful. This means if you allowed incoming access via a rule to a port, you do not have to have another rule for outgoing traffic for the same port in the reverse direction thus simplifying the configuration. Taking our earlier example of MySQL related Security Group, an EC2 instance in the valid IP range can establish a database connection. Since the inbound rule permits traffic, an outbound rule is not needed for the connection to return the data coming from the MySQL server to the EC2 instance.
• Each Security Group rule can use any of the following as a source.
  • An IP range
  • A set of IP addresses
  • Another Security Group! This is a very interesting capability. A use case could be you have a load balancer and 2 EC2 instances behind it listening on port 80. You can set up a Security Group rule so that the inbound traffic to the EC2 instance port 80 is only permitted from the load balancer security group. That is, only the load balancer will be able to talk to the EC2 instances port 80 and no one else. This makes the deployment more secure.
• Lastly, a VPC can certainly have multiple Security Groups. It is a good idea to have specific Security Groups than having a single Security Group that contains multiple different types of rules. As this way, you run the risk of over-allowing access to components that may not need it. For example, a web tier may only need access to the application tier, but not to the database. So, it is better to have different security groups for each tier than clubbing all of these into a single Security Group.

VPC Best Practices

Following are some of the key VPC best practices.

• Design your VPCs based on the application and infrastructure needs. For example, the types of subnets, IP ranges, security groups should factor in the components that will be running inside the VPC.
• Use logical names for the resources for ease of organization.
• Use non-overlapping IP ranges when setting up VPCs. If possible, identify IP ranges that will be used by VPCs (at least, in the near future) within your deployment. This will avoid running into issues, such as not able to do VPC peering because of IP range conflict (both VPCs having IP overlaps).
• Avoid/minimize the use of the default VPC as it is too generic and permits quite open access. Preferably use this for initial learning only and from that point use specifically designed VPCs based on the application and deployment needs.
• Use multi-AZ capabilities to set up VPC infrastructure and to deploy your stack for high availability purpose.
• Use Peering Connection to enable private IP-based communication between VPCs and avoid unnecessarily exposing components publicly.
• Be conservative when specifying the Security Group rules and grant only the required access.
• Create Security Groups in advance and avoid creating these on the fly as part of provisioning resources, such as creating an EC2 instance.
• Use specific Security Groups, such as based on application tier and avoid clubbing multiple different types of rules into one Security Group.
• Use Security Group as a source for the rule to enable a more secure configuration.

 

Happy learning!
– Nitin

 

 

---

Also published on Medium.