Being able to configure routers and switches is not enough to be a networker. Before you implement a network, you need to plan its structure. In other words, you need to create a design that translates business requirements into technical specifications. Cisco defines design guidelines to create solid infrastructures, and among them, we have the Three-Tier Architecture. With this design, you can create scalable and manageable networks, that can adapt to business growth while providing robustness.
A Structured Approach
The Challenges
Being a network engineer can be quite challenging, specifically when you design new networks. In general, you need to create technical solutions that meet real-world business needs. However, you are going to deal with strict constraints such as tight budgets. Of course, we want to deliver the best possible solution for a given price. To do it, we need to understand what are the options the market is offering, and how we can use them in this very context.
But knowing the technologies out there is not enough. We need to select one over the other, identify which ones are mandatory for our deployment. Implementing unnecessary features will only harm the budget, while not giving any business-appreciable improvement. To avoid that, you must assess the requirements clearly. You can do it by evaluating the indicators described below.
Key Performance Indicators of your network
Before we can implement, we need to assess how good our network is (or how good we want it to be). To do it, we can measure three key items: scalability, resiliency, and manageability. Don’t consider these elements such as black-or-white indicators, but as an area of measurement. Each indicator has its own scale, and you need to identify how much each indicator is implemented.
Each network must meet these three requirements at least to some degree. If we understand what these requirements are about, we can then define a design that implements them in the best way.
- Scalability is the ability of the network to accommodate further growth. In a scalable network, you can add more switches to support more users without having to change the entire design. Often times, this translates into the possibility to add components without service disruption on devices already in place.
- A resilient network can tolerate faults of devices, and keep working even if a primary component fails. This is possible by using secondary devices that can take over the load upon failures. The more faults the network can tolerate, the better the resiliency. Furthermore, consider that before the secondary device can take over there might be some delay. Again, the less this delay is, the better the resiliency.
- Manageability is pretty self-explanatory. It is the ease you have when managing the network. You should be able to upgrade hardware and software, replace components with only limited outages. On top of that, your network should be easy to troubleshoot and work on. You must know in a deterministic way what is happening and why it is happening. All in all, this feature reflects the clarity of the solution.
Introducing the Three-Tier Architecture
A network out of control
Before you can identify a good network, you need to identify a bad network. Sadly, the network is full of them and you are going to deal with them daily. Often times, small or mid-sized companies underestimate the importance of a healthy network. As a result, they do not hire a network specialist for their network. This can only mean a network not following the three requirements defined above, not at all. Here’s an example.
This is a classic example of a horrible network. In jargon, this is a switch chain: a series of switches connected one next to the other. Something we must avoid. Even so, this is the most common deployment you are going to find in arranged networks. The reason for that is simple. At the very beginning, you have your router connecting to the Internet and a switch to connect all the users. At this point, the network is not so bad just yet. However, as the number of users grows your switch can’t accommodate them all. At this point, the “IT Guy” pops in and attach a new switch to serve more users. The business continues to grow, and the network can’t support more users. Then, someone adds another switch to the chain.
This design brings several problems. First of all, we highlighted all switches in red because they are a Single Point of Failure. If a switch dies, all users passing through it to reach the router will be disconnected. And this until someone physically replaces the switch. As a result, you can’t upgrade the software or hardware of a switch without creating outages. But there’s more. In this kind of deployment, it is extremely unlikely that they hired some specialized technicians to run cables. This translates into spaghetti cabling onto the switch. So, when you go there to replace it, you’ll have a very to tough time to reconnect everything in the right place.
The Three Tier Architecture
Also known as Three-layered Hierarchical Model, this is the Cisco flagship design for Campus networks. Its concepts, however, can be applied to any network, including Data Centers. This three-tier architecture is the most scalable and resilient solution. Its clear and simple design makes it also really easy to manage. The combination of these characteristics makes this design the best for large enterprises, where you need to connect thousands of devices on a large campus, even spanning onto several buildings.
The key concept behind this design is to group devices into three tiers. Devices into the same tier have all the same roles. Once the role of each device is clear, we can start to define the interconnections between devices. Even for this, Cisco provides us the best practices and specifications. Here’s a logical picture of the Three-Tier Architecture.
This approach defines three layers: Core, Distribution, and Access. Each layer has its specific purpose, so what we implement into the access layer won’t be implemented on the distribution and so on.
The Access Layer
The Access Layer is the one closer to the users. In fact, at this layer, we find the users themselves and the access-layer switches. The main purpose of this layer is to physically connect users to the network. In other words, there is just a cable between end-user PCs and access-layer switches.
At this layer, we apply network-access policies. These are the security policies we want to enforce in order to allow access to the network. For example, we can configure port-security or Network Access Control to ensure that only our company’s PC can have access to the network. Access switches propagate layer 2 broadcast domains to users, so we need to implement VLANs there. Furthermore, since we are talking about Layer 2, Spanning-Tree is running as a loop-prevention mechanism. We are still on the CCENT Track, while Spanning-Tree is part of the CCNA (ICND2) exam, so we are going to talk about that later on in the series. For now, just know that exists.
Generally speaking, at this layer we generally use simple fixed form-factor Layer 2 switches. This is because if we want to support more users we aren’t going to expand our access switches. Instead, we will simply add an access switch more.
The Distribution Layer
The Distribution layer bridges users to the core layer. It serves as a major spine for all users in an area, so it connects several access switches. In most deployment, Default Gateways for all the VLANs reside at the Distribution layer.
Cisco encourages you to reduce Layer 2 links in your network, having all Layer 3 links but the ones going directly to users. However, this is not yet possible as it requires some advanced access switches. As a result, the compromise implemented often-times is to have Layer 2 links between the Access and Distribution layer. This way, you can have default gateways residing at the distribution layer. Furthermore, here you can apply distribution policies. These policies include filtering traffic to allow some devices to reach only some of the other devices, or policy-routing (customizing routing, CCNP stuff).
While possible to connect server devices directly to the distribution switches, Cisco recommends connecting all end devices to access switches. This way, you reduce the overall cost-per-port of the infrastructure.
At this layer, you have to use multilayer switches. Furthermore, their form factor should be at least stackable, so that you can expand the distribution layer in case you need more ports. In large deployments, you can find modular switches at this layer.
The Core Layer
In the Three-Tier Architecture, the Core Layer is the one coordinating everything. It has only one, simple purpose: connecting all the distribution layers together. In large enterprises, where you have several distribution switches, the core layer is also known as Backbone.
Cisco is very clear about the purpose of this layer. Its only role is to forward traffic, the fastest it can. Here you don’t apply any policy, as you must try to reduce the load of the core so it can focus on routing. It is likely that your core switches talk with distribution switches using dynamic routing protocols, such as OSPF or EIGRP.
At this layer, we find the most advanced and expensive switches, the ones with the modular form factor. These are fully redundant devices supporting advanced Layer 3 switching features and dynamic routing protocols. However, remember that you need to keep your configuration as simple as possible on devices in this role.
Connecting the Three Tier Architecture
Cisco defined all the best practices about their Three Tier Architecture, including how to interconnect layers together. While not a requirement, we suggest using fiber optic on all links but the access links to the users. However, when not possible, you can use aggregated copper links.
As a mandatory requirement, you must have two core devices and a pair of distribution switches for each module (a group of access switches) to be served. This means that each device will have two devices right on top of it. As a result, each access switch must be interconnected to both its distribution switches. Then, each distribution switch must be connected to both cores. This not only increases throughput (with the proper configuration) but adds resiliency. In case a device fails, traffic can flow to the other device remained active smoothly.
Two redundant devices, like the two cores of a pair of distribution switches, might be connected together. As a best practice, this link must be a Layer 3 point-to-point link. If this is not available due to software limitations, you should avoid a Layer 2 link and have no link at all. This way you will prevent loop and spanning-tree convergence time. Sometimes, even in networking the less is more.
With this solution, you can easily add more access switches or a pair of distribution switches. You can easily upgrade the core by adding new modules, and also shut down a device for maintenance while still leveraging the redundant path to avoid outages.
Collapsed Core Architecture
The Three Tier Architecture is a great solution for large enterprises. However, it can be very expensive and its cost won’t be justified in a smaller environment. No worries, Cisco has designed a smaller solution that still provides some degree of scalability, resiliency, and manageability. We are talking about the Collapsed Core Architecture, an underpowered version of the Three-Tier Architecture.
With this design, we merge the core layer and distribution layer into a single layer called collapsed core. As a result, this design has two layers (tiers) instead of three, like in the picture below.
Here, the collapsed core takes over the functionalities of both core and distribution. It connects access switches together and to uplinks (like to the Internet), while applying distribution policies. The role of the access layer is untouched. The design best practices are still the same, each access switch must be connected to both core switches. It is likely that core devices will host the default gateways, doing Inter-VLAN Routing.
In this design, the core may not be considered the backbone, as it just interconnects access switches and no distribution switches. This design still provides a great degree of scalability, as you can expand the core by adding more modules or stack members (in case of stackable switches). Again, you can simply add more access switches as you wish.
You can easily migrate from this design to a full Three Tier Architecture by adding two core devices on top of the collapsed core and then adding more distribution modules at will.
Conclusion
In this article, we covered the design best practices for a campus network according to Cisco. We unveiled the Three Tier Architecture and the Collapsed Core Architecture as well, discussing their benefits. We are now able to identify a network badly-implemented and propose design fixes. To recap, here we have some bullet-points of what you need to remember.
- A network should be scalable, resilient and easy to manage
- To implement these three characteristics, you need to have clear the role of each device in the network. Cisco recommends defining three roles (layers, tiers): access, distribution, and core
- Access layer switches physically connect the users and enforce network-access policies
- Distribution switches connect multiple access switches to the core and enforce policies such as firewalling or policy routing
- The Core layer switches just route traffic between distribution switches, the fastest possible. You shouldn’t apply any policy there
- When you have design constraints, you can collapse together Distribution layer and Core layer
- Cisco recommends preferring Layer 3 links over Layer 2 links when connecting layer together whenever possible
With this knowledge, you are now ready to discuss what other features a network needs to meet modern business requirements, such as mobility and security. Once you’ll see the full picture, you will be ready to implement other cutting-edge technologies and try them with hands-on labs.
