ACI - Best Practices

Cisco ACI Best Practices

When people starting out with Cisco ACI, they often struggle to find the correct or best settings for their ACI Fabric. Further they look for the correct settings, to make sure that the fabric is working properly.

To help people out, we line out some ACI best practices here, which should help people to set some important settings and configuration correctly. We will update this page in the future, to be sure to include new additions as well.

We try to reference to Cisco wherever possible, but some Best Practices are also based on our own experience of nearly 8 years with ACI.

Cisco ACI Global Settings

Cisco ACI AEP & VLAN Pool Best Practice

An AEP or Attachable Entity Profile, stitches together VLAN Pools & Physical Interface. It’s important to make sure that you have no overlapping VLANs within a single AEP.

With the help of a VLAN Pool, ACI decides which VXLAN ID (VNID) is assigned for each VLAN. Each AEP uses a different VNID Pool, which means that VLAN 20 attached to AEP A has a different VNID then VLAN 20 attached to AEP B.

For that reason it’s Best Practice for VLAN Pools to try to not overlap them. Less is definitely more here, we always try to go with one or two (If we do VMM integration).

The same goes for the AEP, it’s Best Practice there as well to try to have only the minimum required AEPs configured. In a lot of cases one can be enough. If you really need the scale and multiple AEPs and overlapping VLAN Pools, be careful!

Since Version 3.2.6 there is an option to enable Enforce EPG VLAN Validation, which prevents you from configuring overlapping VLANs that are attached to the same EPG.

It can be configured under System -> System Settings -> Fabric Wide Setting.

Issues when not following the Best Practices for AEP & VLAN Pool configuration

If you configure overlapping VLANs and then link them together through the same AEP, you can face various issues in the ACI Fabric.

The VNID assignments can become indeterministic and this leads to issues with STP BPDU flooding, endpoint learning and so on.

Cisco ACI Bridge Domain Best Practices

The Best Practices for the Bridge Domain Depends on different factors, there is a distinction between a Layer 2 Bridge Domain (Where the default gateway is outside ACI) and a Layer 3 Bridge Domain (Where ACI is the default gateway).

Layer 3 Bridge Domain

If the Bridge Domain is the Layer 3 default gateway for the Clients in the EPG, we call it Layer 3 Bridge Domain. In this case you have to decide if you need ARP flooding or not. If you have clustered servers, firewalls with some sort of active/standby mechanism, or Loadbalancers in this Bridge Domain, it’s best practice to enable ARP flooding.

These systems and their failover mechanism often rely on GARP to announce a failover from one of the members to the others. With ARP flooding, these messages are always flooded to all clients in the Bridge Domain.

Enabled ARP flooding

Personal Recommendation: Enable ARP flooding, never had an issue with it.

Layer 2 Bridge Domain

In case that the default gateway is outside ACI (e.g. a Firewall or a Router), then we talk about a Layer 2 Bridge Domain. Maybe there isn’t even a default gateway and the Bridge Domain only allows connectivity between some systems to sync/replicate data without ever leaving the network.

For a Layer 2 Bridge Domain, it’s Best Practice to set L2 Unknown Unicast to Flood.

Set L2 Unknown Unicast to Flood is Best Practice for a Layer 2 BridgeDomain

On a Bridge Domain with L2 Unkown Unicast set to Flood, you can’t disable ARP flooding. So this Best Practices is enforced ;)

Cisco ACI Contracts Best Practices

Over the years of deploying and operating ACI Fabrics for and with our customers, we learned one important lesson regarding ACI & Contracts.

Keep it simple!

In the beginning we tried different Designs and levels of Contract complexity, but ACI is no Firewall! It’s nearly impossible to manage larger sets of Contracts in production and the benefit they provide don’t justify this operational burden.

ACI is a great Networking Platform, but leave the Security to other Devices.

We have a great success with PBR designs, where we use a simple redirect Contract to forward all the Traffic to a DC Firewall. This Firewall then makes the security decision, is the traffic allowed or not, and send it back to ACI.

Best Practice for Contracts, keep it simple! A single allow (or PBR) is the best thing most of the time.

Cisco ACI Naming Best Practices

Prefixes in Naming of Objects

This is another topic where experience is key, at least it feels to me like that. In the beginning we often included the object type in the name, as example “T-Network” or “TN-Network” for a Tenant called Network. We prefixed the Name itself with the Type.

But the longer we worked with ACI, the more we had to realize that this just pollutes the names and doesn’t provide any real benefit in the end. We came to this conclusion after realizing where and how we use these names in the end.

Working with the API

If you are working with the API, then it’s anyhow always clear with what type of object you are working.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


{
  "fvTenant": {
    "attributes": {
      "annotation": "",
      "descr": "",
      "dn": "uni/tn-Network",
      "name": "Network",
      "nameAlias": "",
      "ownerKey": "",
      "ownerTag": "",
      "userdom": ":all:"
    }
  }
}

Here you can see the type of the object from the fvTenant class name and also in the DN tn-.

Working in the GUI

In the GUI, it’s always given which object you can choose, so there is never a confusion between a Tenant called “Network” and a Bridge Domain called “Network”.

You only see the Bridge Domain called Network, not the Tenant

Best Practice regarding Prefix in names

Don’t use type prefixes in your object names. Just go with a short and clear Name.

Tenant	VRF
Network	Network

A setup like that is completely valid for a lot of cases, if you have a 1 Zone = 1 Tenant Setup, you normally only need one VRF.

Access Port selectors & Port Policy Groups

This is another topic where we made quite a pivot from the original naming we used. It’s tempting to name your Access Port selectors and the respective Policy Groups after the Devices you connect to them. But what if these servers are renamed? You have to delete and recreate new Objects with different Names.

Probably you will just leave it as it is, and the name is just wrong. Been there, done that ;-)

For that reason we propose generic naming and then just work with the Description (Which is the information you see on the Leafs itself).

Best Practice for Access Port selectors

For Access Port selectors, use something in the style of ETH-1-01 or eth-1-01 or maybe just 1-01. But with that, the information should always be correct. For VPCs we normally just use the first port in the name.

Not saying this is perfect, but for us, it works better than system names.

Best Practice for Policy Groups

Policy Groups are normally required for the connection of a VPC to the ACI Fabric. Here we tend to do the same thing as we are doing for the Access Port Selectors. Use something generic in the format VPC-L111-ETH-1-01 which would be the channel on Leaf 111 (and 112) on Port 1.

Just some lines ago I told you to not use prefixes in names and now that? This is a bit a special case, because with this naming its clear that we are looking at a VPC in the Access Port Selector configuration :)

Every Rule has its exception, right?!

Open Best Practices

There are more to cover, especially the following which are hopefully covered soon:

Cisco ACI Multi Site Best Practices
Cisco ACI Migration Best Practices
Cisco ACI Monitoring Best Practices