EVE-NG Default IDLE PC

As I continue on my quest for the perfect lab I have been messing around with EVE-NG (a competitor to GNS3 and VIRL). One thing I really like about EVE-NG is the ability to use a web client instead of needing a thick client like you do for GNS3 and VIRL. A draw back I’ve run into with EVE-NG is its lack of quick and easy customization. For example, configuring the default IDLE PC for a Cisco IOS node in GNS3 is very straightforward. Doing extensive Google searching only resulted in a single document on the “old” way of configuring the default IDLE PC and a bunch of hits for GNS3 configuration. This post is going to be a quick guide on configuring the default IDLE PC settings for EVE-NG.

First ssh to the EVE-NG CLI and issue the “top” command:Screen Shot 2018-04-17 at 7.31.30 PM.png

Open a second SSH session to the EVE-NG CLI, while leaving the first SSH session open. Now run the IOS from the second CLI, for this tutorial I will be issuing the IDLE PC for a C7200 node:Screen Shot 2018-04-17 at 7.40.08 PM

Once the IOS device completes booting (after exiting the initial configuration dialog) navigate back to the first SSH session with top running and observe the CPU load:Screen Shot 2018-04-17 at 7.50.14 PM

As will be noticed, the CPU is running pretty high for a single IOS device running. To fix this issue, we need to have Dynamips (the software that emulates the Cisco IOS) calculate a better IDLE PC value compared to the default.  To do this, navigate back to the SSH session running the IOS emulation and use the keyboard combination “Ctrl+]” and then hit “i”:Screen Shot 2018-04-17 at 8.02.08 PM

The highest count is usually the best value to use, so quit out of the IOS session with “Ctrl+]” and then”q”. Then start the IOS session over but appended with “–idle-pc=” and the value selected.

Now go back to the SSH session running “top” and see how the CPU is doing:Screen Shot 2018-04-17 at 8.20.35 PM

As can be seen, a single IOS instance is only taking 1% CPU instead of 25% which is much better!

Now time for the fun part. Go ahead and close the session running the IOS node in CLI.

Next edit the template file with vi. For this template I will be using the command “vi /opt/unetlab/html/templates/c7200.php”. If you are editing a different template they are all found within that directory.

Once inside the template, locate the idle PC variable and set it to the determined value:Screen Shot 2018-04-17 at 9.04.53 PM

Save the edited file and any new node created from that template will be created with the configured IDLE PC value.

As can be seen in my environment I am able to run 51 IOS Nodes with only utilizing 50% CPU:Screen Shot 2018-04-17 at 10.25.41 PM

Happy labbing!

Advertisements

Using Raspberry Pi’s for Network Monitoring and Health Status – Part 1 : Dual NIC Network Connectivity

I have recently been able to spend sometime working with Raspberry Pi’s both for personal projects as well as a few things for work. This seems like a great opportunity to do a series of posts on them! The first post will be around utilizing both NICs at the same time to allow for “Out Of Band” management of the Pi while using the other NIC for network testing.

This will be one of hopefully many posts around this topic.

First a quick background:
The Raspberry Pi is an interesting mini computer that comes in a few different form factors and a very low price point. The GPIO (General Purpose Input/Output) connectivity allows for some really neat projects. The Raspberry Pi can run any OS that will run on the CPU, but the primary OS is Raspbian. Raspbian is a version of Linux optimized to work efficiently on the Raspberry Pi hardware platform. The newest version (3 Model B+) includes both built in Gigabit Ethernet NIC and a Dual Band wireless AC NIC.

770a5842-1612x1080

***As always, make sure to have appropriate backups prior to making any changes. It is also highly recommended to do all development work in a test environment not attached to any production environment. Following any of the provided steps or using any of the provided information is at the users own risk.***

Now for the good stuff:
This setup is intended to utilize the Wired NIC as the management interface while all testing will be performed through the wireless NIC. This allows for remote wireless connectivity testing at locations without having to have a onsite tester, as well as provides the ability to perform general RF spectrum health analysis. This is accomplished by having the primary preferred default route over the wireless NIC and a more specific route to the management endpoint (or subnet) over the wired NIC.

The default behavior when connected to both a wired and wireless NIC is to allow both connections with each having a default route, wired having a lower metric and preferred over the higher metric of the wireless NIC:2018-01-12-014702_1824x984_scrot

To start off, and to make sure connectivity is never lost, configure the static route that forces traffic destined to the management PC / subnet over the wired NIC. To do this a new file called “40-route” needs to be created in “/lib/dhcpcd/dhcpd-hooks/” that contains the static route entry:2018-01-12-033146_1824x984_scrot

After the file is created and saved restart dhcpcd services and make sure the route entry is added into the route table:2018-01-12-033626_1824x984_scrot

To prefer a specific interface (such as the Wireless NIC, for testing of the Wireless environment) the metric for the interface needs to be set to a lower value. I also recommend setting the management interface to a higher value so that it is always higher then the preferred testing interface. To do this edit the file “dhcpcd.conf” using your favorite editor to include the interface metric configurations as shown below:2018-04-08-171319_1920x1080_scrot

Once the file is edited and saved, restart dhcpcd services and make sure the new metrics are shown in the route table:2018-04-08-171508_1920x1080_scrot

Once setup and verified all primary traffic not destined for any directly connected subnets or the static route assigned will use the more preferred (lower metric) interface.

Now that the Raspberry Pi is setup, you can deploy a low cost network monitor probe using tools such as iperf, IP SLA, or any other responder software that you want to be forced over a specific medium for testing at a location.

 

 

Recovering an ESXi host with the state archive….

I had the joy of dealing with the USB drive which hosts my ESXi host’s OS and configuration go bad, and naturally without a backup of the configuration.

I did manage to recover enough of the drive to gain access to the “state” archive which contains the host configuration (this is stored as state.tgz).

The following steps will go over how to restore the files located in the state archive to be able to run the host in its original pre-failure state. This will NOT cover how to recover state.tgz from a failed drive as there are far too many variables to be able to cover the different ways of recovery. It is highly recommended to work with copies of all data being manipulated and not the original version. All steps below are to be tested in a lab environment and extreme caution should be used when attempted in production environments.

1. Install ESXi on the host with new media. Steps for this can be found here. To make this easier, configure the host to the point of having username/password and network connectivity. I recommend using a different IP address and username/password combination from the original failed host so that way you can verify the restored configuration takes affect.

2. Copy state.tgz to a directory on the ESXi host such as /tmp I used WinSCP, but you are welcome to whatever method is desired.

3. Navigate to the location state.tgz is stored and extract the contents (local.tgz): Step3

4. Verify if local.tgz is already located in your root directory and rename it if it is:Step 4

5.  Copy the local.tgz file from /tmp to the root directory:Step 5

6. Extract the contents of local.tgz with tar -xvzf local.tgz:Step 6

7.  Run auto-backup.sh to have the extracted contents replaced with the current host’s configuration:Step 7

8. Reboot and verify your failed host has its settings restored:step-8

9. Setup proper backups for your virtual and physical server environment!  (Not included in this post).

 

Happy labbing!

802.1x VLAN User Distribution (VLAN Group)

In this blog post, I will be going over 802.1x VLAN User Distribution (sometimes referred to as “VLAN Groups”) in Cisco IOS and a use case scenario that involves Cisco ISE (Identity Services Engine).

First, some background around VLAN Groups. Based on my research it seems there are two major types of VLAN Groups: The Firewall Service Module (FWSM) on the 6500 and on Cisco IOS & IOS XE Switches. It appears to possibly have other functionality within the Wireless Space for user assignment, but I did not do extensive research on that aspect to find an inclusive answer. In the world of IOS a VLAN group is simply a group that has a name assigned to it that can contain one or more VLANs assigned to that group.

The main purpose of 802.1x VLAN User Distribution is to dynamically provide VLAN load balancing by having the RADIUS server dictate the VLAN Group name within attribute 81 (Tunnel-Private-Group-ID) in the RADIUS response instead of a regular VLAN ID/Name. When the switch receives the VLAN Group name, it will assign the endpoint to the least populated configured VLAN for that group. Prior to IOS release 12.2(33)SXI1, this was accomplished by having multiple VLAN names specified under attribute 81.

A use case I have found outside of VLAN distribution load balancing (and the reason I know about VLAN Groups) is to provide a way to dynamically assign a preconfigured VLAN that does not have a uniform number across the enterprise from ISE. This case in particular was to have a predefined VLAN, that would span multiple different VLAN numbers, specific for Cisco IP Phones not tied to a Cisco CM dynamically assigned once the appropriate device profile in ISE was determined. This allows for the ability to have a different option 80 fields in the DHCP response to direct the phones to their non-Cisco based Call Manager.

To take advantage of this configuration, the VLAN group assigned with your desired VLAN(s) must be configured on the switch and the authorization profile that will be applied from ISE must be configured with RADIUS attribute 81 set to the VLAN group name.

To configure a VLAN group in IOS perform the following task:
SW1(config)# vlan group group-name vlan-list vlan-list

To note:

  • A VLAN Group name can be up to 32 characters
  • A VLAN Group name must start with a letter
  • Group members can be specified as a single VLAN ID, a list of VLAN IDs, or a VLAN ID range. Multiple entries are separated by a hyphen (-) or a comma (,) similar to the interface range command.
  • To remove a VLAN from the VLAN group, use the no version (no vlan group group-name vlan-list vlan-ID).
  • The VLAN Group will be removed once the last VLAN ID is removed from the group.

Configuring a VLAN Group on a Cisco Switch:rh3cn9l
vlan group TEST_VG vlan-list 410

Configuring VLAN Group assignment in ISE:
image002
Navigate to Policy Elements > Results > Authorization > Authorization Profiles > Profile
Select VLAN and Enter VLAN Group Name

Once a endpoint is authenticated against the switch via 802.1X and the appropriate authorization profile is assigned, the VLAN configured on the switch for the VLAN group is assigned:Verfication

Some bonus verification information:

When a VLAN is statically assigned via 802.1X, the VLAN assignment can be seen across multiple switchport / VLAN status commands.

The first command is show vlan.

Before dynamic VLAN assignment (port configuration):verification1

After dynamic VLAN assignment (via 802.1X with VLAN Group):verification2

The second command is show interface interface-name switchport:

Before dynamic VLAN assignment (port configuration):verification3

After dynamic VLAN assignment (via 802.1X with VLAN Group):verification4

Remote Lab Access and Control

A requirement I’ve quickly come to realize with building my lab is remote access into lab my equipment. This requirement is two fold, I don’t feel like always sitting in my basement to build topologies and I’m not always home when I will be studying. This need naturally led me to acquiring a terminal server, which was very helpful in fulfilling my first need of not having to always hang out in the basement when studying. I didn’t like the idea of always leaving my lab equipment on wasn’t exciting to me as I don’t like wasting electricity, so I found a Remote Power Control (RPC) unit also known as a switched PDU.

I enjoyed setting everything up so I figured I’d share the configuration steps I took to get two devices communicating with each other and functioning. The two devices I used were a Opengear IM7200 terminal server and a Avocent (Cyclades) PM10. The setup is pretty straightforward with minimal steps.

First you need to make sure the RPC unit is cable properly, for the PM10 a serial console connection is made of a UTP straight through cable from one of the serial ports on the Opengear terminal server to the “In” port on the PM10. You can daisy chain multiple PM10s together by going from the “out” port to the “in” port on the next PM10, but I recommend setting up each PM10 as an individual serial port on the terminal server. This gives more flexible control and you won’t lose multiple RPCs if you have a failure “up stream” in the daisy chain. After the cabling is taken care of its time to move on to the fun part, configuration!

The first configuration component is to configure the serial port on the IM7200 to the PM10. To do so navigate to the Serial Port configuration section: ynwxbpg

The next step is to configure the serial port connected to the PM10 by editing the port on the IM7200:um6g9qy

The following settings are specific to the PM10 connection and need to be configured on the serial port on the IM7200 for connectivity:rbf9hxzoihsmtcSettings include:

  • Label – Port name you would like
  • Baud Rate – 9600
  • Data Bits – 8
  • Parity – None
  • Stop bits – 1
  • Flow control – None
  • Port Pinout – Cisco Straight (X1)
  • Terminal Type – ansi

In addition to the required serial settings, the serial port must be set to a device type of “RPC” so that the terminal server knows how to handle the port:r8igm0v

Next navigate to the RPC configuration under Serial & Networks:mj3mucf

Next click on ” Add RPC”:ijx5dch

Next setup the RPC configuration on the IM7200 withe the following settings:dly8d0pSettings include:

  • Connected via – Serial Port previously configured
  • RPC Type – Cyclades PM10
  • Name – Whatever you would like to name it
  • Outlets (optional) – set it to 10 or leave it as default for auto-probing
  • Username / Password – Set to admin/password for PM10
  • Log Status – Enabled (Checked)
  • Log Rate – Setting you would like

The next step is to configure serial ports connected to console ports on the devices controlled by the RPC with the Power Menu enabled:twmrllo

The last step is to setup a Managed Device for each device to be controlled by the RPC, to do so navigate to “Managed Devices” under Serial & Networks:kawabbv

Click “Add Device”:fr3x9gy

Finally configure the device with a name, assigned console port, and assigned RPC port:f8ohwtg

After configuration the devices can be managed under devices:tlbf8dt

Or right from the console sessions via the terminal server:yla5om8

Happy labbing!

A short little time lapse I made…

I decided that I wanted to almost double the time it took to re-cable my CCIE Lab so I made a time lapse video out of it. I think it turned out pretty well!

 

 

The layout is pretty straight forward, I have a 2801 and 3560 as a “hub” which acts is a central place of connectivity for 5 other “pods” that consist of a 1841 and 3560. A diagram will follow I’m sure.

 

Enjoy 🙂

Cisco ISE REST API & Python

I’ve been faced with a fun little challenge on how to make sure our ISE deployment has every NAD (Network Access Device) configured appropriately to allow for successful EAP communications. Originally I was planning on utilizing a CSV and the bulk import tool to regularly import new devices into ISE as they were built. This allows for a number (small or large) of devices to be imported into ISE without taking too much time. This has worked well in the past but creates an reliance on making sure the CSV is proper and that someone (me) still has to manually login and import the file. With that I decided to look into other possibilities to remove the “me” from the process flow. At first I was looking into ways to automatically populate the CSV and then script out away to login to ISE and force the bulk import. While that option would work, it seemed to be too complicated to really deploy and rely on. I finally decided to give another whack at using the REST API ( I had previously tried years prior with ACS but did not have much luck).

There are a two things that need to be done on ISE prior to being able to utilize the REST API. The below screens and settings are based on ISE 2.2 but are similar between all recent releases of ISE:

  1. Create an account that will be utilized for the REST calls.  To do this navigate to: Administration > Admin Access > Administrators > Admin Users and click on “Add”:ialbmj4
    Currently there are two different access types you can assign: Read/Write or ReadOnly. For the code about to be run, we need Read/Write.
  2. Enable ERS (External RESTful Services) to allow REST calls. To do this navigate to: Administration > System > Settings > ERS Settings then select “Enable ERS for Read/Write” under the Primary Administration Node:fsdtf17
    This setting must be enabled after each upgrade as its set to disabled during the upgrade. If you plan to utilize the REST API, I recommend adding to your upgrade documentation / process that the REST API is enabled at the end of the upgrade.

After a user account is created and ERS is enabled the REST API can be utilized via HTTPS on port 9060. API documentation can be found at: https://ISE-PAN-IP:9060/ers/sdk

Now that the API is exposed its now for some fun! But first some cautions / warnings..

  1. This is by no means a tutorial about REST API or Python.
  2. You really should have a good understanding about REST API before enabling it. I’m still skeptical about the security around access when it comes to REST.
  3. You should never use a production system to develop code that makes changes to it.
  4. Use the code shown at your own risk!

And a few notes about the code…

  1. The below code is not complete, and needs tweaking to be functional. Its intention is simply to show a proof of concept for automating device creation.
  2. The code calls ‘nad.xml’ which is a separate XML file (can be found on my github repository). I will not be going over the file in this tutorial, but can be manipulated for actual use.
  3. The final output is not pretty and may not be complete depending on the number of devices being imported.
  4. The code below is a picture due to me not knowing how to easily paste code that looks nice on wordpress. A copy of the code can by found on my github repository.
  5. The IP address or FQDN of your ISE PAN needs to be updated prior to running the code.
  6. A proper authorization key needs to be added prior to running the code. This will be from the account you created earlier.
  7. It would be a good idea to create a variable for the ISE PAN information to use for multiple URLs.
  8. It would be a good idea to create a variable for the authorization information to use for multiple calls.

Now its really time for the fun part!

The below code is intended to do two things: Bulk create network devices in ISE and to verify the status of the bulk job:

kdtfnke

Code Breakdown:

  • Lines 2 – 8 are simply to deal with importing the XML file. You could just include it in the script and assign it to the payload variable (referenced in line 18) but that doesn’t make this usable in a production environment.
  • Line 11 is the URL used for the first API call. Don’t forget to update with your ISE PAN information.
  • Line 15 is where you should update your authorization information.
  • Line 18 has an extra variable in the request which is “verify” set to “False”. This lets you ignore certificate warnings. In my lab I didn’t bother deploying trusted certs so I needed this.
  • Line 18 is the actual API call being pushed. If you do not care about the status you could simply end here or just print the response.
  • Line 21 grabs just the location header from the response from the API call. The location header is a URL containing the BULK ID that is parsed from the entire URL to use for the second portion of the script.
  • Line 28 is the URL used for the second API call + the BULK ID. Don’t forget to update with your ISE PAN information.
  • Line 32 is where you should update your authorization information.
  • Line 36 has an extra variable in the request which is “verify” set to “False”. This lets you ignore certificate warnings. In my lab I didn’t bother deploying trusted certs so I needed this.

Lets see the code in action!

First we will look to see whats configured in ISE for network devices:9aczrfjNow lets run the script:hghklfn

As can be seen the XML containing 10 network devices was still in progress when the status check was run. If this was production code there are multiple options to avoid this such as a timer could be implemented, the user could be asked when to run the check, or constant checking until it completes.

Now lets see what we have in ISE:qghpxe9Ten brand new devices!

The API in Cisco ISE has many different functions that can allow for the creation, modification, or deletion of several different objects outside of network devices. This is just one example of the power that is available for automating functions within ISE that have been around for a while.