Automating Network Configuration with Device Lists from File
In the previous post, already talk about automation using Multithreading and automation using configuration files. So, in this post let’s talk about automating network configuration using multithreading and device definition form files. Previouly all device definition using hard-coded mechanisme. Hard-coded device definitions come with several disadvantages. Any time a host changes, the script must be updated manually, making the process inflexible and possibility lead to human error. A better approach is to store device definitions in external source file, such as a CSV or JSON file. Its will be easy to maintain devices directly in the CSV file without changing the python script directly. In this post device definition using CSV file. Same as previous post, Network automation in this post using netmiko that running on pyhton version 3. Device type that used is Cisco IOS-XE and Cisco IOS-XR. The automation to show OSFP routes and verify adjacancy.
Becase device definition using CSV file, than create as below with file name is devicelist.csv
hostname,device_type,host,username,password
XRV1,cisco_xr,11.10.10.1,admin,password
XRV2,cisco_xr,11.10.10.2,admin,password
R3,cisco_ios,11.10.10.3,cisco,cisco
R4,cisco_ios,11.10.10.4,cisco,cisco
R5,cisco_ios,11.10.10.5,cisco,cisco
After that, call the CSV file and turn into dictionary. Create an empty dictionary to store device definition using variable devices_dict = {}. Open the devicelist.csv in read mode using 'r'. The csv.DictReader read the CSV file and convert each row becomes dictionary. The headers form CSV will be the keys and data each row will be the The Values.
# Device definitions
devices_dict = {}
with open ('devicelist.csv','r') as file:
reader = csv.DictReader(file)
Current device definition as below:
{'hostname': 'XRV1', 'device_type': 'cisco_xr', 'host': '11.10.10.1', 'username': 'admin', 'password': 'password'}
{'hostname': 'XRV2', 'device_type': 'cisco_xr', 'host': '11.10.10.2', 'username': 'admin', 'password': 'password'}
{'hostname': 'R3', 'device_type': 'cisco_ios', 'host': '11.10.10.3', 'username': 'cisco', 'password': 'cisco'}
{'hostname': 'R4', 'device_type': 'cisco_ios', 'host': '11.10.10.4', 'username': 'cisco', 'password': 'cisco'}
{'hostname': 'R5', 'device_type': 'cisco_ios', 'host': '11.10.10.5', 'username': 'cisco', 'password': 'cisco'}
Then, remove the hostname keys and insert its value as dictionary key and the remaining row data as the value of devices_dict.
for row in reader:
name = row.pop('hostname')
devices_dict[name] = row
So, the device definitiong from device_dict variable. The key is hostname and the rest become the values. details as below:
{
'XRV1': {'device_type': 'cisco_xr', 'host': '11.10.10.1', 'username': 'admin', 'password': 'password'},
'XRV2': {'device_type': 'cisco_xr', 'host': '11.10.10.2', 'username': 'admin', 'password': 'password'},
'R3': {'device_type': 'cisco_ios', 'host': '11.10.10.3', 'username': 'cisco', 'password': 'cisco'},
'R4': {'device_type': 'cisco_ios', 'host': '11.10.10.4', 'username': 'cisco', 'password': 'cisco'},
'R5': {'device_type': 'cisco_ios', 'host': '11.10.10.5', 'username': 'cisco', 'password': 'cisco'}
}
As the execution function later call the devices_dict for each items. And device_info is the value device_dict, in order to call specific value it must use devices_dict.items(). So, in the execution function able to pass the the value for device_name which is hostname and others value is device_info. Example, in order to get the value of host, which is an IP Address. So, it must be called using device_info ['host'].
execution(device_name, device_info):
#become:
execution("XRV1", {"device_type": "cisco_xr", "host": "11.10.10.1", …})
This thread pool will run simultaneously maximum 5 devices using max_workers=5. This multithread run the function execution (device_name, device_info) concurrently for all host in devices_dict.items(). After the execution complete this syntax will calculate the total of duration execution time.
# Main execution
if __name__ == "__main__":
start_time = datetime.now()
with ThreadPoolExecutor(max_workers=5) as executor:
multi_thread = [executor.submit(execution, device_name, device_info) for device_name, device_info in devices_dict.items()]
for thread in as_completed(multi_thread):
print(thread.result())
end_time = datetime.now()
The execution function using device_name and device_info arguments. This syntax will have two commands will be running from each device concurently, to check OSPF Adjacny and OSPF routes based on device_type. After all command execute in one device, SSH will be automatically close connection using net_connect.disconnect()
def execution(device_name, device_info):
try:
output_result = f"\nConnecting to {device_name} ({device_info['host']})"
net_connect = ConnectHandler(**device_info, fast_cli=False)
commands = command_list.get(device_info['device_type'], [])
commands2 = command_list2.get(device_info['device_type'], [])
for cli in commands:
output = net_connect.send_command(cli)
output_result += f"\n Output of '{cli}' on {device_info['host']}:\n{output}"
for cli in commands2:
output = net_connect.send_command(cli)
output_result += f"\n Output of '{cli}' on {device_info['host']}:\n{output}"
net_connect.disconnect()
output_result += "\n##############################################"
return output_result
except Exception as e:
return f"Failed to connect to {device_info['host']}: {e}"
Results
Duration for the same scenario is only 3 seconds and give more flexibility to changing device list without changing the automation script.
Connecting to R5 (11.10.10.5)
Output of 'show ip ospf neighbor' on 11.10.10.5:
Neighbor ID Pri State Dead Time Address Interface
11.10.10.1 1 FULL/BDR 00:00:33 172.16.0.18 GigabitEthernet0/0
11.10.10.4 1 FULL/BDR 00:00:33 172.16.0.13 GigabitEthernet0/2
Output of 'show ip route ospf' on 11.10.10.5:
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is not set
11.0.0.0/32 is subnetted, 5 subnets
O 11.10.10.1 [110/2] via 172.16.0.18, 23:03:44, GigabitEthernet0/0
O 11.10.10.2 [110/3] via 172.16.0.13, 23:27:50, GigabitEthernet0/2
O 11.10.10.3 [110/65537] via 172.16.0.18, 23:03:44, GigabitEthernet0/0
O 11.10.10.4 [110/2] via 172.16.0.13, 23:31:37, GigabitEthernet0/2
172.16.0.0/16 is variably subnetted, 7 subnets, 2 masks
O 172.16.0.0/30
[110/65536] via 172.16.0.18, 23:03:44, GigabitEthernet0/0
O 172.16.0.4/30
[110/65537] via 172.16.0.18, 23:03:44, GigabitEthernet0/0
[110/65537] via 172.16.0.13, 23:27:50, GigabitEthernet0/2
O 172.16.0.8/30 [110/2] via 172.16.0.13, 23:31:37, GigabitEthernet0/2
O 192.168.100.0/24 [110/2] via 172.16.0.13, 23:31:37, GigabitEthernet0/2
##############################################
Connecting to R4 (11.10.10.4)
Output of 'show ip ospf neighbor' on 11.10.10.4:
Neighbor ID Pri State Dead Time Address Interface
11.10.10.5 1 FULL/DR 00:00:37 172.16.0.14 GigabitEthernet0/2
11.10.10.2 1 FULL/BDR 00:00:39 172.16.0.9 GigabitEthernet0/1
Output of 'show ip route ospf' on 11.10.10.4:
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is not set
11.0.0.0/32 is subnetted, 5 subnets
O 11.10.10.1 [110/3] via 172.16.0.14, 23:01:16, GigabitEthernet0/2
O 11.10.10.2 [110/2] via 172.16.0.9, 23:25:19, GigabitEthernet0/1
O 11.10.10.3 [110/65537] via 172.16.0.9, 23:25:19, GigabitEthernet0/1
O 11.10.10.5 [110/2] via 172.16.0.14, 23:29:10, GigabitEthernet0/2
172.16.0.0/16 is variably subnetted, 7 subnets, 2 masks
O 172.16.0.0/30
[110/65537] via 172.16.0.14, 23:01:16, GigabitEthernet0/2
[110/65537] via 172.16.0.9, 23:25:19, GigabitEthernet0/1
O 172.16.0.4/30
[110/65536] via 172.16.0.9, 23:25:19, GigabitEthernet0/1
O 172.16.0.16/30 [110/2] via 172.16.0.14, 23:29:10, GigabitEthernet0/2
##############################################
Connecting to R3 (11.10.10.3)
Output of 'show ip ospf neighbor' on 11.10.10.3:
Neighbor ID Pri State Dead Time Address Interface
11.10.10.2 1 FULL/BDR 00:00:38 172.16.0.6 GigabitEthernet0/0
11.10.10.1 1 FULL/BDR 00:00:37 172.16.0.1 GigabitEthernet0/1
Output of 'show ip route ospf' on 11.10.10.3:
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is not set
11.0.0.0/32 is subnetted, 5 subnets
O 11.10.10.1 [110/2] via 172.16.0.1, 23:03:48, GigabitEthernet0/1
O 11.10.10.2 [110/2] via 172.16.0.6, 23:27:49, GigabitEthernet0/0
O 11.10.10.4 [110/65537] via 172.16.0.6, 23:27:49, GigabitEthernet0/0
O 11.10.10.5 [110/65537] via 172.16.0.1, 23:03:48, GigabitEthernet0/1
172.16.0.0/16 is variably subnetted, 7 subnets, 2 masks
O 172.16.0.8/30
[110/65536] via 172.16.0.6, 23:27:49, GigabitEthernet0/0
O 172.16.0.12/30
[110/65537] via 172.16.0.6, 23:27:49, GigabitEthernet0/0
[110/65537] via 172.16.0.1, 23:03:48, GigabitEthernet0/1
O 172.16.0.16/30
[110/65536] via 172.16.0.1, 23:03:48, GigabitEthernet0/1
O 192.168.100.0/24
[110/65537] via 172.16.0.6, 23:27:49, GigabitEthernet0/0
##############################################
Connecting to XRV1 (11.10.10.1)
Output of 'show ospf neighbor' on 11.10.10.1:
Sun Aug 10 16:37:08.489 UTC
* Indicates MADJ interface
# Indicates Neighbor awaiting BFD session up
Neighbors for OSPF 100
Neighbor ID Pri State Dead Time Address Interface
11.10.10.5 1 FULL/DR 00:00:30 172.16.0.17 GigabitEthernet0/0/0/0
Neighbor is up for 23:03:48
11.10.10.3 1 FULL/DR 00:00:34 172.16.0.2 GigabitEthernet0/0/0/1
Neighbor is up for 23:03:48
Total neighbor count: 2
Output of 'show route ospf' on 11.10.10.1:
Sun Aug 10 16:37:09.197 UTC
O 11.10.10.2/32 [110/65537] via 172.16.0.2, 23:03:47, GigabitEthernet0/0/0/1
O 11.10.10.3/32 [110/65536] via 172.16.0.2, 23:03:47, GigabitEthernet0/0/0/1
O 11.10.10.4/32 [110/65537] via 172.16.0.17, 23:03:47, GigabitEthernet0/0/0/0
O 11.10.10.5/32 [110/65536] via 172.16.0.17, 23:03:47, GigabitEthernet0/0/0/0
O 172.16.0.4/30 [110/65536] via 172.16.0.2, 23:03:47, GigabitEthernet0/0/0/1
O 172.16.0.8/30 [110/65537] via 172.16.0.17, 23:03:47, GigabitEthernet0/0/0/0
O 172.16.0.12/30 [110/65536] via 172.16.0.17, 23:03:47, GigabitEthernet0/0/0/0
O 192.168.100.0/24 [110/65537] via 172.16.0.17, 23:03:47, GigabitEthernet0/0/0/0
##############################################
Connecting to XRV2 (11.10.10.2)
Output of 'show ospf neighbor' on 11.10.10.2:
Sun Aug 10 16:37:07.451 UTC
* Indicates MADJ interface
# Indicates Neighbor awaiting BFD session up
Neighbors for OSPF 100
Neighbor ID Pri State Dead Time Address Interface
11.10.10.3 1 FULL/DR 00:00:37 172.16.0.5 GigabitEthernet0/0/0/0
Neighbor is up for 23:27:55
11.10.10.4 1 FULL/DR 00:00:32 172.16.0.10 GigabitEthernet0/0/0/1
Neighbor is up for 23:27:52
Total neighbor count: 2
Output of 'show route ospf' on 11.10.10.2:
Sun Aug 10 16:37:08.317 UTC
O 11.10.10.1/32 [110/65537] via 172.16.0.5, 23:03:49, GigabitEthernet0/0/0/0
O 11.10.10.3/32 [110/65536] via 172.16.0.5, 23:27:54, GigabitEthernet0/0/0/0
O 11.10.10.4/32 [110/65536] via 172.16.0.10, 23:27:49, GigabitEthernet0/0/0/1
O 11.10.10.5/32 [110/65537] via 172.16.0.10, 23:27:49, GigabitEthernet0/0/0/1
O 172.16.0.0/30 [110/65536] via 172.16.0.5, 23:27:54, GigabitEthernet0/0/0/0
O 172.16.0.12/30 [110/65536] via 172.16.0.10, 23:27:49, GigabitEthernet0/0/0/1
O 172.16.0.16/30 [110/65537] via 172.16.0.10, 23:27:49, GigabitEthernet0/0/0/1
O 192.168.100.0/24 [110/65536] via 172.16.0.10, 23:27:49, GigabitEthernet0/0/0/1
##############################################
Duration: 0:00:03.472142