How to optimize your workload with scripts

If you are thinking of expanding your network to support more clients, then you need to think about making your network more efficient and at the same time more user friendly for your clients. However, this is easier to say it than to implement it.
In today’s networks, together with the more complex requirements from clients, it is hard to keep the balance between a stable network and an efficient service. With the fast pace of life, each client demands a service that won’t cause any financial impact and will improve their business. In addition, they require the Service Provider to provide them with the quickest possible fault resolution so the impact to their business is minimized. This is where the network automation and the implementation of various scripts come in place. These processes will help reduce the workload and pressure of your team and network and will provide your clients with the “remote control” for their service. And this is something they are going to be pleased about.

What are scripts and why are they useful?

The most common definition of scripts, the one you will find anywhere on the Internet, is that they are a list of specific commands put together to be executed by some kind of a scripting engine. Most often these are used with the goal to help us optimize our workload by providing process automation. These scripts are text written in a scripting language. One of those scripting languages, is PHP. PHP, among its many uses, is often used for web development because of the fairly easy way to use it with HTML. By hosting these scripts on your local server, they are easy to be incorporated in your NOC, since people will find it easy to use them by simply opening a certain website. Fairly simple right?

The use cases for these scripts in the networking world are endless. You need to be creative and really think of what they need to do and how they can simplify your day to day work responsibilities.

The use of scripts in the Network Operation Centre

The Network Operation Centre (NOC) is the main engine behind the network architecture of a company. Its responsibilities include keeping the network stable, safe and future proof. The hardest thing to achieve in the NOC is the team’s efficiency. With the nature of the IT world and the development of new technologies, the whole NOC team should be able to work as efficient as possible. That is why various automation solutions are commonly implemented within the NOC. In this article we will go through some of them.

You would agree that today, the most important part of the IT world is the access to the Internet. This would mean that when people are left with no Internet connection, they will most probably call the NOC. The NOC would then need to gather information about the issue, investigate why the issue occurred, find and test the possible solution and in the end implement the solution and thus resolving the issue. In today’s fast paced world this is a fairly long and time consuming process. Especially if these issues multiply by hundred in case of a major outage. That is why you can implement some kind of automation that will precisely do half of the steps mentioned above for you, in 90% less time. Different vendors have different solutions to offer this possibility. With Juniper that is called NETCONF.

Juniper’s NETCONF

The PHP NETCONF solution provided by Juniper is a tool which offers users CLI capabilities via an automation environment. Here is a brief overview from Junipers documentation:

“The NETCONF XML management protocol is an XML-based protocol that client applications use to request and change configuration information on routing, switching, and security devices. It uses an Extensible Markup Language (XML)-based data encoding for the configuration data and remote procedure calls. The NETCONF protocol defines basic operations that are equivalent to configuration mode commands in the command-line interface(CLI). Applications use the protocol operations to display, edit, and commit configuration statements (among other operations), just as administrators use CLI configuration mode commands to perform those operations.”

These capabilities include:

Remote connectivity and management of Junos devices.
Retrieve various information from the device such as software-version, serial-number, interface and protocol status etc.
Retrieve configuration information with the goal to simplify the troubleshooting and maintenance process
Make various simple and complex configuration changes in unstructured and structured ways

Let’s take for example you have dozen Juniper devices that require software upgrades. If you choose the manual way you will need to log on each device and configure it to pull the upgrade file from your tftp or any other server. Within the NOC, where the number of devices is much larger than dozen, this is time and resource consuming. Therefore it is recommended to use the NETCONF solution to do this automatically. The script is as simple as the below:

<!?php
include('netconf/Device.php');

$param= array("hostname"=> "Router1","username"=>"Juniper",
"password"=>"JuniperPass");
$d= new Device($param);


    $d->setReplyTimeout(35);
    $d->setConnectTimeout(10);

$d->connect();

$rpc = "<rpc>";
 $rpc.="<request-package-add>";
 $rpc.= "<package-name>";
 $rpc.= "....path/of/package.......";
 $rpc.= "</package-name>";
 $rpc.= "<no-copy/>";
 $rpc.="no-validate/>";
 $rpc.="</request-package-add>";
 $rpc.="</rpc>";
 $rpc.="]]>]]>\n";
echo "\nexecuting rpc \n\n";

$reply=$d->execute_rpc($rpc);
echo $reply->to_string();

echo"\n\nrebooting the system\n";
$d->reboot();

echo"\nclosing\n";
 $d->close();
?>

In short, this script would connect to the Router1 and use the username Juniper and password JuniperPass to authenticate. Then it will set connect and reply timeouts to 10 and 35 seconds. After these steps are completed the script will start the RPC protocol (which uses XML to encode the transmission and HTTP as a transport mechanism) to locate the upgrade package and request the installation of it. After this the script requests reboot of the device and logs off. This automation process saves you the headache of finding resources to perform software upgrade on each device in your network.

Another neat thing that scripts offer you is the possibility to retrieve various configuration information from your devices. Let’s take the NETCONF script below for an example:

#!/usr/bin/expect
set timeout 60
set Loop_Back_Address__c [lindex $argv 0]

spawn /usr/bin/ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no
admin@$Loop_Back_Address__c -s netconf
#spawn /usr/bin/ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no
admin@100.115.135.55 -s netconf
expect "password:" { send -- "JuniperPass\n" }

#expect "]]>]]>" {send --
"<rpc><get-interface-information></get-interface-information></rpc>\n"}
expect "]]>]]>" {send -- "<rpc><get-configuration></get-configuration></rpc>\n"}
#expect "]]>]]>" {send --
"<rpc><get-interface-information><extensive/></get-interface-information></rpc>\n"}
expect "]]>]]>" {send -- "<rpc><close-session/></rpc>\n\n"}
#expect "</configuration>" {send -- "<rpc><close-session/></rpc>\n\n"}
expect "<!-- session end"

This script has been developed so it can retrieve interface stats from a Juniper device. If you look closely you will notice the script is accessing the Juniper via SSH, it uses pre-configured credentials to log on the device and then runs a couple of commands to retrieve the interface configuration. This script can later be programmed to automatically inform the engineer what to look for in the fault finding process.

One of the main reasons behind the use of NETCONF are in fact the limitations of SNMP. You will notice that SNMP is most commonly known as a monitoring protocol. This is mainly because of its complexity and inability to locate the correct MIB modules the device is using. This happens because SNMP lacks an automatic discovery process which can discover the various parameters needed automatically. This is why engineers decided to use SNMP only for monitoring purposes and switch to configurations via the CLI or in this case NETCONF.

Fortinet API

With so many vendors offering their networking solutions on the market, it’s understandable that you will run into various automation solutions. This is the case with Fortinet. Fortinet uses a different programming language than Juniper, it’s called Python. One of the cool features that Fortinet provides you with is their python library. Even though it was much harder to gain access to this library in the past, now it’s as simple as knowing two people with @fortinet.com e-mail addresses. This library is used to make API calls and if you don’t know what API calls are, and why they are needed, please click here.

When it comes to the code itself, the sky is the limit. Fortinet gives you the power to do whatever you need to do via these scripts. From configuring various policies, to pulling system information, you have the convenience to play with almost all the parameters of the device. The short code below is the proof of how simple it is to pull certain data from a Fortinet device, in this case system status:

import python
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('1.1.1.254',username='admin',password=’admin’)
stdin, stdout, stderr=ssh.exec_command("get system status")
type(stdin)
stdout.readlines()

Yes, it is literally that simple. However, like almost all technologies out there, this one is not the exception when it comes to limitations. Sometimes you will find the Fortigate API to be a bit foolish. This is most noticeable with the scripting paths. Most often you will find some illogical dots in the path code which will cause you a lot of headaches when you find out that your code is not working and you have to locate the reason why. In addition you will notice that not all coding fields are being edited in the same way. For some unknown reason you are not able to configure different firewall rules as a group. Instead you will need to edit them one by one, which can be time consuming. On the contrary, you can edit a whole list of prefix lists with a single command.

There are dozens of different options you can use to simplify your workload and optimize your network. The most important thing to remember is to find the option which will better suit your needs. But please, don’t forget how to be a network engineer!

Next steps

Interested in having validation tasks like these automated for you? Check out the automation scripts Indeni can run on your behalf by downloading Indeni today.

Thank you to community member Bojan Janevski for contributing this article! If you found the information helpful please share on social media by clicking the share links at the top of this page.