Share: bash script to force a "connection request" on all registered CPEs

1

scenario: We have about 5k of CPE and our provisioning disables periodic inform. (the way I see it, if we need to configure a CPE right now, then we force a connection request using the appropriate API).

objective: I want, at least once a week, having all available CPE to contact the ACS just to say “hello! I’m here!”.-

skills and tools required:

  • Linux
  • bash
  • curl
  • python with json.tool module installed
  • grep / egrep
  • cron

(default install of Linux Slackware comes with all of them)

#!/bin/bash

lista=$(curl -s 'http://ACS.com:7557/devices/?projection=_id'  -X GET )

nada="[

]"

[[ "$lista" == "$nada" ]] && exit 0


logger INICIO CONNREQ

lista2=$(echo $lista| python -m json.tool |egrep "_id" |egrep -o '(":\s)(.*)' |egrep -o '\s(.*)' | egrep -o '[^" ]*')

while IFS= read -r devid
do
    curl -s 'http://ACS.com:7557/devices/'$devid'/tasks?connection_request' -X POST 
    sleep 2
done < <(echo "$lista2")

logger FIN CONNREQ

the sleep 2 is important here. Not using it or using a low value can create a bottleneck and eventually leave a large trail of faults on the ACS (also, you might want to use it to test the ACS load but in that case you can also use genieacs-sim, provided you have a large amount of RAM). I think value of 2 is a safe zone. In my case, with 5k of customers, it takes about 3 hours to complete. If we ever reach 500k of customers we will have several ACSs runnings.

once it runs as it should, a crontab could look something like this:

0 5 * * 1 /root/bin/acs/connreq.sh >/dev/null 2>&1

edit: sry about the Spanish mix

2 Likes
2

Interesting that you encounter issues without the sleep statement. What hardware are you running your ACS on?

I have a similar script I use to refresh a param (in the case where I add a new vparam and want to update it immediately) for the CPEs and have no sleep statement. I can do 3500 CPEs in 15 minutes.

$ time php8.1 refreshParameter.php
Refreshing 3568 devices

Refreshed 3568 devices %100.00

real    15m24.511s
user.   0m25.307s
sys.    0m1.620s

I monitored the load average on our ACS, and it never jumped more than half a point. For reference, our ACS VM is 8 CPU’s @ 2,300 MHz with 16gb of RAM, and 8 CWMP worker processes and also runs our mongo server.

Here is my script. I change the $filter and $refreshItems depending on what needs to be done. In the case of my testing above, I ran this script as is.

<?php

$nbi = 'https://ACS.com:7557';

$now = new DateTime();
$date = $now->modify('1 days ago');

$filter = [
    '_lastInform' => ['$gt' => $date->format(DateTimeInterface::ATOM)],
];

$devices = queryNbi($nbi, $filter);
$count = count($devices);

printf("Refreshing %d devices\r\n", $count);

$index = 0;

$refreshItems = [
    'VirtualParameters.PPPoEUsername',
    'VirtualParameters.Location',
];

foreach ($devices as $id) {
    $uri = sprintf('%s/devices/%s', $nbi, rawurlencode($id));

    if (!refresh($uri, $refreshItems, false)) {
        print "Error refreshing parameter on device {$id}\r\n";
    }

    printf("\t%d of %d - %%%0.2f\r", ++$index, $count, ($index / $count) * 100);
}

printf("Refreshed %d devices\r\n\r\n", $count);

function refresh(string $uri, array $parameters, bool $connectionRequest = true): bool {
    $url = $uri . '/tasks' . ($connectionRequest ? '?timeout=1&connection_request' : '');

    $data = [
        'name'           => 'getParameterValues',
        'parameterNames' => $parameters,
    ];
    /** @var string $dataString */
    $dataString = json_encode($data, JSON_THROW_ON_ERROR);

    $ch = createCurl($url);

    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
    curl_setopt($ch, CURLOPT_POSTFIELDS, $dataString);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
            'Content-Type: application/json',
            'Content-Length: ' . mb_strlen($dataString),
    ]);

    curl_exec($ch);

    $statusCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

    return $statusCode >= 200 && $statusCode <= 299;
}

function queryNbi(string $nbi, array $filter): array {
    $projection = '_id';

    $uri = sprintf('%s/devices/?query=%s&projection=%s', $nbi, rawurlencode(json_encode($filter, JSON_THROW_ON_ERROR)), $projection);

    $curl = createCurl($uri);

    $data = json_decode(curl_exec($curl), true, 512, JSON_THROW_ON_ERROR);

    curl_close($curl);

    return array_map(static fn($row) => $row['_id'], $data);
}

function createCurl(string $url): CurlHandle {
    $curl = curl_init();

    curl_setopt($curl, CURLOPT_TIMEOUT_MS, 20000);
    curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, true);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);

    curl_setopt($curl, CURLOPT_URL, $url);

    return $curl;
}
1 Like
3

you make a valid point. I probably wrote the script when we were accessing our database via API extension call. I do remember having faults from tests I made using genieacs-sim and that’s the reason I ended up using sleep. But I will make a test later without it and see what happens on ACS side.

our ACS runs under a virtual machine.

root@acs3:~# uname -a
Linux acs3 5.10.0-14-amd64 #1 SMP Debian 5.10.113-1 (2022-04-29) x86_64 GNU/Linux

root@acs3:~# lscpu |grep Model
Model:                           63
Model name:                      Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz

root@acs3:~# free -h
               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       3.1Gi       3.2Gi       3.0Mi       1.5Gi       5.0Gi
Swap:          974Mi          0B       974Mi
4

tested it without the sleep command and it was as you said. took about 10-15 mins, processor load peaked about 40% and usually was under 10%. If cpu load is not an issue then I might leave it as is, but I will check with the guy in charge of the datacenter first. If it is, I might replace the sleep with usleep command just to give the processor some breathing time.

2 Likes