Caching strategy

#1

Hello,

I’ve lately been developing some extensions to connect to our billing system using the REST API and fetch data. I started with the iss.js extension code and worked from there.

I have everything working, but the only thing I am worried about is the load on our billing server. Sometimes I am doing a REST query that contains multiple fields that I want to store as VirtualParameters, and so I pull the one value I need for the VirtualParameter and then have to rerun the same query almost immediately for the other VirtualParameter.

I saw the caching code in the iss.js but it looked to me like it was a global cache since it is for the International Space Station, so it doesn’t really matter what device it is that pulls the data, so caching makes sense. I am wondering if there are any recommendations for strategies that I could use to do similar caching that would work for more specific REST queries, so that if a duplicate REST query is performed shortly thereafter, that it uses the cached response instead.

I’m not necessarily looking for a complete walkthrough here - just advice on a general approach should be enough to get me started. Or, even better, example code from people who are doing similar things.

Thanks!

#2

GenieACS is designed to handle 100k+ CPEs. A few queries from your billing system won’t cause performance issues :slight_smile:.

#3

It is not queries from our billing system, it is to it. GenieACS is using our billing system’s REST API to pull the information for each unit. It has to fill multiple fields so it has to rerun the same query multiple times for different virtual parameters. Our billing system is cloud hosted and I’m just worried about hitting some kind of rate-limit on the API of the billing system.

#4

Okay, now I follow. So what about using a provision script that fetches the values once and then sets the values in the appropriate vparams? This is how I accomplish this.

1 Like
#5

Yes, that might work - can you show an example? What does the Virtual Parameter script look like when you do it that way?

#6

Here is a simple v param:

let value;

if ("value" in args[1]) {
    // Set declared value
    value = args[1].value;
} else if ("value" in args[3]) {
    // No declared value, keep current value
    value = args[3].value;
} else {
    // No current value, use default
    value = ["", "xsd:string"];
}

return {writable: true, value: value};
#7

OK, thanks - and those arguments are being passed directly by the provision script somehow?

#8

Yes, just like you set any other parameter.

declare('VirtualParameters.ParamName', {value: now}, {value: 'someValue'});
#9

Thanks for the info! How do you put multiple arguments? I see your VirtualParameter script references args[1] and args[3], but your example just has ‘someValue’? Presumably that will be args[0] but how would you get the args[1] and args[3]?

#10

Those are automatically created by GenieACS when calling a vparam.

#11

OK, so then I would do something like this:

Provision:

let sonarInstance = declare(
  "VirtualParameters.Sonar-Instance",
  {value: Date.now()}).value[0];

let uplinkMac = declare(
  "VirtualParameters.Uplink-MAC",
  {value: Date.now()}).value[0];

let sonarAccount = "";

if (sonarInstance != "") {
  sonarAccount = ext("sonar-v1-search-mac-serial.js", "searchSonar", sonarInstance, uplinkMac);
  declare('VirtualParameters.Sonar-Account-Num', {value: now}, {value: [sonarAccount.accountNum, "xsd:string"]});
  declare('VirtualParameters.Sonar-Account-Name', {value: now}, {value: [sonarAccount.accountName, "xsd:string"]});
}

return {writable: false, value: true};

Then my code for VirtualParameters.Sonar-Account-Num would be something like:

let value;

if ("value" in args[1]) {
    // Set declared value
    value = args[1].value;
} else if ("value" in args[3]) {
    // No declared value, keep current value
    value = args[3].value;
} else {
    // No current value, use default
    value = ["", "xsd:string"];
}

return {writable: true, value: value};

And my code for VirtualParameters.Sonar-Account-Name would be the same?

let value;

if ("value" in args[1]) {
   // Set declared value
   value = args[1].value;
} else if ("value" in args[3]) {
   // No declared value, keep current value
   value = args[3].value;
} else {
   // No current value, use default
   value = ["", "xsd:string"];
}

return {writable: true, value: value};

Am I understanding this correctly?

#12

Sorry I realized I misunderstood something I think - edited the post above.

OK so I tried this and it does work. The only negative thing is that it only seems to work if the VirtualParameter is set to writable: true. If I change that to false, it no longer works in that way. Especially in the new GenieACS 1.2.x with the pen icon that appears next to every writable parameter I am concerned that that might confuse people.

#13

This is by design. Some VParams are read only. This vparam I use for getting the Manufacturer. The CPE will not allow you to change its Manufacturer name. So this vparam is set to writable: false.

let keys = [
    'InternetGatewayDevice.DeviceInfo.Manufacturer',
    'Device.DeviceInfo.Manufacturer'
];

let result = {writable: false, value: [getParameterValue(keys, 'UNKNOWN'), 'xsd:string']};

return result;

function getParameterValue(keys, def) {
    for (let key of keys) {
        let d = declare(key, {value: Date.now()});

        for (let item of d) {
            if (item.value && item.value[0]) {
                return item.value[0];
            }
        }
    }

    return def;
}
#14

Right, but especially in the new GenieACS versions it will show a pen icon next to the info which can mislead the technician into thinking that they can change that parameter.

Instead of using a provision to set the virtual parameter, it might be better to store the entire record as json in a single virtual parameter, and then have the other virtual parameters parse the json to get their info.

#15

And thats a valid strategy. I do that to track which diagnostic test has been initiated so I know which parameters to refresh and send to our subscriber management server.

I don’t know what you are using for subscriber management (ours is home grown), but what I did was add hooks to the ACS from our subscriber management server (SMS) to the ACS. This way our techs never have to go into the ACS and then they can’t screw stuff up :slight_smile: .

-dan

#16

Thanks.

Related question - are all parameters xsd:string? Is integer ever used?

#17

The param type is what ever you want it to be, as long as it an xml schema attribute type.
Supported ones are:

$ grep -oh "xsd\:\w*" /opt/genieacs/lib/* | sort | uniq
xsd:base64
xsd:boolean
xsd:dateTime
xsd:hexBinary
xsd:int
xsd:string
xsd:unsignedInt