Initial setup within walled garden, catch 22 scenario

dcabanaw · July 22, 2022, 7:33pm

Hello! I’m new to genieacs and I’m trying to setup a workflow where the CPE devices initially connects with PPPoE into a walled garden (our LNS handles this with a realm identifier), gets it’s configuration from the within the walled garden, then can communicate publicly after that.

I have the ACS server setup with multiple interfaces:

Management, UI, and NBI (private management network)
CWMP and FS (walled garden)
CWMP and FS (public internet)

The server is running genieacs 1.2.8 (built manually)

Through a combination of routing and a nginx reverse proxy, I have the services listening on the various interfaces and ports.

So far with some presets and provisions I can get some basic things configured, both within the walled garden and public interfaces.

The “catch 22” issue I’m running into is the order of operations…

Within my provision script I basically want to do the following:

setup lan config
setup wifi
setup wan config (pppoe)
change management url

But in doing so, I run into a preset_loop fault condition. Based on a tcpdump capture it looks like most of the time the management url (InternetGatewayDevice.ManagementServer.URL) changes before the pppoe config applies (or vice versa) which results with the CPE no longer able to communicate with the ACS or the ACS unable to make a connection request to the CPE.

I have tried changing the order in which things are declared and have tried putting commit() in various places (maybe I have a misunderstanding of this?)

I have also tried adjusting the some the ENV variable (like the batch size).

Is there something I can do to work around this or is this just a limitation of CWMP in general? Any insight or suggestions would be helpful.

Thanks.

References

Test provision scirpt

// TEST provisioning script

const now = Date.now();
log("\nDEBUG LOG: Provision started at " + now + "\n\n");

function setup_wan(now, args) {
    log("setup_wan");
    log(JSON.stringify(args));

    //Ensure we have a WANPPPConnection instance
    log("Creating WANPPPConnection (if necessary)");
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*", null, { path: 1 });

    //Refresh the node...
    log("Setting up WANPPPConnection");
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.*", { path: now });

    //Setup static parameters
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.Name", { value: now }, { value: "pppoe_eth0" });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.ConnectionType", { value: now }, { value: "IP_Routed" });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.X_BROADCOM_COM_IfName", { value: now }, { value: "ppp0.1" });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.NATEnabled", { value: now }, { value: true });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.X_BROADCOM_COM_FirewallEnabled", { value: now }, { value: true });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.Enable", { value: now }, { value: true });

    //Setup customer parameters
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.Username", { value: now }, { value: args.ppp_username });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.Password", { value: now }, { value: args.ppp_password });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.Mtu", { value: now }, { value: args.ppp_wan_mtu });
    declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.*.PPPoEServiceName", { value: now }, { value: "internet" });

    // ip masked out....  it's a public ip
    declare("InternetGatewayDevice.ManagementServer.URL", { value: now }, { value: "http://***.***.***.***:8085" });
}

function setup_lan(now, args) {
    log("setup_lan");
    log(JSON.stringify(args));

    // refresh node
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.*", { path: now });

    //
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.DHCPServerEnable", { value: now }, { value: args.dhcp_en });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.DHCPServerConfigurable", { value: now }, { value: true });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.DHCPLeaseTime", { value: now }, { value: 86400 });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.DHCPRelay", { value: now }, { value: false });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.DNSServers", { value: now }, { value: "0.0.0.0,0.0.0.0" });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.MinAddress", { value: now }, { value: args.dhcp_range.from });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.MaxAddress", { value: now }, { value: args.dhcp_range.to });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.SubnetMask", { value: now }, { value: args.netmask });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.TftpServerEnable", { value: now }, { value: false });
    //declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.X_COMTREND_COM_ReservedIPSize", { value: now }, { value: args.dhcp_range_size });

    //Ensure we have an IPInterface instance
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.*", null, { path: 1 });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.1.Enable", { value: now }, { value: true });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.1.IPInterfaceAddressingType", { value: now }, { value: "Static" });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.1.IPInterfaceIPAddress", { value: now }, { value: args.gateway });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.1.IPInterfaceSubnetMask", { value: now }, { value: args.netmask });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.1.X_BROADCOM_COM_IfName", { value: now }, { value: "br0" });
    declare("InternetGatewayDevice.LANDevice.1.LANHostConfigManagement.IPInterface.1.X_BROADCOM_COM_DhcpDefaultGateway", { value: now }, { value: "0.0.0.0" });
}

function setup_wifi(now, args) {
    log("TODO: Setup Wi-Fi");
    log("setup_lan");
    log(JSON.stringify(args));
}

function setup_firewall(now, args) {
    log("TODO: Firewall");
    log("setup_firewall");
    log(JSON.stringify(args));
}

let provisioned = declare("Tags.Provisioned", { value: 1 });
if (provisioned.value !== undefined) {
    log("CPE is (allegedly) provisioned, returning");
    return;
}

// get details of CPE to uniquiely identify it...
let model = declare("InternetGatewayDevice.DeviceInfo.ModelName", { value: 1 }).value[0];
let serialNumber = declare("DeviceID.SerialNumber", { value: 1 }).value[0];
let productClass = declare("DeviceID.ProductClass", { value: 1 }).value[0];
let oui = declare("DeviceID.OUI", { value: 1 }).value[0];
let args = { serial: serialNumber, productClass: productClass, oui: oui };

//Allow LAN+WAN Ping
declare("InternetGatewayDevice.X_BROADCOM_COM_AppCfg.IcmpCfg.NetworkAccess", null, { value: "LAN or WAN" });

setup_lan(now, ext("cpe-config", "get_lan_config", args));
// setup_wifi(now, ext("cpe-config", "get_wifi_config", args));
// setup_firewall(now, ext("cpe-config", "get_firewall_config", args));
commit();
setup_wan(now, ext("cpe-config", "get_pppoe_config", args));
commit();


/**
 * Done confing, set Provisioned tag
 */
log('Done configuring. Setting provisioned tag');
declare("Tags.Provisioned", null, { value: true });

env file

NODE_OPTIONS=--enable-source-maps

GENIEACS_CWMP_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-cwmp-access.log
GENIEACS_NBI_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-nbi-access.log
GENIEACS_FS_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-fs-access.log
GENIEACS_UI_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-ui-access.log

GENIEACS_CWMP_LOG_FILE=/var/log/genieacs/genieacs-cwmp.log
GENIEACS_NBI_LOG_FILE=/var/log/genieacs/genieacs-nbi.log
GENIEACS_FS_LOG_FILE=/var/log/genieacs/genieacs-fs.log
GENIEACS_UI_LOG_FILE=/var/log/genieacs/genieacs-ui.log

GENIEACS_DEBUG=TRUE
GENIEACS_DEBUG_FORMAT=yaml
GENIEACS_DEBUG_FILE=/var/log/genieacs/genieacs-debug.yaml

GENIEACS_EXT_DIR=/opt/genieacs/ext
GENIEACS_UI_JWT_SECRET=<its a secret>

GENIEACS_UI_PORT=8080
GENIEACS_UI_INTERFACE=127.0.0.1

GENIEACS_NBI_PORT=7557
GENIEACS_NBI_INTERFACE=127.0.0.1

GENIEACS_CWMP_PORT=7547
GENIEACS_CWMP_INTERFACE=127.0.0.1

GENIEACS_FS_PORT=7567
GENIEACS_FS_INTERFACE=127.0.0.1

GENIEACS_SESSION_TIMEOUT=600
GENIEACS_GPN_NEXT_LEVEL=4
GENIEACS_GPV_BATCH_SIZE=128
GENIEACS_MAX_DEPTH=8
GENIEACS_MAX_COMMIT_ITERATIONS=128
# GENIEACS_CONNECTION_REQUEST_TIMEOUT=3000
# GENIEACS_DEVICE_ONLINE_THRESHOLD=6000  probably don't set this?

nginx config

# /etc/nginx/sites-available/genieacs.conf
# Nginx configuration for genieacs services
# Ensure you change the listening IP address
# If you're allergic to security, you can remove the `add_header`
#   lines without breaking functionality

upstream genieacs-gui {
    server 127.0.0.1:8080;
}

upstream genieacs-cwmp {
    server 127.0.0.1:7547;
}

upstream genieacs-nbi {
    server 127.0.0.1:7557;
}

upstream genieacs-fs {
    server 127.0.0.1:7567;
}

server {
    listen 80 default_server;
    server_name genie-acs.domain.tld;

    add_header Content-Security-Policy "default-src 'self'" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "sameorigin" always;
    add_header X-XSS-Protection "1; mode block" always;

    location /.well-known {
        alias /var/www/html/.well-known;
    }

    location / {
        return 301 https://$server_name$request_uri;
    }
}

server {
    listen 443 ssl http2;
    server_name genie-acs.domain.tld;

    ssl_certificate_key /path/to/server.key;
    ssl_certificate /path/to/certificate.pem;

    access_log /var/log/nginx/genieacs-web-access.log combined;
    error_log /var/log/nginx/genieacs-web-error.log;

    client_max_body_size 50M;

    root /opt/genieacs/public;

    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header Content-Security-Policy "default-src 'self' 'unsafe-inline'" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "sameorigin" always;
    add_header X-XSS-Protection "1; mode block" always;

    location /.well-known {
        alias /var/www/html/.well-known;
    }

    location / {
        try_files $uri $uri/index.html @app;
    }

    location @app {
        proxy_pass http://genieacs-gui;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
        proxy_redirect off;
    }
}

server {
    listen 10.42.0.2:8085;

    access_log /var/log/nginx/genieacs-cwmp-access.log combined;
    error_log /var/log/nginx/genieacs-cwmp-error.log;

    client_max_body_size 50M;

    add_header Content-Security-Policy "default-src 'self'" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "sameorigin" always;
    add_header X-XSS-Protection "1; mode block" always;

    location / {
        proxy_pass http://genieacs-cwmp;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Authorization "";
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/ms-htpasswd;
    }
}

server {
    listen <public_ip>:8085;

    access_log /var/log/nginx/genieacs-cwmp-access.log combined;
    error_log /var/log/nginx/genieacs-cwmp-error.log;

    client_max_body_size 50M;

    add_header Content-Security-Policy "default-src 'self'" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "sameorigin" always;
    add_header X-XSS-Protection "1; mode block" always;

    location / {
        proxy_pass http://genieacs-cwmp;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Authorization "";
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/ms-htpasswd;
    }
}

server {
    listen 192.168.7.189:7557 ssl;
    server_name genie-acs.domain.tld;

    ssl_certificate_key /path/to/server.key;
    ssl_certificate /path/to/certificate.pem;

    access_log /var/log/nginx/genieacs-nbi-access.log combined;
    error_log /var/log/nginx/genieacs-nbi-error.log;

    client_max_body_size 50M;

    add_header Strict-Transport-Security "max-age=31536000" always;
    add_header Content-Security-Policy "default-src 'self'" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "sameorigin" always;
    add_header X-XSS-Protection "1; mode block" always;

    location / {
        proxy_pass http://genieacs-nbi;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Authorization "";
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/ms-htpasswd;
    }
}

server {
    listen 10.42.0.2:7567;

    access_log /var/log/nginx/genieacs-fs-access.log combined;
    error_log /var/log/nginx/genieacs-fs-error.log;

    client_max_body_size 50M;

    add_header Content-Security-Policy "default-src 'self'" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "sameorigin" always;
    add_header X-XSS-Protection "1; mode block" always;

    location / {
        proxy_pass http://genieacs-fs;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Authorization "";
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/ms-htpasswd;
    }
}

interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug ens192
iface ens192 inet static
        address 192.168.7.189/24
        gateway 192.168.7.1
        dns-nameservers 192.168.7.55 192.168.7.100

allow-hotplug ens224
iface ens224 inet static
        address 10.42.0.2/24
        post-up ip route add 10.99.0.0/24 via 10.42.0.1
        pre-down route del 10.99.0.0/24

allow-hotplug ens256
iface ens256 inet static
        address <public_ip>/26
        post-up ip route add <cpe_pool_1>/20 via <default_gateway>
        post-up ip route add <cpe_pool_2>/24 via <default_gateway>
        post-up ip route add <cpe_pool_3>/18 via <default_gateway>
        post-up ip route add <cpe_pool_4>/22 via <default_gateway>        
        pre-down route del <cpe_pool_1>/20
        pre-down route del <cpe_pool_2>/24
        pre-down route del <cpe_pool_3>/18
        pre-down route del <cpe_pool_4>/22

akcoder · July 22, 2022, 8:33pm

Changing this will cause the CPE to send a 0 BOOTSTRAP event. This is not what you want.

From Event Basics | qa | cafe

0 BOOTSTRAP

The most fundamental is 0 BOOTSTRAP. This event tells the ACS that the CPE is contacting it either for the first time or because the URL of the ACS has changed. When an ACS receives a BOOTSTRAP event, it will often begin a series of initial configuration steps on the CPE.

dcabanaw · July 25, 2022, 5:38pm

Thanks. I’ll start looking at other ways handle this.

dcabanaw · July 29, 2022, 2:35pm

I have found a work around for this. Instead of using an IP for the management URL, we’re using a proper domain name and when the CPE connects with pppoe, we use DNS to have it resolve to an IP in walled garden network or the public network. To make sure there’s no DNS caching issues, we reboot the CPE after the new configuration is applied.

akcoder · August 8, 2022, 8:09pm

This is a much better solution . FWIW, I’ve found rebooting the CPE after provisioning it eliminates a lot of issues.

Topic		Replies	Views
Genieacs-cwmp: CPE not picking up Tasks	2	3425	April 29, 2019
Push WAN profile from ACS to CPE	16	4522	October 16, 2023
About configuring acs url	2	2589	March 21, 2022
Pushing configration to CPE	1	920	October 31, 2019
FactoryReset Loop	8	2495	April 12, 2021