Add start of jfrog CICD polling tool and notes.

NOTES.txt

@@ -24,3 +24,48 @@ layouts), a new directory in testbeds/ should be created, and populated with tes
 LANforge scenario and test configuration files.  See 'ben-home' for examples.
 See lanforge/lanforge-scripts/gui/README.txt for details on ways to grab the appropriate text,
 but you will also need familiarity with LANforge to accomplish this easily.
+
+
+# Test controller setup.  Assuming Fedora-30 or similar is installed.
+
+# Enable vnc, install some things, disable selinux so vnc will work.
+./lf_kinstall.pl --do_vnc --do_pkgs --do_selinux 0
+
+# As user lanforge (or other user should work too)
+# You will need to put the proper keys in ~/.ssh/ in order to
+# do the clone.
+mkdir ~/git
+cd ~/git
+git clone [email protected]:Telecominfraproject/wlan-testing.git
+git clone [email protected]:Telecominfraproject/wlan-lanforge-scripts.git
+cd wlan-testing/lanforge
+ln -s ../../wlan-lanforge-scripts lanforge-scripts
+
+# Make a copy of a test bed similar to what you have, for instance:
+cd ~/git/wlan-testing/testbeds
+cp -ar example_test_bed my_new_test_bed
+
+# Edit the new test bed details as needed, this may be somewhat tricky
+# if your test bed is different from others.
+# See ~/git/wlan-lanforge-scripts/gui/README.txt for some hints on setting up
+# scenarios and such.
+
+
+
+# On LANforge machine, make sure GUI is in auto-start mode:
+https://www.candelatech.com/cookbook/misc/autostart
+
+* Create uplink-nat profile.
+* Create upstream DUT
+* Create AP DUT
+* Create scenario that uses uplink-nat profile and upstream-dhcp to provide VR+NAT access to the world.
+* Add stations for the radios, properly linked to DUT radio(s).
+
+* Create WCT test, verify it works OK, save scenario
+*  Interleave sort, set pass/fail limits
+
+* Create AP-Auto test
+ * configure DUT
+ * configure radios
+ * configure advanced (dual-band only in our case)
+ * configure stability settings

cicd/README.txt

@@ -0,0 +1,50 @@
+Potential polling method for CICD integration.
+
+* Polling should decrease network security head-aches such as setting
+  up VPN access for all test beds.
+
+***
+
+Implementation:
+
+* Web server accessible to all CICD test beds runs a 'test orchestrator' logic, henceforth TO
+  *  This TO will periodically query jfrog for latest openwrt builds (see jfrog.pl)
+  *  If new build is found, a text file containing pertinent info, including the HW platform
+     will be created, example:
+
+CICD_URL=https://tip.jfrog.io/artifactory/tip-wlan-ap-firmware/
+CICD_FILE_NAME=ea8300-2020-04-24-046ab4f.tar.gz
+CICD_URL_DATE=24-Apr-2020 18:28 
+CICD_HW=ea8300
+CICD_FILEDATE=2020-04-24
+CICD_GITHASH=046ab4f
+
+  *  TO has manually configured list of test-beds, with some info about each test
+     bed, including the DUT HW platform and testing capabilities.
+  *  It picks a test bed that matches the new build HW.
+  *  That test bed will have a URL directory for it and it alone.
+  *  The TO writes a new test configuration file into this directory.
+     The test configuration file will have the info above, and also have other
+     info including the tests to run and where to upload results when complete.
+
+
+* Test bed polling:
+  *  The test-bed (hence forth TB) will poll its directory on the TO web server to look for new jobs.
+  *  When new job is found, the TB will download the test config file, and use scp to upload a file
+     to the TO to indicate it is working on the test.
+  *  When test is complete, TB will upload results to TO server.  TO now knows test bed is available
+     for more jobs.
+
+* TO Polls periodically for results from test-beds, and when found it will re-generate historical
+  graphs and reports.  If feasible, it could also email or otherwise notifiy whoever is interested in
+  these results.  It could poke the results into testrails or similar at this point.
+
+
+* If we can implement something like CTF, then it could cause the test config files to be placed into
+  the test-bed directory, potentially with URLs pointing to user-specified locations for testing private
+  builds.
+
+
+
+
+

cicd/jfrog.pl

@@ -0,0 +1,99 @@
+#!/usr/bin/perl
+
+# Query jfrog URL and get list of builds.
+
+use strict;
+use warnings;
+use Getopt::Long;
+
+my $user = "cicd_user";
+my $passwd = "";
+my $url = "https://tip.jfrog.io/artifactory/tip-wlan-ap-firmware/";
+my $files_processed = "jfrog_files_processed.txt";
+my $next_info = "jfrog_files_next.txt";
+my $help = 0;
+
+my $usage = qq($0
+  [--user { jfrog user (default: cicd_user) }
+  [--passwd { jfrog password }
+  [--url { jfrog URL, default is OpenWrt URL: https://tip.jfrog.io/artifactory/tip-wlan-ap-firmware/ }
+  [--files_processed { text file containing file names we have already processed }
+  [--next_info { output text file containing info about the next file to process }
+
+Example:
+$0 --user cicd_user --passwd secret --url https://tip.jfrog.io/artifactory/tip-wlan-ap-firmware/ \
+   --files_processed jfrog_files_processed.txt --next_info jfrog_files_next.txt
+
+);
+
+GetOptions
+(
+  'user=s'                 => \$user,
+  'passwd=s'               => \$passwd,
+  'url=s'                  => \$url,
+  'files_processed=s'      => \$files_processed,
+  'next_info=s'            => \$next_info,
+  'help|?'                 => \$help,
+) || (print($usage) && exit(1));
+
+if ($help) {
+  print($usage) && exit(0);
+}
+
+if ($passwd eq "") {
+   print("ERROR:  You must specify jfrog password.\n");
+   exit(1);
+}
+
+my $i;
+
+#Read in already_processed builds
+my @processed = ();
+my $listing = `cat $files_processed`;
+my @lines = split(/\n/, $listing);
+for ($i = 0; $i<@lines; $i++) {
+   my $ln = $lines[$i];
+   chomp($ln);
+   print("Already processed: $ln");
+   push(@processed, $ln);
+}
+
+my $cmd = "curl -u $user:$passwd $url";
+print ("Calling command: $cmd\n");
+$listing = `$cmd`;
+@lines = split(/\n/, $listing);
+for ($i = 0; $i<@lines; $i++) {
+   my $ln = $lines[$i];
+   chomp($ln);
+
+   if ($ln =~ /href=\"(.*)\">(.*)<\/a>\s+(.*)\s+\S+\s+\S+/) {
+      my $fname = $1;
+      my $name = $2;
+      my $date = $3;
+
+      if ( grep( /^$fname\s+/, @processed ) ) {
+         # Skip this one, already processed.
+         next;
+      }
+
+      open(FILE, ">", $next_info);
+      print FILE "CICD_URL=$url\nCICD_FILE_NAME=$fname\nCICD_URL_DATE=$date\n";
+
+      if ($fname =~ /^(\S+)-(\d\d\d\d-\d\d-\d\d)-(\S+).tar.gz/) {
+         my $hw = $1;
+         my $fdate = $2;
+         my $githash = $3;
+         print FILE "CICD_HW=$1\nCICD_FILEDATE=$fdate\nCICD_GITHASH=$githash\n";
+      }
+
+      close(FILE);
+
+      print("Next: File Name: $fname  Display Name: $name  Date: $date\n");
+      print("To download: curl --location -o /tmp/$fname -u $user:$passwd $url/$fname\n");
+      exit(0);
+   }
+
+   #print "$ln\n";
+}
+
+exit 0;