Customized Scan Scope for Large Size Jira/Confluence/Asana Sites

[blupants]

This PR implements a solution for the following issues:

• Issue Pause and resume scan from a space #13
• Issue Handle error: JiraError HTTP 400 #20
• Issue Confluence scanning: for large sites not all spaces/pages are scanned without error #26
• Issue Scan specifica Confluence Pages & JIRA Tickets Only. #29

Summary

The solution is based on mapping the target site using the new flag --map. It will generate a JSON file with a list of all tickets/pages/issues from the site organized hierarchically. Then you can customize the mapping file to run scans with a narrower coverage.

Usage

In order to run a scan with a customized coverage, 2 steps are required:

1. Generate the map file
2. Run the scoped scan based on a map file

Generating the map file

You can generate a full map file or a map with only the highest namespace (i.e projects for Jira, workspaces for Confluence and Asana)

Full Mapping

n0s1 jira_scan --map --map-file jira_map.json

Will generate the following "jira_map.json" file:

{
  "projects": {
    "AS": {
      "AS-1": {}
    },
    "DLP": {
      "DLP-7": {},
      "DLP-6": {},
      "DLP-5": {},
      "DLP-4": {},
      "DLP-3": {},
      "DLP-2": {},
      "DLP-1": {}
    },
    "GTMS": {
      "GTMS-4": {},
      "GTMS-3": {},
      "GTMS-2": {},
      "GTMS-1": {}
    },
    "IT": {
      "IT-3": {},
      "IT-2": {},
      "IT-1": {}
    },
    "ITSAMPLE": {
      "ITSAMPLE-12": {},
      "ITSAMPLE-11": {},
      "ITSAMPLE-10": {},
      "ITSAMPLE-9": {},
      "ITSAMPLE-8": {},
      "ITSAMPLE-7": {},
      "ITSAMPLE-6": {},
      "ITSAMPLE-5": {},
      "ITSAMPLE-4": {},
      "ITSAMPLE-3": {},
      "ITSAMPLE-2": {},
      "ITSAMPLE-1": {}
    },
    "MAR": {
      "MAR-2": {},
      "MAR-1": {}
    }
  }
}

Quick Mapping

Alternatively you can create a mapping with only the highest namespace by using the --map 1 flag:
n0s1 jira_scan --map 1

It will generate the following "n0s1_map.json" file:

{
  "projects": {
    "AS": {},
    "DLP": {},
    "GTMS": {},
    "IT": {},
    "ITSAMPLE": {},
    "MAR": {}
  }
}

Scoped Scans

You can customize the scan coverage by manually editing the map file, or splitting the mapping in equal parts by using the --scope c/n flag (where "c" is the chunk number you want to scan, and "n" the total number of chunks you are splitting the map).

Manual Scope Customization

You can edit the map file and remove tickets or entire projects to perform a faster scan. You can customize the scan coverage by editing the map file (e.g. saving it to "smaller_jira_map.json") and running a new scan with the --map-file flag:
n0s1 jira_scan --map-file smaller_jira_map.json

Using the following "smaller_jira_map.json" as an example:

{
  "projects": {
    "DLP": {
      "DLP-7": {},
      "DLP-5": {}
    },
    "GTMS": {
    },
    "IT": {
      "IT-3": {},
      "IT-2": {},
      "IT-1": {}
    },
    "ITSAMPLE": {
      "ITSAMPLE-12": {},
      "ITSAMPLE-11": {},
      "ITSAMPLE-10": {},
      "ITSAMPLE-9": {},
      "ITSAMPLE-8": {}
    }
  }
}

The tickets to be scanned are:

• "DLP-7"; "DLP-5";
• all "GTMS" tickets;
• "IT-1" to "IT-3"
• "ITSAMPLE-8" to "ITSAMPLE-12"

Everything else will be ignored..

Auto Scope Customization

Another alternative is to split the scan in several chunks with the same size by using the new flag --scope. For instance, if you provide the original mapping file "jira_map.json" and set the flag --scope 2/5, it will split the mapping in 5 equal parts, and scan the 2nd chunk.

It allows you to scan a massive database by diving the scan in "n" steps:
n0s1 jira_scan --map --map-file jira_map.json
n0s1 jira_scan --scope 1/4 --map-file jira_map.json --report-file report_1_4.json
n0s1 jira_scan --scope 2/4 --map-file jira_map.json --report-file report_2_4.json
n0s1 jira_scan --scope 3/4 --map-file jira_map.json --report-file report_3_4.json
n0s1 jira_scan --scope 4/4 --map-file jira_map.json --report-file report_4_4.json