From 24b78e0399e5792de4061f485ce43e06c9f65135 Mon Sep 17 00:00:00 2001 From: Parth Date: Sat, 11 Nov 2023 21:31:02 -0500 Subject: [PATCH 1/7] wsl 2 docs --- Topics/Development_Process.md | 2 +- Topics/Development_Process/WSL.md | 113 ++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+), 1 deletion(-) create mode 100644 Topics/Development_Process/WSL.md diff --git a/Topics/Development_Process.md b/Topics/Development_Process.md index 7aaf37faf..d146529b4 100644 --- a/Topics/Development_Process.md +++ b/Topics/Development_Process.md @@ -8,7 +8,7 @@ - [Automated Testing](./Development_Process/Automated_Testing.md) ### [Getting Started With Docker](./Development_Process/Docker.md) - +### [Getting Started With WSL 2](./Development_Process/WSL.md) ## Build requirements ### [Requirements.txt](./Development_Process/Build_Requirements/Requirements_txt.md) diff --git a/Topics/Development_Process/WSL.md b/Topics/Development_Process/WSL.md new file mode 100644 index 000000000..52755ac78 --- /dev/null +++ b/Topics/Development_Process/WSL.md @@ -0,0 +1,113 @@ +# Learning WSL 2 Usage + +## Table of Contents +### [Introduction](#introduction-1) +### [Installation](#installation-1) +### [Useful Commands](#useful-commands-1) +### [WSL 1 Support](#wsl-1-support-1) +### [Terminology](#terminology-1) +---- + +## Introduction + +**NOTE:** We assume your computer is already virtualization ready with technologies like Hyper-V Threading enabled. If you have a computer which may not obviously satisfy this (remote server, old systems, custom installations), we recommend 1) Enabling virtualization from BIOS/UEFI and then 2) Turning on Hyper-V Windows feature. A simple search with the mentioned keywords would suffice the exact steps. + +This article will help Windows users setup and use WSL 2 and save their time solving issues. Before we begin, it is highly recommended to use the modern [Windows Terminal](https://apps.microsoft.com/detail/windows-terminal/9N0DX20HK701?hl=en-us&gl=US) app on Windows for Command Line operations, which allows much more customization and ability to have different types of terminals open at the same time in different tabs (For example, Powershell, Windows Command Prompt and Azure Shell simultaneously). + +WSL is a relatively lightweight virtualization tool dedicated for Linux, to run it on top of Windows. It is used by developers not wanting to have a dedicated Linux machine or not wanting to setup dual-boot machines which includes Linux as one of the operating systems. Developers can work and build in Linux environment using WSL on top of Windows, which is also useful when you quickly want to do something in a Linux environment, but it comes with its limitations. The official documentation for WSL by Microsoft can be found [here](https://learn.microsoft.com/en-us/windows/wsl/). + +---- + +## Installation + +It is possible to lookup WSL on Microsoft Store or indirectly install WSL by installing a flavor of Linux directly, but such installations have different default settings (precise information is available on official documentation). It is recommended to follow the following steps instead: + +The following command will print the available distributions. + + wsl --list --online + +Pick a distribution of your choice (Ubuntu is usually a fair choice), and run the following: + + wsl --install + +If you installed multiple distributions, you can use the following command to set one as default: + + wsl --set-default + +So, when you just run `wsl`, it will launch that distribution. +To launch specific distribution, run: + + wsl --distribution + +It's this simple to get started with WSL! We will talk about other settings, features and limitations in the rest of the document. + +---- + +## Useful Commands + +### Handling Distributions + +To see the state of your local WSL, run: + + wsl --list --all -v + +You can see all distributions and their states using this command. To terminate a specific distribution, run: + + wsl --terminate + +To shutdown WSL and thus all running distributions on it, run: + + wsl --shutdown + +To delete a distribution, run: + + wsl --unregister + +### WSL Help, Update, and Version + +To update WSL, run: + + wsl --update + +To see WSL version: + + wsl --version + +To see possible commands with WSL, run: + + wsl --help + +### Distribution Specific Configuration Files + +"/etc/wsl.conf" is the configuration file read by WSL in startup when booting the distribution, if it exists. A simple search in the documentation for available configuration options can help automate things when you are launching WSL ! Note that more features have been added in newer builds, so version is important to take care of, which is available [here](https://learn.microsoft.com/en-us/windows/wsl/release-notes). + +---- + +## WSL 1 Support + +Microsoft changed the way they parse Windows Path files between WSL 1 and WSL 2, and it created a deadlock where trying to run Docker on WSL 2 will lead to Path errors, and Docker cannot be run on WSL 1 due different virtualization capabilities. There is no visible solution to the author's best knowledge and extensive research to bypass setup scripts involved. It is recommended to use Docker only on Linux instead, especially for the sake of best compatibility. Thus, we are going to discuss how to run WSL 1 for any similar reasons that may require it. + +Following the installation steps in this document, it is assumed by WSL that the user wants to use the latest version, version 2. To change this default setting, run: + + wsl --set-default-version 1 + +To change the version of a specific distribution instead, run: + + wsl --set-version + +This allows backwards compatibility. To change WSL 1 distribution launch settings: + +1) Run regedit.exe +2) Navigate to HKCU\Software\Microsoft\Windows\CurrentVersion\Lxss +3) Use [WSL_DISTRIBUTION_FLAGS](https://learn.microsoft.com/en-us/windows/win32/api/wslapi/ne-wslapi-wsl_distribution_flags) to change launch settings for distributions. For example, 0x7 implies all flags are enabled (7 = 1 + 2 + 4). + +**NOTE:** Modifying registers can potentially damage and crash you system and data. It is important to be responsible and think twice before implementing changes. + +This information should be mostly sufficient, and further information is available on official documentation. + +---- + +## Terminology +- [**Virtualization**](https://www.ibm.com/topics/virtualization): the action of creating an abstraction layer over computer hardware that allows the hardware elements of a single computer like processors, memory, storage and more to be divided into multiple virtual computers, commonly called virtual machines (VMs). This action allows you to run Linux on top of Windows without shutting it down! + +- **Distribution**: The operating system being virtualized on your Windows machine, usually some flavor of Linux. From 709eaf86f43bf635bf00c13ee8c71f6ecebb76d4 Mon Sep 17 00:00:00 2001 From: Parth Date: Sat, 11 Nov 2023 22:30:58 -0500 Subject: [PATCH 2/7] refined limitations details and how to enable virtualization --- Topics/Development_Process/WSL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Topics/Development_Process/WSL.md b/Topics/Development_Process/WSL.md index 52755ac78..55bc20852 100644 --- a/Topics/Development_Process/WSL.md +++ b/Topics/Development_Process/WSL.md @@ -10,7 +10,7 @@ ## Introduction -**NOTE:** We assume your computer is already virtualization ready with technologies like Hyper-V Threading enabled. If you have a computer which may not obviously satisfy this (remote server, old systems, custom installations), we recommend 1) Enabling virtualization from BIOS/UEFI and then 2) Turning on Hyper-V Windows feature. A simple search with the mentioned keywords would suffice the exact steps. +**NOTE:** We assume your computer is already virtualization ready with technologies like Hyper-V Threading enabled. If you have a computer which may not obviously satisfy this (remote server, old systems, custom installations), we recommend 1) Enabling virtualization from BIOS/UEFI and then 2) Turning on Hyper-V Windows feature and Windows Subsystem for Linux feature. A simple search with the mentioned keywords would sufficiently provide the exact steps. This article will help Windows users setup and use WSL 2 and save their time solving issues. Before we begin, it is highly recommended to use the modern [Windows Terminal](https://apps.microsoft.com/detail/windows-terminal/9N0DX20HK701?hl=en-us&gl=US) app on Windows for Command Line operations, which allows much more customization and ability to have different types of terminals open at the same time in different tabs (For example, Powershell, Windows Command Prompt and Azure Shell simultaneously). @@ -20,7 +20,7 @@ WSL is a relatively lightweight virtualization tool dedicated for Linux, to run ## Installation -It is possible to lookup WSL on Microsoft Store or indirectly install WSL by installing a flavor of Linux directly, but such installations have different default settings (precise information is available on official documentation). It is recommended to follow the following steps instead: +It is possible to lookup WSL on Microsoft Store or indirectly install WSL by installing a flavor of Linux directly, but such installations have different default settings and version (precise information is available on official documentation). It is recommended to follow the following steps instead: The following command will print the available distributions. @@ -85,7 +85,7 @@ To see possible commands with WSL, run: ## WSL 1 Support -Microsoft changed the way they parse Windows Path files between WSL 1 and WSL 2, and it created a deadlock where trying to run Docker on WSL 2 will lead to Path errors, and Docker cannot be run on WSL 1 due different virtualization capabilities. There is no visible solution to the author's best knowledge and extensive research to bypass setup scripts involved. It is recommended to use Docker only on Linux instead, especially for the sake of best compatibility. Thus, we are going to discuss how to run WSL 1 for any similar reasons that may require it. +Microsoft changed the way they parse Windows Path files between WSL 1 and WSL 2, and it created a deadlock where trying to run Windows Docker Desktop based on WSL 2 will lead to Path errors, and Windows Docker Desktop cannot be run on WSL 1 due different virtualization capabilities. There is no visible solution to the author's best knowledge and extensive research to bypass setup scripts involved. It is recommended to use Docker only on Linux instead, especially for the sake of best compatibility (Docker would still work in a distribution running on WSL). Thus, we are going to discuss how to run WSL 1 for any similar reasons that may require it. Following the installation steps in this document, it is assumed by WSL that the user wants to use the latest version, version 2. To change this default setting, run: From 17155d1d19271fa3e5939ad46b8c01b8512aa0e4 Mon Sep 17 00:00:00 2001 From: parth <35137360+parth2324@users.noreply.github.com> Date: Tue, 14 Nov 2023 00:24:59 -0500 Subject: [PATCH 3/7] Updated WSL.md based on feedback --- Topics/Development_Process/WSL.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/Topics/Development_Process/WSL.md b/Topics/Development_Process/WSL.md index 55bc20852..b5d715bba 100644 --- a/Topics/Development_Process/WSL.md +++ b/Topics/Development_Process/WSL.md @@ -6,11 +6,12 @@ ### [Useful Commands](#useful-commands-1) ### [WSL 1 Support](#wsl-1-support-1) ### [Terminology](#terminology-1) +### [Other Notes](#other-notes-1) ---- ## Introduction -**NOTE:** We assume your computer is already virtualization ready with technologies like Hyper-V Threading enabled. If you have a computer which may not obviously satisfy this (remote server, old systems, custom installations), we recommend 1) Enabling virtualization from BIOS/UEFI and then 2) Turning on Hyper-V Windows feature and Windows Subsystem for Linux feature. A simple search with the mentioned keywords would sufficiently provide the exact steps. +**NOTE:** We assume your computer is already virtualization ready with technologies like Hyper-V Threading enabled. If you have a computer which may not obviously satisfy this (remote server, old systems, custom installations), we recommend 1) [Enabling virtualization from BIOS/UEFI](https://support.microsoft.com/en-us/windows/enable-virtualization-on-windows-11-pcs-c5578302-6e43-4b4b-a449-8ced115f58e1) and then 2) Turning on [Hyper-V Windows feature](https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v) and [Windows Subsystem for Linux feature](https://learn.microsoft.com/en-us/windows/wsl/install-manual). This article will help Windows users setup and use WSL 2 and save their time solving issues. Before we begin, it is highly recommended to use the modern [Windows Terminal](https://apps.microsoft.com/detail/windows-terminal/9N0DX20HK701?hl=en-us&gl=US) app on Windows for Command Line operations, which allows much more customization and ability to have different types of terminals open at the same time in different tabs (For example, Powershell, Windows Command Prompt and Azure Shell simultaneously). @@ -108,6 +109,13 @@ This information should be mostly sufficient, and further information is availab ---- ## Terminology + - [**Virtualization**](https://www.ibm.com/topics/virtualization): the action of creating an abstraction layer over computer hardware that allows the hardware elements of a single computer like processors, memory, storage and more to be divided into multiple virtual computers, commonly called virtual machines (VMs). This action allows you to run Linux on top of Windows without shutting it down! - **Distribution**: The operating system being virtualized on your Windows machine, usually some flavor of Linux. + +---- + +## Other Notes + +It is useful to note that WSL integration on Windows allows a lot of mixing of commands between the distribution and Windows since the Windows %PATH% is imported to WSL CLI space by default. If running WSL yields any errors, initially make sure all prerequisites are met as dicussed, and make sure your Windows and WSL are up to date. If the problem persists, search the error code for resolution, and try to sort out the type of issue while trying to determine whether it's due to the distribution or WSL. The author has had WSL hang on him at times, for which a simple and patient `wsl --shutdown` followed by `wsl` works well. From 508bce696ba2319e5278312c4f6a7587e4b54b97 Mon Sep 17 00:00:00 2001 From: William Jarvis-Cross Date: Tue, 14 Nov 2023 13:03:15 -0500 Subject: [PATCH 4/7] add required files for url sanitization --- Topics/Development_Process.md | 3 + .../Development_Process/URL_Sanitization.md | 106 ++++++++++++++++++ 2 files changed, 109 insertions(+) create mode 100644 Topics/Development_Process/URL_Sanitization.md diff --git a/Topics/Development_Process.md b/Topics/Development_Process.md index 7aaf37faf..aea88a749 100644 --- a/Topics/Development_Process.md +++ b/Topics/Development_Process.md @@ -16,6 +16,9 @@ ## React Testing Library ### [React Testing Library](./Development_Process/React_Testing_Library.md) +## URL Sanitization +### [URL Sanitization](./Development_Process/URL_Sanitization.md) + ## SOLID PRINCIPLES: SOLID is a mnemonic acronym that represents a set of five very important software development principles which lead to code that is easier to read, maintain, and extend, leading to higher-quality software that is easier to evolve over time. diff --git a/Topics/Development_Process/URL_Sanitization.md b/Topics/Development_Process/URL_Sanitization.md new file mode 100644 index 000000000..ac823bc1c --- /dev/null +++ b/Topics/Development_Process/URL_Sanitization.md @@ -0,0 +1,106 @@ +# URL Sanitization: What is it? Why should we do it? How do we do it? + +## Table of Contents: +### [Introduction](#introduction-1) +### [What is sanitization and why does it matter?](#what-is-sanitization-and-why-does-it-matter-1) +### [How should we sanitize URLs?](#how-should-sanitize-urls-1) +### [Examples of how to use](#examples-of-how-to-use-1) +### [Testing your sanitization implementation](#testing-your-sanitization-implementation-1) + + +## Introduction + +In today's digital landscape, web applications are essential tools for businesses and individuals. However, they are also susceptible to various cyber threats, including attacks through manipulated URLs. To safeguard against potential vulnerabilities, it's imperative to understand and implement proper URL sanitation practices. As a student learnging software engineering, sanitization of URLs is a key concept when building a new web application. + + +## What is sanitization and why does it matter? + +URL sanitation is the process of validating, cleaning, and securing incoming URLs in a web application. There should be some level of sanitation for every web application. Here are some reasons why it is needed: + +-Guarding against Security Threats: Unsanitized URLs can be gateways for security threats such as cross-site scripting (XSS) (This is the process of injecting malicious scripts in websites. More information can be found here: https://owasp.org/www-community/attacks/xss/), SQL injection (Malicious code used to access and modify backend databases. More information about this can be found here: https://www.imperva.com/learn/application-security/sql-injection-sqli/#:~:text=SQL%20injection%2C%20also%20known%20as,lists%20or%20private%20customer%20details.), and other malicious attacks. Sanitizing URLs mitigates these risks. + +-Protecting User Data: Proper sanitation ensures the safety of user data by avoiding potential exposure to attackers who might exploit vulnerabilities in URLs to access sensitive information. This is vital for owning a website that users can trust. + +-Maintaining Application Integrity: By sanitizing URLs, you maintain the integrity and functionality of your application, reducing the risk of unexpected behaviors or compromises. + + +## How should we sanitize URLs? + +Implementing URL sanitation involves several key steps and best practices: + +Input Validation: Validate incoming URLs against a strict set of rules and expected patterns. Ensure they conform to standard URL formats and accepted protocols (HTTP/HTTPS). + +Encoding and Escaping: Encode special characters in URLs using proper encoding mechanisms such as [encodeURIComponent()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent) in JavaScript or server-side functions like urlencode() in PHP. Additionally, escape output when displaying URLs on web pages to prevent interpretation as executable code. Just remember that you may have to decode the URL parameters after if you need parameter values in your code. This can be done using [decodeURIComponent()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent). + +Whitelisting: Define a whitelist of allowed characters, protocols, and URL patterns. Reject any URL that does not match the predefined criteria, effectively filtering out potentially harmful input. Here is an example of what characters could be on the whitelist: (A-Za-z0-9-._~:/?#[]@!$&'()*+,;=%). Notice that this does not include angle brackets (<>) or curly braces ({}) as those are not needed in a url and can be used maliciously. + +Regular Expressions: Use regular expressions to match and filter URLs based on expected patterns. Regular expressions can help validate and sanitize URLs effectively. + +Utilize Security Libraries: Leverage trusted URL sanitization libraries or frameworks available in your programming language or framework. These libraries often provide specific methods to clean and validate URLs effectively. The best one I know of is the [sanitize-url](https://www.npmjs.com/package/@braintree/sanitize-url) library. This is good for general sanitization, but if you need specific cases checked, then it may be more effective to build a sanitization function from scratch. + + +## Examples of how to use + +Example of using sanitize-url library: + +First, you have to install the library using this: npm install -S @braintree/sanitize-url + + + +```javascript +var sanitizeUrl = require("@braintree/sanitize-url").sanitizeUrl; + +sanitizeUrl("https://example.com"); // 'https://example.com' +sanitizeUrl("http://example.com"); // 'http://example.com' +sanitizeUrl("www.example.com"); // 'www.example.com' +sanitizeUrl("mailto:hello@example.com"); // 'mailto:hello@example.com' +sanitizeUrl( + "https://example.com" +); // https://example.com + +sanitizeUrl("javascript:alert(document.domain)"); // 'about:blank' +sanitizeUrl("jAvasCrIPT:alert(document.domain)"); // 'about:blank' +sanitizeUrl(decodeURIComponent("JaVaScRiP%0at:alert(document.domain)")); // 'about:blank' +// HTML encoded javascript:alert('XSS') +sanitizeUrl( + "javascript:alert('XSS')" +); // 'about:blank' +```javascript + +The more recommended method is to make your own function, here is an example of one that I made: + +```javascript +export const sanitizeInput = (input, isUrl = false) => { + if (typeof input !== 'string') { + return input + } + let paramInput = input + let splitUrl + if (isUrl && input) { //splitting url so we only sanitize parameters + splitUrl = input.split('?') + paramInput = splitUrl.length > 1 ? splitUrl[1] : '' + } + //force incoming url to math this regex pattern + const sanitizedInput = paramInput.replace(/[^a-zA-Z0-9\s.,!?_&=%<>"']/g, '') + // Input encoding + const htmlEntities = { + '<': '<', + '>': '>', + '"': '"', + "'": ''', + } + + const encodedInput = sanitizedInput.replace(/[<>"']/g, char => htmlEntities[char]) + + // HTML tag filtering + const filteredInput = encodedInput.replace(/<\/?script>/gi, '') + if (isUrl && input) { // recombining url + const sanitizedPart = splitUrl.length > 1 ? (`?${filteredInput}`) : '' + return splitUrl[0] + sanitizedPart + } + return filteredInput +} +```javascript +Of course, every situation is different, and you might need to add or remove different characters from your whitelist, or you may have to add more encoding and decoding. + +## Testing your sanitization implementation \ No newline at end of file From 18f4a80643aff4b5d2827749ce05dda7ae6adb97 Mon Sep 17 00:00:00 2001 From: William Jarvis-Cross Date: Tue, 14 Nov 2023 13:11:11 -0500 Subject: [PATCH 5/7] fix markdown formatting --- Topics/Development_Process/URL_Sanitization.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/Topics/Development_Process/URL_Sanitization.md b/Topics/Development_Process/URL_Sanitization.md index ac823bc1c..1b001767c 100644 --- a/Topics/Development_Process/URL_Sanitization.md +++ b/Topics/Development_Process/URL_Sanitization.md @@ -5,7 +5,6 @@ ### [What is sanitization and why does it matter?](#what-is-sanitization-and-why-does-it-matter-1) ### [How should we sanitize URLs?](#how-should-sanitize-urls-1) ### [Examples of how to use](#examples-of-how-to-use-1) -### [Testing your sanitization implementation](#testing-your-sanitization-implementation-1) ## Introduction @@ -65,7 +64,7 @@ sanitizeUrl(decodeURIComponent("JaVaScRiP%0at:alert(document.domain)")); // 'abo sanitizeUrl( "javascript:alert('XSS')" ); // 'about:blank' -```javascript +``` The more recommended method is to make your own function, here is an example of one that I made: @@ -100,7 +99,5 @@ export const sanitizeInput = (input, isUrl = false) => { } return filteredInput } -```javascript +``` Of course, every situation is different, and you might need to add or remove different characters from your whitelist, or you may have to add more encoding and decoding. - -## Testing your sanitization implementation \ No newline at end of file From 0ea81b614d3e814b2ee426b810ba6d96dfbe0499 Mon Sep 17 00:00:00 2001 From: William Jarvis-Cross Date: Tue, 14 Nov 2023 16:08:56 -0500 Subject: [PATCH 6/7] add testing instructions --- .../Development_Process/URL_Sanitization.md | 41 +++++++++++++++++-- 1 file changed, 37 insertions(+), 4 deletions(-) diff --git a/Topics/Development_Process/URL_Sanitization.md b/Topics/Development_Process/URL_Sanitization.md index 1b001767c..7de1c9bdb 100644 --- a/Topics/Development_Process/URL_Sanitization.md +++ b/Topics/Development_Process/URL_Sanitization.md @@ -5,11 +5,13 @@ ### [What is sanitization and why does it matter?](#what-is-sanitization-and-why-does-it-matter-1) ### [How should we sanitize URLs?](#how-should-sanitize-urls-1) ### [Examples of how to use](#examples-of-how-to-use-1) +### [How to test the implementation of your sanitization function](#how-to-test-the-implementation-of-your-sanitization-function-1) +### [Errors you might encounter](#Errors-you-might-encounter-1) ## Introduction -In today's digital landscape, web applications are essential tools for businesses and individuals. However, they are also susceptible to various cyber threats, including attacks through manipulated URLs. To safeguard against potential vulnerabilities, it's imperative to understand and implement proper URL sanitation practices. As a student learnging software engineering, sanitization of URLs is a key concept when building a new web application. +In today's digital landscape, web applications are essential tools for businesses and individuals. However, they are also susceptible to various cyber threats, including attacks through manipulated URLs. To safeguard against potential vulnerabilities, it's imperative to understand and implement proper URL sanitation practices. As a student learning software engineering, sanitization of URLs is a key concept when building a new web application. ## What is sanitization and why does it matter? @@ -31,7 +33,7 @@ Input Validation: Validate incoming URLs against a strict set of rules and expec Encoding and Escaping: Encode special characters in URLs using proper encoding mechanisms such as [encodeURIComponent()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent) in JavaScript or server-side functions like urlencode() in PHP. Additionally, escape output when displaying URLs on web pages to prevent interpretation as executable code. Just remember that you may have to decode the URL parameters after if you need parameter values in your code. This can be done using [decodeURIComponent()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent). -Whitelisting: Define a whitelist of allowed characters, protocols, and URL patterns. Reject any URL that does not match the predefined criteria, effectively filtering out potentially harmful input. Here is an example of what characters could be on the whitelist: (A-Za-z0-9-._~:/?#[]@!$&'()*+,;=%). Notice that this does not include angle brackets (<>) or curly braces ({}) as those are not needed in a url and can be used maliciously. +Whitelisting: Define a whitelist of allowed characters, protocols, and URL patterns. Reject any URL that does not match the predefined criteria, effectively filtering out potentially harmful input. Here is an example of what characters could be on the whitelist: (A-Za-z0-9-._~:/?#[]@!$&'()*+,;=%). Notice that this does not include angle brackets (<>) or curly braces ({}) as those are not needed in a URL and can be used maliciously. Regular Expressions: Use regular expressions to match and filter URLs based on expected patterns. Regular expressions can help validate and sanitize URLs effectively. @@ -44,7 +46,7 @@ Example of using sanitize-url library: First, you have to install the library using this: npm install -S @braintree/sanitize-url - +Below are a couple of examples of implementations for sanitization methods. Remember if you are using this to sanitize URL parameters, the parameters should be sanitized before used for anything else. ```javascript var sanitizeUrl = require("@braintree/sanitize-url").sanitizeUrl; @@ -100,4 +102,35 @@ export const sanitizeInput = (input, isUrl = false) => { return filteredInput } ``` -Of course, every situation is different, and you might need to add or remove different characters from your whitelist, or you may have to add more encoding and decoding. +Of course, every situation is different, and you might need to add or remove different characters from your whitelist, or you may have to add more encoding and decoding. This document specifically talks about URL sanitization, but this method is also effective for any type of data coming into your application. This even includes text fields that users enter data into. + + +## How to test the implementation of your sanitization function + + +To test your new function, you will want to pass different URLs into the function (this example is for JavaScript React). Here are a couple of examples for sanitizeInput(): +```javascript +describe('sanitizeInput function', () => { + it('should return the input string as is when it is not a string', () => { + const input = 123; // Input is a number + const result = sanitizeInput(input); + expect(result).toEqual(input); + }); + + it('should sanitize URL parameters and remove unwanted characters', () => { + const inputUrl = 'https://example.com/?param1=¶m2=abc'; + const sanitizedUrl = sanitizeInput(inputUrl, true); + expect(sanitizedUrl).toEqual('https://example.com/?param1=¶m2=abc'); + }); +}); +``` + +## Errors you might encounter + +When the sanitization change is merged, check if the function is working correctly. This can be done by adding an alert in a URL for your website (example: 'https://example.com/?param1=¶m2=abc'). If an alert pops up on the window, then this means the sanitization is not working correctly and you are probably missing something important in your sanitization function (you might have to alter your whitelist to include less characters). + +If you notice that some of the URL parameters are becoming altered by this sanitization, then there could be 2 reasons for this: + +-Your whitelist in the sanitization function is too strict and is not letting normal characters pass through. + +-You are using improper characters in your URL parameters. Here is a [good reference](https://www.freecodecamp.org/news/url-encoded-characters-reference/) which describes which characters should and should not be put in URL parameters. \ No newline at end of file From 2fe5466ac87c80b48c4f26bdda1a1547aa716188 Mon Sep 17 00:00:00 2001 From: William Jarvis-Cross Date: Tue, 14 Nov 2023 16:22:24 -0500 Subject: [PATCH 7/7] fix formatting --- Topics/Development_Process/URL_Sanitization.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Topics/Development_Process/URL_Sanitization.md b/Topics/Development_Process/URL_Sanitization.md index 7de1c9bdb..1788af4f5 100644 --- a/Topics/Development_Process/URL_Sanitization.md +++ b/Topics/Development_Process/URL_Sanitization.md @@ -3,10 +3,10 @@ ## Table of Contents: ### [Introduction](#introduction-1) ### [What is sanitization and why does it matter?](#what-is-sanitization-and-why-does-it-matter-1) -### [How should we sanitize URLs?](#how-should-sanitize-urls-1) +### [How should we sanitize URLs?](#how-should-we-sanitize-urls-1) ### [Examples of how to use](#examples-of-how-to-use-1) ### [How to test the implementation of your sanitization function](#how-to-test-the-implementation-of-your-sanitization-function-1) -### [Errors you might encounter](#Errors-you-might-encounter-1) +### [Errors you might encounter](#errors-you-might-encounter-1) ## Introduction