Skip to content

Commit

Permalink
Merge pull request #366 from Azure/dev
Browse files Browse the repository at this point in the history
3.4.0 Release - Sync dev branch to master
  • Loading branch information
vinjiang authored Dec 26, 2019
2 parents df5bf1d + 1c5e332 commit 6d444ef
Show file tree
Hide file tree
Showing 127 changed files with 26,238 additions and 1,661 deletions.
22 changes: 22 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
.vscode/**
node_modules/**
swagger/**
__blobstorage__/**
__queuestorage__/**
__*.json
__*__
*.tgz
*.vsix
*.log
*.png
*.env
*.yml
temp
swagger
.git
.gitattributes
.gitignore
.lintstagedrc
.vscode
.vscodeignore
dist
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto eol=lf

# Denote all files that are truly binary and should not be modified.
*.png binary
*.jpg binary
6 changes: 5 additions & 1 deletion .npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,8 @@ debug.log
__*__.json
__*__
*.env
temp
temp
.gitattributes
.dockerignore
README.mcr.md
icon.png
26 changes: 23 additions & 3 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,27 @@
{
"type": "node",
"request": "launch",
"name": "Azurite Queue Service",
"name": "Azurite Service - Loki",
"cwd": "${workspaceFolder}",
"runtimeArgs": ["-r", "ts-node/register"],
"args": ["${workspaceFolder}/src/azurite.ts", "-d", "debug.log"],
"env": { "AZURITE_ACCOUNTS": "" },
"outputCapture": "std"
},
{
"type": "node",
"request": "launch",
"name": "Azurite Service - Loki, Loose",
"cwd": "${workspaceFolder}",
"runtimeArgs": ["-r", "ts-node/register"],
"args": ["${workspaceFolder}/src/azurite.ts", "-d", "debug.log", "-L"],
"env": { "AZURITE_ACCOUNTS": "" },
"outputCapture": "std"
},
{
"type": "node",
"request": "launch",
"name": "Azurite Queue Service - Loki",
"cwd": "${workspaceFolder}",
"runtimeArgs": ["-r", "ts-node/register"],
"args": ["${workspaceFolder}/src/queue/main.ts", "-d", "debug.log"],
Expand All @@ -32,7 +52,7 @@
"runtimeArgs": ["-r", "ts-node/register"],
"args": ["${workspaceFolder}/src/blob/main.ts", "-d", "debug.log"],
"env": {
"AZURITE_DB": "mariadb://root:[email protected]:3306/azurite_blob",
"AZURITE_DB": "mysql://root:[email protected]:3306/azurite_blob",
"AZURITE_ACCOUNTS": ""
},
"outputCapture": "std"
Expand Down Expand Up @@ -82,7 +102,7 @@
],
"env": {
"AZURITE_ACCOUNTS": "",
"AZURITE_TEST_DB": "mariadb://root:[email protected]:3306/azurite_blob_test"
"AZURITE_TEST_DB": "mysql://root:[email protected]:3306/azurite_blob_test"
},
"internalConsoleOptions": "openOnSessionStart",
"outputCapture": "std"
Expand Down
5 changes: 5 additions & 0 deletions .vscodeignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ azure-pipelines.yml
.lintstagedrc
.gitignore
.npmignore
.dockerignore
Dockerfile
__blobstorage__
typings
Expand All @@ -20,5 +21,9 @@ debug.log
*.tgz
__*__.json
__*__
__queuestorage__
__blobstorage__
__queueTestsPersistence__
__*
*.env
temp
11 changes: 9 additions & 2 deletions BreakingChanges.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

> Note. This file includes breaking changes after 3.0.0-preview. For legacy Azurite changes, please goto GitHub [releases](https://github.com/Azure/Azurite/releases).
2019.11 Version 3.3.0-preview
## 2019.12 Version 3.4.0

- [Breaking] This version cannot guarantee compatible with persisted database models file by previous version. Remove previous metadata file and restart Azurite in case any errors.
- [Breaking] By default Azurite will block requests with unsupported headers or parameters which may impact data integrity.
- Skip this by switching to loose mode by Azurite configuration parameter `--loose`.
- [Breaking] Azurite updates underline metadata schema which does not compatible with previous versions.
- This version cannot guarantee compatible with persisted database models file by previous version. Remove previous metadata file and restart Azurite in case any errors.

## 2019.11 Version 3.3.0-preview

- [Breaking] This version cannot guarantee compatible with persisted database models file by previous version. Remove previous metadata file and restart Azurite in case any errors.
15 changes: 7 additions & 8 deletions CONTRIBUTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,34 +15,33 @@ In the root of the repository, we have provided pre-defined debugging scripts. T
Manually follow following steps to build and run all services in Azurite:

```bash
npm install
npm ci
npm run azurite
```

Or build and run a certain service like Blob service:

```bash
npm install
npm ci
npm run blob
```

## Debug with SQL based Persisted Metadata Storage

By default, Azurite leverages loki as metadata database.
However, loki limits Azurite's scalability and extensibility.
Set environment variable `AZURITE_DB=dialect://[username][:password][@]host:port/database` to make Azurite blob service switch to a SQL database based metadata storage, like MySql, SqlServer, MariaDB.
Set environment variable `AZURITE_DB=dialect://[username][:password][@]host:port/database` to make Azurite blob service switch to a SQL database based metadata storage, like MySql, SqlServer.

For example, connect to MariaDB, MySql or SqlServer by set environment variables:
For example, connect to MySql or SqlServer by set environment variables:

```bash
set AZURITE_DB=mariadb://root:[email protected]:3306/azurite_blob
set AZURITE_DB=mysql://localhost:3306/azurite_blob
set AZURITE_DB=mysql://root:[email protected]:3306/azurite_blob
set AZURITE_DB=mssql://username:password@localhost:1024/azurite_blob
```

> Note. Need to manually create database before starting Azurite instance.
> Tips. Create database instance quickly with docker, for example `docker run --name mariadb -p 3306:3306 -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mariadb:latest`.
> Tips. Create database instance quickly with docker, for example `docker run --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:latest`. And grant external access and create database `azurite_blob` using `docker exec mysql mysql -u root -pmy-secret-pw -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION; FLUSH PRIVILEGES; create database azurite_blob;"`. Notice that, above commands are examples, you need to carefully define the access permissions in your production environment.
## Develop for Visual Studio Code Extension

Expand All @@ -57,7 +56,7 @@ We also provide a predefined Visual Studio Code debug configuration "Current Moc
Or manually execute all test cases:

```bash
npm install
npm ci
npm run test
```

Expand Down
54 changes: 50 additions & 4 deletions ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,49 @@

> Note. This file includes changes after 3.0.0-preview. For legacy Azurite changes, please goto GitHub [releases](https://github.com/Azure/Azurite/releases).
2019.11 Version 3.3.0-preview
## 2019.12 Version 3.4.0

- Return the list of containers will be in sorted order.
- Fixed a bug that get/download blob snapshot fails.
- Check input request "x-ms-version" Header, only valid version are allowed.
- Fixed a race condition that GC will delete active write extents.
- Force flush data into disk before data upload request returns.
- [Breaking] By default Azurite will block requests with unsupported headers or parameters which may impact data integrity.
- Skip this by switching to loose mode by Azurite configuration parameter `--loose`.

Blob:

- [Breaking] Azurite updates underline metadata schema which does not compatible with previous versions.
- This version cannot guarantee compatible with persisted database models file by previous version. Remove previous metadata file and restart Azurite in case any errors.
- List blocks will filter the returned block list with input BlockListingFilter.
- Added support for CORS.
- AllowedHeaders and ExposedHeaders are optional now when setting CORS.
- Added support to create block blob with empty block list.
- Stage block cannot have blockID longer than 64.
- Fix the issue that Copy Blob will overwrite the destination blob Lease status.
- Fix the issue that Change Lease fail when blob lease id only matches the input ProposedLeaseId.
- Fix the issue that UploadPage, ClearPage will fail on leased Page blob, even input correct lease id.
- Update some lease error codes to align with Azure Storage.
- Fixed a bug that set blob tier doesn't work with account SAS.
- Fixed a bug that Azurite Blob service cannot start in Mac as Visual Studio Extension.
- Fixed a bug that persistency location cannot be customized through -l parameter.
- Fixed a bug that GC will remove uncommitted blocks.
- Fixed a bug that download page blob doesn't return content range header.
- Fixed a bug that uncommitted block blob invalid length.
- Fixed a bug that SetHTTPHeaders, SetMetadata won't update blob etag.
- Remove double quotation marks from list blob request returned blob etag, to align with Azure Server behavior.
- Fixed a bug that BlobTierInferred not change to false after SetBlobTier.
- Blocked set tier for page blob which requires premium storage account where Azurite provides standard storage account.
- GetPageRangesDiff API (incremental snapshot) now returns NotImplementedError.
- Fixed a bug that listing containers won't honor prefix with marker when using external metadata database.

Queue:

- AllowedHeaders and ExposedHeaders are optional now when setting CORS.
- Fix Put message fail with max messagettl.
- Updated message size calculation when checking 64KB limitation.

## 2019.11 Version 3.3.0-preview

- Azurite now supports customized account names and keys by environment variable `AZURITE_ACCOUNTS`.
- Improved logging for underlayer operations, such as persistency data read and write operations.
Expand All @@ -12,6 +54,7 @@
- [Breaking] This version cannot guarantee compatible with persisted database models in Azurite workspace used by previous version. Clean Azurite workspace folder and restart Azurite in case any errors. Notice that, data will be lost after cleaning Azurite workspace folder.

Blob:

- Fixed a bug that snapshot blob doesn't honor metadata options.
- Force alphabetical order for list blob results.
- Updated Azure Storage API version to 2019-02-02, and added following new features:
Expand All @@ -23,21 +66,24 @@ Blob:
- Added GC support for blob SQL metadata store.

Queue:

- Responses now includes x-ms-client-request-id when request provided client request ID.

2019.08 Version 3.2.0-preview
## 2019.08 Version 3.2.0-preview

- Updated repository link to https to compatible with Visual Studio Code.

Blob:

- Fix listblobs order when filtering by prefix.

Queue:

- Added Azure Storage Queue Service features (API version: 2019-02-02).
- Decoupled persistence layer into service metadata storage and extent file storage.
- Supported Cors and Preflight in Queue service.

2019.06 Version 3.1.2-preview
## 2019.06 Version 3.1.2-preview

- Integrated Azurite with Visual Studio Code as an extension.
- Added Visual Studio Code extension usage guidelines.
Expand All @@ -47,6 +93,6 @@ Queue:
- Fixed an issue that metadata doesn't get copied when copy blob.
- Fixed GetBlockBlob missing Content-Range header

2019.05 Version 3.0.0-preview
## 2019.05 Version 3.0.0-preview

- Initial Release of Azurite V3.
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ VOLUME [ "/data" ]
COPY . .

RUN npm config set unsafe-perm=true
RUN npm install
RUN npm ci
RUN npm run build
RUN ls -l
RUN npm install -g

# Blob Storage Port
Expand Down
Loading

0 comments on commit 6d444ef

Please sign in to comment.