Skip to content

Commit

Permalink
Restricting storage access by IP & vnet, support vnet service endpoin…
Browse files Browse the repository at this point in the history 
…ts (#545)

* Restricting storage access by IP and vnet, support vnet service endpoints.

* Cleanup tests

* Stop emitting empty service endpoints on vnets.

* Adds subnetSpec for carving a vnet into complex subnets

* Moved vnet changes up in release notes

* Separate vnet builders in documentation
  • Loading branch information
@ninjarobot
ninjarobot authored Jun 11, 2021
1 parent 5617600 commit 6d9a265
Show file tree
Hide file tree
Showing 20 changed files with 575 additions and 84 deletions.
1 change: 1 addition & 0 deletions Farmer.sln
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ ProjectSection(SolutionItems) = preProject
samples\scripts\postgresql.fsx = samples\scripts\postgresql.fsx
samples\scripts\aks.fsx = samples\scripts\aks.fsx
samples\scripts\bastion.fsx = samples\scripts\bastion.fsx
samples\scripts\vnet.fsx = samples\scripts\vnet.fsx
samples\scripts\container-instance.fsx = samples\scripts\container-instance.fsx
EndProjectSection
EndProject
Expand Down
3 changes: 3 additions & 0 deletions RELEASE_NOTES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ Release Notes
=============
## 1.6.0
* Added support for nesting resource groups
* Storage: Support for firewall to restrict storage account network access to virtual network subnets, IP addresses, and CIDR prefixes.
* Virtual Network: Support for creating service endpoints on subnets.
* Virtual Network: Support for assigning existing service endpoint policies to subnets.

## 1.5.3
* CDN: Support for CDN rules
Expand Down
132 changes: 107 additions & 25 deletions docs/content/api-overview/resources/vnet.md
View file
Original file line number Diff line number Diff line change
@@ -1,33 +1,64 @@
---
title: "Virtual Network"
date: 2021-01-09T11:22:17-05:00
date: 2021-06-09T10:31:36-05:00
chapter: false
weight: 21
---

#### Overview

The Virtual Network builder (`vnet`) is used to create Azure Virtual Network instances.
The virtual network builder is used to deploy virtual networks and their subnets.

- Virtual Network (`Microsoft.Network/virtualNetworks`)
- Subnets (`Microsoft.Network/virtualNetworks/subnets`)

The Virtual Network module contains four builders

- The `vnet` builder is used to create Azure Virtual Network instances.
- The `subnet` builder is used within the `vnet` builder to define subnets.
- The `addressSpace` builder can be used to automatically generate subnets based on the sizes of networks needed within the address space.
- The `subnetSpec` builder is used to define the automatically generated subnets, with the primary different from the `subnet` builder being that you define the `size` for the prefix, and not the address.

#### Builder Keywords

| Resource | Keyword | Purpose |
| -------------- | -------------------- | ------------------------------------------------------------------ |
| vnet | name | Sets the name of the virtual network. |
| vnet | add_address_spaces | Adds address spaces to the virtual network. |
| vnet | add_subnets | Adds subnets to the virtual network. |
| vnet | build_address_spaces | Automatically builds address spaces for subnet names and sizes. |
| vnet | add_tags | Adds a set of tags to the resource |
| vnet | add_tag | Adds a tag to the resource |
| subnet | name | Name of the subnet resource |
| subnet | prefix | Subnet prefix in CIDR notation (e.g. 192.168.100.0/24) |
| subnet | add_delegations | Adds one or more delegations to this subnet. |
| addressSpace | space | When using `build_address_space` this specifies the address space. |
| addressSpace | subnets | Specifies the subnets to build automatically. |
| addressSpace | build_subnet | Specifies the name, size, and service delegations for the subnet. |
##### Virtual Network: `vnet`

| Keyword | Purpose |
| --------------------------------------- | -------------------------------------------------------------------- |
| name | Sets the name of the virtual network. |
| add_address_spaces | Adds address spaces to the virtual network. |
| add_subnets | Adds subnets to the virtual network. |
| build_address_spaces | Automatically builds address spaces for subnet names and sizes. |
| add_tags | Adds a set of tags to the resource |
| add_tag | Adds a tag to the resource |

##### Subnet: `subnet`

| Keyword | Purpose |
| --------------------------------------- | -------------------------------------------------------------------- |
| name | Name of the subnet resource |
| prefix | Subnet prefix in CIDR notation (e.g. 192.168.100.0/24) |
| add_delegations | Adds one or more delegations to this subnet. |
| add_service_endpoints | Adds one or more service endpoints to this subnet. |
| associate_service_endpoint_policies | Associates a subnet with an existing service policy. |

##### Automatically build out an address space: `addressSpace`

| Keyword | Purpose |
| --------------------------------------- | -------------------------------------------------------------------- |
| space | When using `build_address_space` this specifies the address space. |
| subnets | Specifies the subnets to build automatically. |


##### Specify subnets in automatic address space: `subnetSpec`

| Keyword | Purpose |
| --------------------------------------- | -------------------------------------------------------------------- |
| name | Specifies the name of the subnet to build. |
| size | Specifies the size of the subnet to build, default is /24. |
| add_delegations | Adds service delegations for the subnet. |
| add_service_endpoints | Adds service endpoints for the subnet. |
| add_service_endpoint_policies | Associates the service endpoint policies with the subnet. |

#### Example - Manual Subnets

Expand All @@ -53,6 +84,9 @@ let myVnet = vnet {
add_delegations [
SubnetDelegationService.ContainerGroups
]
add_service_endpoints [
EndpointServiceType.Storage, [Location.NorthEurope; Location.WestEurope]
]
}
subnet {
name "databases"
Expand Down Expand Up @@ -86,20 +120,68 @@ let myVnet = vnet {
addressSpace {
space "10.28.0.0/16"
subnets [
buildSubnet "vms" 26
buildSubnet "services" 24
buildSubnet "corporate-west" 18
buildSubnet "corporate-east" 18
buildSubnet "GatewaySubnet" 28
buildSubnetDelegations "containers" 27 [ SubnetDelegationService.ContainerGroups ]
subnetSpec {
name "vms"
size 26
}
subnetSpec {
name "services"
size 24
}
subnetSpec {
name "corporate-west"
size 18
}
subnetSpec {
name "corporate-east"
size 18
}
subnetSpec {
name "corporate-east"
size 18
}
subnetSpec {
name "GatewaySubnet"
size 28
}
subnetSpec {
name "containers"
size 27
add_delegations [SubnetDelegationService.ContainerGroups]
add_service_endpoints [
EndpointServiceType.Storage, [
Location.NorthEurope
Location.WestEurope
]
]
}
]
}
addressSpace {
space "10.30.0.0/16"
subnets [
buildSubnet "remote-office" 23
buildSubnet "reserved" 24
buildSubnet "GatewaySubnet" 28
subnetSpec {
name "remote-office"
size 23
}
subnetSpec {
name "applications"
size 24
add_service_endpoints [
EndpointServiceType.Storage, [
Location.NorthEurope
Location.WestEurope
]
]
}
subnetSpec {
name "reserved"
size 24
}
subnetSpec {
name "GatewaySubnet"
size 28
}
]
}
]
Expand Down
23 changes: 15 additions & 8 deletions samples/scripts/vnet.fsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,26 @@

open Farmer
open Farmer.Builders
open Farmer.Network

let serviceEndpointPolicy = Farmer.Arm.Network.serviceEndpointPolicies.resourceId "svc-endpt-policy"


let privateNet = vnet {
name "my-vnet"
build_address_spaces [
addressSpace {
space "10.28.0.0/16"
build_subnet "services" 24
build_subnet "corporate-west" 18
build_subnet "corporate-east" 18
build_subnet "GatewaySubnet" 29
build_subnet_delegated "containers" 27 [ SubnetDelegationService.ContainerGroups ]
build_subnet_service_endpoints "can-use-servicebus" 28 [ EndpointServiceType.ServiceBus, [Location.EastUS] ]
build_subnet_service_endpoint_policies "can-use-storage" 28 [ EndpointServiceType.Storage, [Location.EastUS] ] [ serviceEndpointPolicy ]
subnets [
buildSubnet "vms" 26
buildSubnet "services" 24
buildSubnet "corporate-west" 18
buildSubnet "corporate-east" 18
buildSubnet "GatewaySubnet" 29
buildSubnetDelegations "containers" 27 [ SubnetDelegationService.ContainerGroups ]
buildSubnet "more-services" 24
buildSubnetDelegations "more-containers" 27 [ SubnetDelegationService.ContainerGroups ]
]
}
addressSpace {
Expand All @@ -28,10 +35,10 @@ let privateNet = vnet {
}

let deployment = arm {
location Location.NorthEurope
location Location.EastUS
add_resource privateNet
}

deployment
|> Writer.quickWrite "output"
|> Writer.quickWrite "vnet-sample"

41 changes: 34 additions & 7 deletions src/Farmer/Arm/Network.fs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@ let expressRouteCircuits = ResourceType ("Microsoft.Network/expressRouteCircuits
let networkInterfaces = ResourceType ("Microsoft.Network/networkInterfaces", "2018-11-01")
let networkProfiles = ResourceType ("Microsoft.Network/networkProfiles", "2020-04-01")
let publicIPAddresses = ResourceType ("Microsoft.Network/publicIPAddresses", "2018-11-01")
let serviceEndpointPolicies = ResourceType ("Microsoft.Network/serviceEndpointPolicies", "2020-07-01")
let subnets = ResourceType ("Microsoft.Network/virtualNetworks/subnets", "")
let virtualNetworks = ResourceType ("Microsoft.Network/virtualNetworks", "2018-11-01")
let virtualNetworks = ResourceType ("Microsoft.Network/virtualNetworks", "2020-07-01")
let virtualNetworkGateways = ResourceType ("Microsoft.Network/virtualNetworkGateways", "2020-05-01")
let localNetworkGateways = ResourceType ("Microsoft.Network/localNetworkGateways", "")

Expand All @@ -35,11 +36,22 @@ type PublicIpAddress =
| None -> null |}
|} :> _

type SubnetDelegation =
{ Name : ResourceName
ServiceName : string }

type Subnet =
{ Name : ResourceName
Prefix : string
Delegations : SubnetDelegation list
ServiceEndpoints : (Network.EndpointServiceType * Location list) list
AssociatedServiceEndpointPolicies : ResourceId list }

type VirtualNetwork =
{ Name : ResourceName
Location : Location
AddressSpacePrefixes : string list
Subnets : {| Name : ResourceName; Prefix : string; Delegations: {| Name: ResourceName; ServiceName: string |} list |} list;
Subnets : Subnet list;
Tags: Map<string,string> }
interface IArmResource with
member this.ResourceId = virtualNetworks.resourceId this.Name
Expand All @@ -53,11 +65,26 @@ type VirtualNetwork =
{| name = subnet.Name.Value
properties =
{| addressPrefix = subnet.Prefix
delegations = subnet.Delegations
|> List.map (fun delegation ->
{| name = delegation.Name.Value
properties = {| serviceName = delegation.ServiceName |}
|})
delegations =
subnet.Delegations
|> List.map (fun delegation ->
{| name = delegation.Name.Value
properties = {| serviceName = delegation.ServiceName |}
|})
serviceEndpoints =
if subnet.ServiceEndpoints.IsEmpty then
Unchecked.defaultof<_>
else
subnet.ServiceEndpoints
|> List.map (fun (Network.EndpointServiceType(serviceEndpoint), locations) ->
{| service = serviceEndpoint
locations = locations |> List.map (fun location ->location.ArmValue) |})
serviceEndpointPolicies =
if subnet.AssociatedServiceEndpointPolicies.IsEmpty then
Unchecked.defaultof<_>
else
subnet.AssociatedServiceEndpointPolicies
|> List.map (fun policyId -> {| id = policyId.ArmExpression.Eval() |})
|}
|})
|}
Expand Down
57 changes: 57 additions & 0 deletions src/Farmer/Arm/Storage.fs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,55 @@ let tables = ResourceType ("Microsoft.Storage/storageAccounts/tableServices/tabl
let managementPolicies = ResourceType ("Microsoft.Storage/storageAccounts/managementPolicies", "2019-06-01")
let roleAssignments = ResourceType ("Microsoft.Storage/storageAccounts/providers/roleAssignments", "2018-09-01-preview")

[<RequireQualifiedAccess>]
type NetworkRuleSetBypass =
| None
| AzureServices
| Logging
| Metrics
static member ArmValue = function
| None -> "None"
| AzureServices -> "AzureServices"
| Logging -> "Logging"
| Metrics -> "Metrics"
[<RequireQualifiedAccess>]
type RuleAction =
| Allow
| Deny
member this.ArmValue =
match this with
| Allow -> "Allow"
| Deny -> "Deny"
type VirtualNetworkRule =
{ Subnet : ResourceName
VirtualNetwork : ResourceName
Action : RuleAction }
type IpRuleValue =
| IpRulePrefix of IPAddressCidr
| IpRuleAddress of System.Net.IPAddress
member this.ArmValue =
match this with
| IpRulePrefix (cidr) -> cidr |> IPAddressCidr.format
| IpRuleAddress (address) -> address.ToString()
type IpRule =
{ Value : IpRuleValue
Action : RuleAction }
type NetworkRuleSet =
{ Bypass : Set<NetworkRuleSetBypass>
VirtualNetworkRules : VirtualNetworkRule list
IpRules : IpRule list
DefaultAction : RuleAction }

/// Needed to build subnet resource ids for ACLs.
let private subnets = ResourceType ("Microsoft.Network/virtualNetworks/subnets", "")

type StorageAccount =
{ Name : StorageAccountName
Location : Location
Sku : Sku
Dependencies : ResourceId list
EnableHierarchicalNamespace : bool option
NetworkAcls : NetworkRuleSet option
StaticWebsite : {| IndexPage : string; ErrorPage : string option; ContentPath : string |} option
Tags: Map<string,string>}
interface IArmResource with
Expand Down Expand Up @@ -72,6 +115,20 @@ type StorageAccount =
| Cool -> "Cool"
| _ ->
null
networkAcls = this.NetworkAcls |> Option.map (fun networkRuleSet ->
{| bypass = networkRuleSet.Bypass |> Set.map NetworkRuleSetBypass.ArmValue |> Set.toSeq |> String.concat ","
virtualNetworkRules =
networkRuleSet.VirtualNetworkRules
|> List.map (fun rule ->
{| id = subnets.resourceId(rule.VirtualNetwork, rule.Subnet).Eval()
action=rule.Action.ArmValue |})
ipRules =
networkRuleSet.IpRules
|> List.map (fun rule ->
{| value = rule.Value.ArmValue
action=rule.Action.ArmValue |})
defaultAction = networkRuleSet.DefaultAction.ArmValue |})
|> Option.defaultValue Unchecked.defaultof<_>
|}
|} :> _
interface IPostDeploy with
Expand Down
1 change: 1 addition & 0 deletions src/Farmer/Builders/Builders.Functions.fs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,7 @@ type FunctionsConfig =
Location = location
Sku = Storage.Sku.Standard_LRS
Dependencies = []
NetworkAcls = None
StaticWebsite = None
EnableHierarchicalNamespace = None
Tags = this.Tags }
Expand Down
Loading

0 comments on commit 6d9a265

Please sign in to comment.