Showing posts with label Cloud. Show all posts
Showing posts with label Cloud. Show all posts

Getting Familiar with Microsoft Azure

I'd like to summarize what I've learned in the past couple years of using MS Azure for personal and professional software development. Keep in mind this is coming from the perspective of a developer; Azure can be used for many interesting things outside the scope of just deploying, hosting, and scaling software in the cloud.


The Azure Portal UI is intuitive, constantly being updated (for the better), and contains tools to create and configure nearly anything you can imagine  


First, it's a bit of a maze.

Then it's amazing.


Starting out

The idea of any cloud provider is to enable IaaS, SaaS and PaaS among other XaaS's. Instead of having to provision physical machines, network equipment and associated hardware, and go out to dozens of different vendors to manage service agreements for the various services a company uses- nowadays a company can move most of that distributed mess into their own private cloud and just manage everything in one place.

And that one place is highly secure, geo-redundant and hosted on some of the best and newest hardware available.





Azure resources

In Azure you have the concept of resources which consume resource units. Anything can be a resource: a network card, a virtual machine, a firewall security policy- they are all resources in the world of Azure. You can create, modify and delete resources virtually at will- or on a schedule through automation scripts that operate on what are known as ARM (Azure Resource Manager) templates which are basically representations of Azure resources in the form of JSON.

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "adminUsername": {
      "type": "string",
      "metadata": {
        "description": "Username for the Virtual Machine."
      }
    },
    "adminPassword": {
      "type": "securestring",
      "minLength": 12,
      "metadata": {
        "description": "Password for the Virtual Machine."
      }
    },
    "dnsLabelPrefix": {
      "type": "string",
      "defaultValue": "[toLower(concat(parameters('vmName'),'-', uniqueString(resourceGroup().id, parameters('vmName'))))]",
      "metadata": {
        "description": "Unique DNS Name for the Public IP used to access the Virtual Machine."
      }
    },
    "publicIpName": {
      "type": "string",
      "defaultValue": "myPublicIP",
      "metadata": {
        "description": "Name for the Public IP used to access the Virtual Machine."
      }
    },
    "publicIPAllocationMethod": {
      "type": "string",
      "defaultValue": "Dynamic",
      "allowedValues": [
        "Dynamic",
        "Static"
      ],
      "metadata": {
        "description": "Allocation method for the Public IP used to access the Virtual Machine."
      }
    },
    "publicIpSku": {
      "type": "string",
      "defaultValue": "Basic",
      "allowedValues": [
        "Basic",
        "Standard"
      ],
      "metadata": {
        "description": "SKU for the Public IP used to access the Virtual Machine."
      }
    },

    "OSVersion": {
      "type": "string",
      "defaultValue": "2019-Datacenter",
      "allowedValues": [
        "2008-R2-SP1",
        "2012-Datacenter",
        "2012-R2-Datacenter",
        "2016-Nano-Server",
        "2016-Datacenter-with-Containers",
        "2016-Datacenter",
        "2019-Datacenter",
        "2019-Datacenter-Core",
        "2019-Datacenter-Core-smalldisk",
        "2019-Datacenter-Core-with-Containers",
        "2019-Datacenter-Core-with-Containers-smalldisk",
        "2019-Datacenter-smalldisk",
        "2019-Datacenter-with-Containers",
        "2019-Datacenter-with-Containers-smalldisk"
      ],
      "metadata": {
        "description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version."
      }
    },
    "vmSize": {
      "type": "string",
      "defaultValue": "Standard_D2_v3",
      "metadata": {
        "description": "Size of the virtual machine."
      }
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    },
    "vmName": {
      "type": "string",
      "defaultValue": "simple-vm",
      "metadata": {
        "description": "Location for all resources."
      }
    }
  },
  "variables": {
    "storageAccountName": "[concat('bootdiags', uniquestring(resourceGroup().id))]",
    "nicName": "myVMNic",
    "addressPrefix": "10.0.0.0/16",
    "subnetName": "Subnet",
    "subnetPrefix": "10.0.0.0/24",
    "virtualNetworkName": "MyVNET",
    "subnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]",
    "networkSecurityGroupName": "default-NSG"
  },
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "apiVersion": "2019-06-01",
      "name": "[variables('storageAccountName')]",
      "location": "[parameters('location')]",
      "sku": {
        "name": "Standard_LRS"
      },
      "kind": "Storage",
      "properties": {}
    },
    {
      "type": "Microsoft.Network/publicIPAddresses",
      "apiVersion": "2020-06-01",
      "name": "[parameters('publicIPName')]",
      "location": "[parameters('location')]",
      "sku": {
        "name": "[parameters('publicIpSku')]"
      },
      "properties": {
        "publicIPAllocationMethod": "[parameters('publicIPAllocationMethod')]",
        "dnsSettings": {
          "domainNameLabel": "[parameters('dnsLabelPrefix')]"
        }
      }
    },
    {
      "type": "Microsoft.Network/networkSecurityGroups",
      "apiVersion": "2020-06-01",
      "name": "[variables('networkSecurityGroupName')]",
      "location": "[parameters('location')]",
      "properties": {
        "securityRules": [
          {
            "name": "default-allow-3389",
            "properties": {
              "priority": 1000,
              "access": "Allow",
              "direction": "Inbound",
              "destinationPortRange": "3389",
              "protocol": "Tcp",
              "sourcePortRange": "*",
              "sourceAddressPrefix": "*",
              "destinationAddressPrefix": "*"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/virtualNetworks",
      "apiVersion": "2020-06-01",
      "name": "[variables('virtualNetworkName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"
      ],
      "properties": {
        "addressSpace": {
          "addressPrefixes": [
            "[variables('addressPrefix')]"
          ]
        },
        "subnets": [
          {
            "name": "[variables('subnetName')]",
            "properties": {
              "addressPrefix": "[variables('subnetPrefix')]",
              "networkSecurityGroup": {
                "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"
              }
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/networkInterfaces",
      "apiVersion": "2020-06-01",
      "name": "[variables('nicName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPName'))]",
        "[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]"
      ],
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "privateIPAllocationMethod": "Dynamic",
              "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPName'))]"
              },
              "subnet": {
                "id": "[variables('subnetRef')]"
              }
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines",
      "apiVersion": "2018-10-01",
      "name": "[parameters('vmName')]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]",
        "[resourceId('Microsoft.Network/networkInterfaces', variables('nicName'))]"
      ],
      "properties": {
        "hardwareProfile": {
          "vmSize": "[parameters('vmSize')]"
        },
        "osProfile": {
          "computerName": "[parameters('vmName')]",
          "adminUsername": "[parameters('adminUsername')]",
          "adminPassword": "[parameters('adminPassword')]"
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "MicrosoftWindowsServer",
            "offer": "WindowsServer",
            "sku": "[parameters('OSVersion')]",
            "version": "latest"
          },
          "osDisk": {
            "createOption": "FromImage",
            "managedDisk": {
              "storageAccountType": "StandardSSD_LRS"
            }
          },
          "dataDisks": [
            {
              "diskSizeGB": 1023,
              "lun": 0,
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('nicName'))]"
            }
          ]
        },
        "diagnosticsProfile": {
          "bootDiagnostics": {
            "enabled": true,
            "storageUri": "[reference(resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))).primaryEndpoints.blob]"
          }
        }
      }
    }
  ],
  "outputs": {
    "hostname": {
      "type": "string",
      "value": "[reference(parameters('publicIPName')).dnsSettings.fqdn]"
    }
  }
}
An example of an ARM template- this one is for deploying/updating a Windows Server VM resource


For a trial period of (currently 12 months) most all of the really useful stuff is free (be careful not to accidentally deploy Azure Co$mos though...😳 ...that is not free, and that is not cheap). After the trial period, the cost was still relatively cheap for the services that I use most in Azure (an App Service hosting a handful of .NET Core apps with SSL, 1 powerful virtual machine, a DNS zone, a vNet)- all for about $30/month.


Development

For development, much like it is with Git, the Visual Studio integration with Azure is pretty seamless and enables deployments directly from the IDE. You can also enable an Azure object explorer to view your Azure cloud instance' resources within VS.

Most all established companies are going to want to- for security reasons- (or will have to for incompatibility reasons)- keep at least some legacy software and/or infrastructure on-prem.


Azure ARC connects your On-Prem to your Cloud



And that is why there is Azure ARC- an incredibly simple way to bridge cloud and on-prem resources to create a hybrid virtual network. ARC is essentially a service that you run on your on-prem machines that connects them to your Azure subscription where the machines can then be configured as if they were an Azure resource and can enable on-prem devices to communicate with cloud resources.

"Arc works by running an agent on your non-azure resources; this is a service on VM's and a Kubernetes pod on Kubernetes cluster. Once you install this service, the machine registers with Azure and is ready for management." -samcogan.com

Additionally, virtual machines (Windows OSs or approved Linux distros) can be accessed via SSH or RDP and are as amazingly fast or as tortuously slow as you configure them to be. You can choose from preconfigured database server or application server templates or build your virtual machines completely à la carte.

The ARM template paradigm is easy to understand and develop with, and there are 2 CLI options- Azure CLI and the new PowerShell "Az" module.


How to see "gains"

To see savings from using the cloud, instead of purchasing a new server or physical license, you can rent the computing power you need to power your apps and services and you can even move your worker machines onto the cloud where they can be more easily managed (we are indeed moving back to a thin client/dumb terminal world).

You can move from the physical Exchange mail server model to Outlook365. You can move all of your physical Office subscriptions to Office365.

If your computing needs are seasonal or time-sensitive, you can scale up when needed and pay a high price for short bursts of computing power, while scaling back down to a much lower-budget level until the next scale-up need arises. The configuration of the usage of resources in Azure is highly granular and lends itself to squeezing out a lot of efficiency for those who can monitor and manage it correctly in accordance with organizational needs.

Azure Hybrid Benefit also provides credits for customers who already have an on-prem SQL Server software license. Want to see that MSSQL2019 Enterprise for Linux instance in the cloud? 🙂


Monitor usage

Monitor your cloud resource usage as you can inadvertently requisition a resource that behaves in ways you did not expect and in turn end up ringing up a lot of expensive RUs (resource units). Azure allows you to configure a budget and alerts when you have reached certain thresholds toward or beyond the budget number so that you can configure an alert which will email you a warning message if you have reached 105% of your monthly budget, for example.


The current Azure offerings are plentiful and powerful enough to outfit even the most complex IT infrastructure

Azure, like any cloud provider, forces you to take a fine-grain look at every single resource you are using. It is amazing how much stuff we don't actually use.

It is only when you begin to pay for usage of each resource and see the numbers rising daily do you really understand how much you are utilizing your various resources.

Powerful computing machines began as a timeshare because of the realization that it is madness to let expensive machines sit idle. And though the resources to share and provision among users has become far more complex, we are returning to that model.


Conclusion

Whether you use Azure to explore different kinds of technology or to implement an IT infrastructure completely in the cloud to connect and supercharge your applications and/or workforce- the tech is now there and the costs are comparable to AWS.

My two criticisms of Azure are that (1) Azure seems to excessively spotlight/push certain features forward and other features (many practical, free and cheap things you would think are "essential"- like setting up DNS zones) remain sort of in the shadows/awaiting help links waiting to be discovered... And (2), sadly other things, little things like logging analytic insights that you would think are free are in fact Azure resources that charge RUs. 😕

These aspects suck but are tolerable in light of all of the awesome functionality Azure provides.

Microsoft continues to improve an industry-leading cloud platform that executives, management, engineers, developers, and system admins alike can all learn to love. 💖

PowerShell Commands

The origins of PowerShell lie in the Monad project which you can learn about here: https://www.jsnover.com/Docs/MonadManifesto.pdf


PowerShell Base CmdLets and associated Pipeline parsing CmdLets  provide powerful Windows administration tools



These can be used on-the-fly to glean and share information about a problem or the state of your machine(s) and network or they can be crafted into useful scripts that run on a schedule to report on the status of applications and services, run backup and ETL tasks as well as myriad other (often critical) scheduled jobs that happen routinely behind the scenes to keep IT operations organized and running.
You may for instance, want to have a script run every few hours that gathers statistics about throughput and storage and then alert admin users if a certain threshold is exceeded. Or, in Azure, you may want to utilize a scripted template (like Azure ARM and associated CLI commands) to configure new Azure resources and their environments.


Useful cmdlets:

#check security level
Get-ExecutionPolicy

#elevate security access level
Set-ExecutionPolicy Unrestricted

#get information on any service
Get-Service -Name PhoneSvc

#get the same log info seen in eventvwr
Get-EventLog -Log "Application" 

#get process information
Get-Process -ComputerName MYCOMPUTER

#stop process like cmd.exe kill
Stop-Process -Name “notepad”

#get drive information of the drives connected to the current PS session
Get-PSDrive 

#get information on any powershell cmdlt, function or module
Get-Help -Name Streaming

#get all the installed commands
Get-Command

#connect to your azure account with the "az" azure cmdlt Connect-AzureRmAccount #upload blob content to storage Set-AzStorageBlobContent -File "D:\_TestImages\Image002.png" ` -Container $containerName ` -Blob "Image002.png" ` -Context $ctx -StandardBlobTier Cool #download blob content from storage Get-AzStorageBlobContent -Blob "Image002.png" ` -Container $containerName ` -Destination "D:\_TestImages\Downloads\" ` -Context $ctx

#stop a sql server instance
Stop-SqlInstance -ServerInstance MSSQL01

#clear screen
Clear-Host

#ping
Test-NetConnection

#telnet
Test-NetConnection -Port

#tracert
Test-NetConnection -TraceRoute

#ipconfig
Get-NetIPAddress

#nslookup
Resolve-DnsName -Name "Hostname"

#netstat
Get-NetTCPConnection

#flushdns
Clear-DnsClientCache

#ip release/renew
Invoke-Command -ComputerName -ScriptBlock {ipconfig /release} Invoke-Command -ComputerName -ScriptBlock {ipconfig /renew}

#disable/enable network card
Invoke-Command -ComputerName -ScriptBlock {ipconfig /release} Invoke-Command -ComputerName -ScriptBlock {ipconfig /renew}



Additionally, it is often useful to implement piping of command output, especially in CI/CD toolchain scripting where scripts feed their output (which become the next script's argument(s)) to the next script in the chain.


For example: 

#export a list to .csv file
Get-Service | Export-CSV c:\20200912_ServiceSnapshot.csv

#be more selective with Select-Object module and pipe that to the csv
Get-Service | Select-Object Name, Status | Export-CSV c:\20200912_ServiceStatusSnapshot.csv

#get event information and pipe method (of each log event) info to the console
Get-EventLog -log system | gm -MemberType Methods

#get a process and stop it
Get-Process notepad | Stop-Process

#delete all files matching some Regex pattern
Get-ChildItem $Path | Where{$_.Name -Match "someFileName.txt"} | Remove-Item



References: