Blog

, , , , , , , , , 0

Azure Tips ‘n Tricks – Azure VM monitoring

 Azure Monitoring

Cloud applications are complex with many moving parts. Monitoring provides data to ensure that your application stays up and running in a healthy state. It also helps you to stave off potential problems or troubleshoot past ones. In addition, you can use monitoring data to gain deep insights about your application. That knowledge can help you to improve application performance or maintainability, or automate actions that would otherwise require manual intervention.

The following diagram shows a conceptual view of Azure monitoring, including the type of logs you can collect and what you can do with that data:
Azure monitoring diagram

Enable Azure diagnostics in a virtual machine with the Azure portal

Enabling monitoring on Azure VMs is fairly easy:

  1. In the Azure portal, navigate to the virtual machine, select Optional Configuration, then Diagnostics and set Status to On.Upon completion, the VM has the Azure Diagnostics extension installed and running.
    This extension is responsible for collecting your diagnostics data.
  2. Enable monitoring and configure event logging on an existing VM. You can enable diagnostics at the VM level.
    To enable diagnostics and then configure event logging, perform the following steps:

    1. Select the VM.
    2. Click Monitoring.
    3. Click Diagnostics Settings.
    4. Set the Status to ON.
    5. Configure the storage account you want to use.
    6. Select each diagnostics log that you want to collect.
    7. Click OK.

Azure diagnostics

As soon as this had been set, all the selected data will be captured and stored in the storage account.

Monitoring Use cases:

Visualize

Visualizing your monitoring data in graphics and charts helps you find trends far more quickly than looking through the data itself.

A few visualization methods include: 

    • Use the Azure portal
    • Route data to Azure Application Insights
    • Route data to Microsoft PowerBI
    • Route the data to a third-party visualization tool using either live streaming or by having the tool read from an archive in Azure storage 

Archive

Monitoring data is typically written to Azure storage and kept there until you delete it.

A few ways to use this data:

    • Once written, you can have other tools within or outside of Azure read it and process it.
    • You download the data locally for a local archive or change your retention policy in the cloud to keep data for extended periods of time.
    • You leave the data in Azure storage indefinitely, though you have to pay for Azure storage based on the amount of data you keep. 

Query

You can use the Azure Monitor REST API, cross platform Command-Line Interface (CLI) commands, PowerShell cmdlets, or the .NET SDK to access the data in the system or Azure storage+

Examples include:

    • Getting data for a custom monitoring application you have written
    • Creating custom queries and sending that data to a third-party application. 

Route

You can stream monitoring data to other locations in real time.

Examples include:

    • Send to Application Insights so you can use the visualization tools there.
    • Send to Event Hubs so you can route to third-party tools to perform real-time analysis. 

Automate

You can use monitoring data to trigger alerts or even whole processes. Examples include:

  • Use data to autoscale compute instances up or down based on application load.
  • Send emails when a metric crosses a predetermined threshold.
  • Call a web URL (webhook) to execute an action in a system outside of Azure
  • Start a runbook in Azure automation to perform any variety of tasks

Caveat

While this is pretty amazing what you can achieve with this, there is 1 caveat that you need to be aware of ..

If you have IIS applications on Azure VM’s and these applications store custom logs that need to be available for consulting on the VM itself, make sure you store these outside of the default IIS log path:

IIS manager 

The reason herefor is that the Azure IIS monitoring moves ALL files it finds in this location to the diagnostic storage account!
While you are able to use tools to read out Azure Storage accounts (Azure Storage Explorer), this might not be a wanted situation.

Big thanks to Paul De Jong for pointing this out to me!

Nutanix OMS Monitoring
, , , , , , , 0

OMS monitoring Nutanix “Cloud in a Box”

Intro

As an early adaptor of Operation Management Suite (OMS) I have seen great progress over the last years, what once was a simple SCOM add-on has now grown into a mature product on its own. Recently I noticed that 3rd party providers started to offer prebuilt OMS solution packs and the one that catched my eye was the Nutanix monitoring solution from http://www.comtradesoftware.com

3rd party integration in OMS

3rd party integration

Over the last years I had the pleasure to get to know Nutanix a little better as the partner I’m working with (Synergics) was one of the first Nutanix customers in Belgium and they became partners soon after that. Recently Synergics & Lenovo have partnered up to launch the concept of  ‘Cloud in a Box’ to the Benelux market. (CIAB is a Private Cloud based upon the hyper converged Nutanix platform that has a 2 hour setup, takes 4U in a rack and is almost self maintaining) Reason enough for me as Cloud Solution Architect to see what benefits the solution could offer.
As we wanted to hear from these guys themselves we setup a call to hear their sales pitch:

Prism is greatly designed console and we do not want to compete with it. Instead, we wanted the Nutanix OMS solution to build on key strengths of the OMS platform and to complement Prism. The following are main complement areas:
•Syslog analytics:
Nutanix OMS solution captures syslog’s from all Nutanix servers and makes them available for searching using the OMS search language. In final version, we plan to develop out-of-box queries that will help you identify specific issues only visible via details captures in syslog files.
•Application Awareness:
Nutanix OMS solution goes beyond capturing only VM information.
Application Awareness module looks inside each VM and identifies which supported applications are installed on them.
Then it groups related VMs together, forming an application-centric view of your deployment.
Supported applications today include: Citrix, SQL Server, and Exchange. If you install Citrix on Nutanix, then the Application Awareness will identify VMs that are delivery controllers, license servers, SQL servers for, storefront server, server OS and desktop OS machines, and group them together by the name of the Citrix site they belong to.
Instead of looking at 100s of VMs, now you have an application centric view across your complete deployment hosted on Nutanix.
•Low-level HW monitoring using Redfish API:
we often get questions – how about low-level hardware monitoring?
Since new HW boxes are getting equipped with the Redfish API that is designed to provide API access to low-level HW information, we plan to also capture this data in OMS. That way, you are able to detect fan or power supply failures, etc.

Nutanix OMS solution of course also captures all important metrics that you can see in Prism.
This enables creation of any query to be created across Redfish HW data, host, storage, VM data, and application data + syslog on top. Pre-packages queries and visualizations deliver what we feel will be useful to most customers.
But, on all this vast captured information there is probably one more, a key value-add for service providers such as Synergics, that is:
•Custom OMS dashboards:
based on your experience and knowledge of your customers, you can custom develop dashboards that meet very specific customer needs.

 

Azure Portal

Nuff said, let’s get our hand dirty!

First thing you need to do is to activate the solutions through the Azure MarketPlace, there are 5 different modules available where you can pick from. For this POC we will try them all.

create and link to OMS workspace

create and link to OMS workspace

In the wizard you can select an existing OMS workspace or you could create a new one and even while there is no real interaction with the Azure Automation Account, you need to select one. Same here, if you don’t have a AAA already you can create one and as you can see in the image above, it cannot be linked to any other OMS workspaces. Repeat the process for the remaining modules.

Nutanix OMS Collector

Moving over to the Nutanix OMS Collector installation

Nutanix OMS Collector

Nutanix OMS Collector

Requirements

For this component you will need to provision a CentOS or RedHat Linux server, this server can either be virtual or physical.
The collector should be able to talk with OMS over the internet and with PRISM over port 9440
it should also accept incoming traffic from the Nutanix hosts on port 8080, 8443 and 54100.

Download

Now download the package and the license (sent by mail) to your collector

download package

download package

Install

open the command shell and run following:

rpm -i ComtradeNutanixOMS-<Version>.x86_64.rpm

open internet browser and browse to the local server

https://computer.company.com:8443

Add the Nutanix Prism DNS address

add datasource

add datasource(s)

enter workspace ID and primary key

enter workspace ID and primary key

and add the license file

license file

add license file

Nutanix Syslog forwarding

open a command shell and SSH connect to one of the CVM’s on the Nutanix cluster

type following commands

ncli

rsyslog-config set-status enable=false

rsyslog-config add-server name=<HostName> ip-address=<IPaddress> port=54100 network-protocol=UDP
<Hostname> is the name of the collector server and the <IPaddress>, is … well guess)

rsyslog-config add-module server-name=<HostName> module-name=<ModuleName> level=<Level> include-monitor-logs=true

<HostName> is fully qualified domain name of the system where Nutanix OMS Data Collector is installed;

<ModuleName> is a Nutanix module name. Available modules are: cassandra, cerebro, curator, genesis, prism, stargate, zookeeper;

<Level> is one of the following severity levels: INFO, WARNING, ERROR, FATAL; the specified level defines the minimum severity level that is still included in data forwarding.

Example:

rsyslog-config add-server name=computer.company.com ip-address=10.17.100.100 port=54100 network-protocol=UDPrsyslog-config add-module server-name=computer.company.com module-name=cassandra level=WARNING include-monitor-logs=truersyslog-config add-module server-name=computer.company.com module-name=cerebro level=WARNING include-monitor-logs=truersyslog-config add-module server-name=computer.company.com module-name=curator level=WARNING include-monitor-logs=truersyslog-config add-module server-name=computer.company.com module-name=genesis level=WARNING include-monitor-logs=truersyslog-config add-module server-name=computer.company.com module-name=prism

Results in OMS

After a while you will see data in the OMS workspace:

OMS results

OMS results

Each module also offers a drilldown

drilldown

drill down

 

Notes:

You can host the OMS collector where you want, on the Nutanix cluster itself as VM, in Azure as VM or on premises as a physical server, just make sure all communication can take place.

It does not matter what Hypervisor you are running on Nutanix, OMS will report on all

If you host the VM in Azure, you will have a server without GUI and getting one activated, well, it takes some puzzling

The latest release of the application will update itself when needed.

Conclusion:

The solution starts to give results as soon as everything is configured correctly, the things you can do with it are still limited in this phase but these guys are doing a good job and the work progresses fast. As company or IT PRO, its nice to have a view on your environment at your fingertips (OMS mobile app) and available over the internet with all the other benefits OMS can provide. Hope to see a version 1.0 soon with reasonable pricing.

ASM to ARM
, , , , , , , , , , 0

ASM to ARM with ASR – Part VIII Final fixes

Welcome to part 8, the last and final blog in the ASM to ARM with ASR series 🙂

Now that we have our VM’s up and running in ARM, there are still some things to configure.
Some main items still need our direct attention and that’s what will be covered in this blog:
Public IP, Load Balancers and Availability Sets
When it comes down to load balancers, VMs deployed within a cloud service boundary can be grouped to use a load balancer.
In the ASM model a public IP address and a FQDN are assigned to the cloud service itself.
The load balancer does port translation and load balances the network traffic by using the public IP address for the cloud service.

In the Resource Manager deployment model there is no such thing as a Cloud service, the load balancer is created to route traffic among multiple virtual machines. A public IP address is an individual resource that has a domain label (DNS name) and the public IP address is associated with the load balancer resource. Load balancer rules and inbound NAT rules use the public IP address as the Internet endpoint for the resources that are receiving load-balanced network traffic.

Besides the Load Balancers, we also need to put the Web VMs in an availability set if we want to make use of fault tolerance.
It would be a great addition if you could specify this in the ASR replication policy, as you will see that we need to delete and recreate the VMs to make this happen.

Thanks to the comnunity and in this case especially SAMIR FARHAT for his script that does this for us: https://gallery.technet.microsoft.com/Set-Azure-Resource-Manager-f7509ec4/view/Discussions#content
This script will guide you through the whole process, the only thing you might want to do is to create the Availability Set that you want to use:

Starting the script:

Checking if the WebVMs show that they are member of the Availability Set that we specified:

and they are, moving on to the next step: creating the Load Balancer with it’s associated Pubic IP.
I prefer to create the Public IP first (you are able to do this while creating the Load Balancer but then you lack the options to specify FQDN)

create public ip

Give your resource a name, select ‘Static’ mode for your IP and assign a DNS name.

Create the Load Balancer

create loadbalancer

create loadbalancer

give the resource a name, make sure the type is set to Public and select the Public IP you previously created

create backend pool

click add

create backend pool

assign a name to the pool

create backend pool

select the Availability Set and Web VM’s and click OK on all the steps

Before we can add the load balancing rule for HTTP, we first need to configure a health probe.
For this I select the HTTP probe as this one also checks if the website is healthy (TCP probe just checks if ping requests get answered)

create health probe

create HTTP health probe

Now we can move over to adding the HTTP rule to the Load Balancer

create load balancer rule

click add

create load balancer rule

create the rule to balance incoming HTTP traffic on port 80

Now lets take the DNS name of the public IP assigned to the load balancer and paste it in a browser

verify web connectivity

WebVM2 is responding

verify web connectivity

and so is WebVM1

Hope you have enjoyed the walk through!

Some items that I encountered during this POC but what’s not published:

  • There was a difference in the naming of the destination VM’s so I also had to change the SQL connection string in the ‘web.config’ file
  • The SQL VM had a virtual disk (Storage Spaces) that I needed to recreate on the destination VM

In a production environment it will make more sense to make use of the SQL application replication mechanism, if that’s not possible then make sure you have a working SQL backup that you can restore on the target VM.

Previous blogs from this serie

part 1: create the ASM resources
part 2: create the recovery vault  
part 3: Network Configuration  
part 4: Mobility Agent installation 
part 5: Prepare failover infrastructure 
part 6: create the recovery plan 
part 7: Planned and Unplanned failover 

ASM to ARM
, , , , , , , , , 1

ASM to ARM with ASR – Part VII Performing Failover

Welcome to part 7 of the ASM to ARM with Azure Site Recovery.

In this blog we will do the actual failover,
before we start it’s advisable to check what VM sizes ASR will provide for your target VMs, in my case the DS2V2 was resized to A5:

Test Failover

Running Test Failovers allow to do some pre checking without interupting the production environment and gives you good indication on potential issues.

Test failovers consume resources, so you should clean up the test failover environments if you don’t want large Azure bills at the end of the month 😉

and confirm you want to delete

Unplanned failover

Now instead of selecting test failover plan, we choose unplanned failover were the old environment should be brought down and start the ARM VMs.

 

Now hold on to your seat, cross your fingers and follow the progress in the portal:

If you configured everything well, you should see that everything is now running in the ARM model:

(don’t mind the shutdown problem, this behaviour has another root cause that is out of scope of this topic)

Checking the new resource group shows that all components are in place 🙂

There are still a few things we need to do here, and that will be discussed in the next and last part of this blog series.
part 8: Final fixes

Previous blogs in this serie:

part 1: create the ASM resources
part 2: create the recovery vault  
part 3: Network Configuration  
part 4: Mobility Agent installation 
part 5: Prepare failover infrastructure 
part 6: create the recovery plan 

 

 

ASM to ARM
, , , , , , , 0

ASM to ARM with ASR – Part VI Create the Recovery Plan

Welcome to part 6 on migrating ASM resources to ARM with Azure Site Recovery.
In this blog we will guide you through the process of creating the ASR Recovery Plan and it’s replication settings.

Step 1: Prepare the Infrastructure

To failover from Azure to Azure, you will need to select the ‘physical server/VMware’ scenario to make this work

Select your process server

verify if your newly created storage account and VNET is ready to be used. Most important is to make sure you select the ‘Resource Manager’ deployment option

Specify RPO, recovery points and application consistent snapshots

The platform now links the plans to the process server

verification on the capacity planning

Step 2: Create Replication Policy

In here we need to specify in which subnet our VMs will be replicated to, so we will do this twice as we want our SQL server to land on the data subnet.

skipping SQL server for now

doing the same for the SQL Server, but now we select the data subnet

3: Create the recovery plan

If you want to check if things are A OK:

In the following blog we will test the failover 🙂

part 7: Planned and Unplanned failover 

Previous blogs in this serie:

part 1: create the ASM resources
part 2: create the recovery vault  
part 3: Network Configuration  
part 4: Mobility Agent installation 
part 5: Prepare failover infrastructure 

ASM to ARM
, , , , , , , 1

ASM to ARM with ASR – Part IV Mobility Agent

Welcome back to part 4,
in this blog we will cover how to install the mobility service agent on our source VMs.
The installation process can be done manually or be pushed from the process server to Windows/Linux VMs.
To show what happens behind the curtins we will focus on the manual approach.
RDP to your process server and navigate to following file path:
C:\Program Files (x86)\Microsoft Azure Site Recovery\home\svsystems\pushinstallsvc\repository

location mobility agent

location mobility agent

mobility agent wizard step 1

select install mobility service

mobility agent wizard step 2

Process Server Info

in case you have lost your passphrase, run following command to display the key:

mobility agent wizard step 3

installation path

mobility agent wizard step 4

last step, installing the software

In the following part we will have a look at creating the prerequisites for the fail-over location.

part 5: Prepare failover infrastructure 

Previous blogs in this serie:

Part 1: Creating ASM resources through ARM portal  
Part 2:  Create ARS Vault and configure process server
Part 3: Create VNET connectivity

ASM to ARM
, , , , , 0

ASM to ARM with ASR – Part III Network configuration

Welcome to part 3 on migrating Azure resources from ASM to ARM with the help of Azure Site Recovery.
In this part we will go over 2 options on how to connect the ASM network to the ARM network.
This is needed as we want our source VMs to communicate with the process server!

VNET Peering
this feature is currently under preview (Let’s not forget that preview also means FREE 😉 ) and is by far the most simple way to connect VNETs.
With VNET Peering you can also connect ASM VNET to ARM VNET without the use of VNET gateway site to site VPN connections.
Today this feature is limited to connect VNETs if they are:
– in the same region (might change later, but today a VNET’s boundary is the region it was created in)
– subnets IP ranges don’t overlap
– for connecting ASM to ARM networks, VNETS must be in same subscription
You can peer networks in different subscriptions if:
– the privileged user must have rights in both subscriptions AND
– both subscriptions are connected to the same Azure Active Directory.
Create peering takes less than 5 minutes:

almost instantly:

The other option requires a little more configuration as we need to create a Site to Site VPN Connection between the 2 VNET gateways in different deployment models and that requires some PowerShell to make this work.
You will need to select this option if:
– you need to connect networks in different regions
– you need to connect to network in another subscription that is not linked to the same Azure AD
The whole process is already very nicely documented and you can find it here: https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-connect-different-deployment-models-portal

In the following chapter we will talk about installing the mobility agent on our source servers

part 4: Mobility Agent installation 

In case you landed on this page, you can find the other topics here:
Part 1: Creating ASM resources through ARM portal  
Part 2:  Create ARS Vault and configure process server

 

Load More