Saturday, January 19, 2019

Targeted Alerts Using Azure Event Grid

I came across something interesting the other day while trying to refine a governance strategy for Azure at an enterprise level.  The specific scenario is centered around an organization that has a very large Azure estate with many virtual machines as well as a wide swath of platform-as-a-service components.  We had seen some upticks in password resets coming from the Azure portal for VMs, and didn't discover this until we decided to go in and audit some activities.  When looking through the Activity Log, these types of activities aren't immediately apparent as they display as deployments instead of password reset events.  After digging through the event metadata, I was able to locate a common convention for these events--under the authorization section of the event details, the scope value will end with "PasswordReset" and the timestamp it was initiated.
This got me would likely be good to capture this and alert operations if anyone is using the Azure portal to reset VM credentials.  Typically the account that is reset using the portal will be an administrative non-domain user, and multiple resets may indicate a larger issue.   But what is the best way to alert on this activity?
Using Log Analytics is one approach to solve this problem.  To start with, you will need to add the Activity Log to the list of data sources for the Log Analytics workspace.  Depending on how large the subscription is, and how active people are in that subscription, this can mean a large amount of data.  When using a per-gigabyte instance of LA, you may end up seeing a large usage charge related to importing all of this data.  Getting the information using a LA query is not terribly difficult:

AzureActivity | where Resource contains 'PasswordReset'

From there, you can wire up an Azure Alert to fire when this condition is met.  Again, this operates on the premise that you have imported all of the Activity Log data into your LA workspace.
Another option, one that I found to be more interesting, is to use Azure Event Grid to listen for specific messages and fire when the intended message is found.  To set up an Event Grid subscription, you can use the Azure portal, Azure PowerShell, or Azure CLI.  One caveat here is that the ability to use complex filters on event subjects is not supported currently when using PS or CLI.  You can use a "starts with" or "ends with" filter, but nothing that captures something akin to "contains".  The use case I'm after here looks to use a wildcard operator to find the phrase that pays within the event subject, in this case "PasswordReset".  The only way to create this type of filter currently is to use the portal.  During creation you can specify the type of action to take when the event subscription picks a winner:
  • Web Hook
  • Azure Storage Queues
  • Event Hub
  • Hybrid Connections (Azure Automation)
Before we rush to get the event subscription created, let's take a moment to think through the mechanism that will send the alert.  Using a serverless component such as a Logic App or Azure Function could be an easy way to tee up an email to the ops team.  For the purposes of this example, using a serverless component will rule out the use of the Hybrid Connection endpoint.  While actions can be invoked from Storage Queues, I want to explore using an Event Hub as the recipient of these messages.  Both Logic Apps and Functions have triggers for receiving events from Event Hubs, and setting up a new Logic App to handle these events should be fairly straightforward.  Why Event Hubs, though?  My thought here is that using Event Hubs will allow me to send multiple types of messages to a Hub, and with low retention settings as well as lower per-transaction cost, it seems like something I should at least try out, right?
Let's start by creating the Event Hub namespace, along with a hub.  The simplest way to do this is to use the Azure CLI to create the namespace, then the hub.

az eventhubs namespace create -g rg-shared-services -n governance
az eventhubs eventhub create -g rg-shared-services --namespace-name governance -n auditalerts

You can set the message retention here as well using the --message-retention flag, as the default setting is 7 days.

Once the namespace and hub are created, we are ready to set up an Event Grid subscription that will push specific events to the hub.  In the Azure Portal, go to All Services -> Event Grid Subscriptions.  The screen below will display.

To create a new subscription, click the "Create one" button if you have no subscriptions already, or click the "+ Event Subscription" button in the command bar.  The next blade that appears will have basic information.  Enter in the information as seen below.  Keep in mind that you will need to select Subscription Events from the dropdown for Topic Details, and Event Hubs for the Endpoint Details section.  You will be able to specify the auditalerts hub once you select the appropriate Event Hub Namespace.

Next, click on the Filters tab.  This will allow you to set filters on the event subject in a basic way (starts with / ends with) or in an advanced manner.  We are after the advanced filter since we intend to use a wildcard search term.  Click on "Add new filter" to add a new advanced filter.

 In the Key field, type in "subject".  For the Operator field, select "String contains" from the drop down.  Then, click on the area that says "Add new value" to enter the filter expression.

In the text box that appears, type in "PasswordReset".

Click "Create" to create the subscription.  This will now send a message to your Event Hub whenever a password reset happens via the portal.  To finish the job, you need to create a Logic App that will listen for activity on the hub and send an email when it receives an event.  You can create a new Logic App in the Azure Portal directly, or by using Visual Studio with the Azure Logic App Tools for Visual Studio extension.  Once you have created a new Logic App, search for Event Hubs in the trigger section and add the "When events are available to Event Hub" trigger. 

You can choose the polling interval for whatever you like, and it will default to 5 minutes.  For extra granularity, you can add a Parse JSON step and parse the event.  To get the schema for an event, copy any activity from the Activity Log and enter that into the popup you receive when clicking on "Generate schema from sample payload" on the expanded JSON task.  After this, you can add a task for Office365, Exchange, SendGrid, or any other email provider you wish and add the details from the event to the body of the email.  Include any email addresses (individuals or distribution lists) that need to be informed. 
To test, use a test VM in the subscription where the Event Grid subscription was set to listen.  Reset the password for the VM using the Portal and wait for the email to be sent out.  It will not fire immediately, so do not be alarmed if you aren't immediately emailed.

That's all there is to it.  You can expand this to include routing to Event Hub consumer groups or even use Event Grid Domains to further expand your reach.  Happy eventing!

Monday, December 10, 2018

My book is live!

I'm excited to announce that my new book, Migrating to Azure, has officially launched!  Copies can be found at select bookstores and always on  This is the reason I've not been very active on this site for a while.  Enjoy!

Saturday, September 10, 2016

Require is required but it can't find require.d.ts

I've just released a fairly significant update to the VSTS Extension Project Templates extension.  Recently a few of the ALM Rangers noticed that there was a problem with a missing TypeScript definition file that seemed to crop up out of nowhere.  The file (require.d.ts) was not in the template's file system previously as it had not been needed by the underlying VSS SDK definition files (VSS.d.ts and TFS.d.ts).  A recent change, however, has revealed that a new dependency was introduced:
To remedy this, I added the DefinitelyTyped d.ts file for RequireJS.  Many thanks to Wouter de Kort and Utkarsh Shigihalli for reporting this, and to David Corbin for helping me beta test this fix!

Now, this release wasn't just a hotfix.  There were several other enhancements and updates that made it in, thanks to suggestions from my friend Max Knor:
  • Changed PowerShell execution to PowerShell3 in task.json
  • Included vsts-task-sdk in Build/Release Task template
  • Added Trace-VstsEnteringInvocation and Trace-VstsExitingInvocation to example task PowerShell scripts
  • Updated versions for Grunt, Jasmine 
For some great contributions to the VSTS extension pool, check out Max's extensions on the VS Marketplace:

Patching previous versions
If you've run into a roadblock with running or building your existing extension project, the easiest way to get back on track is to create a folder under the typings folder in the project for requirejs, and in that folder place a copy of the require.d.ts file:

Friday, April 29, 2016

Dashboard Widgets and Charting

Recently I was asked a question by someone in the community around how to properly construct a dashboard widget that contained custom charting/graphing components.  Specifically, the high-level functional requirements centered around:

  • Using TypeScript to manage the code versus inline JS in the widget's HTML
  • Using a charting library like Chart.js or HighCharts
The choice was made by the inquiring party to look into using HighCharts.  Now as a caveat, please keep in mind that using HighCharts for personal use is completely free.  If you plan on publishing a widget to the Marketplace, however, you will need to purchase the appropriate license to ensure you are in compliance.  Licensing in general can be a complex topic and is well beyond the scope of this post.
The idea behind this example is to make sure that someone with little experience can adequately use the VSTS Extension Project Template to: 
  • create a new extension
  • add a dashboard widget to that project
  • add the appropriate libraries, and 
  • wire up the widget to consume what it needs
Let's start with the basics, after creating a new extension project.  

Right-click on the project and go to Add -> New Item.  We're going to add a new custom dashboard widget.
Name the widget whatever you like.  For the sake of this example, I have named it "SampleChartWidget".  The item template will add a generic image and generic HTML file underneath a folder named SampleChartWidget.  Let's go ahead and right-click on that new folder and add a new TypeScript file, which we'll call SampleChartWidget.ts for the sake of simplicity.
Now that we have the basic skeleton set up, let's update a couple of items that we'll need to make HighCharts work.  First stop will be the package.json file, where we will need to add a reference to "highcharts": "^4.2.4" in order to pull in the latest JS file the next time the project is built.  Once that's been saved, the next stop will be the Grunt file.  There are two things in here that will need to be altered.
One will be the task that performs the typings initialization (exec:tsdinit), which will require running that task after the file has been saved:

The other will be the task that copies the needed library file to the scripts folder (copy:main):
As a reminder, the Grunt file is already set up to run the npm update, copy step, and package step anytime you issue a build command in Visual Studio.

Once that editing is complete, build the project to pull in the newly added dependencies.  This will pull down the HighCharts library from npm, copy the HighCharts.js file to your scripts folder, and pull down the typings file for HighCharts as well.  We still need to make sure they are a part of the project, so don't forget to include them in the project by clicking on "Show all Files":
This will show you the newly copied JS file in your scripts directory (above) as well as the typings folder that you will need for constructing the TypeScript file:
Now that we have our prerequisites, we can move on to wiring up the chart.  Let's first start by adding some meat to our widget's TypeScript file.  We will want to add in two reference lines at the top, one to pull in the typings referenced in the tsd.d.ts file (consolidated links for the VSS, TFS and RMO typings supplied by the vss-web-extension-sdk) and the other for the typing file we just downloaded for HighCharts.  Once that has been added, a class declaration can be added for a very simple widget class, with one method for initializing the chart.  Next, we will add a variable to the exports section that will hold a new instance of our SampleChartWidget class, making it ready to be used by the VSS SDK.  Finally, we can dump in some simple initialization code to turn a named element into a chart.  Sample code is courtesy of the HighCharts JSFiddle.
Having completed the working bits, we can now move on to setting up the HTML that will display this new chart.  We need to wire up the section that registers the widget, allowing for the new TypeScript file to be injected via Require.  We also need to include an element in the body that will act as the target for HighCharts to render the chart to.  Once that's complete, ensure that the references to VSS.SDK.js and highcharts.js are properly marked in your HTML file.  I've optionally added the rel to a central stylesheet.

A couple of additional caveats to pay attention is the sizing of the widget itself.  Some graphs or charts may require a specific amount of screen real estate.  Being mindful of how you're rendering the graph is important as you have a maximum of four rows and four columns on any dashboard.  I chose to use a 4x3 for this example, but this can be adjusted based on need.  Resizing the widget will mean needing to resize the container element as well, so extra care and testing needs to go into that prior to releasing anything into the wild.  Another thing to be mindful of is the project's TypeScript build settings.  In order to have this solution build and create the right JS, you will need to make sure that the build setting is pointed at AMD.

Again, this is an example and is not meant to be taken as anything but an illustrative way to begin hooking up charting and graphics libraries in dashboard widgets.  Please be mindful of any licensing and/or component performance concerns while developing.  Official documentation from Microsoft in relation to authoring widgets with charts and graphs will be forthcoming, so stay tuned and watch the space.

For those interested, source code can be found here.  

Tuesday, April 19, 2016

Version 2.4.0 of the VSTS Extension Project Template released

I've just released version 2.4.0 of the VSTS Extension Project Template extension for Visual Studio.  This release includes a new Item Template for the Dashboard Widget.  Special thanks to Utkarsh Shigihalli for the suggestion!  The template itself is quite basic but will get you moving in the right direction.  For more information on dashboard widget development, please refer to the fabulous VS integration documentation.  You can also refer to the ALM Rangers blog post on creating widgets.

As always, candid feedback is welcome!

Saturday, February 27, 2016

A Two-Fer Release

I've just released version 2.3 of the VSTS Extension Project Templates, following a silent release of version 2.2 a few days ago.  Updates include:
  • Updated manifest for build extension to highlight specifics on task contributions
  • Modified gruntfile in build extension to allow for task publish, extension package, and extension publish
  • Added support for parameters with gruntfile in both project templates
Version 2.4 will include item templates for VSTS dashboard widgets.  Stay tuned!

Sunday, February 21, 2016

Updates to Build.vNext agent Docker images

I've just released some new updates to my VSTS Build.vNext Docker images, including:
  • Support for the Azure xplat CLI
  • Specific image for Ruby-based applications
Head over to the Docker Hub to check it out, or simply run 'docker pull jgarverick/vsoagent'!