Windows Azure Diagnostics Note

refer to : Microsoft Windows Azure Development Cookbook

The configuration of Windows Azure Diagnostics is performed at the instance level. The code to do that configuration is at the role level, but the diagnostics configuration for each instance is stored in individual blobs in a container named wad-control-container located in the storage service account configured for Windows Azure Diagnostics.

There is no need for application data and diagnostics data to be located in the same storage service account. Indeed, a best practice from both security and performance perspectives would be to host application data and diagnostic data in separate storage service accounts.

Windows Azure Diagnostics supports record-based data buffers that are persisted to Windows Azure tables and file-based data buffers that are persisted to Windows Azure blobs.

Windows Azure Diagnostics supports the following record-based data buffers:

1. Windows Azure basic logs
2. Performance counters
3. Windows Event Logs
4. Windows Azure Diagnostic infrastructure logs

The Windows Azure basic logs data buffer captures information written to a Windows Azure trace listener. In the Using the Windows Azure Diagnostics trace listener recipe, we see how to configure and use the basic logs data buffer. The performance counters data buffer captures the data of any configured performance counters. The Windows Event Logs data buffer captures the events form any configured Windows Event Log. The Windows Azure Diagnostic infrastructure logs data buffer captures diagnostic data produced by the Windows Azure Diagnostics process.

Windows Azure Diagnostics supports the following file-based data sources for the Directories data buffer:

1. IIS logs
2. IIS Failed Request Logs
3. Crash dumps
4. Custom directories

The Directories data buffer copies new files in a specified directory to blobs in a specified container in the Windows Azure Blob Service. The data captured by IIS Logs, IIS Failed Request Logs, and crash dumps is self-evident. With the custom directories data source, Windows Azure Diagnostics supports the association of any directory on the instance with a specified container in Windows Azure storage. This allows for the coherent integration of third-party logs into Windows Azure Diagnostics. We see how to do this in the Implementing custom logging recipe.

The Windows Azure Diagnostics module is imported into a role by the specification of an Import element with a moduleName attribute of Diagnostics in the Imports section of the service definition file (ServiceDefinition.csdef). This further requires the specification, in the service configuration file (ServiceConfiguration.cscfg), of a Windows Azure Storage Service account that can be used to access the instance configuration for diagnostics. This configuration is stored as an XML file in a blob, named for the instance, in a container named wad-control-container in the storage service account configured
for diagnostics.

On instance startup, the diagnostics configuration for the instance can be set as desired in the overridden RoleEntryPoint.OnStart() method. The general idea is to retrieve the default initial configuration using DiagnosticMonitor.GetDefaultInitialConfiguration() and modify it as necessary before saving it using DiagnosticMonitor.Start().This name is something of a relic, since Windows Azure SDK v1.3 and later, the Diagnostics Agent service is started automatically.

Another way to modify the diagnostics configuration for the instance is to use RoleInstanceDiagnosticManager.GetCurrentConfiguration() to retrieve the existing instance configuration from wad-control-container. This can be modified and then saved using RoleInstanceDiagnosticManager.SetCurrentConfiguration(). This method can be used both inside and outside the role instance.  For example, it can be implemented remotely to request that an on-demand transfer be performed.

Windows Azure Diagnostics stores the diagnostics configuration for an instance as an XML file in a blob, named for the instance, in a container namedwad-control-container in the Windows Azure Storage Service account configured for diagnostics. When an instance is started for the first time, a default instance configuration is inserted in the container. This can be modified programmatically, either local to the instance or remotely using methods in the Windows Azure SDK.

Local configuration typically occurs in the OnStart() method for the role and is used to further specify the information captured by Windows Azure Diagnostics and the schedule with which diagnostic data is persisted in the Windows Azure Storage Service

Remote configuration can also be used to modify the information captured. However, it is used normally to initiate an on-demand transfer of data from an instance when a problem must be investigated before the next scheduled transfer of data to the storage service.

Windows Azure SDK v1.3 introduced a way to specify declaratively the initial configuration of Windows Azure Diagnostics for a role. This requires the placement of a file, named diagnostics.wadcfg , in a specific location in the role package. When an instance is started for the first time, the Diagnostic Agent reads the file and initializes the diagnostic configuration for the instance in wad-control-container with it, instead of using the default configuration.
During instance startup, the Diagnostics Agent uses the following sequence to search for a diagnostics configuration to use:

1. Diagnostic configuration for the instance in wad-control-container.

2. Configuration specified programmatically when the instance is started.

3. Configuration specified in diagnostics.wadcfg.

4. Default configuration

Note that requesting an on-demand transfer does not require a direct connection with the hosted service.The request merely modifies the diagnostic configuration for the instance. This change is then picked up when the Diagnostic Agent on the instance next polls the diagnostic configuration for the instance. The default value for this polling interval is 1 minute.This means that a request for an on-demand transfer needs to be authenticated only against
the storage service account containing the diagnostic configuration for the hosted service.

Implementing custom logging

The directories data buffer comprises a set of data sources. These are instances of the DirectoryConfiguration class that exposes the following properties:

1. Container
2. DirectoryQuotaInMB
3. Path

A data source maps a path in the local file system with a container in the Windows Azure Blob Service. The DirectoryQuotaInMB reserves a specified amount of space in local storage for the specified data source. A scheduled transfer period is specified at the level of the directories data buffer. The Diagnostics Agent persists to the configured container any files added to the specified path since the last transfer.

Tool to use: from envykok

Windows Azure Management Tool  http://wapmmc.codeplex.com/

Example for custom diagnostics

http://convective.wordpress.com/2009/12/08/custom-diagnostics-in-windows-azure/

 when a file is inserted in the directory specified indirectoryConfiguration.Path it will be persisted automatically as a blob in an Azure Storage container named wad-my-container when the ScheduledTransferPeriod TimeSpan expires. 

protected void UseCustomLogs()
{
    DiagnosticMonitorConfiguration diagnosticMonitorConfiguration =
        DiagnosticMonitor.GetDefaultInitialConfiguration();
    diagnosticMonitorConfiguration.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);    LocalResource localResource = RoleEnvironment.GetLocalResource(“MyLogs”);
    DirectoryConfiguration directoryConfiguration = new DirectoryConfiguration();
    directoryConfiguration.Container = “wad-my-container”;
    directoryConfiguration.DirectoryQuotaInMB = localResource.MaximumSizeInMegabytes;
    directoryConfiguration.Path = localResource.RootPath;
    diagnosticMonitorConfiguration.Directories.DataSources.Add(directoryConfiguration);    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.DevelopmentStorageAccount;
    DiagnosticMonitor diagnosticMonitor = DiagnosticMonitor.Start(cloudStorageAccount,
         diagnosticMonitorConfiguration);    Byte[] bytesToWrite = new byte[] { 65, 122, 117, 114, 101 };
    using (FileStream fileStream = new FileStream(localResource.RootPath + @”logFile1″, FileMode.Create))
    {
        fileStream.Write(bytesToWrite, 0, bytesToWrite.Length);
    }
}