Developer Chronicles on Azure, Cloud, IoT, Modern Javascript and .NET

Global Office 365 Developer Bootcamp–Letterkenny-Nov’10 2018–Register Now

October 19, 2018 Boot Camp, Dev Community, Global Office 365 Developer Bootcamp, Microsoft, Office 365, Share Point, VisualStudio, Windows No comments

We have got the opportunity to host Global Office 365 Developer Bootcamp in Letterkenny as part of Letterkenny DotNet Azure User Group (LK-MUG).

Global Office 365 Developer Bootcamp – Overview?

Following the success of last year, Global Office 365 Developer Bootcamp now becomes an annual event.

  • It is a free, one-day, hands-on training event led by Microsoft MVPs with support from Microsoft and local community leaders.
  • Developers worldwide are invited to attend the bootcamp to learn the latest on Office 365 platform including Microsoft Graph, SharePoint Framework, Microsoft Teams, Office Add-ins, Connectors and Actionable Messages and apply what you learn to your future projects.
  • Watch the video to hear from Jeff Teper and Microsoft MVPs on 2018 Global Office 365 Developer Bootcamp.
  • Global Office 365 Developer Bootcamp will take place between October 1 and November 30, 2018.

[Quoting from: Official Site]

Letterkenny is first venue in Ireland and among many other 59 venues announced so far.

You can get a glance at all the venues at: http://aka.ms/O365DevBootcamp

Letterkenny – Global Office 365 Developer Bootcamp – November 10th 2018

Below is the event announcement artwork and agenda. Looking forward to the event.

image

image

Seats are limited. If you would like to join us, please join us using the registration link below:

REGISTER & RSVP: https://www.meetup.com/lk-mug/events/255066993/ 

Event Website: http://lk-mug.org/wp-event/global-office-365-developer-bootcamp-letterkennyireland/ 

About LK-MUG?

LK-MUG is a Microsoft recognized user community established in Letterkenny, Donegal,  Ireland under the full name “Letterkenny DotNet Azure User Group”.  We are currently being operated  with support of .NET foundation and Microsoft. This community is for everyone interested in Microsoft .NET, Office-365, SharePoint, Azure cloud platform and other Microsoft Open Source initiatives.

  • All skill levels are welcome.
  • We are committed to helping you learn and share things about .Net, Office-365, SharePoint and Azure Cloud.
  • Our community brings all students, enthusiasts, experts and professionals working in and around Donegal county in Ireland.
  • Time to time we will be seeing events organized by Microsoft and MVPs to provide best-in-class learning experience for Microsoft Technology enthusiasts.

User Group URLs:

  Meet Up:  https://www.meetup.com/lk-mug/
  Facebook:  https://www.facebook.com/mugdonegal 
  Twitter: https://twitter.com/lkmsug
  LinkedIn: https://www.linkedin.com/groups/12121376/
  Website: http://www.lk-mug.org
  Email : lk-mug@outlook.com

Azure Cosmos DB – TTL (Time to Live) – Reference Usecase

October 9, 2018 .NET, .NET Core, .NET Framework, Analytics, Architecture, Azure, Azure, Azure Cosmos DB, Azure Functions, Azure IoT Suite, Cloud Computing, Cold Path Analytics, CosmosDB, Emerging Technologies, Hot Path Analytics, Intelligent Cloud, Intelligent Edge, IoT Edge, IoT Hub, Microsoft, Realtime Analytics, Visual Studio 2017, VisualStudio, VS2017, Windows No comments

TTL capability within Azure Cosmos DB is a live saver, as it would take necessary steps to purge redudent data based on the configurations you may. 

Let us think in terms of an Industrial IoT scenario, devices can produce vast amounts of telemetry information, logs and user session information that is only useful until we operate on them and take action on them, to be specific up to finate period of time. Once that data becomes surplus, we need an application logic that purges these old records.

With the “Time to Live” or TTL, Microsoft Cosmos DB provides an ability to have your documents automatically purged from database storage after a certian period if time(which you configured)

  • This TTL by default can be set on a document collection level and later can be overridden on a per document basis.
  • Once the TTL is set, Cosmos DB service will automatically remove the documents when its lifetime is over.
  • Inorder to track TTL, Cosmos DB uses an offset field to check when it was last modified.  This field is identifiable as “_ts”, which exists in every document you create.  Basically it is a UNIX epoch timestamp. This field is updated everytime when the document is modified. [Ref: Picture1]

image

[Picture1]

Enabling TTL on Cosmos DB Collection:

You can enable TTL on a Cosmos DB collection simply by using Azure Portal –> Cosmos DB collection setting for existing or during creation of  a new collection)

TTL value needs to be set in seconds – if you need 90 days => 60 sec * 60 min * 24 hour * 90 days = 7776000 seconds

image

[Picture2]

Below is a one of the reference architecture in which Cosmos DB – TTL would be essentially useful and viable to any Iot business case:

image

[Picture3]

Hope that was helpful to get some understanding. For more references visit:  Cosmos DB Documentation

Azure Cosmos DB–Multi Master

October 8, 2018 .NET, .NET Core, .NET Framework, ASP.NET, Azure, Azure CLI, Azure Cosmos DB, CosmosDB, Data Consistancy, Data Integrity, Microsoft, Multi-master, Performance, Reliability, Resilliancy, Scalability, Scale Up No comments

During the Ignite 2018, Microsoft has announced the general availability of Multi-Master feature being introduced to Azure Cosmos DB to provide more control into data redundancy and elastic scalability for your data from different regions with multiple writes and read instances.

What is Multi-Master essentially?

Multi-master is a capability that provided as part of Cosmos DB, that would provide you multiple write regions and provides an option to handle conflict resolution automatically through different options provided by the platform. Most of the major scenarios you would encounter the conflict can be resolved with these simple configurations.

A sample diagram depicting a use case of load balanced web app writing to respective regional master:-

image

With multi-master, Azure Cosmos DB delivers a single digit millisecond write latency at the 99th percentile anywhere in the world, and now offers 99.999 percent write availability (in addition to 99.999 percent read availability) backed by the industry-leading SLAs.

image

Wow! That’s an amazing performance Cosmos DB guarantees to provide so that your mission-critical systems will have zero downtime, if they start using Cosmos DB.

 

How to Enabled Multi-Master support in your Cosmos DB solutions?

Currently multi-master can only be enabled for new Cosmos DB instances using “Enable Multi-Master” option in Azure Portal or through PowerShell or ARM templates or through SDK.

These options are detailed below with necessary examples:

1.) Azure Portal – Enable Multi-region writes and Enable geo-redundancy

image

2.) Azure CLI 
Set the “enable-multiple-write-locations” parameter to “true”

az cosmosdb create \
   –-name "thingx-cosmosdb-dev" \
   --resource-group "consmosify-dev" \
   --default-consistency-level "Session" \
   --enable-automatic-failover "true" \
   --locations "EastUS=0" "WestUS=1" \
   --enable-multiple-write-locations true \

3.) AzureRM PowerShell
In AzureRM PowerShell cmdlet – Set enableMultipleWriteLocations parameter to “true”

$locations = @(@{"locationName"="East US"; "failoverPriority"=0},
             @{"locationName"="West US"; "failoverPriority"=1})

$iprangefilter = ""

$consistencyPolicy = @{"defaultConsistencyLevel"="Session";
                       "maxIntervalInSeconds"= "10";
                       "maxStalenessPrefix"="200"}

$CosmosDBProperties = @{"databaseAccountOfferType"="Standard";
                        "locations"=$locations;
                        "consistencyPolicy"=$consistencyPolicy;
                        "ipRangeFilter"=$iprangefilter;
                        "enableMultipleWriteLocations"="true"}

New-AzureRmResource -ResourceType "Microsoft.DocumentDb/databaseAccounts" `
  -ApiVersion "2015-04-08" `
  -ResourceGroupName "consmosify-dev" `
  -Location "East US" `
  -Name "thingx-cosmosdb-dev" `
  -Properties $CosmosDBProperties

4.) Through CosmosDB SDK
Setting connection policy in DocumentDBClient and set UseMultipleWriteLocations to true.

ConnectionPolicy policy = new ConnectionPolicy
{
   ConnectionMode = ConnectionMode.Direct,
   ConnectionProtocol = Protocol.Tcp,
   UseMultipleWriteLocations = true,
};
policy.PreferredLocations.Add("East US");
policy.PreferredLocations.Add("West US");
policy.PreferredLocations.Add("West Europe");
policy.PreferredLocations.Add("North Europe");
policy.PreferredLocations.Add("Southeast Asia");
policy.PreferredLocations.Add("Japan East");
policy.PreferredLocations.Add("Japan West");

Azure Cosmos DB multi-master configuration is the game changes that really makes it a true global scale database with automatic conflict resolution capabilities for data synchronization and consistancy.

In my later sessions I will write examples to cover how conflict resolutions can be configured and used in realtime scenarios.

Useful Refs:

Azure Cosmos DB–Reserved Capacity

October 7, 2018 Azure, Azure Cosmos DB, Billing, CosmosDB, Enterprise Agreement, Infrastructure, Microsoft, Pay-as-you-go No comments

Azure Cosmos DB is a planet scale global document database which have been available for Azure Customers based on pay-as-you-go. Reserved Capacity is a new long term pre-paid billing commitment customers can get a discounted pricing. Azure Cosmos DB reserved capacity helps you save money by pre-paying for Azure Cosmos DB resources for a period of one year or three years.

  • Reserve capacity allows you to get a optimum discount for the throughput(RUs) provisioned for Cosmos DB resources.
  • For Ex: Databases and Containers (tables/collections/graphs).
  • It can significantly reduce your Cosmos DB costs and it enables you to save up to 65 percent on regular prices with one-year or three-year upfront commitment.
  • Reserved capacity provides a billing discount and does not affect the runtime state of your Cosmos DB resources.
  • It is available to all supported APIs (including MongoDB, Cassandra, SQL, Gremlin and Azure Tables) and all regions worldwide.

    • Scope of Reservation:

    A reservation is scoped to a single subscription or shared across all your enrolment:

    • Scoped to a Single Subscription: that means a set of cosmos db account resources such as database or containers within the selected subscription.
    • Shared/Scoped to an enrolment: Billing benefit can be shared across any subscription in the enterprise agreement or pay-as-you-go subscriptions.

    cosmos-db-reserved-capacity

    Who can opt-in for Reserved Capacity?

    This feature is currently only applicable for enrolment (Enterprise Agreement customers) or account (Pay-as-you-go customers). Users with MSDN Subscription benefits are not applicable for the Reserved Capacity benefits.

    You can buy Azure Cosmos DB reserved capacity from the Azure portal.

    1. Sign in to the Azure portal.
    2. Select All services > Reservations > Add and Select Product Type > Cosmos DB

    image

    8fafbeff-f11c-4894-98e0-e03d79d1787d

    For more details review reserved capacity documentation.

    Azure Cosmos DB – 429 Too Many Requests

    October 6, 2018 .NET, Azure, CosmosDB, Document DB, Microsoft, Performance, Reliability, Resilliancy, Scalability, Visual Studio 2017, VisualStudio, VS2017 No comments

    Recently while I was doing Performance Testing in one of the APIs interacting with Cosmos DB, I encountered a problem as Azure Cosmos DB API’s started returning Http Code 429.  Http Status Code 429 indicates that too many request been received or request rate is very large. This error would happen when we have concurrent users trying to write or read from same cosmos db collection.

    Following diagram covers the architecture of the performance test I am performing:

    image

    Based on analysis it found out to be the Throttling happening from Azure Cosmos DB, as we make requests that may use more than provisioned Request Units(RU) per second. We were using default Cosmos DB configuration for a fixed collection of 1000 RU’s per second which is sufficient enough for a 500 reads and 100 writes for a 1 kb file. You can refer more about Request Units from Azure Docs.

    image

     

     

     

    Solution(s):

    1. Now first logical step we can do is to get rid off this error by increasing the Throughput for the collection.  I am going to increase to 10000 RU/s maximum allocatable for a Storage Capacity: Fixed.   This should ideally improve the Throughput for 250 or more virtual users hitting.

    image

    2. Second logical step is to improve the code: Improve the connection parameters in the Document DB SDK –> DocumentDbClient. For this I referred to the Microsoft Docs: Performance tips for Azure Cosmos DB and .NET

    Providing optimum values to the following Properties in RetryOption class   to be passed as parameter to Connection Policy.

    image

     

    In my case I provided a value of 30 to give ultimate results:

    new RetryOptions() { MaxRetryAttemptsOnThrottledRequests = 30, MaxRetryWaitTimeInSeconds = 30  }

    That should resolve most of the 429 issues when dealing with Cosmos DB SDK

    Azure Database for MariaDB: Public Preview

    October 4, 2018 Azure Database for MariaDB, Managed Services, MariaDB, OpenSource No comments

    During Ignite 2018, Microsoft has announced the availability of Maria DB support in Azure Database services. Today it has been opened for Public Preview for all Azure customers.

    mariadbhero

    What is MariaDB?

    MariaDB is a community-developed fork of the MySQL relational database management system intended to remain free under the GNU GPL.Development is led by some of the original developers of MySQL, who forked it due to concerns over its acquisition by Oracle Corporation.Wikipedia

    Azure Database for MariaDB: Public Preview Availability

    The Azure Database for MariaDB service is now available in preview. It offers an enterprise-ready, fully managed database service that based on the Community Edition of MariaDB.

    The service features open-source compatibility, built-in high availability, dynamic scaling, and flexible pricing. Customers can lift and shift to the cloud and use languages and frameworks of their choice, leveraging the power of MariaDB running on Azure.

    To learn more about the service, view the service page, pricing, and documentation.

    You can create a MariaDB server by using the Azure portal or Azure CLI.

    More References: