Skip to main content

ASP.MVC Real-time data processing with Sharded RavenDB and SignalR

In a move that some call "Radical” The Weather Channel recently swapped their enterprise Oracle and MySql Databases for NoSQL MongoDB.  One word that could describe the rapid adoption of NoSQL databases compared to relational once is “sharding.” Unlike relational databases, NoSQL systems allow developers to be active participants in the data storage process by providing a way to create logic (sharding strategy) that outlines how and where data could be stored, resulting in better scalability and improved performance.

RavenDB, an open source NoSQL system for .NET that is easy to deploy (both as a standalone/embedded system) for an ASP.MVC application. SingalR is a tool that one can use in a .NET environment to add real-time client-server processing in an application. SignalR allows a developer to broadcast messages to all (or some) clients from a centralized client and/or server call.

I was looking for a quick data-sharding example with embedded RavenDB that I can download and run and wasn't able to find one. After some trail and error I was able to set up a sample solution with sharded-embedded RavenDB and SignalR using the steps below. If you are exploring NoSQL systems and/or real-time data processing with SignalR, I hope the following sample application could be helpful. All of the resources listed in this post are available on NuGet and can be installed using NuGet's Package manager.


If you want to download and run the code you can get the solution on GitHub.

NuGet packages you will need.

Note: the following packages are needed for .NET 4.5 application (if your application uses .NET 4.0 you might need to install different package versions for some of the packages/dependencies below)

Install-Package Microsoft.AspNet.SignalR
Install-Package RavenDB.Embedded -DependencyVersion Highest
Install-Package Microsoft.AspNet.Identity.Core -Version 2.0.1
Install-Package Microsoft.AspNet.Identity.Owin -Version 2.0.1
Adding Data Model
I am only using one class, namely, customer class as an example model for the sample application. The customer class has properties FirstName, LastName, Address, and Region.

public class Customer
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public string Address { get; set; }
        public string Region { get; set; }
        public Customer() { }
    }

Adding RavenDB
Configuration
As establishing a RavenDB database connection is a very expensive operation you would want to connect with RavenDB once in your application life-cycle in Global.asax file in the “Application _Start” method. A singleton “IDocumentStore” variable holds all the connection properties needed for RavenDB.

public static IDocumentStore Store;
Sharding Strategy
Sharding is a way of telling RavenDB what criteria should be used to persist data. In the example application, there are four separate file-system based data storages that are responsible to storing data for various regions (the file system storage can be upgraded to a http/server based system using a non-embedded flavor of RavenDB). In the example data for various regions Asia, Australia, Europe, and North America are stored in separate files.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Mvc;
using System.Web.Routing;
using System.Reflection;
using Raven.Client.Embedded;
using Raven.Client.Document;
using Raven.Client.Indexes;
using Raven.Client.Shard;
using Raven.Client;
using RavenSignalRTest.Models;

namespace RavenSignalRTest
{
    public class MvcApplication : System.Web.HttpApplication
    {
        public static IDocumentStore Store;
        protected void Application_Start()
        {
            AreaRegistration.RegisterAllAreas();
            
            RouteConfig.RegisterRoutes(RouteTable.Routes);

            //File based sharding example for customer data
            var shards = new Dictionary<string, IDocumentStore>
                {
                    {"North America", new EmbeddableDocumentStore { DataDirectory = "App_Data\\North-America" }},
                    {"Europe", new EmbeddableDocumentStore { DataDirectory = "App_Data\\Europe" }},
                    {"Asia", new EmbeddableDocumentStore { DataDirectory = "App_Data\\Asia" }},
                    {"Australia", new EmbeddableDocumentStore { DataDirectory = "App_Data\\Australia" }},
                };

            var shardStrategy = new ShardStrategy(shards)
                 .ShardingOn<Customer>(customer => customer.Region);

            Store = new ShardedDocumentStore(shardStrategy).Initialize();

            IndexCreation.CreateIndexes(Assembly.GetCallingAssembly(), Store);

        }
    }
}

At this point RavenDB should be up and running. You can submit data storage requests and query data in RavenDB storage.

Adding SignalR
Configuration
SignalR (for .NET 4.5) uses Orwin set up for initial configuration. You would need to add the following enteries into your Orwin StartUp file to map SignalR.

using System;
using Microsoft.AspNet.Identity;
using Microsoft.AspNet.Identity.Owin;
using Microsoft.Owin;
using Owin;
using RavenSignalRTest.Models;

namespace RavenSignalRTest
{
    public partial class Startup
    {
        public void ConfigureAuth(IAppBuilder app)
        {
            app.MapSignalR();
        }
    }
}
Adding Server-Side Code
A SignalR Hub class exposes methods that allow you to send broadcast messages to clients connected to your application. You can add a SignalR class by right clicking your project file and adding a new “SignalR Hub Class (V2)” class.
Once you add the class you can update it as follows.


using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using Microsoft.AspNet.SignalR;
using Microsoft.AspNet.SignalR.Hubs;
using RavenSignalRTest.Models;
using Raven.Client;
using Raven.Client.Document;
using Raven.Client.Indexes;

namespace RavenSignalRTest.Services
{
    [HubName("signalRTestHub")]
    public class SignalRTestHub : Hub
    {
        //Singleton customers object 
        private static List<Customer> customers = new List<Customer>();

        //First method that gets called when SignalR hub starts
        public override System.Threading.Tasks.Task OnConnected()
        {
            getCustomers();
            return base.OnConnected();
        }

        //Method that executes when hub reconnects
        public override System.Threading.Tasks.Task OnReconnected()
        {
            getCustomers();
            return base.OnReconnected();
        }
        //List of all customers (All Raven Shards)
        public void getCustomers()
        {
            IDocumentSession RavenSession = MvcApplication.Store.OpenSession();
            customers = RavenSession.Query<Customer>().ToList();

            //Broadcast to clients
            Send();

        }

        //Add a new customer
        public void addCustomer(Customer customer)
        {
            IDocumentSession RavenSession = MvcApplication.Store.OpenSession();
            RavenSession.Store(customer);
            RavenSession.SaveChanges();

            //All new customer to the singleton list and broadcast new customer to all clients
            customers.Add(customer);
            Send(customer);
        }

        //Broadcast all customers
        public void Send()
        {
            // Call the addNewMessageToPage method to update clients.
            var context = GlobalHost.ConnectionManager.GetHubContext<SignalRTestHub>();
            context.Clients.All.allCustomers(customers);
        }

        //Overloaded method to send one customer
        public void Send(Customer customer)
        {
            var context = GlobalHost.ConnectionManager.GetHubContext<SignalRTestHub>();
            var newCustomer = new List<Customer>();
            newCustomer.Add(customer);
            context.Clients.All.allCustomers(newCustomer);
        }
    }
}
The "addCustomer” method handles adding entries into RavenDB.
The overloaded “Send” method will broadcast new data entries to all clients.

Adding Client-Side Code
SingnalR JavaScript libraries create proxies for calling and receiving server side messages to a client. Add the following links to SignalR JavaScript libraries to a page/partial you are planning to call SignalR server-side calls.

<script src="~/Scripts/jquery.signalR-2.1.2.js"></script>
<script src="/signalr/hubs"></script>

Linking client-side events and proxies to make server-side Call
The main event we want to handle for the application is being able to add a new record to RavenDB and simultaneously broadcast the changes to all clients.
We need a method to query and display all customers and another method to add a customer to our RavenDB.

<script type="text/javascript">
    var customers = {
        signalRConnect: $.connection.signalRTestHub,
        init: function () {
            //Loads all customers
            customers.signalRConnect.client.allCustomers = function (customerList) {
                for (i = 0; i < customerList.length; i++) {
                    $(".list-group").append("<li class='list-group-item'>" 
                        + "<div class='row'>"
                        + "<div class='col-lg-4'>"
                        + customerList[i].FirstName + " "
                        + customerList[i].LastName
                        + "</div>"
                        + "<div class='col-lg-4'>"
                        + customerList[i].Address + "</i>"
                        + "</div>"
                        + "<div class='col-lg-4'>"
                        + customerList[i].Region 
                        + "</div>"
                        + "</div>" 
                        + "</li>");
                }
            };
            $.connection.hub.start();
        },
        addCustomer: function () {
            // Adds new customer
            var newCustomer = {
                FirstName: $("#FirstName").val(),
                LastName: $("#LastName").val(),
                Address: $("#Address").val(),
                Region: $("#Region").val()
            };
            customers.signalRConnect.server.addCustomer(newCustomer);
            $.connection.hub.start();
        }
    };
    customers.init();
</script>
To test the application you can download the source code from GitHub and run it while having multiple browser windows open.

Comments

Post a Comment

Popular posts from this blog

Turning WCF Service into .asmx for debugging

Even though .asmx web services are becoming dinosaurs of the new .NET world of WCF. I missed the simplicity of debugging code right in visual studio without: Creating a client to consume WCF service Attaching w3p.exe process and Adding break points  One quick solution: Turn WCF service into .asmx service with few lines of code, debug your code with asmx, and turn .asmx off during deployment.  Detail steps: 1- First take your WCF class and add WebService attribute to it Code Snippet /// <summary> /// Dual mode with .ASMX and WCF /// </summary> [ WebService (Namespace = "http://www.yourdomain.com/" )] 2- Then add WebMethod attribute to a function you want to expose in .asmx Code Snippet [ WebMethod ] public List < PageController . Page > DualFunction() { 3- Take the .svc file from your solution - copy and rename the copied file [YourOriginalWCFFile.asmx]. Open up the copied file and rename "ServiceHost...
Post a Comment
Read more

Processing ASP MVC Web API Requests in Multi-threaded JS Web Worker

Unlike an asynchronous Ajax call, HTML5 Web workers provide an opportunity to run a Multi-threaded JavaScript code in modern browsers that support them . Each worker spawns an isolated thread with dedicated JavaScript Event Loop, Stack and Heap memory. For example a regular Ajax Call to MVC Web API service call goes through the following asynchronous call cycle. The JavaScript Event Loop in this case could be interrupted by events that are being executed on the UI; for instance, a "window.alert", could possibly stop all scripts on the page from executing until a user responds. Replacing the Ajax Call with HTML5 web worker provides a great way to run long running scripts in separate threads so that asynchronous code execution is not interrupted by UI events. Here is the a JavaScript worker implementation of the same MVC Web API call using a JavaScript web worker. Despite the advantages of using a web worker, implementing one requires working with some constr...
1 comment
Read more