A while ago I wrote a post about an interesting Twitter behaviour – if you start a tweet with an @<username> only people who follow you & that person can see the tweet (if you unsure see the post which explains it). The question I had today was to find out who is in that list – or put who follows us both on Twitter?

Oddly, I could not find anything (besides some tools that cost money) to do this?! So I built my own awesome little tester you can use below:

For the developers among you who want to see how this all works? Check it out on bitbucket: https://bitbucket.org/rmaclean/twittercoalesce

Known issues:

• My website uses an older version of jQuery, and I use a newer one in this code. If the loading gives an error or gets stuck on the screen, try refreshing the page (seems to solve it).
• If you have too many people shared between the two names - it will break. I am looking into how to solve this. If this happens - loading will get stuck :|
• Twitter limits clients to 150 calls per hour. If the rate limit is exceeded the loading will get stuck.

Lightswitch Recap

For you to understand the rest of this post it is vital you have a high level understanding of Lightswitch and how it works. Lightswitch is a RAPID development platform from Microsoft that makes development of line of business (LOB) apps really easy. The team at Microsoft often talk about citizen developers – i.e. people who are not full time developers, but are the “IT guy” in the department or company that need to put together a great looking solution. The team also talk about no-code solutions – where you can build great systems without code.

Both statements from the team are true and false at the same time. Sure your accountant can build a CRM system with no code in Lightswitch, but Lightswitches true value is that it is a professional development tool, and in reality unless it is a really simple solution you will need a touch of code.

What is great is that Lightswitch allows the citizen developer to write a system that can be matured by professional developers later on – it’s power is that it does not lock you into being too simple or too complex a development system.

For me the value proposition is that you get REAL rapid development, that citizen developers can put together and extend solutions that are well architected and that when the need is there a professional developer can extend that solution and hand it back over to the citizen developer – it is the circle of Lightswitch.

Architecture

When you craft a (avoiding the development term here on purpose) Lightswitch create a multi-tier architecture, that is either two tier (client & database) or three tier (client, server & database). Two tier is really three tier but the server & client are just one package.

The database can be any support by Lightswitch, the middle tier is OData and the front end is Silverlight. The choice of front end has recently hurt Lightswitch because Silverlight is dying. However if you step back for a second and think about it Lightswitch provides the easiest and fastest way to build a complete (and I mean complete, authentication, methods, proper designed) OData solution… you could always ignore the client portion and build on top of the OData server.

Making a HTML Client

The HTML client mode for Lightswitch is a recently announced new feature that allows you to build a client that runs in a browser, and not just Internet Explorer on Windows (Dynamics CRM I am looking at your shameful behaviour) but pretty much any browser, say on an iPad:

This is possible because of two things, the OData server which allows really any technology to connect to it, and the second piece of the Lightswitch system the LSML file.

I hope you have never heard of the LSML file, as it is not a nice place to go to – it is a MASSIVE (even simple demo’s I build are thousands of lines) XML file that stores ALL of the Lightswitch system in a “Lightswitch domain language”. This enables the team to take that information, parse it and produce output based on it. So the concept of producing a parser that creates HTML rather than Silverlight is really simple… just build the parser.

What do we know about this HTML client so far?

It is early days, in fact there are no bits available yet, but we do know some things from the demo’s and screen shots that are available.

• Multiple themes will be supported (their is a dark & a light at least) – thanks to the jQuery Mobile that powers it.
• It is a separate client – so you will have a Silverlight experience and then also have the HTML experience added in.
• It follows the true Lightswitch model of being easy to build with no code, but if you need that little extra, the JavaScript can be edited.

The Important Two Issues

To wrap this up it is a very exciting point in time for the Lightswitch world with so much happening that I think it is important to take a step back and find a few key aspects about this amazing feature that will help position it. There are two that really stand out to me from all the announcements:

Separate Client

This is not a Silverlight to HTML generator – it is separate. This means that awesome Silverlight chart you use today will not magically work in the HTML client. This has both advantages and disadvantages, but if you think about the dying of Silverlight I am very glad that they have a whole new growth path.

It also allows for the real scenario of supporting a rich experience in Silverlight in a company (where we control all the machines and know we can run Silverlight for a long time still) and having a mobile or companion experience in HTML for those people on the road. Sure they do not get the great sales forecast chart but they can still capture their sales on their iPad.

Web Developers

A recent did an survey of app developers looked at what they are building today, what they were building and what they intend to build in the future (future = one year in this survey). Interestingly there are only TWO platforms that are getting growth in the future? HTML & Windows Phone. Android, iPhone and many others are all expected to decline.

If you think about those numbers and add in the MASSIVE investments in HTML development that are in Windows 8, it should not surprise you that web development is a MAJOR area in the future of all developers. It also means that web developers can start to have way more opportunities in the market outside of building websites & portals, and that is very exciting as that little garage web designer company today could be a major line of business developer in a few years.

With Visual Studio 2012 there is an increase importance placed on the Extension Manager component, not only does it provide a great integrated experience to the Visual Studio gallery for downloading and updating extensions but it will also be used to delivery updates for Visual Studio itself!

At work it would constantly fail to work with error 417 – expectation failed, so working with our facilities team we were able to identify the problem as an issue with the proxy server we use, Squid.

Squid seems unable to handle the HTTP status code 100, and will then fail with the error 417. To solve this you simple need to add the following to your squid.conf file: ignore_expect_100 on

Yesterday Rudi Grobler & I had awesome fun with a full room of amazing people who took time off work to attend a full day of free Windows 8 training. The audience was amazing, breaking a lot of my expectations of how audiences react at free events, which really honoured Rudi & I to have most people stay to the very end of the day.

For those people who attended the training, or those who didn’t but want the content too:

• You can get the slides: http://bit.ly/win8-slides 
• You can get videos from Build on the topics we discussed: http://bit.ly/win8-videos
• The demo’s I built are available to download from below.

Introduction

Culture settings in .NET are a very important but often ignored part of development, they define how numbers, dates and currencies are displayed and parsed and you application can easily break when it is exposed to a new culture.

In .NET 4, we could do three things:

• Ignore it
• Set it manually everywhere
• If we were using threads, we could set the thread culture.

So lets see how that works:

// uses the system settings
Console.WriteLine(string.Format("1) {0:C}", 182.23));

// uses the provided culture
Console.WriteLine(string.Format(CultureInfo.InvariantCulture, "2) {0:C}", 182.23));

// spin up a thread - uses system settings
new Thread(() =>
    {
        Console.WriteLine(string.Format("3) {0:C}", 182.23));
    }).Start();

// spin up a thread - uses thread settings
var t = new Thread(() =>
{
    Console.WriteLine(string.Format("4) {0:C}", 182.23));
});

t.CurrentCulture = new CultureInfo("en-us");

t.Start();

Console.ReadLine();

You can see in the capture above lines 1 & 3 use the South African culture settings it gets from the operating system. What if I want to force say an American culture globally? There was no way before .NET 4.5 to do that.

What .NET 4.5 adds?

By merely adding one line to the top of the project, all threads, including the one we are in get the same culture:

CultureInfo.DefaultThreadCurrentCulture = new CultureInfo("en-us");

Introduction

While the regular expression passing in .NET is damn fast, there are times where it can take too long for your needs. Until now there hasn’t been much you can do but wait. In .NET 4.5 we get the ability to timeout regular expressions if they took too long.

Problem

So lets look at a really silly example to start off with, checking a string fifty million characters (where only one is different) against regular expression which is looking for fifty million letters. As I said it is silly, but to get a truly slow reg ex is pretty hard.

static Regex match = new Regex(@"\w{50000000}", RegexOptions.None);
static void Main(string[] args)
{
    var sw = Stopwatch.StartNew();
    Console.WriteLine(match.IsMatch(String.Empty.PadRight(49999999, 'a') + "!"));
    sw.Stop();
    Console.WriteLine(sw.Elapsed);
    Console.ReadLine();
}

This 13.5secs on my machine!

Solution

All we need to do to take advantage of the new timeouts is modify the constructor of the Regex, by adding a third parameter.

static Regex match = new Regex(@"\w{50 000 000}", RegexOptions.None, TimeSpan.FromSeconds(5));

Now after five seconds a RegexMatchTimeoutException is raised.

Introduction

We have seen the IDataFlowBlock, in three different implementations and now we will look at a few more.

BroadcastBlock<T>

In the BatchBlock we saw that if you had multiple subscribers, messages are delivered to subscribers in a round robin way, but what about if you want to send the same message to all providers? The solution is the BoardcastBlock<T>.

static BroadcastBlock<string> pubSub = new BroadcastBlock<string>(s =>
    {
        return s + " relayed from publisher";
    });

static async void Process()
{
    var message = await pubSub.ReceiveAsync();
    Console.WriteLine(message);
}

static void Main(string[] args)
{
    // setup 5 subscribers
    for (int i = 0; i < 5; i++)
    {
        Process();
    }

    pubSub.Post(DateTime.Now.ToLongTimeString());

    Console.ReadLine();
}

TransformBlock<TInput,TOutput>

The next interesting block is the transform block which works similar to the action block, except that the input and output can be different types and so we can transform the data internally.

static TransformBlock<int, string> pubSub = new TransformBlock<int, string>(i =>
    {
        return string.Format("we got: {0}", i);
    });

static async void Process()
{
    while (true)
    {
        var message = await pubSub.ReceiveAsync();
        Console.WriteLine(message);
    }
}

static void Main(string[] args)
{
    Process();
     
    for (int i = 0; i < 10; i++)
    {            
        pubSub.Post(i);
        Thread.Sleep(1000);
    }

    Console.ReadLine();
}

Introduction

We have spent a number of posts looking at IDataFlowBlock, BatchedBlocks and some other interesting blocks but as a final interesting bit for blocks is the ISourceBlock interface which is implemented on the “publisher” like blocks and it has a very interesting method: LinkTo. The goal of LinkTo is to chain blocks together to do interesting and amazing things!

It's in the name

The final interesting point, and likely the most important is in the name of the assembly all this goodness comes from: System.Threading.Tasks.DataFlow. All of this is handled my multiple tasks (and likely multiple threads) under the covers without you needing to worry about it. When we look at the TPL in .NET 4 and how it made working with multi-threaded processing easier, the final goal of this is to make multi-threaded data processing easier - and it does!

Demo LinkTo

In this demo what we want to do is take in DateTime’s as fast as possible, then convert them to strings, and send them in batches to the three subscribers. Sounds complex, but with the ability to LinkTo this is very easy.

static BufferBlock<DateTime> bufferPublisher = new BufferBlock<DateTime>();
static TransformBlock<DateTime, string> transformPublisher = new TransformBlock<DateTime, string>(d =>
    {
        return d.ToLongTimeString();
    });
static BatchBlock<string> batchPublisher = new BatchBlock<string>(5);

static ActionBlock<string[]> subscriber1 = new ActionBlock<string[]>(s =>
    {
        Console.WriteLine("Subscriber 1: {0}", s);
    });
static ActionBlock<string[]> subscriber2 = new ActionBlock<string[]>(s =>
{
    Console.WriteLine("Subscriber 2: {0}", s);
});

static ActionBlock<string[]> subscriber3 = new ActionBlock<string[]>(s =>
{
    Console.WriteLine("Subscriber 3: {0}", s);
});


static void Main(string[] args)
{
    batchPublisher.AsObservable().Subscribe(subscriber1.AsObserver());
    batchPublisher.AsObservable().Subscribe(subscriber2.AsObserver());
    batchPublisher.AsObservable().Subscribe(subscriber3.AsObserver());

    transformPublisher.LinkTo(batchPublisher);
    bufferPublisher.LinkTo(transformPublisher);

    for (int i = 0; i < 100; i++)
    {
        bufferPublisher.Post(DateTime.Now);
        Thread.Sleep(200);
    }

    Console.ReadLine();
}

Introduction

We previously looked at the IDataFlowBlock interface in it’s simplest implementation ActionBlock<TInput>, today though we are going to look at more complex implementations that give amazing powers.

BatchBlock<T>

Where ActionBlock<TInput> could only be a subscriber, BatchBlock<T> is more than that – it can be a subscriber and a publisher rolled into one! As such the usage is rather different, we do not pass in the receive method in the constructor. Rather we call the Receive method to get the latest messages past to the subscriber.

The interesting thing about BatchBlock is it batches up messages into groups, so rather than getting each message one at a time, you get groups of messages. As such, you could wait a long time for enough the messages to arrive, so thankfully they also include a ReceiveAsync which works with C# new async methods.

In the sample below you can see how I create a BatchBlock with a batch size of 3, so messages get processed in groups of three. This also means the last two messages (since we are sending 20) are never processed.

static BatchBlock<string> pubSub = new BatchBlock<string>(3);
static int counter = 0;

static async void Process()
{
    while (true)
    {
        var messages = await pubSub.ReceiveAsync();
        foreach (var item in messages)
        {
            counter++;
            Console.WriteLine("Got message number {0}: {1}", counter, item);
        }
    }
}

static void Main(string[] args)
{
    Process();

    for (int i = 0; i < 20; i++)
    {
        Task.Factory.StartNew(() =>
        {
            Thread.Sleep(new Random().Next(200, 1000));
            pubSub.Post(DateTime.Now.ToLongTimeString());
        });
    }

    Console.ReadLine();
}

BatchedJoinBlock<T1,T2>

The second block implementation we look at is the BatchedJoinBlock<T1,T2> which is similar to BatchBlock<T> except, that it has multiple type inputs. This allows you to have a single publisher that can send messages to different subscribers based on type! The batching works the same as before, but be careful as batch sizes are based on all messages regardless of types.

static BatchedJoinBlock<string, int> pubSub = new BatchedJoinBlock<string, int>(3);
static int stringCounter = 0;
static int intCounter = 0;

static async void Process()
{
    while (true)
    {
        var messages = await pubSub.ReceiveAsync();
        foreach (var item in messages.Item1)
        {
            stringCounter++;
            Console.WriteLine("Got STRING message number {0}: {1}", stringCounter, item);
        }

        foreach (var item in messages.Item2)
        {
            intCounter++;
            Console.WriteLine("Got INT message number {0}: {1}", intCounter, item);
        }
    }
}

static void Main(string[] args)
{
    Process();

    for (int i = 0; i < 20; i++)
    {
        Task.Factory.StartNew(() =>
        {
            Thread.Sleep(new Random().Next(200, 1000));
            pubSub.Target1.SendAsync(DateTime.Now.ToLongTimeString());
        });

        Task.Factory.StartNew(() =>
        {
            Thread.Sleep(new Random().Next(200, 1000));
            pubSub.Target2.SendAsync(new Random().Next(1, 99));
        });
    }

    Console.ReadLine();
}

Multiple Subscribers

So what happens when you add multiple subscribers to the system? It handles each processing in a round robin like way. The sample below is the same as the BatchBlock<T> about, but  has three subscribers (A, B & C) and a batch size of two.

static BatchBlock<string> pubSub = new BatchBlock<string>(2);
static int counter = 0;

static async void Process(string id)
{
    while (true)
    {
        var messages = await pubSub.ReceiveAsync();
        foreach (var item in messages)
        {
            counter++;
            Console.WriteLine("{2} - Got message number {0}: {1}", counter, item, id);
        }
    }
}

static void Main(string[] args)
{
    Process("A");
    Process("B");
    Process("C");

    for (int i = 0; i < 11; i++)
    {
        Task.Factory.StartNew(() =>
        {
            Thread.Sleep(new Random().Next(200, 1000));
            pubSub.Post(DateTime.Now.ToLongTimeString());
        });
    }

    Console.ReadLine();
}