On Tuesday night I gave a presentation on Apache Kafka at the Sydney ALT.NET meet up.

What is Kafka?

Kafka is publish-subscribe messaging rethought as a distributed commit log. Originally developed at LinkedIn in 2011 it has been adopted by some big companies since such as Twitter, Netflix and Microsoft to provide high throughput, low latency messaging.

Most common use cases for Kafka include:

• Pub/Sub
• Activity tracking
• Metrics
• Log aggregation
• Stream processing
• Event sourcing
• Commit logs

How do I use it?

There are a bunch of libraries available for a lot of languages. Unfortunately the .NET ones are quite immature. Recently a few of the boffins at Microsoft open-sourced the library they’ve used internally called CSharpClient-for-Kafka. This library is targeted at version 0.8, but should soon support 0.9. It is fast and with some work will be quite reliable.

However, it does not support the new consumer protocol that was released with v0.9. If you want something that does, you’ll have to go for ah-/RDKafka-DotNet. This is a wrapper around the librdkafka c library that’s the most commonly used outside of Java-land. The downside of this is that due to interop it’s not quite as performant as the native .NET TCP communication of the MS library. In my tests I’ve found publishing with CSharpClient-for-Kafka to be ~.1ms per message/batch and consuming ~20ms. RDKafka-DotNet performs these at ~2ms and ~150ms with similar parameters.

Also, if you think that TCP is too hard core, you can give the Confluent REST Proxy or Kafka Pixy a go… These are services that allow you to publish to and subscribe from Kafka via HTTP APIs. Not the most performant option, but definitely easier to debug…

If you’re looking for samples of using these clients you can check out a couple of my repositories –

• kafka-basic – a wrapper for CSharpClient-for-Kafka that provides simple producer/consumer abstractions
• rdkafka-tests – a test console for RDKafka-DotNet

Where do I learn more?

The Apache Kafka site has a lot of great information to get you up and running. Confluent are the commercial entity providing supported distributions and whose founders were originally responsible for the development of Kafka at LinkedIn.

My presentation is available from Tuesday night on slideshare with a few funky animations to help visualise message delivery and rebalances.

Microsoft have started a new Patterns & Practices project name Liike (LEEE-keh) with the aim of delivering guidance for building mobile web solutions.

http://liike.github.com/

How do they plan to do this…?

Well they’ve got a few FTE’s who are creating a mobile version of a previous project as a reference implementation so they can make the mistakes we’ve all made in the past. They’ve also taken on an advisory board consisting of a few members of the community (including myself) for some ideas and extra guidance.

One thing that’s been made clear so far is that reusable code is not a major deliverable. There are many people out there who’ve done a bang-up job of building UI frameworks and plugins for mobile web, so they don’t want to re-invent the wheel – they just want to reduce the squeak in yours by giving you the oil you need to run smoothly.

So, how can you help…?

1. There’s a uservoice site available for you to throw your ideas for what you think are the most important challenges: http://liike.uservoice.com/forums/136038-mobile-web-dev-guidance

2. Start a conversation with myself or anyone else on the advisory board! I’m always available on twitter and am happy to take comments on my blog. If you want to get me on email, just leave a comment on this post and I’ll get back to you.

Before Mango it was a little difficult to put a generic status message at the top of your page. The Mango tools have made this a bit easier by giving developers (more) access to the System Tray. That’s this part:

You are still a bit limited in what you can do (i.e. you can’t actually add or remove icons), but there’s a lot more flexibility than before.

You can access the system tray either in code or XAML using the Microsoft.Shell.SystemTray class. This class has the following dependency properties:

• IsVisible
• Opacity
• BackgroundColor
• ForegroundColor
• ProgressIndicator

While it’s quite exciting to be able to style the status bar, I cried out of joy when I saw the ProgressIndicator property. Why…? Because now I can add progress information to the top of my pages quickly and easily! J

So, I want to add a downloading message and progress bar to the top of my page (above the application title) as follows:

Using XAML, I can add text and a progress bar to a page by dropping the following into the page:

<shell:SystemTray.ProgressIndicator>
    <shell:ProgressIndicator IsIndeterminate="true" IsVisible="True" Text="Click me..." />
</shell:SystemTray.ProgressIndicator>

Or, using code, I can accomplish the same thing by dropping this into my code:

ProgressIndicator progress = new ProgressIndicator
{
    IsVisible = true,
    IsIndeterminate = true,
    Text = "Downloading details..."
};
SystemTray.SetProgressIndicator(this, progress);

Feel free to download my sample application showing how to manipulate the System Tray:

Recently I released a little open source project with the aim of making it easier for developers to write services that push information to Windows Phone 7 devices called MetroPimp.

This library is available via nuget and on bitbucket.

Based on a blog post I wrote when WP7 was still in beta, MetroPimp provides a simple and consistent API that allows you to push Toast, Tile (single and double sided) and Raw push notifications to WP7 devices using the Microsoft Push Notification Service (MPNS).

As developers who have tried this already may know, there is quite a bit of work that goes into setting up push notifications. First you have to register for notifications on the device by opening a channel. Then you have to get the URI provided by MPNS and send it to a service of your own. Lastly, using this URI you need to form XML packets and craft HTTP requests with the right headers to push the data to the device.

MetroPimp simplifies the last step of this process by providing a model that represents the messages you want to send to devices, a single method call to handle serializing the message and sending it up the wire and a simple result which, in the case of failure, identifies the reason for the failure and provides informative message.

Enough talking… show us the code!!!

The main entry point of MetroPimp is the Pusher class (which can be mocked via the IPusher interface). It has a single method named Send that accepts a Message and returns a SendMessageResponse. To create a Pusher, simply new it up (or inject it using your preferred IOC container).

var pusher = new Pusher();

The following sections will define the classes you can use to push various notification formats to a device. For more information on message formats, please refer to Sending Push Notifications for Windows Phone on MSDN.

Raw

To send raw data to the phone (i.e. a notification that is not a toast or tile update), create an instance of Raw and send it using the pusher.

var rawMessage = new Raw {
    Uri = /* insert MPNS uri here */,
    Content = "hello world!"
    };
pusher.Send(rawMessage);

MPNS does not limit you to sending strings in the raw message. However, seeing as just about anything can be represented as a string, I decided this may be more universal. Under the covers, MetroPimp simply converts the string to its UTF8 representation and drops it into the request body.

For more information on sending and receiving raw notifications, refer to How to: Send and Receive Raw Notifications for Windows Phone on MSDN.

Toast

Sending a toast message is similar to sending a raw message in that you simply create an instance of a Toast object and pass it into Send. Toast has 3 fields that differ it from a raw:

• Text1 – The
  first bit of bold text that appears in the toast (required)
• Text2 – The second bit of text that appears in the toast (optional)
• Param – A parameter string to be passed to the application if the user taps the notification (optional)

var toastMessage = new Toast {
    Uri = /* insert MPNS uri here */,
    Text1 = "Hello World!",
    Text2 = "This is a toast.",
    Param = "helloWorldToasted"
    };
pusher.Send(toastMessage);

For more information on sending and receiving toast notifications, refer to How to: Send and Receive Toast Notifications for Windows Phone on MSDN.

Tile

Tile notifications can be specified to be either single or double sided tiles. As double sided tiles are only supported by WP7 Mango, the functionality has been split into two classes. Sending a single sided tile is done by specifying properties available on the Tile class.

• BackgroundImageUri – The URI to the background image for the tile. Can be either embedded or remote. (required)
• Title – The text to use as the title of the tile. (required)
• Count – The number to show in the blue bubble. Not defining or using 0 will clear the bubble. (optional)

var tileMessage= new Tile {
    Uri = /* insert MPNS uri here */,
    BackgroundImageUri = "background.png",
    Title = "Hello World",
    Count = 9
    };
pushed.Send(tileMessage);

Double sided tiles are sent by specifying properties on the DoubleSidedTile class. This class has the same set of properties as Tile for the front side of the tile, along with the following properties for the back side:

• BackBackgroundImageUri – The URI to the background image for the back of the tile.
• BackTitle – The text to use as the title for the back of the tile.
• BackContent – The text to display over the top of the background image on the back of the tile.

var doubleSidedTileMessage = new DoubleSidedTile {
    Uri = /* insert MPNS uri here */,
    BackgroundImageUri = "background.png", 
    Title = "Hello World", 
    Count = 9, 
    BackBackgroundImageUri = "back_background.png",
    BackTitle = "Hello Back",
    BackContent = "This is the back"
    };
pushed.Send(doubleSidedTileMessage);

For more information on sending and receiving toast notifications, refer to How to: Send and Receive Tile Notifications for Windows Phone on MSDN.

Auxiliary Properties

Raw, Toast, Tile and DoubleSidedTile all inherit the base Message class. This class provides the following optional properties to give you greater control over your notification messages:

• Id – a GUID value representing the message identifier.
• DeliveryInterval – Defines when the message should be delivered to the device. This can either be immediate (default), within 450 seconds or within 900 seconds.

SendMessageResponse

As soon as you’ve started trying to send notifications to devices, you’ll want to know if it gets through and/or why it failed. To handle this, Pusher.Send returns a SendMessageResponse object with the following properties:

• ErrorMessage – The message returned from MPNS on the HTTP response.
• MessageId – The
  identifier of the message. Only included if specified when sending.
• HttpStatusCode – The HTTP status code (e.g. 200, 400, 401, etc.) of the HTTP response returned from MPNS.
• HttpStatusDescription – The description of the HTTP status code returned from MPNS.
• StatusCode – The actual status of the notification’s delivery returned from MPNS (i.e. Received, Queue Full, Suppressed or Dropped).
• DeviceConnectionStatusCode – The status of the device’s connection to MPNS (i.e. Connected, Temporarily Disconnected, Inactive or Disconnected).
• SubscriptionStatusCode – The status of the device’s subscription to MPNS (i.e. Active or Expired).
• DetailsStatusDescription – The detailed description of the notification message’s status given the HTTP status code, MPNS status code, connection and subscription status according to the MSDN documentation.

For more information on response codes, refer to Push Notification Service Response Codes for Windows Phone on MSDN.

At the request of a colleague, I’m writing a little review on my precious baby… 🙂

Before I start boring most, I’m going to go out there and say I agree with *almost* everything this guy says, so I’ll just rehash a few things and give my opinion.

Firstly, some quick specs:

• Processor: Intel Core i7 820QM (8MB 1.73GHz – 3.066GHz)
• Memory: 8GB – 2DIMM 1333MHz DDR3
• HDD: 128 GB G.Skill Falcon SSD (aftermarket – orignially with 500GB 7200RPM)
• Graphics: 1GB ATI Mobility Radeon HD 4670
• Display: 16.0" 1080p Full HD RGBLED LCD with 2.0 MP Webcam
• Optical Drive: 4X Blu-ray Disc Combo Drive (DVD/CD +/- RW +BD Read)
• OS: Windows 7 Ultimate 64-bit
• Wireless: Intel WiFi Link 5300 (802.11a/g/n)
• Battery: 9-cell
• Dimensions: 2.5-5cm x 38cm x 26cm with 9-cell battery (H x W x D) 

Build and Design

Pros

Just like the rest of the (Studio) XPS range, this laptop just looks sexy. It’s sleek, shiny, dressed in a little leather, and has all the shiny touch-sensitive media buttons to match. The backlit keyboard is an absolute pleasure to use with its big spaced keys. The unit has very little flex in it at all. The 16” frameless glossy RGBLED screen is so gorgeous it makes staring at code even more fun. All the ports are in the right place (security lock, VGA, Ethernet, DisplayPort, HDMI, 2x USB, 2x headphone, microphone, USB/eSATA combo, FireWire, ExpressCard and card reader). And the sound is awesome…

Cons

The shininess of the plastic means that fingerprints really stand out. The Synaptics multi-touch track pad is hard to use and can get very annoying when trying to do simple things like crop using in a picture editor. The gloss on the screen makes it pretty hard to use outside on a bright day. The slot-loader drive is not my favourite as it tends to be a bit noisy (compared to tray-loaders) and quite slow. Also, the battery doesn’t quite line up with the rest of the unit so there is a bit of a gap constantly staring me in the face as I’m typing.

Most importantly… THERE’S NO SEPARATE RAM AND HDD COVER ON THE MAIN PANEL!!! This really got on my nerves when I switched in my SSD, because it meant what is usually a 2 minute job on most laptops became substantially more due to the funny screw placements and fear of “screwing” something up.

Performance

I ran a few benchmarks in an earlier post, and found my PCMarks at 9018 and WEI at 6.7. The review on notebookreview had the PCMarks at 6303. It also had the WPrime 32M result at 31.827s, which I bested at 15.257s. I think it’s safe to say that this baby ain’t no slug… 🙂

However, there is usually one downfall to all this power that not even the XPS 16 could surpass… HEAT! This thing gets sooo hot that putting it to good use means it can’t be used as a “laptop”. Noise was not too bad, but probably because the fan isn’t actually doing much.

Also, the battery life is a bit too short for my liking. My 9-cell only gets me about 2.5hrs, which some might think is quite high given what it’s running, but I think could be improved.

Conclusion

I love  it. It’s sexy and fast. It falls down on the build quality a bit like most Dell laptops and can be used to help you get through the winters, but the bang for buck this machine offers more than makes up for it in my eyes.

I finally received my new laptop… A Dell XPS 16. Specs:

So, first thing’s first… unbox it!

What next…? pull it apart of course!

Firstly, I’m not taking the credit for this one. It was Paul Stovell’s idea, but because he’s currently blogless I’m going to post it on mine… 🙂

The other day Paul was talking about a way of utilising the new VHD features of Windows 7 to keep all his documents and important stuff in a single location so that he can back them all up by copying one file. He had created a VHD and written a script that mounts it as a drive at start up. I thought this was a great idea, so I reproduced it and am now sharing it.

Firstly, create the VHD either using Virtual PC or the Disk Management console Action > Create VHD. Attach the VHD in Disk Manager then initialise and format it. Detach it and we can start scripting the attach process.

Create a text file named “Attach VHD.txt” with the following contents in %windir%\System32\GroupPolicy\User\Scripts\Logon entering the location of the VHD:

SELECT VDISK file="<Location of VHD>"
ATTACH VDISK
SELECT DISK 4
ASSIGN LETTER=U

NOTE: I’ve used U as the drive letter. You can change this if it will cause a conflict or you just don’t like it.

In the same location, create a batch file named “Attach VHD.cmd” with the following contents:

DISKPART /s "Attach VHD.txt"

Open the group policy editor (Hit Start then type “group policy”) and drill down to User Configuration > Windows Settings > Scripts (Logon).

Open the the Logon script properties and add the batch file you just created (it should open the location you created the files in by default).

Now, log out and on again. It may take a few seconds, but the autorun screen should pop up when the drive is attached. Open Windows Explorer and you will see a new drive in your tree.

That’s it!

I’ve actually now moved all my user folders (i.e. Documents, Music, etc.) to the VHD, but if you don’t want to do that you can just use the libraries to include a folder and set it as the save location.

Ok, so here’s the current version of my colour scheme…

What’s it look like?

Hope you like it. 🙂