Thursday Night

Paul Betts’s personal website / blog / what-have-you

Calling Web Services in Silverlight using ReactiveXaml

View Comments

One of the scenarios that our summer intern Roberto Sonnino pointed out over the summer that was somewhat annoying in ReactiveXaml (my MVVM library that integrates the Reactive Extensions for .NET), was making RxXaml more friendly to asynchronous web service calls in Silverlight. Since this is a pretty important scenario, I decided to hack on it some (it’s always been possible, just not as easy).

Getting our function prototype correct

Remember from an earlier post that an IObservable can be used as a Future, a “box” that will eventually contain the result of a web service call or other asynchronous function, or the error information. So to this end, we’d really like our web service calls to all be vaguely of the form:

IObservable[Something] CoolWebServiceCall(object Param1, object Param2 /*etc*/);

However, we know that neither HttpWebRequest nor the web service generated code looks anything like that. Normal Silverlight objects use an OnCompleted event to fire a callback once the call completes. The Rx framework designers saw this coming however, and gave us a pretty handy function to deal with it: Observable.FromAsyncPattern(). This function will take a BeginXXXX/EndXXXX pair that follows the .NET Asynchronous Pattern and in return, will give us a Func that follows the form above, without us having to write some boilerplate code to connect the async method to an IObservable.

An example: looking at the Bing Translation API

One of the most simple public web services is the Bing Translation API, so its Translate() method is a good candidate for our example. The synchronous signature is:

string Client.Translate(string appId, string text, string fromLang, string toLang);

Here’s how we could take this and turn it into an Rx-friendly Async function – don’t be scared off by the five template parameters, they’re just the types of the parameters and return value, in the same order as a Func<T>:

var client = new LanguageServiceClient();
var translate_func = Observable.FromAsyncPattern[string,string,string,string,string]
    (client.BeginTranslate, client.EndTranslate);

IObservable[string] future = translate_func(appId, "Hello World!", "en", "de");
string result = future.First();   // This will *wait* until the call returns!!
>>> "Guten Tag, Welt!"

//
// Let’s try with an array…
//

var input = new[] {"one", "two", "three"};

// Fire off three asynchronous web service calls at the same time
var future_items = input.ToObservable()
    .SelectMany(x => translate_func(appId, x, "en", "fr"));

// This waits for *all* the web service calls to return
string[] result_array = future_items.ToArray()
>>> ["un", "deux", "trois"]

An important note for Silverlight!

Silverlight’s web service generated client code does something a bit annoying – it hides away the BeginXXXX/EndXXXX calls, presumably to make the Intellisense cleaner. However, they’re not gone, the way you can get them back is by casting the MyCoolServiceClient object to its underlying interface (i.e. the LanguageServiceClient object has a generated ILanguageServiceClient interface that it implements)

Turning this into a Command

Once we’ve got the function, turning it into a command is easy via a new method introduced to ReactiveAsyncCommandRegisterObservableAsyncFunction. This method is almost identical to RegisterAsyncFunction, but instead of expecting a synchronous Func which will be run on the TPL Task pool, it expects a Func that returns an IObservable as described above. Here’s a simple example of a good ViewModel object that demonstrates this:

public class TranslateViewModel : ReactiveObject
{
    //
    // Input text
    //

    string _TextToTranslate;
    public string TextToTranslate {
        get { return _TextToTranslate; }
        set { RaiseAndSetIfChanged(x => x.TextToTranslate, value); }
    }

    //
    // The "output" property we bind to in the UI
    //

    ObservableAsPropertyHelper[string] _TranslatedText;
    public string TranslatedText {
        get { return _TranslatedText.Value; }
    }

    public ReactiveAsyncCommand DoTranslate { get; protected set; }

    const string appId = "Get your own, buddy!";
    public TranslateViewModel()
    {
        var client = new LanguageServiceClient();
        var translate_func = Observable.FromAsyncPattern[string,string,string,string,string](
                client.BeginTranslate, client.EndTranslate);

        // Only one web call at a time please!
        DoTranslate = new ReactiveAsyncCommand(null, 1);

        //
        // ‘x’ is the CommandParameter passed in, which we will use as the
        // source text
        //

        var results = DoTranslate.RegisterObservableAsyncFunction(
            x => translate_func(appId, (string)x, "en", "de"));
           
        _TranslatedText = this.ObservableToProperty(
            results, x => x.TranslatedText);
    }
}

What does that get us?

Let’s review what this fairly short, readable code gets us – using a few simple bindings, we’ll have a fully non-blocking, responsive UI that correctly handles a lot of the edge cases associated with background operations: greying out the Button attached to the Command while the web call is running, saving off the results, then notifying the UI that there is something new to display so that it updates instantly, without any tricky callbacks or mutable state variables that have to be guarded by Lock statements to ensure multithreaded safety. That’s pretty cool.

Written by Paul Betts

September 26th, 2010 at 4:00 pm

Making INotifyPropertyChanged type-safe using Expressions

View Comments

Some folks on the mailing list for ReactiveXaml rightly pointed out some code in ReactiveXaml that could really use some work – my implementation of INotifyPropertyChanged’s “RaiseAndSetIfChanged”. For the impatient, here’s how I did it

Trying to make RaisePropertyChanged less verbose

This method is a helper method designed to encapsulate the following common code pattern when implementing a property:

  1. Make sure that the property is different than the original value
  2. Set the property to the new value
  3. Call RaisePropertyChanged with the name of the property that has changed

Doing this via a helper is a little bit tricky – the naive approach often forgets #1 entirely, or will try to do #2 and #3 in the opposite order, resulting in WPF/Silverlight not correctly updating. Another problem is that we usually have to pass in a string literal to RaisePropertyChanged (as well as RaiseAndSetIfChanged). While I’m a Rubyist and string literals don’t bother me, most .NET developers are annoyed by it, and it also can lead to really evil-to-debug issues after refactoring.

Using Expression Trees to implement a better version

Here’s what the new syntax for defining properties looks like:

string _SomeProperty;
public string SomeProperty {
    get { return _SomeProperty; }
    set { _SomeProperty = this.RaiseAndSetIfChanged(x => x.SomeProperty, value); }
}

Much improved, and fairly clean (we could do way better if C# had Macros, but I digress). Compare this with the old, dumb looking syntax:

string _SomeProperty;
public string SomeProperty {
    get { return _SomeProperty; }
    set { _SomeProperty = this.RaiseAndSetIfChanged(_SomeProperty, value,
        x => _SomeProperty, "SomeProperty");
    }
}

Or doing this by-hand:

string _SomeProperty;
public string SomeProperty {
    get { return _SomeProperty; }
    set {
        if (_SomeProperty == value)
            return;

        _SomeProperty = value;
        RaisePropertyChanged("SomeProperty");
    }
}

How can we do this? Expression Trees! When we write Expression<Func<T>> as a type instead of Func<T>, instead of getting a compiled method pointer, we get an object representing the AST of the lambda function. We’ll use this to grep out the property name.

Astute readers will notice an issue – how can we correctly decide T so that Intellisense works correctly? If we naïvely choose ‘ReactiveObject’ as T, none of our properties will be defined, since they’re all really defined on the derived class, not on ReactiveObject itself. To solve this, we’ll use a Mixin plus a cleverly typed generic function – since the first parameter of an extension method is the type itself, the compiler will automatically bolt this on to any ReactiveObject while still treating it as the correct derived type:

public static class ReactiveObjectExpressionMixin
{
    public static TRet RaiseAndSetIfChanged[TObj, TRet](this TObj This,
            Expression[Func[TObj, TRet]] Property,
            TRet Value)
        where TObj : ReactiveObject
    {
    }
}

Some tricky caveats

It’s not all biscuits and gravy though, there are a few subtle implementation details that might catch you when using this – remember though, that you can always fall back to writing it by-hand via RaisePropertyChanged, this method is solely a helper:

  1. You must name the field that backs your property as _TheFieldName – we use Reflection to set the backing field
  2. Writing the ‘this’ in ‘this.RaiseAndSetIfChanged’ isn’t optional – otherwise the extension method won’t be invoked, the old busted version will

Written by Paul Betts

September 19th, 2010 at 1:01 am

ReactiveXaml Series: QueuedAsyncMRUCache – the async version of MemoizingMRUCache

View Comments

A thread-safe, asynchronous MemoizingMRUCache

As we saw in a previous entry, MemoizingMRUCache is great for certain scenarios where we want to cache results of expensive calculations, but one disadvantage is that it is fundamentally a single-threaded data structure: accessing it from multiple threads, or trying to cache the results of several in-flight web requests at the same time would result in corruption. QueuedAsyncMRUCache solves all of these issues, as well as gives us a new method called AsyncGet, which returns an IObservable. This IObservable will fire exactly once, when the async command returns.

What places would I actually want to use this class? Here’s a motivating example: you’re writing a Twitter client, and you need to fetch the profile icon for each message – a naive foreach loop would be really slow, and even if you happened to write it in an asynchronous fashion, you would still end up fetching the same image potentially many times!

Using IObservable as a Future

One of the things that an IObservable encapsulates is the idea of a Future, described simply as the future result of an asynchronous operation. The pattern is implemented via an IObservable that only produces one element then completes. Using IObservable as a Future provides a few handy things:

  • IObservables let us block on the result if we want, via Observable.First().
  • IObservables have built-in error handling via OnError, so we can also handle the case where something goes pear-shaped.
  • We can easily group several IObservables together via Observable.Merge and wait for any (or all) of them.

A difficult problem – preventing concurrent identical requests

Furthermore, QueuedAsyncMRUCache solves a tricky problem as well: let’s revisit the previous example. As we walk the list of messages, we will asynchronously issue WebRequests. Imagine a message list where every message is from the same user:

For the first item, we’ll issue the WebRequest since the cache is empty. Then, we’ll go to the 2nd item – since the first request probably hasn’t completed, we’ll issue the same request again. If you had 50 messages and the main thread was fast enough, you could end up with 50 WebRequests for the same file!

What should happen? When the 2nd call to AsyncGet occurs, we need to check the cache, but we also need to check the list of outstanding requests. Really, for every possible input, you can think of it being in one of three states: either in-cache, in-flight, or brand new. QueuedAsyncMRUCache ensures (through a lot of code!) that all three cases are handled correctly, in a thread-safe manner.

Written by Paul Betts

August 23rd, 2010 at 11:43 pm

The case of the disappearing OnLoad exception – user-mode callback exceptions in x64

View Comments

For the impatient, you can skip to the end of the article to see what you should do about disappearing exceptions in desktop applications

The problem – why doesn’t this crash?

If you’ve got a 64-bit OS on your machine, try the following: open up Visual Studio 2010 and create a new WinForms project. Add an OnLoad event handler and paste in the following code:

namespace WindowsFormsApplication1
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void OnLoad(object sender, EventArgs e)
        {
            throw new Exception("Hey, where’d I go?");
        }
    }

    static class Program
    {
        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        [STAThread]
        static void Main()
        {
            Application.EnableVisualStyles();
            Application.SetCompatibleTextRenderingDefault(false);
            Application.Run(new Form1());

            MessageBox.Show("We should never get here");
        }
    }
}

Run the app – it succeeds! The exception we threw just disappeared into the ether, and the app just went on its merry way. A lot of people think this is a CLR bug or a WinForms bug, but it actually happens everywhere – you can trigger this using WinForms, WPF, and also in MFC / straight Win32. This is a complicated design decision in ntdll / NT kernel that I worked on for Vista and Windows 7, and it’s important to understand if you’re writing desktop applications on Windows. But before I can explain how it all works, we’ll learn some about exceptions in NT.

What is an SEH exception?

Different programming languages all implement their own notion of software exceptions differently, with different semantics (some like C# offer keywords like ‘finally’, for example, and others like VB.NET allow you to run a filter function on thrown exceptions to decide whether to catch them). However, Windows has its own notion of exceptions, called Structured Exception Handling. Almost every language-specific exception framework including C++ and .NET exceptions are really SEH exceptions on Windows.

This concept is built into the NT kernel – remember, that not all exceptions are traps (i.e, invoked by a ‘throw’ statement); exceptions are also generated by the hardware. When you run that *((void*)0x0) = 0x42; statement, the CPU attempts to walk the page tables looking for that memory mapping; when it fails, the CPU raises a hardware page fault exception, and the OS has a chance to take over. The OS will realize that this address is bogus, and it propagates that exception back to the application as an SEH exception with the code STATUS_ACCESS_VIOLATION.

The moral I’m trying to express is, that all of these program crashes like a Null pointer or a divide-by-zero are really all SEH exceptions. You could catch that exception and in fact, that’s exactly how the CLR’s NullReferenceException works – it’s really a reworded access violation, verified to point to NULL. When an exception is thrown, it walks up the stack looking for a handler – when it finally runs out of stack entries to traverse, UnhandledExceptionFilter is invoked, and normally that sends a signal to WER to come clean up the mess.

Here’s the reason that you probably shouldn’t implement exceptions yourself if you’re a language implementer – SEH exceptions are the only ones that understand how to traverse the user-mode / kernel-mode boundary (i.e. think about what happens when you throw an exception inside a user-mode APC for example – the exception has to go through kernel-mode to get back to your WinMain).

How come this doesn’t happen all the time?

Here’s why this seems to happen only on certain window messages – remember that window messages can originate from different sources, anyone(*) can queue a message to a window. However, certain window messages are directly sent via win32k.sys (the most notable one being WM_CREATE) as a direct synchronous result of a user-mode call.

Your app calls CreateWindow(); this results in a system call to NtCreateWindow. This syscall ends up going down to win32k.sys, who does the work of actually creating the window. None of the work is deferred, it’s all done in the context of the running user thread; as part of the work to create and show the window, win32k must call the app’s WndProc function. We’ve went from user mode to kernel mode, and now we have to come back to user mode, all in the same thread stack. Here’s what it looks like:

00 KERNELBASE!RaiseException+0×39
01 clr!RaiseTheExceptionInternalOnly+0×363
02 clr!IL_Throw+0×146
03 WindowsFormsApplication1!WindowsFormsApplication1.Form1.OnLoad+0×70
04 System_Windows_Forms_ni!System.Windows.Forms.Form.OnLoad+0x1a9
05 System_Windows_Forms_ni!System.Windows.Forms.Control.CreateControl+0x1c4
06 System_Windows_Forms_ni!System.Windows.Forms.Control.CreateControl+0×24
07 System_Windows_Forms_ni!System.Windows.Forms.Control.WmShowWindow+0xd8
08 System_Windows_Forms_ni!System.Windows.Forms.Control.WndProc+0x3dd
09 System_Windows_Forms_ni!System.Windows.Forms.Form.WndProc+0×243
0a System_Windows_Forms_ni!System.Windows.Forms.NativeWindow.Callback+0x16c
0b System_Windows_Forms_ni!DomainBoundILStubClass.IL_STUB_ReversePInvoke+0×50
0c clr!UMThunkStubAMD64+0×77
0d USER32!UserCallWinProcCheckWow+0x1ad
0e USER32!DispatchClientMessage+0xc3
0f USER32!__fnDWORD+0x2d
10 ntdll!KiUserCallbackDispatcherContinue
11 USER32!ZwUserShowWindow+0xa
12 clr!DoNDirectCall__PatchGetThreadCall+0x7b
13 System_Windows_Forms_ni!DomainBoundILStubClass.IL_STUB_PInvoke+0×42
14 System_Windows_Forms_ni!System.Windows.Forms.Control.SetVisibleCore+0×179
15 System_Windows_Forms_ni!System.Windows.Forms.Form.SetVisibleCore+0x25d
16 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner+0x1dc
17 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoop+0×81
18 WindowsFormsApplication1!WindowsFormsApplication1.Program.Main()+0×57

Frame 10 and 11 are the ones I’m talking about – we disappeared into the syscall, then reappear in userspace via KiUserCallbackDispatcherContinue, then we showed up into OnLoad in frame 3. SEH will now walk the stack back up but will hit a brick wall at KiUserCallbackDispatcherContinue. For complicated reasons, we cannot propagate the exception back on 64-bit operating systems (amd64 and IA64). This has been the case ever since the first 64-bit release of Server 2003. On x86, this isn’t the case – the exception gets propagated through the kernel boundary and would end up walking the frames back until it ended up at WindowsFormsApplication1.Program.Main().

When this happens, there are only two things we can sanely do: either make the exception disappear, or kill the application: rethrow the exception as noncontinuable, similar to the CLR’s StackOverflowException. Your catch blocks and finallys still run, but any attempts to catch the exception are ignored. The kernel architects at the time decided to take the conservative AppCompat-friendly approach – hide the exception, and hope for the best.

Why would I want to crash my own application?

In a small application, you can sometimes get away with eating the exception – if you can, great! However, most of the time you’re not so lucky; since SEH was in the middle of unwinding the stack and processing the exception then suddenly jumped to another location, the application is usually in a very inconsistent weird state when the call comes back, as if you just suddenly longjmp()’ed to another place. The end result for large apps is chaos – a simple AV turns into an difficult to debug application state corruption – if you’re a developer, you’ll pull your hair out wondering how your structures ended up in such a bizarre state. This also makes it very difficult to use the crash reports from WER or write your own error reporting code, since you’ll never see the true crash, only a later random one that resulted from the corrupted state.

The situation on Server 2003 and Vista

On Server ’03, XP64 (a rebranded Server ’03) and Vista, we kept the swallow exception behavior on both native x64 applications and WOW64 applications (32-bit programs running on a 64-bit OS), in an attempt to keep programs working. However, this was never an ideal solution – we needed a better way to cater to both modern applications by crashing and giving good bug reports, as well as to legacy applications that just happened to work correctly.

Windows 7 fixes this…kind of

The solution? In Windows 7, when a native x64 application crashes in this fashion, the Program Compatibility Assistant is notified. If the application doesn’t have a Windows 7 Manifest, we show a dialog telling you that PCA has applied an Application Compatibility shim. What does this mean? This means, that the next time you run your application, Windows will emulate the Server 2003 behavior and make the exception disappear. Keep in mind, that PCA doesn’t exist on Server 2008 R2, so this advice doesn’t apply.

What does this mean to you as a developer? This means, if you’re writing a new application, you always want to have a Win7-compatible manifest. Using a Win7 manifest tells Windows not to treat you like an older application and always use the latest OS features.

Completing the story with KB976038

I called out in the Win7 section that this applies to native x64 applications – well, what about 32-bit applications (WOW64 processes)? Unfortunately in Win7 RTM, WOW64 applications still have the Server ’03 behavior: exceptions are always swallowed. However, if you install this hotfix from Microsoft, WOW64 will now act just like x64 on Windows 7 – Win7-manifested x86 applications will crash, just like their x64 counterparts.

This fix also gets you more ways to control when we swallow or rethrow user-mode callback exceptions: first, for debugging purposes only, this fix adds a way to control the behavior via Image File Execution Options. You can enable/disable this option system-wide, or per-application (via the EXE name). I want to mention again though, that this option is for developer machines. If you set this key in an installer, you are Doing it Wrong and will make me sad.

Another way that you can enable/disable exception swallowing is via a new public API in Kernel32.dll – since this won’t be available in the SDK headers until Win7 SP1, you’ll have to dynamically invoke the API call via LoadLibrary and GetProcAddress. Here’s the definitions of these functions:

//
// If this flag is set, the exception will be *swallowed* (i.e. the Server ’03
// behavior)
//

#define PROCESS_CALLBACK_FILTER_ENABLED     0×1

BOOL
WINAPI
SetProcessUserModeExceptionPolicy(
    __in DWORD dwFlags
    );

BOOL
WINAPI
GetProcessUserModeExceptionPolicy(
    __out LPDWORD lpFlags
    );

So, the best future-proof way to call this function is:

DWORD dwFlags;
if (GetProcessUserModeExceptionPolicy(&dwFlags)) {
    SetProcessUserModeExceptionPolicy(dwFlags & ~PROCESS_CALLBACK_FILTER_ENABLED); // turn off bit 1
}

Hey, there’s a Vista package too

On Vista, the story’s a bit different: since PCA doesn’t have the proper support on Vista, the hotfix will add this behavior only for applications (either native or WOW64) who specifically ask for it via the public API or the IFEO key. Since this is a hotfix for an older, stable operating system, the hotfix tries to be more conservative so existing apps aren’t broken.

In Summary – ways to control user-mode exceptions

Here’s the tl;dr; version of this article:

  • If you’re writing desktop applications, install this hotfix from Microsoft on all your development machines until Win7 SP1 comes out.
  • Mark all your new applications as Win7-compatible.
  • If the manifest doesn’t work for you for some reason, or you’re shipping for Vista, try to call the public API via GetProcAddress.

Written by Paul Betts

July 20th, 2010 at 11:42 pm

VS2010 Snippets for ReactiveXaml

View Comments

One of the folks using RxXaml is our intrepid intern, Roberto Sonnino (he’s the one who came up with the short name RxXaml!) – he’s created some handy snippets that will expand out to the standard forms of a read-write ReactiveObject property, as well as a property backed by ObservableAsPropertyHelper. I’m personally against snippets since I want typing extra code to cost me personally as much as possible, encouraging me to be more general and brief! However, I think in this case it’s a good idea – here are the snippets:

Written by Paul Betts

July 20th, 2010 at 12:35 am

ReactiveXaml series: Using MemoizingMRUCache

View Comments

Memoization and Caching

One thing that is useful in any kind of programming is having a look-up table so that you don’t have to spend expensive calls to fetch the same data that you just had recently, since fetching the data and passing it around via parameters often gets ugly. A better way is to use a cache – store values we’ve fetched recently and reuse them. So, a naïve approach would be to store the data off in a simple Dictionary. This might work for awhile, but you soon realize as Raymond Chen says, “Every cache has a cache policy, whether you know it or not.” In the case of a Dictionary, the policy is unbounded – an unbounded cache is a synonym for ‘memory leak’.

To this end, one of the things that comes with ReactiveXaml is a class called MemoizingMRUCache. As its name implies, it is a most recently used cache – we’ll throw away items whose keys haven’t been requested in awhile; we’ll keep a fixed limit of items in the cache, unlike other approaches involving WeakReference that keep references to items only if they’re used on some other thread at the time. Since most desktop / Silverlight applications aren’t so massively multithreaded as a web application, using a WeakReference approach means we’ll just get constant cache misses.

Using MemoizingMRUCache

Really when it comes down to it, you can just think of MemoizingMRUCache as just a proxy for a function – when you call Get, it’s going to invoke the function you provided in the constructor. One thing that’s important to understand with this class, is that your function must be a function in the mathematical sense – i.e. the return value for a given parameter must always be identical. Another thing to remember is that this class is not implicitly thread-safe – unlike QueuedAsyncMRUCache, if you use it from multiple threads, you have to protect it via a lock just like a Dictionary or a List.

Here’s a motivating sample:

// Here, we’re giving it our "calculate" function – the ‘ctx’ variable is
// just an optional parameter that you can pass on a call to Get.
var cache = new MemoizingMRUCache[int, int]((x, ctx) => {
    Thread.Sleep(5*1000);     // Pretend this calculation isn’t cheap
    return x * 100;
}, 20 /*items to remember*/);

// First invocation, it’ll take 5 seconds
cache.Get(10);
>>> 1000

// This returns instantly
cache.Get(10);
>>> 1000

// This takes 5 seconds too
cache.Get(15);
>>> 1500

Maintaining an on-disk cache

MemoizingMRUCache also has a feature that comes in handy in certain scenarios: when a memoized value is evicted from the cache because it hasn’t been used in awhile, you can have a function be executed with that value. This means that MemoizingMRUCache can be used to maintain on-disk caches – your Key could be a website URL, and the Value will be a path to the temporary file. Your OnRelease function will delete the file on the disk since it’s no longer in-use.

Some other useful functions

  • TryGet – Attempt to fetch a value from the cache only
  • Invalidate – Forget a cached key if we’ve remembered it and call its release function
  • InvalidateAll – Forget all the cached keys and start from scratch

Written by Paul Betts

July 13th, 2010 at 10:48 pm

ReactiveXaml series: Implementing search with ObservableAsPropertyHelper

View Comments

Implementing an auto-search TextBox using Rx and ReactiveXaml

One of the most important classes in ReactiveXaml called ObservableAsPropertyHelper is a class that allows you to take an IObservable and convert it into a read-only, change notifying property. This class is really useful for exposing the results of your code (i.e. the “output”). This class makes it easy to complete the scenario that was described in the previous blog post about ReactiveAsyncCommand. One of the cool things about ObservableAsPropertyHelper, is that it guarantees that it will run notifications on the UI thread via the Dispatcher so that you don’t have to think about what thread the observable notification came in on.

The sample app




Click on the image to download the sample project.

Going through the code

First, let’s look at our main data item – a Flickr search result item. Since we will never change these objects, we don’t need any INotifyPropertyChanged goo, just regular old auto-properties:

public class FlickrPhoto {
    public string Title { get; set; }
    public string Description { get; set; }
    public string Url { get; set; }
}

Now, the app data model – there’s two real bits; the current search text, and the List of FlickrPhoto results. In ReactiveXaml, all of this code below is boilerplate – these code chunks are just some stuff to memorize or put into a snippet and never look at it again. Note: once again because of a syntax highlighting glitch, generics are using [] instead of < >

public class AppViewModel : ReactiveValidatedObject
{
    //
    // This is the canonical way to make a read-write property
    //

    string _SearchTerm;
    public string SearchTerm {
        get { return _SearchTerm; }
        set { this.RaiseAndSetIfChanged(x => x.SearchTerm, value); }
    }

    //
    // This is the canonical way to make a read-only property whose value
    // is backed by an IObservable
    //

    ObservableAsPropertyHelper[List[FlickrPhoto]] _Photos;
    public List[FlickrPhoto] Photos {
        get { return _Photos.Value; }
    }

    ObservableAsPropertyHelper[Visibility] _SpinnerVisibility;
    public Visibility SpinnerVisibility {
        get { return _SpinnerVisibility.Value; }
    }

    public ReactiveAsyncCommand ExecuteSearch { get; protected set; }
}

Now here’s the interesting part

Our goal is to write a search box which automatically issues searches in the background as the user types, similar to what most browsers do with the address bar. However, there are a number of tricky aspects to this:

  • We don’t want to issue too many requests, especially when the user is still typing, so wiring something directly to KeyUp would be lousy.
  • Don’t issue queries for empty strings, and don’t issue the same query 2x (for example, if the user types “foo”, then quickly hits Backspace, then retypes ‘o’, we should realize that we already have the right results)
  • The delay should be consistent, so having a global timer won’t work because sometimes the user will hit the key right before the timer fires, so the delay will vary wildly between the max time and instantaneous.

Implementing this properly using traditional methods would be absolutely awful. Here’s the code on how we do it, and it’s 5 lines in the constructor:

public AppViewModel()
{
    ExecuteSearch = new ReactiveAsyncCommand(null, 0);

    //
    // Take the inflight items and toggle the visibility
    //

    var should_spin = ExecuteSearch.ItemsInflight.Select(x => x > 0 ? Visibility.Visible : Visibility.Collapsed);

    //
    // This was described last time too, we actually do the async function
    // here and RegisterAsyncFunction will return an IObservable which
    // gives us the output, one item per invocation of ExecuteSearch.Execute
    //

    var results = ExecuteSearch.RegisterAsyncFunction(
        term => GetSearchResultsFromFlickr((string)term));

    //
    // Here’s the awesome bit – every time the SearchTerm changes
    // throttled to every 800ms (i.e. drop changes that are happening
    // too quickly). Grab the actual text, then only notify on unique
    // changes (i.e. ignore "A" => "A"). Finally, only tell us when
    // the string isn’t empty. When *all* of those things are true,
    // fire ExecuteSearch and pass it the term.
    //

    this.ObservableForProperty[AppViewModel, string]("SearchTerm")
        .Throttle(TimeSpan.FromMilliseconds(800))
        .Select(x => x.Value).DistinctUntilChanged()
        .Where(x => !String.IsNullOrWhiteSpace(x))
        .Subscribe(ExecuteSearch.Execute);

    //
    // This code is also boilerplate, it’s the standard way to take our
    // observable and wire it up to the property, giving it an initial
    // value.
    //

    _SpinnerVisibility = new ObservableAsPropertyHelper[Visibility](
        should_spin, x => RaisePropertyChanged("SpinnerVisibility"), Visibility.Collapsed);

    _Photos = new ObservableAsPropertyHelper[List[FlickrPhoto]](
        results, _ => RaisePropertyChanged("Photos"));
}

Here’s the code that actually does the work as an aside, it’s not nearly as pretty:

//
// If you don’t understand this code, don’t worry about it, I just got lazy.
// We’re just hack-parsing the RSS feed and grabbing out title/desc/url and
// newing up the list of FlickrPhotos while blatantly abusing Zip.
//

public static List[FlickrPhoto] GetSearchResultsFromFlickr(string search_term)
{
    var doc = XDocument.Load(String.Format(CultureInfo.InvariantCulture,
        "http://api.flickr.com/services/feeds/photos_public.gne?tags={0}&format=rss_200",
        HttpUtility.UrlEncode(search_term)));
    if (doc.Root == null)
        return null;

    var titles = doc.Root.Descendants("{http://search.yahoo.com/mrss/}title")
        .Select(x => x.Value);
    var descriptions = doc.Root.Descendants("{http://search.yahoo.com/mrss/}description")
        .Select(x => HttpUtility.HtmlDecode(x.Value));
    var items = titles.Zip(descriptions,
        (t, d) => new FlickrPhoto() { Title = t, Description = d }).ToArray();

    var urls = doc.Root.Descendants("{http://search.yahoo.com/mrss/}thumbnail")
        .Select(x => x.Attributes("url").First().Value);

    var ret = items.Zip(urls, (item, url) => { item.Url = url; return item; }).ToList();
    return ret;
}

Written by Paul Betts

July 5th, 2010 at 5:26 pm

ReactiveXaml series: ReactiveObject, and why Rx is awesome

View Comments

ViewModels via ReactiveObject

Like any other MVVM framework, ReactiveXaml has an object designed as a ViewModel class. This object is based on Josh Smith’s ObservableObject implementation in MVVM Foundation (actually, many of the classes’ inspiration come from MVVM Foundation, Josh does awesome work!). The Reactive version as you can imagine, implements INotifyPropertyChanged as well as IObservable so that you can subscribe to object property changes.

Other things that are nice to have

ReactiveObject also does a few nice things for you: first, when you compile ReactiveXaml in Debug mode, it will print debug messages using its logging framework whenever a property changes. Another example is, implementing the standard pattern of a property that raises the changed event is a few lines shorter:

int _someProp;
public int SomeProp {
    get { return _someProp; }
    set { this.RaiseAndSetIfChanged(x => x.SomeProp, value);}
}

Compared to the traditional implementation which is a few lines longer:

int _someProp;
public int SomeProp {
    get { return _someProp; }
    set {
        if (_someProp == value)
            return;
        _someProp = value;
        RaisePropertyChanged("SomeProp");
    }
}

Some philosophy

A lot of examples of the Reactive Extensions make its domain appear really constrained, like the only thing it’s ever useful for is either handing web service requests and implementing drag-and-drop. However, here’s the key thing to realize – a property change notification is an event. Once you realize that Reactive Programming applies to any time an object changes state, Rx suddenly becomes far more applicable to any programming domain – really, Rx is a library to model state machines that are context-free with respect to threading.

Abstracting away context is critical for a multicore + cloud world

What do I mean by “context-free”? Well, just as LINQ abstracts away the idea of a for loop into these operations like “Select” and “Where”, where the implementation is separate from the semantics, Rx does the same thing. When you call Where() on an IEnumerable, the default implementation is a simple loop – but add an .AsParallel(), and suddenly the same code is now running on multiple cores. Use Dryad, and the same code is running on multiple machines in an HPC cluster.

In traditional imperative programming, we are always explicit about the thread context in which we were running in – to put work on another thread, we had to call new Thread(). The really interesting thing about Rx, and the reason that Rx is made by the “Cloud Programmability Group” inside MS, is that Rx is also an abstraction above context – in Rx, you usually don’t care what thread you’re running on, and you only have to specify it when you explicitly do care via ObserveOn (like to deal with WPF’s thread affinity). Observables are essentially a way to declaratively write dependencies between incoming data sources without explicitly specifying the mechanism of their synchronization, only their semantics.

Do you see where I’m going with this? If the concept of “Lock” and “Thread” are no longer concretely tied to a kernel thread and a Critical Section, this means that you can write the same Rx code, and go from a single thread, event-based model with no locks, to a multicore model that synchronizes via locking, to a cluster of computers in a lab which synchronize via a message-passing model, to a giant cloud computing array that synchronizes via a service bus. I think that’s awesome.

Written by Paul Betts

July 1st, 2010 at 10:31 pm

ReactiveXaml: A compelling combination of MVVM and Reactive Extensions (Rx)

View Comments

I’ve been hacking on a library in my spare time (Hah!) that I really think has the potential to change how folks write Silverlight/WPF applications and I’m really excited about it. After testing concepts and refining the interface using several different sample apps, I believe I’m confident enough in the concept that it needs to be seen by more people. I also have a newfound respect for the folks who worked on the BCL and the PMs in DevDiv – writing a library to get something done is easy; writing a library that gets stuff done and is elegant and straightforward is decidedly not.

This library is an exploration I’ve been working on for several weeks on combining WPF Model-View-ViewModel paradigm with the Reactive Extensions for .NET (Rx). Combining these two make managing concurrency as well as expressing complicated interactions between objects possible in a declarative, functional way. Put simply, if you’ve ever had to chain events / callbacks together and declare state ints/booleans to keep track of what’s going on, Reactive Extensions provides a sane alternative.

I’m going to be posting quite a bit more about this library as well as a sample application, but for now, check out the code to ReactiveXaml on Github.

What’s in this library

ReactiveCommand – an implementation of ICommand that is also a Subject whose OnNext is raised when Execute is executed. Its CanExecute can also be defined by an IObservable<bool> which means the UI will instantly update instead of implementations which rely on RequerySuggested.

ReactiveAsyncCommand – a derivative of ReactiveCommand that encapsulates the common pattern of “Fire asynchronous command, then marshal result back onto dispatcher thread”. Allows you to set a maximum level of concurrency as well (i.e. “I only want 3 inflight requests” – when the maximum is reached, CanExecute returns false).

ReactiveObject – a ViewModel object based on Josh Smith’s implementation, that also implements IObservable as a way to notify property changes. It also allows a straightforward way to observe the changes of a single property.

ReactiveValidatedObject – a derivative of ReactiveObject that is validated via DataAnnotations by implementing IDataErrorInfo, so properties can be annotated with their restrictions and the UI will automatically reflect the errors.

ObservableAsPropertyHelper – a class that easily lets you convert an IObservable<T> into a property that stores its latest value, as well as fires NotifyPropertyChanged when the property changes. This is really useful for combining existing properties together and replacing IValueConverters, since your ViewModels will also be IObservables.

StopwatchTestScheduler – this class allows you to enforce time limits on items scheduled on other threads. The main use for this is in unit tests, as well as being able to say things in Debug mode like, “If any item runs in the Dispatcher scheduler for longer than 400ms that would’ve made the UI unresponsive, crash the application”.

Blend SDK Integration

AsyncCommandVisualStateBehavior – this behavior will watch a ReactiveAsyncCommand and transition its target to different states based on the command’s status – for example, displaying a Spinner while a command is running.

FollowObservableStateBehavior – this behavior will use the output of an IObservable<string> and call VisualStateManager.GoToState on its target; using Observable.Merge makes it fairly straightforward to build a state machine based on the changes in the ViewModel.

ObservableTrigger – this trigger will fire when an IObservable calls OnNext and can be tied to any arbitrary Expression Action.

Other stuff that’s useful

MemoizingMRUCache – this class is non-threadsafe most recently used cache, and can be used to cache the results of expensive lookups. You provide thefunction to use to look up values that aren’t known, then it will save the results. It also allows a “destructor” to be run when an item is released from the cache, so you can use this to manage an on-disk file cache as well (where the “Get” function downloads a file, then the “Release” function deletes it).

QueuedAsyncMRUCache – this class is by far the most complicated in this library, its goals are similar to MemoizingMRUCache, but instead of returning the result immediately, it will schedule a Task to run in the background and return an IObservable representing the result (a Future). Once the Future completes, its result is cached so subsequent requests will come from memory.

The advantage of this class is that subsequent identical requests will block on the outstanding one (so if you ask for “foo.com” on 3 separate threads, one of them will send out the web request and the other two threads will receive the result as well). This class also allows you to place a blocking limit on the number of outstanding requests, so that further requests will block until some of the inflight requests have been satisfied.

IEnableLogger – this is an implementation of a simple logger that combines some of log4net’s syntax with the ubiquity of the Rails logger – any class that implements the dummy IEnableLogger interface will able to access a logger for that class (i.e. this.Log().Warn("Something bad happened!");)

Written by Paul Betts

June 15th, 2010 at 1:56 am

WPF Data Validation using DataAnnotations

View Comments

Data Validation in WPF isn’t that great

As part of trying to put together a “base application” for starting projects at work, I’ve had a look at WPF Data Validation for the first time. Compared to ASP.NET MVC, WPF’s data validation is really primitive – it basically leaves you to do everything by hand. Fortunately, the DataAnnotations DLL is sufficiently decoupled, that we can actually use it – props to Brad Wilson’s Data Annotations article for the inspiration to add it to WPF.

Disclaimer: The DataAnnotation guys would probably tell me I’m Doing It Wrong(r) here, and this code is proof-of-concept and very inefficient. However, it’ll gets the idea across

Using DataAnnotations

First, we’ll make a dummy class:

public class PersonEntry : DataValidable
{
    [StringLength(35, MinimumLength = 3)]
    public string Name {get; set;}

    [StringLength(10, MinimumLength = 10)]
    public string PhoneNumber {get; set;}
}

public class DataValidatable : IDataErrorInfo
{
    public class Foo : IDataErrorInfo
    {
        public string Error{
            get { throw new NotImplementedException(); }
        }

        public string this[string columnName] {
            get { throw new NotImplementedException(); }
        }
    }
}

Now, let’s implement IDataErrorInfo using data annotations:

public string this[string columnName]
{
    get {
        var prop = GetType().GetProperty(columnName);
        var validationMap = prop
            .GetCustomAttributes(typeof(ValidationAttribute, true)
            .Cast<validationattribute>();

        foreach(var v in validationMap) {
            try {
                v.Validate(prop.GetValue(this, null), columnName);
            } catch (Exception ex) {
                return ve.Message;
            }
        }

        return null;
    }
}
</validationattribute>

Making it show up in the view:

To actually see your validations get called, you need some magic words in the binding – here’s how to do it:

Text="{Binding Person.Name, Mode=TwoWay, NotifyOnValidationError=true, ValidatesOnDataErrors=true}"

Written by Paul Betts

April 27th, 2010 at 10:16 pm