A Developer’s Notes

Ayende and Eli, among several others, are having a little back and forth about testability and how that is impacted by design choices. Here’s some more background.

I think the question that Eli poses at the end of his post merits a little more attention. Here it is for simplicity’s sake

Quiz: Which is easier to maintain:

File.WriteAllText(filename, text);

or

using(TextWriter writer = IoC.Resolve<ifilewriterfactory>().Create(filename)) writer.Write(text);

The first approach represents maintainability while the second approach represents “testability” insofar as it has been constructed to allow the injection of stub or mock objects, while the first approach, using a sealed static method, cannot easily have its behavior modified at test execution time.

There are more differences to the two approaches than are generally being discussed, and I think it might be valuable to talk about them a little bit more.

Here’s a quick rundown of the differences.

The File class doesn’t provide deterministic disposability in the same way that the second example’s using statement does. So let’s remove the using statement from the second example.

The File class isn’t injecting any dependencies, so let’s remove the container resolution bit.

After those two steps, we’re left with this (I’ve named the concrete factory on my own):

FileWriterFactory.Create(filename).Write(text);

The signature is a little more verbose, but otherwise, we’re not dealing with a significantly different idiom here. I think it’s safe to say that the fundamental behavior of the two approaches are pretty similar.

Reducing the two approaches to the lowest common denominator reveals that the question itself may be spurious, or at least poorly framed. What value is there in comparing the testability or maintainability of two approaches that in both concrete and abstract terms fulfill two different sets of requirements?

In my opinion, the maintenance-based critique of the second approach as needless complexity is a straw man. You may as well ask whether it’s easier to maintain an algorithm that performs basic arithmetic or one that implements Runge-Kutta approximation of differential equations.

A more productive comparison of would involve setting out a couple of basic requirements and implementing them using dependency injection and without using it. In that case, we might see that the ability to automatically resolve dependencies against services such as authentication (of course we need permissions to write the file, right?) is actually a win in the long term.  But then again, we might not.

The example Eli gave of a development team implementing an unnecessary feature and then being burned by it down the road is a great cautionary tale about why good product management is important, but I am not convinced that it holds much water when it comes to a discussion about whether or not designing your code for testability is a good practice.

I think that framing the discussion to be more about whether or not your approach to designing applications takes into account that various types of software that are out there and what their expectations are in terms of lifecycle, staffing, and maintenance might help to better address the core issue here: is it ever acceptable to put the big guns away and just hack out something that works?

In Part 1 I discussed the logical architecture of a distributed system. Part 2 is dedicated to the physical architecture.

Physical Application Architecture

There are two primary concerns with the physical architecture of a distributed system: reliability and scalability. These two concerns are often viewed as marketing-driven buzzwords, and in fact they are frequently used as such. In an attempt to clear up this ambiguity, I will provide definitions for them.

Reliability & Scalability

In order to be considered reliable, a system must be constructed so that a failure of one component of service does not impact the remaining services. Another way of thinking about reliability is as fault tolerance. If the Listing Manager component goes offline, the Listing Service component should not be adversely impacted.

In order to enhance the reliability of the system, MSMQ queues are used to spool units of work prior to processing. These queues are persistent and provide a level of safety that simple HTTP-based web service invocations can’t provide.

Additionally, each component runs in its own isolated environment. Whether these independent environments are virtual or physical is not important. The important part is that they are separate.

This brings us to scalability. Since each component of our application is isolated from the next, and the interactions are loosely coupled, it is possible for us to use standard load balancing and clustering techniques to scale the application horizontally. In the following diagram you can see what the physical architecture looks like initially, and one option for what it might look like in the future.

By replacing the single instance nodes with clusters, we can dramatically increase the number of transactions the system can process. If it becomes necessary in the future, the database cluster can be replicated to other instances and the entire system can be clustered.

In part 3, I’ll take a closer look at the design of one of the queue processors.

There are many reasons to implement an application as a distributed system, and it’s not a decision to be undertaken lightly. For the moment, let’s assume that this decision has already been made and that it was a good one. Subsequent installments of this series will address the decision making process itself.

Application Overview

The initial requirements for the distributed system calls for a front end web service that accepts data from end users. This web service will expose features for creating a user account on the system and for modifying that account. In addition, it will allow and end user to update account statistics. The fundamental purpose of this service is for people to be able to create an account, assign a set of keywords, and retrieve lists of items for sale on a wide range of e-commerce websites that match those keywords. When a list of items for sale is requested, an XML document is returned that includes the title of the item for sale, a brief description, the price, and a link to that item for sale on the e-commerce site it came from. The link will have been modified to include the system’s affiliate account id. The service will track how much money an account has earned via affiliate sales and manage these funds internally.

It is almost immediately evident that as this service becomes more popular, the volume of traffic handled by the front-end web service will increase. Since it’s important that this service scale, the application will be implemented as a distributed system. This means that the web service front end needs to operate independently from the back-end processes that manage account creation and update, obtain the data from the e-commerce sites, and credit user accounts for successful affiliate purchases.

Logical Application Architecture

The first step in our design process is to articulate a draft version of the logical application architecture. In order to accomplish this, we’ll need to break down the different functional behaviors into discrete units. The logical architecture will help us to divide responsibilities and establish both the method and the means of collaboration between the different components. For example, we’ll be able to determine what communications protocols are used internally by the application, whether the requests will be synchronous or asynchronous, and most importantly, by keeping scalability in mind, potential physical architectures will begin to emerge.

Here is an example of a draft logical application architecture.

One of the first to mention about this architecture is that it’s based on the Unit of Work pattern. Units of work originate from multiple points in the system, but each one is represented as an XML message that’s processed through MSMQ. Queue processors wait for these messages and operate on them as they arrive. The only task that isn’t processed through a queue is the fulfillment of a request for product listings. These listings are retrieved directly from the database. Some of the units of work are also initiated by scheduled jobs within each component, but we’ll overlook that complexity for the time being.

In the next installment, I’ll talk about the physical application architecture.

One of my favorite syntax structures in C# is the using statement. It’s simple, concise, and helpful. Here’s a quick usage guide.

You’re going strong, writing your application, and you get that feeling in the back of your mind that you might need to be thinking more about performance. You’re creating an object that’s a real heavyweight.

One of the first things you should think about in this scenario is whether or not this object is bound to a scarce physical resource such as an I/O stream, a file handle, or a database connection. If your object meets this description, verify that it implements IDisposable. This interface is the key to the using statement. Here’s how it works: any type that implements IDisposable must implement Dispose() with the intention of freeing any unmanaged resources that the type has referenced during its life cycle. The using statement creates a lexical shortcut by initiating a block scope around a group of code. In other words, using operates a lot like try...catch...finally. When execution leaves the scope of that using block, Dispose() is called. This guarantees that you’ve freed any finite resources that your program was referencing.