Internal Project

December 27, 2013 ~ David Cameron ~ Leave a comment

I’ve decided to start a home project to test a few different technologies. The aim is to pick a a relatively simple project and implement this in a few different ways.

The simple project is a website where someone can post a status.

The rough outline is a website where:

Authenticated Users can:

post a status

Anonymous & Authenticated users can:

View current Status
View past statuses and search through them
‘Like’ a Status

I guess you could call this twitter without most of the useful features.

I’m aiming to try a number of different technologies, in decreasing level of comfort.

I’ll also be swapping pieces in and out with new implementations in different technologies.

The current plan

Document the UI – I’ve found this tends to make the implementation clearer
Implement in bootstrap, MVC4 with SQL Server backend
Swap the SQL Server backend for a No SQL database
Design JSON API to access app
Implement JSON api using WebApi backend
Replace MVC app with javascript client side framework (currently planning to try angular)
Replace the WebApi backend with an F# implementation
Replace the WebApi backend with node.js

I want to apply best practices throughout, which might mean learning new best practices as I start using new technologies.

I’ll post all parts of the implementation and any documentation on Github.

Working with External APIs

June 2, 2013 ~ David Cameron ~ Leave a comment

I’ve done a fair bit of work with external APIs. It’s generally a fairly frustrating process. So I’ve collected a list of best practises. Some of these are shamelessly stolen from colleagues.

Some of these are a bit ‘heavy’ and not appropriate for all circumstances. This isn’t a checklist to follow blindly, it’s just a list of some good ideas I’ve picked up.

1. Logging

Log all calls going out and all incoming responses along with dates & times. Any logging should occur as close to the boundary as possible. This is to ensure that you remove as many layers of abstraction. For example if making webservice call, you would want to log the raw request & response, not a serialised version of the data you send.

Logging will help answer questions like:

Response times, current and historical
Is the API still available & responding?
Has something changed at the remote end?
Have we correctly translated internal data to external calls?

2. Mock the API

When trying to develop or test something that depends on an external API, you’ve got a huge dependency on that API. Development and testing can stall if that external API is unavailable. This is also an issue with intermittent faults. A lot of time can be wasted on trying to identify that that API is unavailable or is not working.

If the API has a test environment to work against, typically this is less stable than the production environment. Updates are shipped more frequently, issues are addressed more slowly than the production environment. Response times are also typically slower, as the environment is rarely provisioned like production.

Mocking the API protected you from all of these. Typically this means capturing known good responses. When the mock API is enabled return, these known good responses. This can be extended further to trigger specific error conditions when certain data is passed in, eg a specific customer name will trigger a duplicate name error.

You will need a simple way to switch the mock API on/off, eg a simple config file change.

3. Write Functional tests of the API

This might seem a bit overboard, but doing this can save a huge amount of time. Typically we end up doing this sort of testing manually, but it is worth doing this more systematically.

Writing functional tests helps explore the limits of the API. It can help expose bugs in the API and clarify things that might be unclear from any documentation. It can help to benchmark performance. It can also be a great source of data for mocking the API.

It can also be a great source of protection from changes in that API. If something fails, you can simply run the functional tests to verify whether anything has changed.

Going Functional

May 25, 2013 ~ David Cameron ~ Leave a comment

As more functional features get added to C# I’m writing code rather differently.

Splitting Data & functions

Instead of having classes that combine data and functions, I’m increasing splitting the two. The functions take data as parameters rather than modifying members on the class. This is a move towards more pure functions.

So instead of this:

class Customer
{
public int Id { get; set; }
public string Name { get; set; }

public void Add()
{}
}

I’m more likely to write this:

class Customer
{
public int Id { get; set; }
public string Name { get; set; }
}

class CustomerService
{
public Customer Add(Customer customer)
{}
}

Passing functions to functions

I’m also increasingly using functions as more first class objects to be passed around.

So instead of this (contrived example):

class CustomerService
{
public Customer Add(Customer customer)
{
// add customer
CustomerModificationNotifier.Notify(“customer added”);
}
}

static class CustomerModificationNotifier
{
public static void Notify(string message)
{}
}

I might write this:

class CustomerService
{
public Customer Add(Customer customer, Action notifier)
{
// add customer
notifier(“customer added”);
}
}

static class CustomerModificationNotifier
{
public static void Notify(string message)
{}
}

Everything Old is New Again

This feels all very new and shiny. Except that it really doesn’t feel that new.

Say for example this:

  struct customer
  {
int id;
char name[50];
  };

  void add_customer(customer*);

  void add_customer(customer* cust)
  {
      /* add customer */
  }

Or this:

  struct customer
  {
int id;
char name[50];
  };

  void add_customer(customer*);

  void add_customer(customer* cust, void (*notifier)(int))
  {
notifier(1);
  }

  void notify(int);

  void notify(int number)
  {
      /* add customer */   }

It’s funny how everything that is new really isn’t that new after all.

Sometimes it really feels like all we’ve achieved in the last 40 years is better string management and bounds checking of arrays.

SSDT Limitations

January 11, 2013 ~ David Cameron ~ 6 Comments

In an earlier post I mentioned some of the limitations of SSDT. It’s worth covering them in a little more detail as they are significant.

The most significant issues relate deployment. Deployment for SSDT involves setting up a publish profile with a destination database to upgrade. This gives you the option of either publishing directly to the database or generating a script to run for the deployment.

Maximalist approach (with no control)

SSDT wants to control everything. It isn’t enough for to just control tables, procs, functions and triggers. SSDT also must control Logins, App Roles, Certificates, Groups.

Unfortunately security controls tend to vary by environment. Production will not look like UAT. For SSDT, everything looks the same.

This would all be fine and dandy except that you cannot control whether SSDT will manage these things. You cannot tell SSDT to ignore logins when running upgrades.

This means storing in version control production:

Logins & passwords
app role passwords
master keys for encryption

Other oddments

SSDT does not handle multiple file groups. Because clearly nobody would actually use that in production

Single file generated for upgrades

While SSDT provides the option to upgrade the database in place, but this a somewhat risky option for a production database. It is generally preferred to at least have the option to inspect what will be deployed.

Unfortunately for SSDT the upgrade generates a single file. In practice this means is that if the database upgrade fails for any reason recovery is rather complex. You’ll need to work out where the upgrade script failed and then what action you will take from there.

Conclusion

The product feels rather like a beta release, bringing even a simple project into production will expose some pretty serious limitation.

However the largest issue is more fundamental: the approach where you are expected to trust the magic box to generate a valid upgrade script. It is very easy to generate scripts that will fail to upgrade the database successfully. Earlier versions of Entity Framework had a similar approach, which they’ve now moved away from. The entire approach is fatally flawed.

Visual Studio 2012 Express

October 7, 2012 ~ David Cameron ~ Leave a comment

I just downloaded Visual Studio 2012 Express for home use.

Similar to the 2010 version, you are basically downloading ‘evaluation software’ until you register. This is silly, but Microsoft have managed to take this to a new level.

The new version of VS.Net 2012 Express will not let you register until you enter:

A business name
A business phone number
An address
Technologies you are interested in

I’m using this for personal use, mostly to play with new .Net technologies. This is related to any company. Thanks for making it all a bit more painful than it needs to be.

I’m guessing this was an attempt to gather more information about who is using the express version.

As a result for Microsoft’s purposes I’m now a developer from Tajikistan, interested in classic ASP (a language I haven’t touched in well over 5 years).