Smartphone image processing development made easy!

A very popular open source C# project for image processing, neural networks, machine learning etc. is the AForge.NET Framework, developed by Andrew Kirillow. AForge.NET has been used in numerous applications. It also forms the basis for the Accord.NET Framework, developed by César de Souza, which substantially extends AForge.NET with additional image processing filters, machine learning algorithms, statistical distributions and much much more. Numerous examples of what Accord.NET Framework is capable of are available here.

AForge.NET was developed to run on the .NET Framework and uses the System.Drawing assembly as basis for image manipulation and processing. In particular the dependency on System.Drawing heavily limits the applicability of AForge.NET (and subsequently, Accord.NET) to more modern platforms, like WPF, Windows (for tablet and desktop, as opposed to desktop-only applications), and Windows Phone.

It seemed like a terrible waste that these frameworks would only be available to the outdated range of Windows Forms applications. So, to enable the invaluable functionality of AForge.NET and Accord.NET for modern Microsoft platforms, I started the project of porting these frameworks into Portable Class Libraries, PCL. These efforts are also open sourced and are available on Github: Portable AForge and Portable Accord.

It should be noted that Portable AForge and Portable Accord cover a subset of the assemblies in the pure .NET based frameworks. Video and robotics support is not included, and audio support is limited. Windows Forms UI controls have not been ported. Nevertheless, practically all image processing and scientific computing functionality is included in the portable frameworks.

To facilitate continuous inclusion of updated code for the two frameworks, I have striven to keep the original AForge.NET and Accord.NET code bases as untouched as realistically possible. To cover up for those parts of the framework codes that are not applicable in the portable class libraries, I have therefore developed two support assemblies called Shim and Shim.Drawing.

Shim, which contains simple implementations of non-UI types and methods unavailable in the PCL profiles, is maintained in a separate Github repository, since this assembly is also immediately applicable as a legacy code bridge in other modern target adaptations, such as the port of the fo-dicom library to the (modern) Windows and Windows Phone platforms.

Shim.Drawing is internal to the Portable AForge and Accord assemblies, and is contained in the Portable AForge Github repository. It provides a down-scaled replacement for the System.Drawing assembly. Along with the platforms of primary interest, Shim.Drawing can also be contained in a .NET Framework WPF project as long as System.Drawing is not simultaneously referenced. Portable AForge and Accord is thus a also shortcut to using AForge.NET and Accord.NET in WPF applications.

To enable the portable framework assemblies on a specific target platform, the Shim and Shim.Drawing assemblies come in various flavors. There are for example specific Windows Phone (Silverlight) 8 versions of Shim and Shim.Drawing that should be referenced in a WP8 application using the Portable AForge and Accord assemblies.

Currently, Portable AForge and Portable Accord are available for the following target platforms:

• .NET Framework 4.5 and higher (for WPF applications)
• Windows 8 and higher (modern tablet/desktop applications)
• Windows Phone Silverlight 8 and higher
• Universal applications, i.e. Windows Phone 8.1 + Windows 8.1
• Xamarin.iOS
• Xamarin.Android

Yes, you read it right! My most recent developments also incorporate the ability to target the Xamarin.iOS and Xamarin.Android platforms. The Xamarin assemblies are however not available as open source. If you are interested in developing AForge and Accord based applications for Xamarin.iOS and/or Xamarin.Android, please contact me directly at licenses@cureos.com.

And now, with AForge and Accord available on the three mobile platforms, cross-platform development using Xamarin.Forms has also become a reality! This means that it is practically possible to contain an entire C# code base in a PCL core library that uses AForge and Accord, and consume this core library from end applications for all three mobile platforms, Windows Phone, Xamarin.iOS and Xamarin.Android.

Just to show what can now be very quickly done, I copied the functionality of the Face Detection (Haar Object Selector) sample application from the Accord.NET samples and contained it in a Xamarin.Forms project. All application code, including setting up the graphical layout and the face detection operation, is contained in the PCL core project. The platform specific end application projects only wrap the core assembly.

And this is what the face detection sample application looks like on the three mobile platforms:

A first small step, perhaps, but the possibilities that have now been opened up are practically infinite :-)