S h o r t S t o r i e s

// Tales from software development

Posts Tagged ‘C#

MySql Connector/NET Reading from stream has failed

leave a comment »

We have a legacy .NET Framework 2.0 application that uses MySql Connector/NET 6.3.6.0.

This week, for no obvious reason, this started failing while reading a result set of around 70,000 rows on my development PC.

The top level MySqlException message was “Fatal error encountered during data read.” This had a nested nested MySqlException with the message “Reading from the stream has failed.” This also contained an inner exception; an EndOfStreamException with the message “Attempted to read past the end of the stream.” and a stack trace of:

at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
at MySql.Data.MySqlClient.MySqlStream.LoadPacket()

I tried setting connection and command timeout values in the connection string and in the C# code but the exception kept occurring. An internet search suggested changing the net_read_timeout and net_write_timeout variables in MySql server.

As the client application appeared to be failing because it was reading beyond the end of the result stream it seemed likely that the problem was that MySql server had timed out while writing the stream. So, I tried changing the net_write_timeout value from its default of 30 seconds to 600 on the same connection that I was using to execute the SELECT and immediately before executing the SELECT:

new MySqlCommand("SET net_write_timeout=600;", conn).ExecuteNonQuery();

This fixed the problem.

I still don’t understand why the problem suddenly started occurring in an environment that has been stable for years but I suspect a reboot of the database server might be worth a try.

Written by Sea Monkey

March 1, 2024 at 7:00 pm

Posted in Debugging, Development, MySQL

Tagged with , , ,

When .NET BindingFlags don’t return anything…

leave a comment »

Not for the first time I’ve spent 15 minutes trying to work out why something like this doesn’t return anything:

ConstructorInfo[] constructors = type.GetConstructors(BindingFlags.Instance & BindingFlags.Public & BindingFlags.NonPublic & BindingFlags.DeclaredOnly);

It’s embarrassing enough to admit the mistake never mind how long it took to see it.

ConstructorInfo[] constructors = type.GetConstructors(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.DeclaredOnly);

That’s better.

Written by Sea Monkey

January 12, 2024 at 7:00 pm

Posted in Debugging, Development

Tagged with , ,

Modifying .NET Application Settings in another assembly

leave a comment »

There are some scenarios where .NET Framework assemblies need to access and modify application configuration settings defined in another assembly and scoped as internal to that assembly.

It’s worth saying that the simple solution to this problem is simply to change the scope of the assembly’s application settings from internal to public in the Visual Studio Settings designer. The downside is that you might not want to make the settings public.

My specific scenario is that I want to keep the scope of the settings as internal but also access them and modify them from a separate assembly that implements unit tests against the first assembly.

The unit tests need to change some of the application setting values to ensure that the tests are providing good coverage of the code under test.

It turns out that it’s very straightforward to use .NET reflection to get and set another assembly’s configuration settings even when they’re scoped as internal.

Getting an application setting value

The first step is to emulate the code that is executed when you reference the Default property of the Settings class, for example:

string s = Settings.Default.MySetting;

We need to reference the assembly that the application settings are defined in. A simple way to do this is to use the Assembly property of a type defined in the assembly. For example, if there’s a class called MainForm then we can use:

Assembly assembly = typeof(MainForm).Assembly;

Similarly, the Assembly class’s GetAssembly() method returns the currently loaded assembly that contains a named type, so this is an alternative:

Assembly assembly = Assembly.GetAssembly(typeof(MainForm);

There are other ways to specify the assembly you’re interested in that might be more appropriate for your use case.

Next we need to iterate through the types defined in the assembly to find the Settings class. It’s possible, even if unlikely, that there is another type also called Settings that is not the application configuration settings class that we’re interested in. So, to make sure that we correctly identify the class we want, we’ll check that the class inherits from System.Configuration.ApplicationSettingsBase and that it’s implemented in a namespace that ends with “.Properties”:

foreach (Type type in assembly.GetTypes())
{
    if (type.Name == "Settings" && type.Namespace.EndsWith(".Properties") && type.IsSubclassOf(typeof(System.Configuration.ApplicationSettingsBase)))
    {
        // do something...
    }

Note that while there is an overload of Assembly.GetType() that accepts a type name, it requires the type full name which you might not want to hardcode. The advantage of the foreach loop above is that it will find the Settings class without needing to know the namespace it’s implemented in.

The Settings class implements a static property called Default that provides a reference to a singleton instance of the settings so we need to create a reflection PropertyInfo instance for the property and then use it to invoke the property get accessor:

PropertyInfo defaultProperty = type.GetProperty("Default");
object settings = defaultProperty.GetValue(null, null);

Note that we don’t need to provide an object instance as it’s a static property, or any arguments, in the call to GetValue().

We now have the Settings class instance that provides access to the application settings.

The Settings class that the Visual Studio Settings Designer creates implements a get property accessor for each setting that retrieves the setting value from its dictionary and casts it to the required type which is how we are able to access the typed value directly as a property of the Settings class:

public string MySetting {
    get {
        return ((string)(this["MySetting"]));
     }
}

We could find this property accessor and call it but we could also just implement what the property accessor does by casting the value returned by the Settings class’s indexed property. Because the Settings Designer does not create a set property accessor we will have to use this approach to set the value anyway.

return (string)settings[settingName];

Pulling all this together and implementing it as a generic method to deal with the different setting value types and casting the object reference returned for the Settings.Default property so that we can access the Settings class’s index property for its settings dictionary give this code:

public static T GetValue<T>(Assembly assembly, string settingName)
{
    foreach (Type type in assembly.GetTypes())
    {
        if (type.Namespace.EndsWith(".Properties") && type.Name == "Settings" && type.IsSubclassOf(typeof(System.Configuration.ApplicationSettingsBase)))
        {
            PropertyInfo defaultProperty = type.GetProperty("Default");
            System.Configuration.ApplicationSettingsBase settings = defaultProperty.GetValue(null, null) as System.Configuration.ApplicationSettingsBase;
            return (T)settings[settingName];
        }
    }

    throw new ArgumentException(string.Format("The application settings class was for {0} was not found.", settingName));
}

Setting an application setting value

Implementing a method to modify a Setting value is almost the same as the code above:

        public static void SetValue<T>(Assembly assembly, string settingName, T settingValue)
        {
            foreach (Type type in assembly.GetTypes())
            {
                if (type.Name == "Settings" && type.Namespace.EndsWith(".Properties") && type.IsSubclassOf(typeof(System.Configuration.ApplicationSettingsBase)))
                {
                    PropertyInfo defaultProperty = type.GetProperty("Default");
                    System.Configuration.ApplicationSettingsBase settings = defaultProperty.GetValue(null, null) as System.Configuration.ApplicationSettingsBase;
                    settings[settingName] = settingValue;
                    return;
                }
            }

            throw new ArgumentException(string.Format("The application settings class was for {0} was not found.", settingName));
        }

Concision

The example implementations above are useful step by step code statements that make it obvious what is going on but it’s often interesting to see how concisely code like this can be written.

    public static T GetValue<T>(Assembly assembly, string settingName)
    {
        foreach (Type type in assembly.GetTypes())
        {
            if (type.Name == "Settings" && type.Namespace.EndsWith(".Properties") && type.IsSubclassOf(typeof(System.Configuration.ApplicationSettingsBase)))
            {
                return (T)(type.GetProperty("Default").GetValue(null, null) as System.Configuration.ApplicationSettingsBase)[settingName];
            }
        }

        throw new ArgumentException(string.Format("The application settings class was for {0} was not found.", settingName));
    }

        public static void SetValue<T>(Assembly assembly, string settingName, T settingValue)
        {
            foreach (Type type in assembly.GetTypes())
            {
                if (type.Name == "Settings" && type.Namespace.EndsWith(".Properties") && type.IsSubclassOf(typeof(System.Configuration.ApplicationSettingsBase)))
                {
                    (type.GetProperty("Default").GetValue(null, null) as System.Configuration.ApplicationSettingsBase)[settingName] = settingValue;
                    return;
                }
            }

            throw new ArgumentException(string.Format("The application settings class was for {0} was not found.", settingName));
        }

Written by Sea Monkey

January 10, 2024 at 7:00 pm

Posted in Development

Tagged with , ,

NUnit ExpectedException deprecated

leave a comment »

In the past I’ve used the NUnit ExpectedException attribute to decorate test methods that are expected to throw exceptions.

The problem with the ExpectedException attribute is that the test method it decorates can’t test anything after the exception thrown. It’s true that unit tests should essentially test only one thing but in reality they often need to setup the environment for the test and make assertions about the test environment before performing the actual test. As long as the test always throws the exception as the last part of the test then the ExpectedException attribute does its job.

But there’s the possibility that the exception thrown, even if it’s of the expected type, is not the exception the test writer intended to test for.

The ExpectedException attribute is a blunt tool.

I often write separate test assemblies as executables that function as their own test runners which means I’ve got to write some code to handle the ExpectedException attribute, catch any exceptions thrown, and check that the exception matches the expected exception type. It’s not that much work but it’s another reason to find a better way.

Finally, as of version 3.x, NUnit no longer supports the ExpectedException attribute.

The Assert.Throws() method is a much more useful means to test for exceptions but I found the NUnit documentation a little opaque about how to use it. It took me a few attempts before I realised that it’s really very straightforward.

A typical use case for me would be to execute a single line of code that I expect to throw an exception and an anonymous delegate is perfect for this.

As an example, consider a collection class derived from a generic Dictionary and we want to test that adding a duplicate key throws an exception. We just want to execute the single line of code:

urlDictionary.AddArgument("test", testUrl);

but in a way that allows us to confirm that it throws the expected exception and, ideally, the exception contains the expected error message.

We can use an anonymous delegate to specify the line of code to execute and the Assert.Throws() method returns the exception thrown allowing us to check its Message property:

// Expect an ArgumentException to be thrown with "An item with the same key has already been added."
Exception exception = Assert.Throws(
    typeof(ArgumentException),
    delegate { urlDictionary.AddArgument("test", testUrl); });

Assert.AreEqual("An item with the same key has already been added.", exception.Message);


It’s as simple as that.

Written by Sea Monkey

December 22, 2023 at 7:00 pm

Posted in Development

Tagged with , ,

Real world immutable string performance issues in .NET

leave a comment »

We all know we shouldn’t write .NET code that treats a string value as a mutable type because it isn’t but I guess most of us don’t worry about it too much when we know we’re only applying a few operations to a short string value.

The problem is that each assignment to an existing string value will cause a new allocation of memory for the result and the memory for the existing value will be released. It’s slow and inefficient and may result in the allocations and deallocations of large chunks of memory even when only a single character is being added or removed from the value if it’s large.

Last week our NHapi based HL7 Interface ground to a halt at a customer site. When I investigated it became apparent that the hospital’s Imaging Department was sending huge messages that were causing the NHapi PipeParser to slow to a crawl.

Instead of the typical 1-2kb of report text for a single report for a single patient these messages contained hundreds of concatenated reports for many different patients but the message’s PID segment indicated they were for a single patient. The text was getting longer with each new message sent.

Clearly something was amiss with the Imaging Department’s system but that was someone else’s problem and I was more concerned about the performance of the NHapi message parser.

The processing logs showed that the time taken to parse the messages had jumped from the typical sub-second value for a 1-2kb message to 10-15 minutes for messages around 500kb and up to 90 minutes as the message size neared 900kb. It appeared that as the size of the messages increased the parsing time increased exponentially. I made a note of few message sizes and their parsing times and confirmed that the increase in parsing time was rising as a cube of the message size.

By now we’d stopped the HL7 Interface and were weeding out the large Imaging Department messages from the queue. It appeared that the department had realised they had a problem and no more large messages were arriving. The largest that was sent contained nearly 1.9 million characters in 292,000 words and over 58,000 lines.

I estimated the parsing time based on the data I had from the processing logs: 12 hours 42 minutes. Just to parse one HL7 message!

It didn’t take long to find the culprit.

        private System.String nextToken(char[] delimiters)
{
string token = "";
long pos = this.currentPos;

//skip possible delimiters
while (System.Array.IndexOf(delimiters, this.chars[currentPos]) != -1)
//The last one is a delimiter (i.e there is no more tokens)
if (++this.currentPos == this.chars.Length)
{
this.currentPos = pos;
throw new System.ArgumentOutOfRangeException();
}

//getting the token
while (System.Array.IndexOf(delimiters, this.chars[this.currentPos]) == -1)
{
token += this.chars[this.currentPos];
//the last one is not a delimiter
if (++this.currentPos == this.chars.Length)
break;
}
return token;
}

Switching to using a StringBuilder class was straightforward with only three lines changed:

        //Returns the nextToken wo delimiters
private System.String nextToken(char[] delimiters)
{
StringBuilder token = new StringBuilder();
long pos = this.currentPos;

//skip possible delimiters
while (System.Array.IndexOf(delimiters, this.chars[currentPos]) != -1)
//The last one is a delimiter (i.e there is no more tokens)
if (++this.currentPos == this.chars.Length)
{
this.currentPos = pos;
throw new System.ArgumentOutOfRangeException();
}

//getting the token
while (System.Array.IndexOf(delimiters, this.chars[this.currentPos]) == -1)
{
token.Append(this.chars[this.currentPos]);
//the last one is not a delimiter
if (++this.currentPos == this.chars.Length)
break;
}
return token.ToString();
}

(In fairness to the NHapi project this change was made in the repository a long time ago but we’re using an older version.)

A few quick tests showed that the performance problem was resolved with the 500-800kb messages now parsing in a few seconds rather than 15-90 minutes. I thought it would be worth trying the largest of the messages that had been received. The 1,800kb messages was successfully parsed in 66 seconds rather than the 12.7 hours I had estimated.

While we often take the view that optimising code is best done when it’s needed this is a good example of why string manipulation, especially simple concatenation, is better implemented using the StringBuilder class.

Written by Sea Monkey

December 4, 2023 at 8:00 pm

Posted in Debugging, Development, HL7

Tagged with , ,

Compiling Windows Hosting CScript files with JScript.NET

leave a comment »

1 Potential Issues

1.1 The WScript object is not available

The Windows Scripting host instantiates the WScript object that implements methods such as Echo() and properties such as Arguments. In principle you can create an instance of WScript and many methods will work as expected. However, if you’re only using a few simple methods then it’s worth considering implementing your own WScript object in JScript.NET. An example is shown below in section 3.1.

1.2 The compiled code is for ANYCPU

If your CScript program uses COM objects such as ADODB database objects and the FileSystemObject then it probably needs to run as a 32 bit (x86) process.

The JScript.NET compiler does not offer an option to specify the platform target (such as x86 or x64) and generates an ANYCPU executable. The compiled program will run in a 64 bit process on a 64 bit version of Windows.

If you need to run the compiled executable as a 32 bit process on a 64 bit version of Windows then you can use a small C# x86 program to start a 32 bit process and then call your JScript.NET program. An example is shown below in section 3.2.

1.3 FAST mode is not compatible with prototype modifications

The JScript.NET compiler has a FAST option that is enabled by default. FAST mode improves performance but is not compatible with user prototype modifications such as adding prototype methods to the standard datatypes. No compiler warning or error is issued but any attempt to use the prototype method will cause a “Function not found” run time exception.

This option can be turned off by specifying the /fast- JScript.NET compiler command argument.

2 Compile

Information on how to run the JScript.NET compiler can be found here:

https://learn.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2010/7435xtz6(v=vs.100)

A Windows batch file containing the following command line is a useful starting point:

"C:\Windows\Microsoft.NET\Framework\v2.0.50727\jsc.exe" /target:exe /debug+ "/out:%~dp0myprogram.exe" "%~dp0myprogram.js"

This uses the .NET 2.0 Framework’s JScript.NET compiler.

The /out argument specifies the name of the compiled program.

The last argument is the name of the JScript.NET source code file.

Note that %~dp0 prepends the path of the batch file to the output and source filenames, i.e. the files are expected to be in the same directory as the batch file.

If your source code is implemented in several files then these can just be added to the end of the command line:

"C:\Windows\Microsoft.NET\Framework\v2.0.50727\jsc.exe" /target:exe /debug+ /out:%~dp0myprogram.exe %~dp0myprogram.js "%~dp0myprogram2.js" "%~dp0myprogram3.js"

3. Resources

3.1 WScript.js

This JScript.NET class implements some of the WScript functionality that I needed for my CScript program. It’s a work in progress and only implements a few methods and properties. You can include this in your compiled code by placing it in its own file and adding it to the list of source files passed to the JScript.NET compiler:

jsc.exe /target:exe /debug+ /out:myprogram.exe myprogram.js wscript.js

This has the advantage that you don’t need to modify your CScript program and you can still run it using CScript. Under CScript it will use CScript’s WScript object and when you run the JScript.NET executable it will use the WScript object implemented in the WScript.js file.

import System;
import System.Text.RegularExpressions;

public class WScript {
    
    private static var arguments;
    
    public static var stdIn;
    
    public static var stdOut;
    
    public static function get Arguments() {
        if (WScript.arguments == undefined) {
            WScript.arguments = new WScriptArguments();
        }
        return WScript.arguments;
    }

    public static function get StdIn() {
        if (WScript.stdIn == undefined) {
            WScript.stdIn = new WScriptStdIn();
        }
        return WScript.stdIn;
    }

    public static function get StdOut() {
        if (WScript.stdOut == undefined) {
            WScript.stdOut = new WScriptStdOut();
        }
        return WScript.stdOut;
    }
    
    public static function Echo() {
        Console.WriteLine();
    }
    
    public static function Echo(message : String) {
        Console.WriteLine(message);
    }
    
    public static function Quit() {
        throw new Error(0, "Quit() was called.");
    }
    
    public static function Quit(errorCode) {
        throw new Error(errorCode, "Quit() was called with error code " + errorCode + ".");
    }
    
    public static function Quit(errorCode, errorDescription) {
        throw new Error(errorCode, errorDescription);
    }    
}

public class WScriptArguments {

    public function WScriptArguments() {
        //Console.WriteLine("==> WScriptArguments.ctor()");
    }

    public function Named(name : String) {

        var regex : Regex = new Regex("[-/](?'name'\\w+)[:=](?'value'.*)", RegexOptions.IgnoreCase);
        var args : String[] = Environment.GetCommandLineArgs();
        for (var index in args) {
            var m : Match = regex.Match(args[index]);
            if (m.Success) {
                if (String.Compare(name, m.Groups["name"].Value, true) == 0) {
                    return m.Groups["value"].Value;
                }
            }
        }
        return undefined;
    }
}

public class WScriptStdIn {

    public function WScriptStdIn() {
        //Console.WriteLine("==> WScriptStdIn.ctor()");
    }

    public function Read(value) {
    }

    public function ReadLine() {
        return Console.ReadLine();
    }
}

public class WScriptStdOut {

    public function WScriptStdOut() {
        //Console.WriteLine("==> WScriptStdOut.ctor()");
    }
    
    public function WriteLine(message : String) {
        Console.Out.WriteLine(message);
    }
}

3.2 RunAsX86

If you need to run your JScript.NET program as a 32 bit process on a 64 bit version of Windows then compile this C# program with a Platform target of x86 and use it to execute your JScript.NET program.

The RunAsX86.exe program must be placed in the same directory as your JScript.NET executable.

The first argument is the file name of the JScript.NET program to execute and the remaining arguments are the arguments to be passed to the JScript.NET program.

// 
// This program must be compiled with Platform target = x86.
//
namespace RunAsX86
{
    using System;
    using System.IO;
    using System.Reflection;

    public class Program
    {
        public static int Main(string[] args)
        {
            if (args.Length > 0)
            {
                // Create the fully qualified path for the specified file:
                string assemblyPath = Path.Combine(Path.GetDirectoryName((new Uri(Assembly.GetExecutingAssembly().CodeBase)).LocalPath), Path.IsPathRooted(args[0]) ? Path.GetFileName(args[0]) : args[0]);

                // Check if the file exists:
                if (File.Exists(assemblyPath))
                {
                    // Load the file as an assembly:
                    Assembly assembly = Assembly.LoadFile(assemblyPath);

                    if (assembly.EntryPoint != null)
                    {
                        // Copy the command line arguments to be passed to the assembly's entry point:
                        string[] newArgs = new string[args.Length - 1];
                        Array.Copy(args, 1, newArgs, 0, newArgs.Length);

                        // Call the entry point of the assembly and pass the command line arguments:
                        object returnValue = assembly.EntryPoint.Invoke(null, new object[] { newArgs });
                        return (returnValue == null) ? 0 : (int)returnValue;
                    }
                    else
                    {
                        Console.WriteLine("ERROR: Cannot find the executable entry point in '{0}'.", assemblyPath);
                        return -3;
                    }
                }
                else
                {
                    Console.WriteLine("ERROR: Executable '{0}' not found.", assemblyPath);
                    return -2;
                }
            }
            else
            {
                Console.WriteLine("ERROR: Executable file not specified.");
                return -1;
            }
        }
    }
}

Written by Sea Monkey

October 21, 2022 at 6:00 pm

Posted in Development, Uncategorized

Tagged with , ,

Creating a mock web service with Visual Studio

leave a comment »

It’s possible to create the basis of a mock web service quickly using Microsoft’s WSDL tool and Visual Studio.

I usually have to remind myself of the required WSDL parameters required to create the service interface as the default is to generate client proxy classes rather than the service interface.

This is a good guide: https://ivangrigoryev.com/en/how-to-mock-a-web-service-in-dot-net/

The key point is to specify the ServerInterface option of the WSDL tool so that it generates a source file for the interface that a mock web service needs to implement. The example given in the article referenced above is:

wsdl /language:CS /out:C:\Downloads\ /protocol:SOAP /serverinterface C:\Downloads\webservice.wsdl

Then add the interface source file generated to a blank Visual Studio Web Service project, add a new class and open the source file, and specify that it implements the interface by adding

: interface-name

after the class name. Finally, right-click on the interface name you added and select Implement Interface so that Visual Studio creates method stubs for all the web service’s methods.

Written by Sea Monkey

November 13, 2020 at 10:28 am

Posted in Development

Tagged with ,

HtmlAgilityPack package no longer includes a .NET 2.0 binary

leave a comment »

For no obvious reason the two latest HtmlAgilityPack release on Nuget, 1.11.16 and 1.11.17, don’t contain a .NET 2.0 binary. Hopefully this will be resolved.

Written by Sea Monkey

December 22, 2019 at 12:29 pm

Posted in Development

Tagged with , ,

C# StorageValue type part 2: Struct

leave a comment »

Almost as soon as I started on a struct implementation I decided to change the type used to hold the storage value from the long (System.Int64) type used in the class implementation to a ulong (System.UInt64). I’d considered this previously and decided that a signed value might be useful for storage/memory calculations but, the more I thought about it the more I realised this didn’t make much sense and it would be better to use an unsigned value.

The class implementation of the StorageClass type in the previous post was useful to developing the code required to parse and format storage values. However, I knew that I really wanted the semantics and behaviour associated with the built in value types such as int.

Another change I considered was to use the implicit operator to allow conversion from a string value without an explicit cast.

With the current implementation, assignment looks like this:

StorageValue s1 = new StorageValue(100 * 1024);
StorageValue s2 = new StorageValue("1.5MB");

but I really wanted to be able to do this:

StorageValue s1 = 100 * 1024;
StorageValue s2 = "1.5MB";

In fact, it’s very easy to do this, whether the type is a class or struct, by implementing the implicit operator:

 public static implicit operator StorageValue(int value)
 {
    return new StorageValue(value);
 }
 
 public static implicit operator StorageValue(string value)
 {
    return StorageValue.Parse(value);
 }

Unfortunately, this is represents an abuse of what the implicit operator is designed for. To explain why, it’s worth considering the intent of the explicit operator first. Typically, when a conversion might fail or will potentially result in data loss, the compiler requires an explicit cast, e.g.

long gb = 1024 * 1024 * 1024;
int i = (int)gb // cast required

This conversion using a cast is implemented using the explicit operator. The point of this is to allow the conversion but only when the programmer uses a cast, i.e. it ensures that the programmer is aware of the potential for failure or data loss.

The implicit operator implements the conversion without requiring the cast, i.e. the programmer is not made aware of the possible failure or data loss when the conversion is performed.

So, while it would be nice to allow syntax such as

StorageValue s2 = "1.5GB";

it is not correct use of the implicit operator as this conversion could easily fail if the string is not correctly formatted, e.g.

StorageValue s2 = "Hello world!";

There is a much bigger issue with a struct implementation rather than a class implementation which is simply that it will be passed by value rather than by reference. For this implementation this is probably the correct behaviour but, again, it really comes down to what feels right.

Written by Sea Monkey

January 30, 2017 at 6:18 pm

Posted in Development

Tagged with ,

C# StorageValue type part 1: Class

leave a comment »

I’ve been meaning to write a class for the past few years for dealing with memory storage values.

The clinical data interfaces that I write often need to read and write disk and memory storage values. For example, the Vitality HL7 MLLP software that receives and sends HL7 messages via TCP/IP, writes incoming messages to disk and so it monitors disk free space to ensure that there is always sufficient disk space available. The required disk free space is held as a configuration file value. I could use a long value for bytes, or an int value for MB, but it would be more convenient to have a datatype that allowed me to express the value using the most appropriate units, e.g. as “500MB” or “0.5GB” rather than 524288000.

Conversely, these interfaces typically log the process working set size before and after processing to aid diagnosing memory problems. Again, it’s possible just to log the number of bytes but it’s easier to read and understand a working set size of, for example, 18.56MB rather than 19464932.

So, the class must implement parsing methods to take values such as “1234”, “123.4KB”, “12.34MB”, “1.234GB”, “1.234GB” and to provide formatting methods to display storage values according to a format string or default to an appropriate formatted value.

Most of the code already existed as discrete utility methods in my code. I just needed to bring them all together into a single class and implement the IFormattable interface so that an instance of the class can be formatted in composite formatting methods such as string.Format().

The formatting supported for parsing is a numeric value optionally suffixed with B, KB, MB, GB, or TB. For example:

StorageValue s1 = StorageValue.Parse("150MB");

The IFormattable formatting supported for conversion to string values is any of the suffixes above, optionally with a leading +/- to enable or disable the display of the suffix, and an optional space in front of the suffix to indicate that a space should be placed between the numeric value and the suffix. If not format is specified then an appropriate unit will be used. Examples:

StorageValue workingSetSize = Process.GetCurrentProcess().WorkingSet64;

// Let the StorageValue class choose the most appropriate units:
applicationLog.WriteInfo(string.Format("Working set size: {0}", workingSetSize));

// Use KB:
applicationLog.WriteInfo(string.Format("Working set size: {0:KB}", workingSetSize));

// Use MB with a space between the value and the suffix:
applicationLog.WriteInfo(string.Format("Working set size: {0: MB}", workingSetSize));

// Use MB but don't display the units:
applicationLog.WriteInfo(string.Format("Working set size: {0:-MB} megabytes", workingSetSize));

One of the big design decisions that needs to be made is whether this type should be a class or a struct. A class is the easy option as its behaviour is likely to be more intuitive as it avoids some of the issues like immutability that structs raise. However, because this type is a thin wrapper around the System.Int64 type, a struct might offer more intuitive syntax and semantics. I decided to write the implementation as a class and then re-work it as a struct to see how the two implementations compared.

The initial implementation as a class is below and the struct implementation will follow in a second post. (Hint: Wait for the struct version, I think it’s better…)

(StorageClass.cs):

namespace Vitality.Common.Types
{
    using System;
    using System.Collections.Generic;
    using System.Globalization;
    using System.Text;

    /// <summary>
    /// Represents a memory or disk storage value.
    /// </summary>
    [Serializable]
    public class StorageValue : IFormattable
    {
        #region Private Constants

        /// <summary>
        /// Constant for one terabyte.
        /// </summary>
        private const long TeraByte = 1024 * 1024 * 1024 * 1024L;

        /// <summary>
        /// Constant for one gigabyte.
        /// </summary>
        private const long GigaByte = 1024 * 1024 * 1024L;

        /// <summary>
        /// Constant for one megabyte.
        /// </summary>
        private const long MegaByte = 1024 * 1024L;

        /// <summary>
        /// Constant for one kilobyte.
        /// </summary>
        private const long KiloByte = 1024L;

        /// <summary>
        /// Constant for one byte.
        /// </summary>
        private const long Byte = 1L;

        #endregion Private Constants

        #region Private Members

        /// <summary>
        /// The storage value in bytes.
        /// </summary>
        private long bytes;

        #endregion Private Members

        #region Constructors

        /// <summary>
        /// Initializes a new instance of the StorageValue class.
        /// </summary>
        /// <param name="bytes">The number of bytes for the storage value.</param>
        public StorageValue(long bytes)
        {
            this.bytes = bytes;
        }

        #endregion Constructors

        #region Public Properties

        /// <summary>
        /// Gets or sets the number of bytes in the StorageValue.
        /// </summary>
        public long Bytes
        {
            get { return this.bytes; }
            set { this.bytes = value; }
        }

        #endregion Public Properties

        #region Public Static Methods

        /// <summary>
        /// Converts the string representation of a storage value to an instance of the StorageValue class.
        /// </summary>
        /// <param name="s">The string representing a storage value and optionally specifying B, KB, MB, GB, or TB as a suffix.</param>
        /// <returns>An instance of the StorageValue class.</returns>
        public static StorageValue Parse(string s)
        {
            long conversionFactor = 1;

            if (s.EndsWith("TB", StringComparison.OrdinalIgnoreCase))
            {
                conversionFactor = StorageValue.TeraByte;
                s = s.Substring(0, s.Length - 2);
            }
            else if (s.EndsWith("GB", StringComparison.OrdinalIgnoreCase))
            {
                conversionFactor = StorageValue.GigaByte;
                s = s.Substring(0, s.Length - 2);
            }
            else if (s.EndsWith("MB", StringComparison.OrdinalIgnoreCase))
            {
                conversionFactor = StorageValue.MegaByte;
                s = s.Substring(0, s.Length - 2);
            }
            else if (s.EndsWith("KB", StringComparison.OrdinalIgnoreCase))
            {
                conversionFactor = StorageValue.KiloByte;
                s = s.Substring(0, s.Length - 2);
            }
            else if (s.EndsWith("B", StringComparison.OrdinalIgnoreCase))
            {
                conversionFactor = StorageValue.Byte;
                s = s.Substring(0, s.Length - 1);
            }
            else
            {
                conversionFactor = StorageValue.Byte;
            }

            double value = double.Parse(s.Trim());

            return new StorageValue((long)(value * conversionFactor));
        }

        /// <summary>
        /// Returns a string representation of the storage value.
        /// </summary>
        /// <param name="bytes">The storage value in bytes.</param>
        /// <returns>A formatted string representing the storage value.</returns>
        public static string ToString(long bytes)
        {
            return StorageValue.ToString(bytes, string.Empty, null);
        }

        /// <summary>
        /// Returns a string representation of the storage value.
        /// </summary>
        /// <param name="bytes">The storage value in bytes.</param>
        /// <param name="format">The format string to use.</param>
        /// <returns>A formatted string representing the storage value.</returns>
        public static string ToString(long bytes, string format)
        {
            return StorageValue.ToString(bytes, format, null);
        }

        /// <summary>
        /// Returns a string representation of the storage value.
        /// </summary>
        /// <param name="bytes">The storage value in bytes.</param>
        /// <param name="format">The format string to use.</param>
        /// <param name="formatProvider">The IFormatProvider to use.</param>
        /// <returns>A formatted string representing the storage value.</returns>
        public static string ToString(long bytes, string format, IFormatProvider formatProvider)
        {
            long conversionFactor = 1L;
            string suffix = string.Empty;
            bool includeSuffix = true;
            bool includeSpace = false;
            bool isUnitsSpecified = false;

            // Format: [+/-][ ]units
            // + : include factor (this is the default if not specified), e.g. "10MB"
            // - : omit factor, e.g. "10"
            // : a space before the units indicates that the value and units should be separated by a space, e.g. "10 MB" rather than "10MB"
            // units : TB, GB, MB, KB, B for, respectively, terabytes, gigabytes, megabytes, kilobytes, and bytes.
            //
            // Examples:
            //
            // "GB" --> "10GB"
            // "+GB" --> "10GB"
            // "+ GB" --> "10 GB"
            // "-GB" --> "10"
            //
            // Commas and decimal places will be used when necessary, e.g. "0.1TB", "102,400KB", etc/
            //
            // The factor argument is not case sensitive, e.g. "10mb", "10MB", "10Mb", and "10mB" are all valid.
            if (!string.IsNullOrEmpty(format))
            {
                if (format.StartsWith("+", StringComparison.OrdinalIgnoreCase))
                {
                    includeSuffix = true;
                    format = format.Substring(1);
                }
                else if (format.StartsWith("-", StringComparison.OrdinalIgnoreCase))
                {
                    includeSuffix = false;
                    format = format.Substring(1);
                }

                if (format.StartsWith(" ", StringComparison.OrdinalIgnoreCase))
                {
                    includeSpace = true;
                    format = format.Substring(1);
                }

                if (string.Compare(format, "TB", true) == 0)
                {
                    conversionFactor = StorageValue.TeraByte;
                    suffix = "TB";
                    isUnitsSpecified = true;
                }
                else if (string.Compare(format, "GB", true) == 0)
                {
                    conversionFactor = StorageValue.GigaByte;
                    suffix = "GB";
                    isUnitsSpecified = true;
                }
                else if (string.Compare(format, "MB", true) == 0)
                {
                    conversionFactor = StorageValue.MegaByte;
                    suffix = "MB";
                    isUnitsSpecified = true;
                }
                else if (string.Compare(format, "KB", true) == 0)
                {
                    conversionFactor = StorageValue.KiloByte;
                    suffix = "KB";
                    isUnitsSpecified = true;
                }
                else if (string.Compare(format, "B", true) == 0)
                {
                    conversionFactor = StorageValue.Byte;
                    suffix = "B";
                    isUnitsSpecified = true;
                }
            }

            if (!isUnitsSpecified)
            {
                // No valid explicit units given so attempt to determine the most suitable to use:
                if (bytes > StorageValue.TeraByte)
                {
                    conversionFactor = StorageValue.TeraByte;
                    suffix = "TB";
                }
                else if (bytes > StorageValue.GigaByte)
                {
                    conversionFactor = StorageValue.GigaByte;
                    suffix = "GB";
                }
                else if (bytes > StorageValue.MegaByte)
                {
                    conversionFactor = StorageValue.MegaByte;
                    suffix = "MB";
                }
                else if (bytes > StorageValue.KiloByte)
                {
                    conversionFactor = StorageValue.KiloByte;
                    suffix = "KB";
                }
                else
                {
                    conversionFactor = StorageValue.Byte;
                    suffix = "B";
                }
            }

            // Format the string according to the IFormatProvider the caller specified, otherwise; use the InvariantCulture:
            NumberFormatInfo numberFormatInfo = null;

            if (formatProvider != null)
            {
                NumberFormatInfo formatProviderNumberFormatInfo = formatProvider.GetFormat(typeof(NumberFormatInfo)) as NumberFormatInfo;

                if (formatProviderNumberFormatInfo != null)
                {
                    numberFormatInfo = formatProviderNumberFormatInfo;
                }
            }

            if (numberFormatInfo == null)
            {
                numberFormatInfo = CultureInfo.InvariantCulture.NumberFormat;
            }

            // Use the culture specific thousands and decimal characters:
            string numberFormatString = string.Format("{{0:#{0}##0{1}##}}{{1}}{{2}}", numberFormatInfo.NumberGroupSeparator, numberFormatInfo.NumberDecimalSeparator);

            return string.Format(numberFormatString, (double)(bytes / (conversionFactor * 1.0d)), includeSpace ? " " : string.Empty, includeSuffix ? suffix : string.Empty);
        }

        #endregion Public Static Methods

        #region Public Methods

        /// <summary>
        /// Returns a string representation of the storage value.
        /// </summary>
        /// <returns>A formatted string representing the storage value.</returns>
        public override string ToString()
        {
            return StorageValue.ToString(this.bytes);
        }

        /// <summary>
        /// Returns a string representation of the storage value.
        /// </summary>
        /// <param name="format">The format string to use.</param>
        /// <returns>A formatted string representing the storage value.</returns>
        public string ToString(string format)
        {
            return StorageValue.ToString(this.bytes, format);
        }

        /// <summary>
        /// Returns a string representation of the storage value.
        /// </summary>
        /// <param name="format">The format string to use.</param>
        /// <param name="formatProvider">The IFormatProvider to use.</param>
        /// <returns>A formatted string representing the storage value.</returns>
        public string ToString(string format, IFormatProvider formatProvider)
        {
            return StorageValue.ToString(this.bytes, format, formatProvider);
        }

        #endregion Public Methods
    }
}



Written by Sea Monkey

January 25, 2017 at 5:08 pm

Posted in Development

Tagged with ,