Processing math: 100%
Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / Languages / F#
Print

Simple trend calculation

4.97/5 (14 votes)
15 Aug 2017CPOL6 min read 61.4K   2.1K  
Simple linear trend calculation with different types for X values implemented in C#, VB, and F#

Introduction

I needed to have a simple linear trend calculation for X values either as double or datetime. So basically the Y value for a single item is always a double but the type of the X varies. Beyond that, I needed basic statistics calculated from the values. These included:

  • Slope
  • Y-intercept
  • Correlation coefficient
  • R-squared value

Excerpt from the user interface

Image 1

As an additional step I decided to have my first look at F# so now one implementation of the calculation is done using F#. I have to admit that it isn't yet F#ish and looks more like C# but on the other hand that may help readers to see equivalences (and differences).

Formulas used in calculations

Since the requirement was to do the calculation in a similar way as it would be done in Excel, I used the same variations for formulas as Excel uses. This also made it simple to check the correctness of the calculations. So the formulas are:

Line 

y=mx+b

 

where

  • m is slope
  • x is the horizontal axis value
  • b is the Y-intercept

Slope calculation

s=(xˉx)(yˉy)(xˉx)2

 

where 

  • x and y are individual values
  • accented x and y are averages for the corresponding values

The correlation coefficient 

c=(xˉx)(yˉy)(xˉx)2(yˉy)2

 

where again

  • x and y are individual values
  • accented x and y are averages for the corresponding values

R-squared value 

r2=1(yˆy)2y2(y)2n

 

where

  • y is individual values
  • accented y (with a hat) is the corresponding calculated trend value
  • n is the count of values.
     

Classes for value items

The first thing is to create the classes for the actual value items, both double and datetime. Basically the classes are simple, just properties X and Y. But things get a bit more complicated since the type of the X varies. Instead of using an object property I wanted to have separate classes for the different item types and to be able use double and datetime types instead of object. This approach quickly lead to using an abstract base class with generics.

However, using generics for X introduces a new problem, how to use the same calculation for two different data types. Since I didn’t have any specific requirements concerning the calculation, I decided to convert the X values always to double. In order to use this value in calculation an extra property ConvertedX is defined.

The classes look like following

Abstract (MustInherit) base class for value items

 

namespace TrendCalculus {
   /// <summary>
   /// Base class for value items
   /// </summary>
   /// <typeparam name="TX">Type definition for X</typeparam>
   public abstract class ValueItem<TX> : IValueItem {

      private double _y;

      /// <summary>
      /// Raised when the data in the item is changed
      /// </summary>
      public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;

      /// <summary>
      /// The actual value for X
      /// </summary>
      public abstract TX X { get; set; }

      /// <summary>
      /// The value for X for calculations
      /// </summary>
      public abstract double ConvertedX { get; set; }

      /// <summary>
      /// Y value of the data item
      /// </summary>
      public double Y {
         get {
            return this._y;
         }
         set {
            if (this._y != value) {
               this._y = value;
               this.NotifyPropertyChanged("Y");
            }
         }
      }

      /// <summary>
      /// This method fires the property changed event
      /// </summary>
      /// <param name="propertyName">Name of the changed property</param>
      protected void NotifyPropertyChanged(string propertyName) {
         System.ComponentModel.PropertyChangedEventHandler handler = this.PropertyChanged;

         if (handler != null) {
            handler(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));
         }
      }

      /// <summary>
      /// Creates a copy of the value item
      /// </summary>
      /// <returns>The copy</returns>
      public abstract object CreateCopy();

      /// <summary>
      /// Creates a new trend item
      /// </summary>
      /// <returns>The trend item</returns>
      public abstract object NewTrendItem();
   }
}

Value item class for double values

namespace TrendCalculus {
   /// <summary>
   /// Class for number items where X is double
   /// </summary>
   public class NumberItem : ValueItem<double> {

      private double _x;

      /// <summary>
      /// X actual value of the data item
      /// </summary>
      public override double X {
         get {
            return this._x;
         }

         set {
            if (this._x != value) {
               this._x = value;
               this.NotifyPropertyChanged("X");
            }
         }
      }

      /// <summary>
      /// The value for X for calculations
      /// </summary>
      public override double ConvertedX {
         get {
            return this.X;
         }
         set {
            if (this.X != value) {
               this.X = value;
            }
         }
      }

      /// <summary>
      /// Creates a new trend item
      /// </summary>
      /// <returns>The trend item</returns>
      public override object NewTrendItem() {
         return new NumberItem();
      }

      /// <summary>
      /// Creates a copy of the value item
      /// </summary>
      /// <returns>The copy</returns>
      public override object CreateCopy() {
         return new NumberItem() {
            X = this.X,
            Y = this.Y
         };
      }
   }
}

Value item class for datetime values

namespace TrendCalculus {
   /// <summary>
   /// Class for number items where X is datetime
   /// </summary>
   public class DateItem : ValueItem<System.DateTime> {

      private System.DateTime _x;

      /// <summary>
      /// X actual value of the data item
      /// </summary>
      public override System.DateTime X {
         get {
            return this._x;
         }

         set {
            if (this._x != value) {
               this._x = value;
               this.NotifyPropertyChanged("X");
            }
         }
      }

      /// <summary>
      /// The value for X for calculations
      /// </summary>
      public override double ConvertedX {
         get {
            double returnValue = 0;

            if (this.X != null) {
               returnValue = this.X.ToOADate();
            }

            return returnValue;
         }
         set {
            System.DateTime converted = System.DateTime.FromOADate(value);

            if (this.X != converted) {
               this.X = converted;
            }
         }
      }
      /// <summary>
      /// Creates a new trend item
      /// </summary>
      /// <returns>The trend item</returns>
      public override object NewTrendItem() {
         return new DateItem();
      }

      /// <summary>
      /// Creates a copy of the value item
      /// </summary>
      /// <returns>The copy</returns>
      public override object CreateCopy() {
         return new DateItem() {
            X = this.X,
            Y = this.Y
         };
      }
   }
}

As you might notice the abstract class implements IValueItem interface. This interface is used for collections of data items. The interface helps the collection handling since it defines all the necessary methods and properties and eliminates the need to know the actual data type for X, which would be needed if the abstract class definition would be used. So the interface looks like this

namespace TrendCalculus {
   /// <summary>
   /// Interace which each value item type must implement in order to be usable in calculation
   /// </summary>
   public interface IValueItem {
      /// <summary>
      /// Raised when the data in the item is changed
      /// </summary>
      event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;

      /// <summary>
      /// Returns the value for X for calculations
      /// </summary>
      double ConvertedX { get; set; }

      /// <summary>
      /// Y value of the data item
      /// </summary>
      double Y { get; set; }

      /// <summary>
      /// Creates a copy of the value item
      /// </summary>
      /// <returns>The copy</returns>
      object CreateCopy();

      /// <summary>
      /// Creates a new trend item
      /// </summary>
      /// <returns>The trend item</returns>
      object NewTrendItem();

   }
}

List of values

The next thing is to create a list for the value items. Of course a simple list could do, but to make things more easy to use I wanted to have a collection which would satisfy following requirements

  • Changes in the collection are automatically detected by WPF
  • Only items implementing IValueItem could be added to collection
  • Any change in the collection would case a data change notification. This would include adding or removing items but also changes in the property values of the items.

Because of these I inherited a new class from ObservableCollection as follows

namespace TrendCalculus {
   /// <summary>
   /// List of item values
   /// </summary>
   public class ValueList<TValueItem> : System.Collections.ObjectModel.ObservableCollection<TValueItem>
      where TValueItem : IValueItem {

      /// <summary>
      /// Raised when items in the value list change or data in existing items change
      /// </summary>
      public event System.EventHandler DataChanged;

      /// <summary>
      /// Type of the items in the list
      /// </summary>
      public ValueListTypes ListType { get; private set; }

      /// <summary>
      /// Default constructor
      /// </summary>
      private ValueList() {
         this.CollectionChanged += ValueList_CollectionChanged;
      }

      /// <summary>
      /// Constructor with the list type information
      /// </summary>
      /// <param name="listType"></param>
      internal ValueList(ValueListTypes listType) : this() {
         this.ListType = listType;
      }

      /// <summary>
      /// Handles collection changed events for data items
      /// </summary>
      /// <param name="sender"></param>
      /// <param name="e"></param>
      private void ValueList_CollectionChanged(object sender, 
      System.Collections.Specialized.NotifyCollectionChangedEventArgs e) {
         // Delete PropertyChanged event handlers from items removed from collection
         if (e.OldItems != null) {
            foreach (IValueItem item in e.OldItems) {
               item.PropertyChanged -= item_PropertyChanged;
            }
         }
         // Add PropertyChanged event handlers to items inserted into collection
         if (e.NewItems != null) {
            foreach (IValueItem item in e.NewItems) {
               item.PropertyChanged += item_PropertyChanged;
            }
         }
         this.NotifyDataChanged(this);
      }

      /// <summary>
      /// Handles Property changed events from individual items in the collection
      /// </summary>
      /// <param name="sender">Item that has changed</param>
      /// <param name="e">Event arguments</param>
      private void item_PropertyChanged(object sender, 
      System.ComponentModel.PropertyChangedEventArgs e) {
         this.NotifyDataChanged(sender);
      }

      /// <summary>
      /// Raises DataChanged event
      /// </summary>
      /// <param name="sender">Item that hsa changed</param>
      private void NotifyDataChanged(object sender) {
         System.EventHandler handler = this.DataChanged;

         if (handler != null) {
            handler(sender, new System.EventArgs());
         }
      }
   }
}

As you see the constructor wires the CollectionChanged event so any modification to the collection will be noticed. When the collection is changed the PropertyChanged event for all the items is wired so that if any changes occur in the properties of individual value items, the collection is notified.  Both event handlers raise DataChanged event if any change occur.

The calculation

The calculation is done by the LinearTrend class. The usage is that first the DataItems collection is filled with proper value items and when done, Calculate method is called. The calculation fills the following properties

  • Calculated, the value is true after Calculate has been called. However, the class keeps track of changes in the data item collection by listening DataChanged event so if the source data changes in any way, this property is set to false
  • Slope contains the calculated slope
  • Intercept contains the value for Y when Y axis is crossed
  • Correl contains the correlation coefficient
  • R2 contains the r-squared value
  • DataItems contains the source data
  • TrendItems contains the calculated trend value for each unique X value in the source data
  • StartPoint returns the calculated trend value for the first X value
  • EndPoint returns the calculated trend value for the last X value

So the coding part of the calculation looks like this

      /// <summary>
      /// Default constructor
      /// </summary>
      public LinearTrend() {
         this.DataItems = new ValueList<TValueItem>(ValueListTypes.DataItems);
         this.TrendItems = new ValueList<TValueItem>(ValueListTypes.TrendItems);
         this.Calculated = false;
         this.DataItems.DataChanged += DataItems_DataChanged;
      }

      /// <summary>
      /// Handles DataChanged event from the data item collection
      /// </summary>
      /// <param name="sender">Item that has changed</param>
      /// <param name="e"></param>
      private void DataItems_DataChanged(object sender, System.EventArgs e) {
         if (this.Calculated) {
            this.Calculated = false;
            this.Slope = null;
            this.Intercept = null;
            this.Correl = null;
            this.TrendItems.Clear();
         }
      }

      /// <summary>
      /// Calculates the trendline
      /// </summary>
      /// <returns>True if succesful</returns>
      public bool Calculate() {
         double slopeNumerator;
         double slopeDenominator;
         double correlDenominator;
         double r2Numerator;
         double r2Denominator;
         double averageX;
         double averageY;
         TValueItem trendItem;

         if (this.DataItems.Count == 0) {
            return false;
         }

         // Calculate slope
         averageX = this.DataItems.Average(item => item.ConvertedX);
         averageY = this.DataItems.Average(item => item.Y);
         slopeNumerator = this.DataItems.Sum(item => (item.ConvertedX - averageX) 
                                                     * (item.Y - averageY));
         slopeDenominator = this.DataItems.Sum(item => System.Math.Pow(item.ConvertedX - averageX, 2));

         this.Slope = slopeNumerator / slopeDenominator;

         // Calculate Intercept
         this.Intercept = averageY - this.Slope * averageX;

         // Calculate correlation
         correlDenominator = System.Math.Sqrt(
            this.DataItems.Sum(item => System.Math.Pow(item.ConvertedX - averageX, 2)) 
            * this.DataItems.Sum(item => System.Math.Pow(item.Y - averageY, 2)));
         this.Correl = slopeNumerator / correlDenominator;

         // Calculate trend points
         foreach (TValueItem item in this.DataItems.OrderBy(dataItem => dataItem.ConvertedX)) {
            if (this.TrendItems.Where(existingItem 
                => existingItem.ConvertedX == item.ConvertedX).FirstOrDefault() == null) {
               trendItem = (TValueItem)item.NewTrendItem();
               trendItem.ConvertedX = item.ConvertedX;
               trendItem.Y = this.Slope.Value * item.ConvertedX + this.Intercept.Value;
               this.TrendItems.Add(trendItem);
            }
         }

         // Calculate r-squared value
         r2Numerator = this.DataItems.Sum(
            dataItem => System.Math.Pow(dataItem.Y
            - this.TrendItems.Where(
               calcItem => calcItem.ConvertedX == dataItem.ConvertedX).First().Y, 2));

         r2Denominator = this.DataItems.Sum(dataItem => System.Math.Pow(dataItem.Y, 2))
            - (System.Math.Pow(this.DataItems.Sum(dataItem => dataItem.Y), 2) / this.DataItems.Count);

         this.R2 = 1 - (r2Numerator / r2Denominator);

         this.Calculated = true;

         return true;
      }

As you can see I have used LINQ in calculations. It would have been possible to condense the calculation even more, but in order to help debugging I calculated numerators and denominators separately. But as a side-note, using LINQ here simplifies the code a lot.

The test application

Now in order to test the functionality let’s create a small test application. The application should be able to generate both double and datetime values as test material and also to show the results of the calculation. The window looks like this with double values

Image 2

And an example with datetime values

Image 3

The code is quite simple. The "Generate values" -button creates the test data with Random object and when the test material is created one can press the "Calculate" -button to show the results

namespace TrendTest {
   /// <summary>
   /// Interaction logic for MainWindow.xaml
   /// </summary>
   public partial class TestWindow : System.Windows.Window {

      TrendCalculus.LinearTrend<TrendCalculus.IValueItem> linearTrend
      = new TrendCalculus.LinearTrend<TrendCalculus.IValueItem>();

      public TestWindow() {
         InitializeComponent();

         this.UseDouble.IsChecked = true;
         this.Values.ItemsSource = linearTrend.DataItems;
         this.TrendItems.ItemsSource = this.linearTrend.TrendItems;
      }

      private void GenerateValues_Click(object sender, System.Windows.RoutedEventArgs e) {
         System.Random random = new System.Random();

         linearTrend.DataItems.Clear();

         for (int counter = 0; counter < 10; counter++) {
            if (this.UseDouble.IsChecked.Value) {
               linearTrend.DataItems.Add(new TrendCalculus.NumberItem() {
                  X = System.Math.Round(random.NextDouble() * 100),
                  Y = System.Math.Round(random.NextDouble() * 100)
               });
            } else {
               linearTrend.DataItems.Add(new TrendCalculus.DateItem() {
                  X = System.DateTime.Now.AddDays(System.Math.Round(random.NextDouble() * -100)).Date,
                  Y = System.Math.Round(random.NextDouble() * 100)
               });
            }
         }

      }

      private void Calculate_Click(object sender, System.Windows.RoutedEventArgs e) {
         if (this.linearTrend.Calculate()) {
            this.TrendItems.ItemsSource = this.linearTrend.TrendItems;
            this.Slope.Text = this.linearTrend.Slope.ToString();
            this.Intercept.Text = this.linearTrend.Intercept.ToString();
            this.Correl.Text = this.linearTrend.Correl.ToString();
            this.R2.Text = this.linearTrend.R2.ToString();
            this.StartX.Text = this.linearTrend.StartPoint.ConvertedX.ToString();
            this.StartY.Text = this.linearTrend.StartPoint.Y.ToString();
            this.EndX.Text = this.linearTrend.EndPoint.ConvertedX.ToString();
            this.EndY.Text = this.linearTrend.EndPoint.Y.ToString();
         }
      }

      private void UseDouble_Checked(object sender, System.Windows.RoutedEventArgs e) {
         this.linearTrend.DataItems.Clear();
      }

      private void UseDatetime_Checked(object sender, System.Windows.RoutedEventArgs e) {
         this.linearTrend.DataItems.Clear();
      }

      private void DataItemsToClipboard_Click(object sender, System.Windows.RoutedEventArgs e) {
         System.Text.StringBuilder clipboardData = new System.Text.StringBuilder();

         clipboardData.AppendFormat("{0}\t{1}\t{2}", "Actual X", "Converted X", "Y").AppendLine();
         foreach (TrendCalculus.IValueItem item in linearTrend.DataItems) {
            if (item is TrendCalculus.DateItem) {
               clipboardData.AppendFormat("{0}\t{1}\t{2}",
                  ((TrendCalculus.DateItem)item).X.ToShortDateString(), item.ConvertedX, item.Y);
            } else {
               clipboardData.AppendFormat("{0}\t{1}\t{2}",
                  ((TrendCalculus.NumberItem)item).X.ToString(), item.ConvertedX, item.Y);
            }
            clipboardData.AppendLine();
         }
         System.Windows.Clipboard.SetText(clipboardData.ToString());
      }
   }
}

In order to easily test the calculations a "Copy to clipboard" -button is included that copies the source data to clipboard with tabulators as delimiters so that the data can easily be pasted into Excel.

Remarks

As this was the first trial on F# I understand that changing the mindset and learning to produce good F# is going to be a rocky road. However, having used procedural and OO languages for ages F# seems like a fresh breath so far :)

References

The references concerning the corresponding Excel functions can be found at:

History

  • 22nd May, 2016: Article created
  • 28th May, 2016: VB.Net version added
  • 6th June, 2016: F# version added
  • 15th August, 2017: Replaced pictures of formulas with LaTeX equations

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)