OpenSebJ by DSebJ: August 2009

While trying to hook up a number of development components, I needed to call a method in a custom control that would update the fill of a rectangle. Calling this method, from within the event of a ButtonClick worked fine, however when I attempted to programmatically access the same method within the Custom Control I encountered an error:

InvalidOperationException - “The calling thread cannot access this object because a different thread owns it.”

It’s all due to the multi threaded nature of what’s going on in the background and thankfully the multithreading is going on, giving us our dynamic, performing environment. But there is a slight addition in complexity to resolve this issue, which is the Dispatcher

The Dispatcher is linked to the UI thread to handle events which require access to UI elements, and in the WPF world it looks a bit simpler than the world of Win Forms. To utilise the Dispatcher there is a bit of setup to complete.

To put this example in context, look at the following screen shot:

<Screen shot to come another day, just imagine a piano keyboard for now>

This is part of a larger custom control, representing a keyboard scale. Each key is made up of a rectangle. The fill can be replaced with an alternate fill to indicate that the key is down and subsequently reverted back to the original fill when the key is released. This control has 2 methods to invoke this functionality:

ColourKey(int key);
UnColourKey(int key);

An instance of this control has been created and added on to a WPF window and linked via the MIDI Framework provided by NAudio, to a MIDI Controller Keyboard. These methods on the custom control can’t be called directly from a method not being invoked from the UI – i.e not a button click. As such we need to utilise a delegate, I’ve opted for two in this example.

//// This delegate enables asynchronous calls for setting
//// the Colour of the key
delegate void dColourKey(int key);
delegate void dUnColourKey(int key);

These delegates provide a safe way to call the methods which have been setup on the custom control.:

/// <summary>
/// Called via the delegate after using this.Dispatcher.Invoke
/// </summary>
/// <param name="key">The key to colour on the scale control</param>
private void ColourKey(int key)
{
scale1.ColourKey(key);
}

/// <summary>
/// Called via the delegate after using this.Dispatcher.Invoke
/// </summary>
/// <param name="key">The key to UnColour on the scale control</param>
private void UnColourKey(int key)
{
scale1.UnColourKey(key);
}

To access these methods, via the delegate we need to use the following;

this.Dispatcher.Invoke(new dColourKey(ColourKey),ne.NoteNumber);

A small explanation is in order. The first part is obviously calling the Invoke method on the Dispatcher. The two parameters are where it all happens. The first is initialising the delegate, and passing in the name of the method which is to be called – in this example, the name of the method is ColourKey:

new dColourKey(ColourKey)

The second is the argument which will be passed to the calling method & in this case it’s the number of the note which is being played:

ne.NoteNumber

This small overhead ensures that we don’t have a situation where our InvalidOperationException will be generated.

Extended Learning:

One of the reasons I like to write up items like this is so that I can increase my own understanding about the topic. Being able to explain a concept to some one else not only assists the person who your explaining it to but reinforces your own understanding and through writing this post I’ve opened my own eyes further. I’ve refracted my own code and included the Dispatcher and Delegate code in the custom control; which means it’s no longer required by the calling WPF form. Amazing.

It’s been a while.

So kicking off from where we left off this instalment is going to be looking at MIDI and how we can interoperate control characters sent through the MIDI interface in our application. This tutorial as per the other tutorials in the series assumes that you have read the previous tutorials, as we will be building upon concepts and understanding from each of these sections. Feel free to read through this tutorial cold but if your looking for the background for anything not covered here it would be best to check out the other Tutorials as a first stop.

http://opensebj.blogspot.com/2009/02/introduction-to-using-naudio.html
http://opensebj.blogspot.com/2009/02/naudio-tutorial-2-mixing-multiple-wave.html
http://opensebj.blogspot.com/2009/03/naudio-tutorial-3-sample-properties.html
http://opensebj.blogspot.com/2009/03/naudio-tutorials-minor-note.html
http://opensebj.blogspot.com/2009/03/naudio-tutorial-4-sample-reversing.html
http://opensebj.blogspot.com/2009/04/naudio-tutorial-5-recording-audio.html

As you may by now expect, NAudio has a set of functions for this as well. You will find the useful set of functions under NAudio.Midi;

Setting It

Lets create a class to encapsulate the bulk of the MIDI functionality that we will be calling upon for this tutorial.

using NAudio.Midi;
namespace AudioInterface
{
    public class NAudioMIDI
    {
       public MidiIn midiIn;
       private bool monitoring;
       private int midiInDevice;

Our midiInDevice represents what MIDI device on the system we want to use for this interface; in case you have more than one MIDI device connected to your system. I only have a single MIDI device however going through this process is obviously useful for those who have more than one and it’s useful to check that the MIDI device I have is actually plugged in and switched on.

Once we have defined what MIDI device we will be using, it will be initiated and the midiIn instance will relate to that device.

/// <summary>
/// Get a list of MIDI Devices
/// </summary>
/// <returns>string[] of MIDI Device Names</returns>
public string[] GetMIDIInDevices()
{
     // Get a list of devices
     string[] returnDevices = new string[MidiIn.NumberOfDevices];

     // Get the product name for each device found
     for (int device = 0; device < MidiIn.NumberOfDevices; device++)
     {
          returnDevices[device] = MidiIn.DeviceInfo(device).ProductName;
     }
     return returnDevices;
}

Assuming that we want to allow the user to select a Device from a list of Device’s then we would pass this list back to a control which will populate this list with the available devices. With something like this from our Load method on the form class:

private void NAudioTutorial6_Load(object sender, EventArgs e)
{
<SNIP>

      // Populate the devices available for the MIDI interface
      string[] MIDIDevices = AudioInterface.NAudioInterface.nMIDI.GetMIDIInDevices();
      foreach (string devices in MIDIDevices)
      {
          boxMIDIIn.Items.Add(devices);
      }
      try
      {
          boxMIDIIn.SelectedIndex = 0;
      }catch(Exception except){
          System.Windows.Forms.MessageBox.Show("No MIDI Device Detected");
      }
<SNIP>

Starting It

Brilliant, so now we have a list of available MIDI devices, loaded in to a list box control, that the user can select from. Now we need to know when the user has actually chosen the MIDI control they would like us to monitor; so let’s put in a button on our UI to trigger this.

private void cmbMonitor_Click(object sender, EventArgs e)
{
    // Setup the MIDI interface to start monitoring the selected device
    AudioInterface.NAudioInterface.nMIDI.StartMonitoring(boxMIDIIn.SelectedIndex);

    // Add the event handler, to handle the MIDI messages received
    AudioInterface.NAudioInterface.nMIDI.midiIn.MessageReceived += new EventHandler<MidiInMessageEventArgs>(midiIn_MessageReceived);
}

When StartMonitoring is called we through back to the nMIDI instance we created earlier and (using the selected MIDI device) setup the midiIn device and set the midiIn device to Start – which in turn, kicks NAudio in to gear to start monitoring MIDI messages received from the MIDI device.

public void StartMonitoring(int MIDIInDevice)
{
      if (midiIn == null)
      {
          midiIn = new MidiIn(MIDIInDevice);
      }
      midiIn.Start();
      monitoring = true;
}

Going back to cmbMonitor(…) we next setup the EventHandler for the MIDI messages which are going to be received:

// Add the event handler, to handle the MIDI messages received
AudioInterface.NAudioInterface.nMIDI.midiIn.MessageReceived += new EventHandler<MidiInMessageEventArgs>(midiIn_MessageReceived);

Playing It

For this to work, we need to have an event handler method setup to receive the messages, within the same class. From the line above you should see that the method is midiIn_MessageReceived - which we will have a look at now:

public void midiIn_MessageReceived(object sender, MidiInMessageEventArgs e)
{
     // Exit if the MidiEvent is null or is the AutoSensing command code
     if (e.MidiEvent != null && e.MidiEvent.CommandCode == MidiCommandCode.AutoSensing)
     {
          return;
     }

Assuming that MIDI Event Command Code represents a Note On Event, then we need to interpret what Note On Event has been sent. To do this we need to cast the MidiEvent to a NoteOnEvent:

     if (e.MidiEvent.CommandCode == MidiCommandCode.NoteOn)
     {
          // As the Command Code is a NoteOn then we need
          // to cast the MidiEvent to the NoteOnEvent
          NoteOnEvent ne;
          ne = (NoteOnEvent)e.MidiEvent;

ne is now a NoteOnEvent which has some specific MIDI attributes, such as a NoteNumber, which is an int that represents a single note from the full scale; as well as a Velocity which represents how hard the MIDI note has been played, in this example it how hard was the MIDI controller pressed (assuming that the MIDI controller you have can report this information ala levels of sensitivity).

Each NoteNumber represents an incremental note on the scale, starting with C0 == 0, Db0 == 1 (C# aka D-before-0), D0 ==2, Eb0 ==3 (D#), E0 == 4 etc. this relationship continues on. For practical purposes (read number of samples for the Piano scale that I have, tops out at 96 which is C8) two sets of notes have been mapped within the NAudioInterface class, in a single array. The first set of notes, 0 – 100 are consider mf (quite). Notes 100 – 200 represent the same positions, but contain samples loaded that are ff (loud). Separating by a round 100 makes all the additions and subtractions to interface with these notes rather straight forward. This mapping is contained within the vKeys Class and is a whole heap of excitement, if a long list of static mappings is your thing. A snip-it of the class:

public static class vKeys

<SNIP>

vFFKeysFileNames[48] = "ff.C4.wav";
vFFKeysFileNames[49] = "ff.Db4.wav";
vFFKeysFileNames[50] = "ff.D4.wav";
vFFKeysFileNames[51] = "ff.Eb4.wav";
vFFKeysFileNames[52] = "ff.E4.wav";

<SNIP>

Ohh Ahh..

Back to the velocity, so we have a number, ne.Velocity which represents how hard the note has been played, as such we use that to then work out what sample should be played. If it’s less then 50, then the quite sample is played, else loud.

          if (ne.Velocity < 50)
          {
               AudioInterface.NAudioInterface.Play(ne.NoteNumber);
          }
          else
          {
               AudioInterface.NAudioInterface.Play(ne.NoteNumber + 100);
          }
    }

Stoping It

This means that we can now play a note and conversely we need to be able to stop playing a note. To fulfil this requirement we have the following, which is effectively the converse with the exception that no checking of the Velocity is required, instead all related samples are requested to be faded out, both the loud and the soft. One may ask whys that, basically a model of the real instrument. When a single note in an instrument stops playing, all of the note stops playing. If it had first been played softly and then loudly but then has stoped being played, then the note is no longer being played – regardless of original velocity. To this end, both sets of the notes are Faded Out.

     if (e.MidiEvent.CommandCode == MidiCommandCode.NoteOff)
     {
          NoteEvent ne;
          ne = (NoteEvent)e.MidiEvent;

          AudioInterface.NAudioInterface.FadeOut(ne.NoteNumber);
          AudioInterface.NAudioInterface.FadeOut(ne.NoteNumber + 100);
     }

Changing It

The home stretch and in fact this could easily be left off. This last section relates to a controller value being changed. The controller, at least on the MIDI device I have represents a set of buttons and knobs – the following code is more fixed then you would put in production code but it suits the purpose of a tutorial and most important, scratches an itch.

Determining if this is a ControlChange event and assuming it is, then the MidiEvent needs to be cast to a ControlChangeEvent:

     if (e.MidiEvent.CommandCode == MidiCommandCode.ControlChange)
     {
          ControlChangeEvent cce;
          cce = (ControlChangeEvent)e.MidiEvent;

Similar to the NoteOnEvent, the ControlChangeEvent has a numerical value, the attribute Controller - which is used to determine which Controller’s value has been changed. For this example we are only monitoring one specific controller, 71. The individual notes above also have the attribute of sensitivity, similarly Controllers have a ControllerValue. The ControllerValue is a value in the range of 0 – 127. This controller has been used to define the time out value for the notes which are played. The longer the fade out, the more of the note duration is heard.

          if ((int)cce.Controller == 71)
          {
               int timeOutValue;
               if (cce.ControllerValue < 127)
               {
                  // Calculate a sliding value for the fade out based on the
                    // ControllerValue. This could be drematically improved..
                    // It is meant to be very granular at one end and more extreme
                    // at the other but the calculation could surley be improved.
                    timeOutValue = (int)Math.Exp(Math.Log(cce.ControllerValue) * 1.75);
               }
               else
               {
                    timeOutValue = 100000;
               }
               AudioInterface.NAudioInterface.SetFadeOut(timeOutValue);
          }
     }
}

Finishing It

Tha, tha, that’s all folks.

For more NAudio guidance, please review the other NAudio tutorials in the series.

OpenSebJ by DSebJ

Sunday, August 30, 2009

UI Threading on WPF