Intercontext communication

Hello,

One of the challenges of developing concurrent software, is how to communicate between different contexts. I use the word context as a placeholder for thread, process, or machine. In most desktop application software I have seen, shared data accessed synchronously seems to be the preferred way to communicate. Data access from multiple contexts must be synchronous in order to be safe. In addition to synchronous access of shared data, you can communicate via some kind of channel or port, which allows one context to send data, and another to receive it.

All this is old news. There is plenty of information online which explains thread safe programming, shared memory, BSD sockets and other such things. My point is that all the various mechanisms are there to serve one underlying purpose: communication. Ultimately, whichever mechanism you choose, what you really want to do is communicate some information. Granted, efficiency, capability, and convenience greatly influence your choices, but from a system design perspective, they are hardly relevant. When you design a system at a high level, using say UML, it's unlikely that you will have a model element for each mutex or socket you use.

With this in mind, now think about object oriented programming. Hmm... the wheels in my head are turning. Can we abstract the very concept of communication into an interface? I prefer to reserve judgement, I don't think I am prepared to follow this line of reasoning to its limits. Taking a step backwards, and trying to fit this into my audio processing engine, what consequences does this perspective have on my design? Clearly I have a concurrent application on my hands, as there is one or more realtime audio processing contexts, and one or more control contexts. Well suddenly I have a lot more freedom from the constraints of specific communication methods. Whereas mutex based communication is only available within a single process, or shared memory is only available on a single machine, the concept of communication is not bound by any particular constraints (semioticians, lets not nitpick).

In practice, this means that my code must use some kind of polymorphic communication system, most likely a class hierarchy, but the payback for this abstraction is potentially huge. Various application contexts can reside virtually anywhere, as long as a mechanism exists which can facilitate communication. Theoretically, a context may exist anywhere in space or time, communicating via wormhole, or some other theoretical or yet unimagined mechanism.

Now this is all very dramatic, but hopefully it illustrates how much freedom you can gain with a little abstraction. Now I am off to design a (simple) abstract communication library so that my audio processing system can be free as a (time traveling, teleporting) bird.

Cocoa AudioUnits

Hello,

In order to be useful, an audio processing graph must be made up of processing nodes. Without processing tools such as delays, equalizers and bandpass filters, my audio processing graph becomes an audio IO graph. So if I want to develop and test my system, I need to have some of these tools available. There is a wide assortment of processing libraries available online, including SoundObjects, STK, CLAM, and others. These libraries are mostly GPL licensed, but at least I can use them during my internal testing.

However, that being said, my computer already contains a wide assortment of such tools packaged as plugins, AudioUnit plugins to be precise. So, in order to test my engine and make it useful right out of the gate (on OS X), I have decided to include support for loading AudioUnits as nodes in my processing graph.

You can look on Apple's developer network for a broad and detailed explanation of what AudioUnits are. If you have experience with VST, MAS, or even DirectX plugins, then you already have an idea what AudioUnits are.

AudioUnits can be pro grammatically used by means of the AudioUnit framework, and the AudioToolbox framework. While the AudioUnit framework provides the bare essentials needed to instantiate and use AudioUnits (AU's for short), AudioToolbox builds upon this core and adds some very useful utilities, including the ability to build AU graphs... Hey! Wait a minute!? I am trying to build an audio processing graph engine! No worries, the AU graph system and my own engine have very different goals, and as such, the overlap is is minimal. The differences are as follows:

• Portability: Obviously AU's are only available on OS X.
• Realtime insulation: While the AU graph system does support connection management from a non-realtime context, additional realtime constraints issues are not clearly defined.
• Extensible: The only way to extend the capabilities of an AU graph is to add AU's.

The most important difference to me is the second, that of realtime constraints insulation. To be honest I am quite happy to limit myself to developing on OS X, as it is by far the best platform I have worked on. Regardless, I find enough justification to develop my own audio processing framework.

Cocoa AudioUnits

I am a little surprised that AudioUnits have not been available as a Cocoa library. Perhaps there is little demand, many pro audio applications have their roots in pre-OS X systems, or are developed on a portable layer. Having been frustrated using the C interface to AudioUnits, I came up with a thin Cocoa wrapper, which allows me to treat AudioUnits as first class Cocoa objects. This is very nice when you want to store your AU's in an NSArray, or archive them, or display them in an NSTableView, etc. The wrapper is thin because it tries to model the existing C API in objective-C, with a few convenience functions thrown in. In addition to the ArkAudioUnit class, I have included ArkAudioUnitEditor and ArkAudioUnitManager. The former provides support for displaying AU views using one line of code, while the latter makes it easy to discover and instantiate AU's by category or human readable string.

The Cocoa AudioUnit source code may be found at http://code.google.com/p/audiodeveloper/ and is BSD licensed. There is no documentation yet (you may recall that one of my goals for this project was comprehensive documentation!) but the header files are fairly neat, and the interface so closely mimics the AudioUnit C API that developers should have little trouble understanding it.

Here is a snippet to get you started. Error checking is omitted for brevity.

// Declared elsewhere...
// ArkAudioUnit * output, * mixer, * reverb;

[ArkAudioUnitManager createDefaultAUGraph];

ArkAudioUnitManager * auManager =
[ArkAudioUnitManager defaultManager];

[auManager createAllAudioUnitLists];

output = [auManager createDefaultOutput];

[output initialize];

NSString * defaultMixerName =
[[auManager mixerNames] objectAtIndex:0];

mixer = [auManager createMixerWithName:defaultMixerName];

[mixer initialize];

reverb = [ArkAudioUnit audioUnitWithType:kAudioUnitType_Effect
subType:kAudioUnitSubType_MatrixReverb
manufacturer:kAudioUnitManufacturer_Apple];

[reverb initialize];
[reverb setBypassing:YES]; // Reverb is off by default.

[output connectInput:0 fromAudioUnit:reverb element:0];
[reverb connectInput:0 fromAudioUnit:mixer element:0];

[output retain];
[mixer retain];
[reverb retain];

// ...
// Later, in response to an action, this code will
// display a window with the AudioUnit view inside.
[reverb showEditor];

Not Bad huh? I have used this code for about 4 years in a few different projects. No major bugs have popped up, but as the BSD license says, don't blame me if things go awry.

Anyways, that's enough for now, I suppose I will revise and expand at some point, until then, goodbye!

Wow, welcome to the interweb

Hi,
I don't suppose anyone would want to read this, but I must assume that I am not writing to myself, you know, for sanity's sake. I wanted to call this blog "Lazy Developer", but it was taken. So with that out of the way, expectations should be suitably low.

My main focus on this blog will be to write about my adventures in audio software development. As of now, I am living and working in Shanghai China. My work has nothing to do with software development, or even computers. I teach adorable little demons, er, children at a kindergarten. When I go home, I try and find a few hours to write software, but as often as not, I get caught up in other affairs.

My current project is a portable realtime audio engine written in C++. The focus is to abstract and simplify the creation and manipulation of audio processing networks (or graphs). I do not intend to develop DSP algorithms or synthesis techniques, there are plenty of more qualified and capable people doing that right now.

If you have ever tried to do any audio development, then you know the huge headaches that realtime constraints bring. When I started learning software development, about 7 years ago, accurate and comprehensive information on realtime programming was not easily found on the internet. In fact, the most useful document I found early on was a programming manual for a realtime Unix variant... from 1995! Today there is more information available, but misunderstandings are still very common. Just browse Apple's coreaudio developer mailing list to see what I mean.

Do not be under the impression that I know what I am talking about, take everything with a grain of salt and the understanding that you are reading the words of an experienced hobbyist with no formal education in software development of any kind. Nevertheless, I intend to write accurate information only, and not stray from the bounds of my "expertise" (apologies to professional developers everywhere).