Expanding BeRTOS multimedia features

Of course I can't hope to decode HD videos on an 48 MHz ARM; however, such CPU has enough power to decode various compressed audio formats, which is what I'm focusing on right now.

Motivations

"Why would you do that?" I hear you asking. Well, as I already mentioned, despite my personal interest in multimedia, my experience is limited and this is a good opportunity to expand my knowledge in the audio field.

Among other things, I want to:

• understand and analyze the difference between various codecs. For example, Speex has an ultra wideband mode (32 kHz) which can be interesting to use to compress music on embedded systems in place of Vorbis or mp3;
• understand the issues of audio synchronization
• have fun!

Furthermore, I'm proud to expand BeRTOS features and I'm curious to see what performance can be achieved on really small systems.

Long term objectives

In the long term I'd like to create a framework to make easy adding new codecs, new audio effects and more. I draw inspiration from desktop oriented multimedia frameworks (GStreamer, Jack), where audio processing is done in different blocks linked together by buffers. For example, playing a wav file can be divided into a file reader block, a wav parse block (which reads header information and outputs data) and an audio convert block, to adapt wav sample rate and bit per sample to driver's one.

Also, I'd like to create a simple high level API that allows to easily play audio files, without the need to understand the underlying framework structure. I'm aiming for an interface similar to FMod library, where you just specify the file name to play and the effects you want to apply.

Probably the framework will need to enable the kernel, but I haven't decided yet. It mainly depends on how complex the interface will be without processes. Also, I'm thinking whether enabling dynamic memory (heap) is a good choice. It will make some things easier, for example on-the-fly creation of decoding blocks based on the file type to play, but probably such a use case is not frequent in embedded systems, where usually the file type is known in advance.

Short term objectives

I already have a WAV decoder ready (which was easy to do), so now I'll switch to compressed audio formats, which require much less storage space. The first codec I'd like to import is Speex, which has really interesting features for embedded systems: it encodes audio very efficiently and decoding load is low enough (12-15% of CPU time expected on ARMv4).

Next I want to import as quickly as possible one music decoder, either mp3 or vorbis. I'd like to start with vorbis, both for license issues and personal preference, but after a quick look I found that the fixed point implementation of the vorbis decoder is not in the official release of the library, so I need to estimate how much time will be needed to import it.

You will get updates on the project soon.