Problem

Few days ago, I wrote a post on which is the best audio format we should choose when developing an iPhone app. The conclusion is using IMA4, it’s the best choice if you want to record audio on the device.

Today, I was trying to use the System Sound service to play a short audio and in order to save the spaces, I chose the “AAC” format to encode my sound effects. I just thought it’s would be capable for a system sound and didn’t searched the documentation for which format is suit for System Sound. Everything went fine in the simulator. But, when I load the app into the device, and, BOOM~~, it failed to play the sound effect! My first reaction was somewhere programmatic mistake in my code, and after a careful review, I thought the reason could  be the format. So I changed the format back to IMA4, and this time, it went well!

Apparently, System sound did not support AAC format, and I could not find the appropriate document where described it. Apple just pushes us to use IMA4 and other linear PCM format to reduce the CPU and power consume. Yeah, I understand that, but please, place such kind of important information some where we can find it easily! The API documentation could be a good place.

If I am the director of the documentation team, I would promote such information and place it in a notable place! Damn it.

Why I’m angry?

Because if you using AAC for recording or System Sound, it works fine on the simulator, but will be failed on the device! If the device does not support AAC for recording and System Sound, please disable it on the simulator!

Conclusion

The conclusion is if you want to playback a song or something like this, you can choose AAC as your primary format for it is well supported by hardware codec, but only for this purpose please. If you want to record audio or play system sound effect, please choose IMA4, it’s is the second best format for iPhone.

“AAC” is a good compressed format, but it’s also has its cost.

AVAudioPlayer is providing a very convenient way to play iPhone supported audio file. Especially, if you want to implement fast forward and rewind, AVAudioPlayer is the best choice.

But I just found an odd problem. If I use [AVAudioPlayer pause] method, next time if I invoke [AVAudioPlayer play], it had no responde. But after I changed pause to stop, it worked fine. The problem looks like it somewhere inside AVAudioPlayer to control the AudioQueue Object to resume playing. Whatever, by using [AVAudioPlayer stop], you can archive the same goal as [AVAudioPlayer pause] do, I mean FFW and RWD.

BTW, the document said, pause will return YES/NO, but clearly, it is a no return value instance method. The guy who is responsible for this method might copied the same comment from play!

What a joke. This line was there since iPhone 2.2, and is still there in 3.0b5. Didn’t there anybody reported this problem? Should I fire a bug?

Problem

I’ve been fighting for AVAudioPlayer and AudioQueue object for 2 days,

Here is my Dev environment:
iPhone: 3.0Beta 5
OSX 10.5.7

Here is my scenario:
1. I need to use AVAudioPlayer to play a CAF with IMA4 data format. Including fast forward and rewind.
2. I need to use AudioQueue Object to record audio, because 3.0 has not go public yet.
3. A common scenario is record an audio A, then play the original audio A’(Not the one recorded) or vice versa.

The problem is I could NOT play the A’ file  after I record file A.
[theAVAudioPlayer play] always return NO, but [theAVAudioPlayer prepareToPlay] return YES.
This situation only happened after I used the AudioQueue Object for recording. If I did not record, it always worked fine.

About the clean up.
1. I released the theAVAudioPlayer object and set to nil each time after I finished playing.
2. I closed the audio file and deposed the AudioQueue Object each time after I finished recording.

Solution

Thanks to Vitali Molodtsov[Open the original post Needs login], the solution is reset the AudioSessionCategory to kAudioSessionCategory_MediaPlayback just after finish recording. Then every thing goes fine.

Conclusion

I think this is a bug for AVAudioPlayer. Each time, when we initialize an AVAudioPlayer object, it should set the correct AudioSessionCategory for us silently, but it doesn’t. So I’ve fired a bug for Apple.

Today I met a weird exception from AUGraphStart().

Basically, I was running an old sample code written by myself. The main function is mix 2 audio files a playback them simultaneously. I worked fine before, but when I run it, I got this exception:

[00:23:58.425 <AURemoteIOServer>] AQMEIOBase::DoStartIO: timeout
[00:23:58.701 <AURemoteIOServer>] AQMEDevice::StartIO: AudioOutputUnitStart returned -66681
[00:23:58.701 <0xa084a720>] AUIOClient_StartIO failed (-66681)
AUGraphStart FFFEFB87 

Obviously, it wasn’t came from my code, and I have no idea on this exception now. Did anybody get a same exception recently?

I had tried these:

   1. Restart the XCode. — No effects.

   2. Restart the machine(Mac Pro, 10.5.7), and restart the XCode. — No effects.

   3. Reinstall the most recent SDK(Build 9M2735), restart the XCode. — No effects.

   4. Set the base SDK to 2.2.1 and target to Simulator 2.2.1 Debug. — No effects.

It’s really odd!

Please, if anybody had met similar exception and got a solution, help me out!

Still don’t know how to resolve it now. I need  help!

[Update 2009-06-11]

A week ago, Apple’s engineer had confirmed that they had a bug in UIKit framework.

We do have a simple workaround for this UIKit bug. In UILocalizedIndexedCollation.hm change this:

  UIKIT_EXTERN @interface UILocalizedIndexedCollation : NSObject

to

  UIKIT_EXTERN_CLASS @interface UILocalizedIndexedCollation : NSObject

Having done that, we can build your project and confirm that AUGraphStart returns 0 in the 3.0 simulator.

 

But, after I installed the 3.0 GM Version today, it’s still not included in the GM Version. So that means I still can not play audio in the simulator. I can ONLY do it on the device!

I had never done audio programming before I started my iPhone programming. After starting iPhone programming, I started to learn CoreAudio Framework, Audio Unit for Mac OS X system and iPhone System, and largest problem is which audio format should I choose for best practice?

After iPhone OS 3.0 beta was released, Apple finally introduced AVAudioRecorder class to AVFoundation framework. With which, we can play and record audio more efficiently! For me, I can cut the original playback code down from 400+ lines to 20 lines. And for recording, cut down from 300+ lines to 30 lines.

When testing AVAudioRecorder, the file format will significantly effect the file size. I had tried all the available data format when packed with .CAF file.  Here is the result:

[Updated@2009-05-25] YES, we can record with this format on simulator, but we CANNOT record this on the device! Remember this. While another one we can refer to is Apple’s official QA1615

[Updated@2009-06-15] Please do not use AAC if you want to play a system sound.

[Updated@2009-09-30] iPhone 3GS support AAC recording since iPhone OS 3.1 was released on Sept 9th. You can use hardware assisted encoding to record the AAC formatted audio file. But let me remind you, you can not use AVAudioRecord to record AAC audio file yet. The only option is using the RemoteIO Unit.

• kAudioFormatLinearPCM               = ‘lpcm’,    OK    20.2M    .CAF
• kAudioFormatAppleIMA4               = ‘ima4′,    OK    2.7M     .CAF [Best Choice]
• kAudioFormatMPEG4AAC                = ‘aac ‘,    OK    968K     .CAF [Updated 2009-05-25], [Updated 2009-09-30] Best Choice for iPhone 3GS
• kAudioFormatMACE3                   = ‘MAC3′,    NG            .CAF
• kAudioFormatMACE6                   = ‘MAC6′,    NG            .CAF
• kAudioFormatULaw                    = ‘ulaw’,    OK    5.1M     .CAF
• kAudioFormatALaw                    = ‘alaw’,    OK    5.1M     .CAF
• kAudioFormatQDesign                 = ‘QDMC’,    NG            .CAF
• kAudioFormatQDesign2                = ‘QDM2′,    NG            .CAF
• kAudioFormatQUALCOMM                = ‘Qclp’,    NG            .CAF
• kAudioFormatMPEGLayer1              = ‘.mp1′,    NG            .CAF
• kAudioFormatMPEGLayer2              = ‘.mp2′,    NG            .CAF
• kAudioFormatMPEGLayer3              = ‘.mp3′,    NG            .CAF
• kAudioFormatAppleLossless           = ‘alac’        OK    4M        .CAF
• kAudioFormatMPEG4AAC_LD             = ‘aacl’,    NG            .CAF
• kAudioFormatAMR                     = ’samr’,    NG            .CAF
• kAudioFormatiLBC                    = ‘ilbc’,    NG    Wierd    .CAF

The list here has omitted all the other data format that are not supported by CAF file extension. You can use the console application afconvert to verify each supported file format and data format via typing this:

afconvert -h 

All the tests were recorded in 60 seconds, with sample rate@44100, 2 channels, 96kbps. The RecordSetting Dictionary object is used as below:

recordSetting = [[NSMutableDictionary alloc] init];

//General Audio Format Settings (Necessary for all audio format.)
[recordSetting setValue:[NSNumber numberWithInt: kAudioFormatMPEG4AACkAudioFormatAppleIMA4] forKey:AVFormatIDKey];
[recordSetting setValue:[NSNumber numberWithFloat:44100.0] forKey:AVSampleRateKey];
[recordSetting setValue:[NSNumber numberWithInt: 2] forKey:AVNumberOfChannelsKey];

//Linear PCM Format Settings (only necessary when you want to record Liner PCM format)
[recordSetting setValue:[NSNumber numberWithInt: 32] forKey:AVLinearPCMBitDepthKey];
[recordSetting setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsBigEndianKey];
[recordSetting setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsFloatKey];

//Encoder Settings (Only necessary if you want to change it.)
[recordSetting setValue:[NSNumber numberWithInt:AVAudioQualityMin] forKey:AVEncoderAudioQualityKey];
[recordSetting setValue:[NSNumber numberWithInt:96] forKey:AVEncoderBitRateKey];
[recordSetting setValue:[NSNumber numberWithInt:16] forKey:AVEncoderBitDepthHintKey];

//Sample Rate Conversion Settings (Only necessary when you want to change the sample rate to a value different to the hardware sample rate, AVAudioQualityHigh means no conversion, usually, 44.1KHz)
[recordSetting setValue:[NSNumber numberWithInt:AVAudioQualityHigh] forKey:AVSampleRateConverterAudioQualityKey];

[Updated 2009-09-30], The record settings listed above was for example only, if you want to record IMA4 or other compressed audio format, you are no need to put the Liner PCM part. Other parts are commented for their purpose.

You can see, the MPEG4AAC has the minimum file size and same quality when compare to other file formats. About the AAC format, please refer to [Wikipedia – AAC"]

Apple, you could just give us a best practice guide! So I won’t need to test it one by one myself. OK, I know, you are working out of people, hire me, I would love to do these works.

When Apple use the error enumerations, it used a fancy cool tip to define the error code. (I would like to call this magic error number, whatever apple call it.)

Just look at this:

kAudioFormatUnsupportedDataFormatError 1718449215 = ‘fmt?’
The playback data format is unsupported (declared in AudioFormat.h).
Available in iPhone OS 2.0 and later.

When you run you app and get an error code, the only thing you could see in the console is an error number like 1718449215 or 560030580, if you don’t know what this magic number is, you would have no idea what this f**king number is. But if you paste this number to calculator app(I mean the system calculator application as below), you would see the real meaning in these number.

Paste the magic error number and see the real meaning

Did you see? in the left down corner, there is an string “!act” that’s what this number means.

The main idea behind this magic error number is how to interpret the 4 bytes int32 data type. When we define a number in C or in C++ and Obj-C, you can say:

Int32 a = 'abcd';

This is legal and cool, it gives every number an abbreviated meaning.

Well, Frankly, it is a good tip! But the nightmare is Apple did not use it everywhere, it just adopted some of these mechanism into the current iPhone API, especially, in CoreAudio, so if you find some error code is strange, just copy it, paste it to the calculator, see what you can get. It might be help.