Audio a Must!

[ctcline]

This is a great application, but no audio hurts. There needs to at least be a way to hear audio from the webcams. Without it we are unable to use the application for what we need. Otherwise its great.

[BarryE]

Understood, but from a technical perspective this isn't a simple request. Many cameras do not support audio at all. Those that do may not support streaming of audio in a standardized way. To give you some background:

1) MPEG streams can contain both audio and video but require intense decoding and processing. Live MPEG streams are usually handled with RTSP which is not supported in the iPhone SDK. There are free open source libraries that can do this but it consumes vast amounts of CPU and battery to make this work. MPEG4 video also uses a huge variety of compressors/decompressors and not all of these are free or compatible with Apple hardware.

2) MJPEG video (which Live Cams uses) is a video-only data stream. There isn't any audio in it. If the camera can send a separate PCM stream with audio-only then it's possible to decode and play back, but buffers would be required and the audio will not be synced with the video (likely a 2 or 3 second lag at least).

As you can see, there is a reason why I haven't included audio in Live Cams yet.

[sodamo]

Barry, not a software guy or programmer of any sort, but wonder if there maybe anything to learn from the new iphone iOS4 Facetime or perhaps the Fring app. Seems like if we could connect to a speaker cam and it answer automatically something similar to Facetime would be pretty awesome.

BTW, I really like Live Cams

[BarryE]

Barry, not a software guy or programmer of any sort, but wonder if there maybe anything to learn from the new iphone iOS4 Facetime or perhaps the Fring app. Seems like if we could connect to a speaker cam and it answer automatically something similar to Facetime would be pretty awesome.

BTW, I really like Live Cams

Yeah, understood. It's not a matter of learning how to to audio, but rather my disinterest in undertaking such a large project. I have an understanding of how complex it will be to implement because I've done this type of work in the past and it could be considered a product all it's own I'm going to tackle it though, because everyone is asking for it!

[Jonkie]

Barry, sent you an email as well. I understand your point that it could be difficult to add sound. But it is possible as i also wrote in my mail.ipux cams for iphone delivers sound without any problems. So there you go, it should be not really a big problem, at least for some kind of cams.
Ipux cams however doesn't do full screen and more importent: no landscape either!
Ip cams hd is a good app, but the lack of sound really is bothering in my case...

[BarryE]

Jonkie, I understand what you are thinking but it's not as easy as writing a line of code that says "play audio from camera"

I have to use CoreAudio libraries to fetch sequences of packets, stuff them in buffers, tell the audio system what format the audio packets are in, convert them to a supported format if they are not, etc. The biggest headache is actually not known to most people. Apple doesn't want me to tell people but I don't want their rules to continue making me look stupid. They impose a bandwidth limit on applications that use cellular network connections. I must keep application traffic within 1MB/min. That's why the thumbnails and video seem slower on 3G, I actually force the cameras to send a frame, then calculate the delay I should wait before asking for another frame. If you send high-res video then the frame rate must be very slow (as slow as 2 frames/min). I've successfully avoided this limit with a clever algorithm and engine for the past year, but adding audio will make it even more complex. I will have to either eliminate the option for audio on 3G networks (the simplest and most likely option I would take) or further reduce the video performance to account for the extra traffic on the audio stream.

And that's just a small piece of the puzzle Following rules is fun ain't it?

BTW - I've had my app submissions rejected many times because their tests found that the app was exceeding their limits. They would not tell me what the acceptable limit was for the first six months, only that my app was consuming too much. I had to constantly slow it down, submit it, watch it fail their reviews, repeat the process until they approve it. It was only after speaking to an Apple engineer on the phone last summer that I squeezed this vital piece of information from him.

[mukti]

Hi Barry,

bought the app today and love it!
i was looking for something with sound but it doesn't seem to be out there ..
so my bet is that you will be the first to have it.
i also do some programming (in a different field) so i realize that this will take some time.
just wanted to send you some encouragement

-Mukti

[BarryE]

I'm working on audio support now. I am going to try to build an MPEG decoder and strip the audio frames from the stream. At the moment I'm getting video successfully from MPEG4 but I am still working on the audio playback. When completed I will likely also support MPEG4/H264 streams from certain cameras in addition to the MJPEG support I have now.