symba (779893), страница 16
Текст из файла (страница 16)
AAC is intended to beits successor and provides better compression at lower bit rates. AMR isspecifically designed for encoding speech data.The encoded video image and audio data for a video are often storedor streamed together in a container. For video files, there are a number ofdifferent container formats that are commonly used, such as MP4, 3GP,and AVI. They differ in how the encoded data is stored within the fileand in what encoded data formats they can store.
MP4 and 3GP keepthe information about the size and position of the encoded data unitsseparate from the encoded data itself, allowing easier access to specificunits. AVI groups all the information together.Video image and audio data types are often described using a fourcharacter code, known as a ‘FourCC code’.
For instance, video imagedata encoded using the XVID codec would have a FourCC code of‘XVID’.In addition, the container file format and the video image dataformat can be described using a notation called a MIME type. Forinstance, the AVI container file format is described by the MIME typevideo/msvideo and the MPEG4 video image data format has the MIMEtype video/mp4v-es.SYMBIAN OS VIDEO ARCHITECTURE574.2 Symbian OS Video ArchitectureFigure 4.1 shows an overview of the Symbian OS video architecture. Inthis section, we briefly describe each of the architectural layers.ClientVideo Client APlsMMFMMF FrameworkVideo controllersMDFDevVideoDevSoundDevVideoPlayDevVideoRecordHWDeviceCodecs/ProcessorsKeySymbian-supplied componentThird-party supplier componentFigure 4.1 The Symbian OS video architecture4.2.1 Client APIsAccess to the video functionality is provided through the client APIs.Applications use the APIs directly to perform the required video operations.
One important point to note is that even though the client APIsmay support an operation, there is no guarantee that the lower layers ofthe architecture will do so. Most of the functionality that performs videooperations is handled by the video controllers (see Section 4.2.3) andsome controllers may not handle certain operations. The client APIs aredescribed in more detail in the following sections.Video PlaybackThe client APIs support playback from file, stream, and descriptor.
So youcan open a file by specifying the file name or handle, specify a URL tostream from, or pass a buffer that contains the video data. Other basicvideo operations such as play, pause, and stop are also supported. Thereare methods which allow you to manipulate how the video is displayedon the screen and to perform operations such as rotating the video. There58MULTIMEDIA FRAMEWORK: VIDEOare also methods which allow you to get information about the video,such as frame rate and frame size.Video RecordingAs with video playback, you can record to a file, to a stream, or to adescriptor. You can specify a file name, a stream URL, or a buffer.
Inaddition you are required to give the handle of the camera that youwould like to record from. See Section 3.6 for details about preparing thecamera for recording. You can start, stop and pause the recording. Thereare also methods which allow you to modify various characteristics forthe stored video such as the frame rate, or audio sample rate.4.2.2 MMF FrameworkThe MMF framework provides methods to identify and load the videocontroller that should be used to playback or record a video.
Theframework also provides the mechanism for passing client requests to thecontroller and receiving responses and errors from the controller.4.2.3 Video ControllersWhile this book does not cover how to write a video controller, it isimportant to understand what they do. Video controllers are normallysupplied preinstalled on a mobile phone, although, because the controllers use a plug-in mechanism, it is possible for developers to write andinstall their own controllers if they would like to.A video controller controls the flow of video data from an input toone or more outputs.
For example, a controller might open an MP4file containing video and audio, use a video codec to decode thevideo frames and then write them to the display, while using an audiocodec to decode the audio frames and then play them through theloudspeaker.Each installed video controller advertises the container formats it candeal with. So for instance you might find one controller that can open andplay only AVI files, while another controller, such as RealPlayer, mightbe able to support a whole range of container formats. This allows thecorrect controller to be selected based on what type of file or stream isbeing used.For video playback from file, the chosen controller is responsible foropening the file and parsing any metadata in order to determine the typesof video and audio data streams; the relevant video and audio codecs arethen loaded to decode those streams.
The controller extracts the videoand audio frames into buffers and passes them to the relevant codecs fordecoding. For video, the controller may then pass the decoded framesSYMBIAN OS VIDEO ARCHITECTURE59onto the display device; alternatively, the codec may do this itself andjust return the buffer once it is done.For video recording to file, the controller takes the video frames fromthe camera and passes them to a video codec for encoding.
On somedevices, an external hardware accelerator may encode the video framesbefore they are passed to the controller. At the same time, the audio fromthe microphone is encoded by an audio codec. The controller then writesthe encoded video and audio frames into the container file.Another important role for a video controller is to ensure that thesynchronization of audio and video is correct. For playback, the videoframes must be played at the same time as the associated audio frames.For instance, if a person in the video is speaking, the audio and videoframes need to match up to ensure lip synchronization is correct.From Symbian OS v9.3 onwards, Symbian provides one referencevideo controller that can be used for test purposes.
If you want to usethis for testing, you should first check that the SDK you are using actuallycontains this controller. The controller handles playback from file andrecording to file for the AVI container format. Within the file, video datais expected to use the XVID codec and audio data is expected to use thePCM format.4.2.4 DevVideoDevVideo is used by the video controllers to access video decoding,encoding, pre-processing, and post-processing functionality. It can beused to find, load, and configure codecs and performs buffer passingbetween the controllers and codecs. It is implemented by two separateclasses, DevVideoPlay for playback and DevVideoRecord for recording.4.2.5 DevSoundDevSound is used by the video controllers to access audio decoding andencoding functionality. It performs a similar role to DevVideo.4.2.6 HWDeviceThe HWDevice layer defines the interface that DevVideo and DevSounduse to communicate with the codecs.
The implementation of the codecsthemselves is specific to the hardware and platform, so the HWDevicelayer is needed to ensure a consistent interface is available for DevVideoand DevSound to use.4.2.7 CodecsThe video codecs are responsible for converting between compressed anduncompressed video data. As with the controllers, each codec advertises60MULTIMEDIA FRAMEWORK: VIDEOwhat type of video data it can process. Once a controller has determinedthe type of video data that it needs to handle, it can find and load therelevant codec. At this level of the architecture sit the audio codecs aswell, performing the same job as the video codecs but for audio data.For test purposes, an open source XVID video codec is available fordownload from the Symbian Developer Network website (developer.symbian.com). Full instructions on building and installing the codec areprovided on the website.
This codec is compatible with the reference AVIcontroller described in Section 4.2.3.As with controllers, codecs with their associated HWDevice are loadedas plug-ins so it is possible for developers to write and install theirown codecs if they would like to. One thing to remember though isthat developers do not generally have access to hardware accelerationdevices on the phone and this may mean that the codec’s performance isdisappointing.4.3 Client API IntroductionFor playback, there are two classes that contain all of the APIs thatcan be used, CVideoPlayerUtility (found in videoplayer.h) andCVideoPlayerUtility2 (found in videoplayer2.h).
For eitherclass, you must link against mediaclientvideo.lib.CVideoPlayerUtility can be used with any video controller,whereas CVideoPlayerUtility2 is only available from SymbianOS v9.5 onwards and can only be used with video controllers thatsupport graphics surfaces (see Section 4.6.3 for more information aboutgraphics surfaces). If you use CVideoPlayerUtility2 then it only usescontrollers that support graphics surfaces. If it cannot find a controller thatsupports graphics surfaces for a particular video, then an error is returnedwhen trying to open that video.CVideoPlayerUtility2 is derived from CVideoPlayerUtility which means that the APIs defined in CVideoPlayerUtility arealso available when using CVideoPlayerUtility2; however some ofthe APIs return an error code because the operations are not supportedwhen graphics surfaces are being used.For recording, the CVideoRecorderUtility class, found in videorecorder.h, contains all the APIs that can be used.Most of the calls to the APIs result in operations being requested on thecontrollers.
Certain operations that you request may not be supported onthe particular controller that you are using, in which case an error codeis returned, typically KErrNotSupported.IDENTIFYING VIDEO CONTROLLERS61Some of the API calls are synchronous and some are asynchronous.Where an API is synchronous it means that once the call to the APIhas returned, that operation has been fully completed. Where an APIis asynchronous it means that the requested operation may not becompleted immediately. In this case, a callback is made to an observerfunction that you supply when instantiating the relevant client API class.In normal operation, the call to the API returns and then at some pointin the future your callback function is called to show that the requestedoperation has been completed.
You should note however, that undersome circumstances it is possible that the callback function is calledbefore the call to the API returns.The order of some of the API calls is also important. Some API calls willnot work unless a previous operation has been completed. The majorityof the API methods will not work unless the video open operation hascompleted, and in the text of the book we will highlight any knownexceptions to this. However as behavior can differ between controllers,it is possible that an API method implemented by one controller mayrequire an operation to have completed, while another controller maynot.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
